Women in tech: the trials of unconscious bias

Early last year, my employer, Spektrix, gave me the opportunity to retrain as a software engineer by sending me on a year-long apprenticeship. The first part of the journey was a wonderful 12-week course at Makers Academy followed by nine months back at work as a Software Engineer. There’s then an assessment by an external organisation (BCS in my case). I worked harder than I think I’ve ever worked on anything, receiving glowing feedback from Spektrix about my performance being beyond expectation. Suffice to say, I was feeling confident and thrilled in my new career and was striving for Distinction, the highest of three grades (Pass/Merit/Distinction). I instead received a Pass. It was at this assessment grading phase that I really started to notice the challenges you can face as a woman in tech.

This is the first time I’ve felt that the grade I received wasn’t a true representation of my ability combined with the effort I put in; leading me to the conclusion of bias. It wasn’t the fact that I received a Pass that was the problem. It’s not that I’ve never been disappointed in a grade before: I have and have accepted it. It was the inaccurate feedback and reasoning, many of which were based on working conditions that my male colleague was also subjected to but not equally penalised for, that made me question it.

I admit now that I was living in blissful ignorance before this, as a woman in the arts, where we make up the majority. I couldn’t see the challenges other women were facing, and I wasn’t hearing the conversations that were happening and I wasn’t looking for them. I learned the hard way just how real gender bias is, and how emotionally and mentally draining it can be. I was supported by everyone at Spektrix to question the things that didn’t feel right in my grading. The more I questioned, the more I believed it was gender bias. This has prompted me to do more research to understand why this happens and to spot it sooner and when it’s happening to others.

By sharing my story in its entirety, it’ll hopefully give a real-life, relatable perspective. I hope that this will bring awareness to anyone who was oblivious like me coming into the industry, so they can go in with eyes open, rather than closed like mine were. I’ll go through my initial assessment, the appeal, the appeal escalation and a final conversation with the assessment company.

Along the way, I’ll share some of the research I’ve found that has helped me understand some of the reasons behind gender bias and some of the suggested ways to mitigate it, as signposts in case it prompts someone to read further. I’ve gone back and forward on how much of my personal story to share, and how much to focus on learnings. However, the resources I’ll share are from experts who can make these cases better than I can.

My male colleague and I came from very similar professional backgrounds, the same prior job role at Spektrix and work in similar working environments now. This made inconsistencies in our assessments straightforward to spot. I want to be clear that, by coming to the conclusion of bias, what I mean is — and I believe this post will show why this is true — if I’d submitted the exact same work, with the only difference being that I was male or anonymous, I believe I’d have

A) been treated differently in the interview and been prompted to give the answers they knew I was capable of

B) would’ve been credited more for my achievements and blamed less for my mistakes

and/or

C) wouldn’t have been held to such high levels of merit on the specifics of my work in the first place.

Assessment

Process

To understand the feedback, I want to briefly touch on the components of work which make up the assessment. Throughout the apprenticeship placement at Spektrix, I completed a number of pieces of work:

• Two exams (a programming language and software development processes)
• A portfolio documenting the best of my work against the required standards
• An employer reference backing up my portfolio.

On completion of the nine months on the job, the assessment pieces were sent to BCS for End Point Assessment (EPA). EPA also includes a week-long solo project followed by an interview, which is presented as an informal “professional discussion” designed to fill in any gaps the assessor has and confirm the grade.

I went into the interview — brimming with confidence. I had a solid portfolio and I’d shown in my final project that I was capable of utilising all of the skills and values I’d learned. It didn’t bring up anything concerning that I didn’t feel I could answer, so I thought to myself, “there must not have been many gaps to fill — Distinction, here I come!”

Feedback inaccuracies

I think it’s worth outlining the assessment structure so that the rest of the things I go through are followable, the full details of which can be found here. I’ll refer a lot to the three categories of assessment ‘What’, ‘How’, ‘With whom’ as this makes up the bulk of the feedback. Below is the diagram of what is required to achieve Pass, Merit or Distinction with those categories:

My initial feedback said that I was significantly above the ‘How’, and many of the higher ‘What’ and ‘With whom’ requirements to be considered ‘significantly above’, but not quite, therefore my grade was Pass.

The reasons provided were easily disputable. The reasons and the arguments for why this was disputable were:

Reason for not meeting ‘What’ — I’d not yet acquired the breadth and depth of tools needed and I’d used the same tools in my project as in my day to day job:

I had demonstrated multiple languages, multiple frameworks and many related tools and methods. All demonstrated in my portfolio, project and interview. Feedback indicated the examiner did not have the technical knowledge to know that MVC and Webforms are different tools.

Reason for not meeting ‘What’ — I’d benefit from learning another language:

The requirements don’t state multiple languages are needed; however, my training was in Ruby and my job is C#; I also demonstrated SQL and Javascript throughout the portfolio and/or project.

Reason for not meeting ‘With whom’: I need more exposure to external customers:

My previous role was managing relationships with 100+ of the same clients, and I now work in the Client Impact team dealing with direct requests or working with the Product Owner, which was shown in my portfolio and interview.

Reason for not meeting ‘With whom’: I spoke about ‘tailoring my language’ (a higher requirement of ‘with whom’) only when it came to senior developers:

I chose to focus on this example in my interview because I’d put it as an area I wanted to improve on in my portfolio, while other examples — leading newer developers, sending full company emails about features implemented, full company demo, presenting to a school group about careers in tech, leading a workshop with those kids. Which were all mentioned in the portfolio and demonstrated throughout the apprenticeship.

Reason for not meeting both (I think): I often work in a pair and so that work in my portfolio isn’t showing my competence:

There was lots of evidence of me working solo, taking the lead with newer developers in my pair, and always showing my specific contribution when pairing.

Appeal

Building the appeal

Initially, I had a sense of the feedback and grade being unfair but didn’t automatically think it was because of my gender. I expected mentors at Spektrix to say it didn’t matter about the grade as it wouldn’t affect my future career. Instead, they felt the same outrage as me and added some extra concerns around how my male colleague hadn’t been penalised for the same working conditions. My training provider, Makers, was also supportive of me raising the appeal as they didn’t agree with the assessment either.

Before I began my apprenticeship, I’d received Unconscious Bias training as a hiring manager. One of the key things I took away was how unconscious bias influences the questions you ask in an interview. Unconsciously, if you have a positive bias towards someone and they don’t provide the answer you were looking for, you’ll prompt them further. But if the same happens when you have a negative bias towards them, that just proves your automatic perception. I used this understanding of unconscious bias when drafting my appeal.

The appeal was a seven-page document disputing all of the key reasons given for me not achieving the grade. This included concrete evidence from my portfolio and project, and what I could remember of the interview that directly contradicted the feedback. In addition, I added how the feedback either differed from my colleague’s feedback or directly contradicted it. It also highlighted concerns that I wasn’t asked questions directly relating to the feedback to allow me to fill in those gaps, which I believe was likely led by the unconscious bias I’ve already touched upon. I would’ve liked a second interview, but this was not an option.

Found during my subsequent research, a Harvard Business Review article ‘How to Take Bias Out of Interviews’ makes several arguments for why unstructured interviews should not be an evaluation tool of choice; they are ‘fraught with bias and irrelevant information’.

Advice: Remove any mention of bias

After an initial review, BCS advised me to reword the appeal as there’s no way to request a grade change specifically, only a re-assessment. It instead needed to fit with one of their categories, and we appealed ‘An assessment decision on the basis that we [BCS] did not apply procedures fairly or consistently’.

They also advised me to take out the inconsistencies about my colleague as ‘two individuals cannot be compared’. They said they were giving this feedback to give me the best chance of success, though no guarantee could be made.

Alarm bells were ringing now. How can you be encouraged to not make a complaint of bias? I felt unsettled by this request but I decided to play the game to get the grade and complain about bias separately, so I redrafted. However, my training provider, Makers, who can see all of the work submitted and the feedback, did add a supporting statement on the inconsistencies between the two. They too were unclear on what they, as a training provider, should advise future applicants based on the two pieces of feedback.

Verdict: reasons for escalation

The verdict was that the grade remained unchanged. I’d achieved ‘significantly above’ in two categories, ‘How’ and ‘With whom’, but it was the ‘What’ that needed to change. I believed my appeal was solid and I had such strong arguments and clear evidence that it’d be hard to dispute. I guess, in a way, that was the case: They entirely ignored the evidence in my appeal and instead gave completely new reasons for my grade.

If the appeal verdict had been indisputable, I’d have left it there and accepted the grade. Instead, I would argue that it just added even more concerns.

Inaccurate representation of my work

The assessor stated, “There was no functionality for question answers so that an end-user could actually take a simple quiz.”

Aside from the fact that ‘taking a quiz’ was specifically stated as “outside of scope” in the project brief, I’m not given accurate or full credit for what I implemented:

• View/Add/Edit/Delete quizzes
• View/Add (& custom ordering)/Delete and reordering Questions
• View Answers

From the feedback, it doesn’t appear that they looked closely enough at my code or documentation.

Contradicting interpretations of subjective criteria

The examiner stated that I’d not met the ‘minimum requirements’ for the solo project and so I had not achieved the higher ‘What’. This is not something the original feedback mentioned. In fact, the initial feedback stated: “The project, while incomplete, demonstrated Lauren meeting all the competencies on offer through her chosen specification”.

Due to my own interpretation of the higher ‘What’ requirements, I believed that by focusing more on well-thought-out automated tests than on completing requirements that I’d already shown I could do elsewhere in the project, I’d have been showing more “Distinction-level” skills. I spoke about this decision in my final interview and justified it in my project documentation. I made a choice that the examiner disagreed with, and was penalised for it.

The ‘With whom’ category I disputed in my appeal had actually been increased to award ‘Significantly above’. The new assessor stated, “Lauren demonstrated strong communication skills both in the documentation submitted and her interview. It was evident she is engaged in her work and works proactively with a variety of stakeholders”. Which is very different from the previous, negative feedback for the same assessment category.

With two examiners giving contradicting views on why I didn’t meet the higher grade, not to mention that I’d had an entirely different interpretation, it’s clear that the criteria are highly subjective.

Subjective criteria in performance reviews are open to bias (Why Most Performance Evaluations Are Biased, and How to Fix Them). It appears BCS was looking for any excuse to not award the higher grade, perhaps because I dared raise the topic of gender, or likely based on a biased view of me and my ability which the subjective nature of the criteria has allowed them to uphold.

Inaccurate use of the requirements to justify the grade

Aside from the previous two points being questionable at best, none of it is really relevant because they’ve used the wrong assessment category to justify not raising my grade.

The feedback conclusion was, “Good project management and decisions to meet minimum functional requirements within the specified time frame are required to clearly verify an apprentice meets higher levels of the standard in the “what” criteria.”

In the requirements (final 2 pages) these types of skills are not mentioned in the ‘What’ section, the one needed to raise the grade. Project management is mentioned in the ‘How’ section. It’s so blatantly wrong that I genuinely don’t think that would stand up in a court case if I’d had any intention of taking it there. Despite raising this many times, no-one at BCS or otherwise actually acknowledged or addressed these inaccuracies with the assessment criteria.

It was at this point that I started to realise that the problem was not an individual assessor: I was up against a bigger, systemic issue.

Bias

No credit for accomplishments

Across both pieces of feedback, any positive comments given about my technical skill weren’t even attributed to me, or even inaccurately represented the technical work I’d achieved. The first assessor stated “the emphasis on paired programming enabled her to rapidly upskill from a technical perspective,” and the appeal assessor said, “This collaborative approach has helped Lauren to rapidly upskill her technical skills.” It ignores the fact that I was also leading newer developers and neither accurately represents my own technical ability or hard work.

I’ve since found the Lean In website has a video series about fighting bias. Two videos, Performance Bias and Attribution Bias, cover how women’s performance is often underestimated and men’s overestimated. Because of this, we’re more likely to give them less credit for accomplishment and blame them for mistakes. This rings true for me, from both the assessors while they didn’t credit my technical ability they did pick out anything they could about my soft skills.

Some of the feedback I received vs what my colleague received also demonstrated this. It’s frustrating to see side by side. One (paraphrased) example from my manager’s complaint:

“While he only uses one programming language, here are all of the great things he’s done”

verses

“Here are all the great things she’s done but she only uses one programming language”.

Official complaint

Due to the further inaccuracies and questionable reasons given, Spektrix supported me in escalating this as a complaint to be reviewed by an external body (selected by BCS). We received vague feedback, which stated that they agreed with the assessment. It appeared that they’d had discussions with BCS. It was stated that there was a misunderstanding of the criteria on our side, which is ‘understandable because it is complex’. They stated that they’d informed BCS that they needed to give clearer feedback in future.

There were still no concrete answers.

Up until this point, myself and my manager had tiptoed around explicitly claiming bias. Each response had brought more inaccuracies, inconsistencies and therefore concerns and it was clear they were never going to change my grade so we had nothing to lose.

My manager wrote a formal complaint of bias or discrimination to BCS questioning why two of their employees, whose performances aligned with each other in exceeding all expectations for the apprentice programme, had been awarded different grades with no concrete justifiable feedback matching the assessment criteria. Aside from my experience, how would Spektrix know how to support any apprentices going forward?

“Closure”

Quality exercise

After the escalated complaint, BCS confirmed that they’d run a quality assurance exercise as a result of the appeal escalation. My colleague’s and my work were re-graded, with each assessor apparently agreeing with the grades though I don’t get to see the gradings. Also, some other apprentices that my assessor had graded being re-assessed.

My concerns about unconscious bias in interviews still stand. The appeal assessor can only go on the answers I gave in that interview, which I felt was led based on a preconceived judgement of me. However, BCS believe that, by doing this exercise, they’ve proven that there can have been no bias at play.

In the ‘Let’s Start Talking About It’ chapter of Lean In, Sheryl Sandberg notes that the people who believe they are not biased don’t then correct for it, and it ends up influencing them more. It’s perhaps most frustrating that BCS doesn’t appear to accept the feedback on their process, and the blatant denial could, in fact, be making their process more biased.

Later, to further deny any bias, BCS stated they had “parity” with male and female distinctions in the standard, which is ambiguous. I assume it means that the same percentage of women achieve distinctions as males. BCS doesn’t appear to publish any figures that I could find. I did find the “gender in tech” section on their website where, perhaps coincidentally, the oldest article on that page is the same month as my appeal.

Said article shows that the number of girls choosing computer science at A-level in England has increased by 300% in 5 years to an underwhelming 15%. The study shows that over 34% of girls achieve ‘high’ (A-A*) compared with 26% of men. If this is the case, and the apprenticeship shows ‘parity’, then girls’ achievements are already dropping and it’s only one step closer to a professional career. Plus, gender is only one issue. So can gender parity really be used to deny the process is open to any bias?

Manipulating criteria

BCS offered me a call with a (female) representative to attempt to provide some “closure”. The conversation was unfortunately a frustrating summary of the previous few months. She could only speculate on reasons as she admittedly wasn’t a technical expert and hadn’t seen my interview. These speculations did more harm than good to my feeling of closure. They included the ‘minimum requirements’ the appeal assessor mentioned, but they also included more brand new, possible reasons I didn’t get the grade.

Even if they can justify how not meeting the minimum requirements matches the ‘What’ category, I have repeatedly questioned what the ‘minimum requirements’ were. The project brief had a large list of requirements but it wasn’t ordered or split into essential and additional requirements, or of course, I’d have focused on minimum requirements first. This aside, why is it okay that my colleague didn’t finish all of them either, yet wasn’t similarly penalised? We did prioritise different requirements but I don’t believe that our choice of what requirements to prioritise is the real issue here.

She added that to get Distinction you need to use different tools in your project than those used in your day to day job. I didn’t know this was a requirement, but apparently, these were in some “key indicators” which are not publicly available. Even so, the tools were different and I’d already disputed this in my appeal. To which she speculated that they were possibly not different enough. If this were true, why was it okay that my colleague, who works in the same code-base as me in day to day job, also used the same tools as I did in the project?

The next reason given was that I chose a project which is similar to my day to day job, and apparently, people going for distinction should go for something outside of their comfort zone. The project was similar to one/two of the projects in my portfolio, but also not similar to many other projects in my portfolio. But none of this matters because it’s the same project my colleague chose; what was the difference this time?

Whenever it got to the point where she couldn’t justify the difference in standards I was being held to, she said, ‘well he must have said more in his interview’. Again, back to the unstructured interview.

In Lean In, by Sheryl Sandberg, the chapter titled Let’s Start Talking About It accurately describes what I feel I’ve faced throughout this apprenticeship that I couldn’t find the words for myself. She mentions a study where evaluators for police chief shifted the weight of hiring criteria to fit their biased view. When men possessed a certain quality, they weighted it higher in importance; if they didn’t have that quality they deemed it a less important requirement. The opposite happened when evaluating women:

“The infuriating takeaway from this study is that “merit” can be manipulated to justify discrimination.”

Grand finale

Creating my own closure

If you’ve come this far on the journey with me, I bet you’re thinking that it can’t get any worse…

At the end of my call with the BCS representative, she promised to provide specific clarification which did not materialise, so I needed a way to end the conversation and give myself my own closure. I carefully drafted an email supported by my boss, which would highlight my ongoing concerns about my own experience, and the process as a whole. I also wanted to give them a heads up that I was going to speak out about it. Here it is in full:

Anyone can be biased

The response I received to my email astounded me for many reasons. The email itself looked to be an emotional and defensive response.

It stated first: “As a woman also working in the IT industry I could misinterpret your statement and take exception to the insinuation that a male colleague of yours was in some way advantaged as a male and I am sure that is not an accusation you were making”. This reads as though I’ve made up a strange phenomenon where, in some crazy world, men might be at an advantage, particularly in the IT industry! This comes from the same woman who said on a call, ‘If you were my daughter I’d be right there with you but…’ I was so shocked by this comment that I don’t remember the specific ‘but’, and I can’t understand this logic.

I was, perhaps in my naivety, surprised to find that a woman could so blatantly deny that sexism exists — even after we were in touch because of a complaint of bias. Perhaps this woman has been worn down by the system or has the beliefs so ingrained that she doesn’t see how others around her might be affected. Her response seems to suggest that it can’t be real — and is invalidating my feelings on the matter.

Gaslighting

The email continues:

“I see it as coming from frustration of you trying to compare yourself to a male colleague — don’t compare yourself to anyone you are you.”.

The word ‘gaslighting’ kept coming to my mind, but I thought that was meant for personal relationships or politicians using it to silence a whole group of people. But it was nagging away at me so I did some research and found an article, Feminism 101: What is gaslighting? One sentence resonated with me:

“Gaslighting is a common psychological trick that removes accountability and stifles discourse by laying blame on the victim or convincing them that there is no problem to begin with.”

This solidified my feeling that that’s what was being done to me. Insinuating that I’m comparing myself to a colleague just shows that my concerns hadn’t been heard. His and my work or ability is not what I’m comparing. What I’m comparing is the difference in standards to which we are evidently being held. This aside, it’s blaming me for making myself feel that way and denying any other issue.

Point proven?

The email then added:

“A Pass is a good grade and I understand from [your manager] you excel in your job role but the requirements of the standard, the assessment plan and the occupational brief may not align directly with this[.] [T]he standard is designed for consistency across all sectors and employers.”

I see this statement as truly calling out the crux of the problem. She’s saying that the standards they’ve set don’t align with my high level of performance as perceived by my employer. Can it really be consistent across all sectors and employers — except mine? To me, it solidifies my feeling of them shifting the criteria to suit their view of what a Distinction engineer looks like. They maintain that I was asked the questions to fill in the gaps which I “did not do”. I still don’t know what more I could have said, or what criteria I was even trying to prove.

I should note here that the Institute of Apprenticeships, who write the criteria BCS works to, does appear to recognise that the apprenticeship wasn’t aligning with the values and practices of smaller, less corporate businesses. The apprenticeship requirements are changing and Makers have written a blog about the changes here.

Speaking out

I’d be lying if I said that these last few months haven’t taken their toll mentally and emotionally. Each time I pressed for answers I was filled with hope I’d get some sort of closure, if not a grade change. Instead, I was disappointed and filled with sadness. Then I’d lie awake until the wee hours going over and over the injustices. Then I’d wake up exhausted.

It’d have been so much easier to just take it lying down. But how is anything ever going to change that way? The plus side of this is that I have been so supported by my employer, Spektrix, at all levels in the business. They’ve been as outraged on my behalf as I’ve been and have given me confidence, both in my ability as a software engineer and in speaking out about my experiences.

By speaking out for myself, I hope I’m also speaking out for the future women, and other marginalised groups. I recently read Why I’m No Longer Talking To White People About Race by Reni Eddo-Lodge, where a chapter on feminism stuck in my head. She describes an ideal where feminism stands not only for women but for all groups who are marginalised by the current system. I want to continue to learn and teach myself to spot gender and other types of bias. I know I can’t singlehandedly change the world but I hope this story inspires others to do so. If there’s one thing I’ve learned when trying to call this out, it’s that if something feels off then you’re probably right to question it.

Moving forward

I’d welcome anyone to share their thoughts or experiences with me that I can learn from and I’d be more than happy to talk more about my experiences if it will help someone.

Everyone around me keeps emphasising how it’s a great achievement to have passed and how ultimately the grade won’t affect my career. But because of performance bias, attribution bias, and the fact that requirements can be manipulated, I feel like I needed this Distinction. Had I received a Distinction, I’d have something to prove my accomplishments in my job; something I may need more than my male counterparts if I ever go for an external role.

While this might not be the flying start in the industry that I wanted, I’ve had wonderful training and I work for a fabulous company. I absolutely love what I do now. I’ve got the best professional start I could have hoped for, even if I don’t have the grade to prove it. I’m determined not to accept an unjust system and I plan to keep working hard.