Gender Equality By Design

So Iris, thank you very much for joining us today. Let me start off with an introductory question. Based on your work, what are the biggest issues in gender equality, you would say?

Well, the biggest issues, it’s a big question.  I think I have to answer it in two ways. One is, what are the biggest issues that the kind of work that I do, focussing on behavioural insights, can solve? Then, you know, what are generally the biggest issues? So let me start with the generally biggest issues. I think one of the generally biggest issues, is gendercide, which is sex selective abortion and neglect of girls.

The UN now estimates that around 200 million women and girls are missing around the world. That is more than all of the people who died in the battlefields of the 20^th century, so clearly that must be one of our biggest issues. I’m not claiming that I can solve that one, but because you asked me what the biggest one is, I think that would be very close to the top of my list.

Okay, and which are the issues that you’ve worked on personally?

We work more on issues related to the workplace and to school that have to do with unconscious bias. So there are lots of reasons that we see gender gaps. One of them is unconscious bias, and that’s what we’ve been focussing on.

Can you explain to us what this unconscious bias is?

I might best be able to do this with an example. So about 15 years ago, a colleague of mine, Kathleen McGinn at the business school, wrote a case study on Heidi Roizen. Heidi Roizen is a venture capitalist in Silicon Valley, a real person. The case study is a typical type of case study that people who have done an MBA or an undergraduate degree would be familiar with, where the case talked about how Heidi created her enterprise, invested, built networks in Silicon Valley and became a successful entrepreneur.

The case became interesting a few years later because then colleagues at Columbia Business School took this case and replaced Heidi’s name with ‘Howard’. They didn’t change anything else in the case, so the very same information, but gave half of the students a case with the protagonist being called Howard and the other half with the protagonist being called Heidi. Then at the end of the case they asked students to evaluate how well these different leaders performed, but also how much they liked Heidi and Howard respectively.

This has turned into the favourite tool for all of us in the educational business to teach people about unconscious bias at the moment, because what we find again and again is that people think that both Heidi and Howard did do a good job, so were successful and performed well, but they don’t like Heidi. They don’t like Heidi because Heidi doesn’t conform to our stereotypes of what a venture capitalist looks like, and she doesn’t conform to our stereotypes of what a good woman does.

That’s unconscious bias. So this is unbeknownst to these students and also unintentional. I mean, these students weren’t out there to get Heidi or punish Heidi. You know, maybe some were. Mostly they weren’t, they weren’t aware that they were evaluating Heidi more harshly and differently than Howard, even though both then did exactly the same thing. That’s what we call ‘unconscious bias’.

So that’s basically preconfigured ideas of what a venture capitalist looks like and what certain roles look like?

Yes, it’s these models, images in our minds that have a lot to do with what and who we see. So seeing really is believing. As long as we don’t see more female venture capitalist or male nurses, for that matter, we don’t naturally associate those jobs with women or men respectively.

Obviously it’s a Catch-22 because it’s so much easier to get into it in the first place and then shift the unconscious bias.

Yes, I don’t know if it’s a Catch-22, but yes, you are right. Our mind-sets are super hard to change. I’m not calling it a Catch-22 because I think there are things you can do, but it is not changing that mind-set. It’s not the first training. We definitely have to do other things, in particular going into organisations and de-bias how we do stuff rather than de-bias mind-sets.

Yes, and in your research, you’ve looked at different types of organisations. Have you found significant differences, say between businesses, government institutions, universities and the like?

No, I wouldn’t actually even think that that was a big deal. I mean, most organisations have to recruit people, they have to hire them, evaluate them, evaluate them for promotions, for performance appraisals. Surely some of these systems differed between the public sector and the private sector and NGOs, but fundamentally, everyone has to deal with people. As soon as you do deal with people, those biases tend to creep in.

So whenever I work with an organisation, the first thing we do is just to measure these, just try to understand what is going on in the organisation; do they have issues? Do they have issues with gender but more generally with diversity, at the entry level, at the more senior levels? Then we’re trying to think about the types of interventions that could help them level the playing field and be able to benefit from 100% of the talent pool.

In your recent book, you propose recipes for what can be done, and one of the key insights seems to be, and you’ve alluded to this a bit earlier, is that you want to de-bias organisations rather than individuals?

Yes.

Why is this the better way?

So the first part of the book really tries too hard to collect all evidence that I could possibly find on whether the types of interventions that we have been focussing on in the last 20 to 50 years – I’m saying 20 to 50 depending a bit on the country that we’re in. In this country, in the United States, it started really in the ‘60s that we introduced diversity training programmes in our companies.

So diversity training is one of the interventions that I looked at. The biggest message here is that we typically just don’t mesh. That really is, I mean, the overarching message on those types of interventions. We just do them because they’re best practice, because others in our sector and our industry do them or just because the law might make us check a box. So we introduced these two hour trainings or two day trainings focussed on de-biasing mind-sets. So this is the big message: that we don’t know.

The second insight that I gained was that we probably have about 20 ish, 20 studies which were acutely done in a randomised fashion. We had a control group, which didn’t get the training, and a treatment group, which got the training, and then we actually measured what difference this made in people’s behaviours.

For example, in one experiment, these were students in a school. Some of the students got diversity training and others didn’t and they then evaluated whether students who had gotten diversity training were more likely to play with, interact with, like, hang out with kids who looked different to themselves, in terms of race, in terms of also body, looks, just everything.

They found nothing, basically nothing, so that was quite discouraging. So that is what most of these studies found. So it’s not that I started writing this book trying to prove that diversity training doesn’t work. I started writing this book because I wanted to find evidence of what works. That’s why the book is called ‘What Works?’ When you check, anywhere, and this included diversity training in companies, but also included, for example, reconciliation training in post-conflict countries, really trying to understand how we can get into these minds. That just proves to be super hard.

It’s just not an easy task and I don’t think anyone has figured out that silver bullet. Again, I also looked at other types of interventions, so very much in the US you can trace the type of things that we’ve been doing over the last 50 years. So the first wave of interventions very much focussed on diversity training as a compliance exercise or as a checked box exercise. So maybe we shouldn’t be too surprised if they weren’t particularly effective.

Then the community broadly speaking moved into helping women and other under-represented groups to succeed in the environment they’re in. So these are leadership training, leadership development training, negotiation training, any kinds of development activities. Again, while were some were encouraging, we actually don’t know whether they are working and the little evidence that we do have is certainly quite mixed.

It appears as if details really matter and the more we can turn these types of leadership training into real capacity building exercises, where people then are building the capacity to go back into the organisation and use, for example, the tools and insights they have gained tomorrow, the more likely they will be succeeding. So that’s the leadership training.

Then the last pocket of interventions, which have nothing to do with de-biasing environments, are related to mentoring, sponsorship types of initiatives and networks. More encouraging, again, so the list, I mean, it does get better as I go down this list. Better diversity training to leadership training to now, mentorship, sponsorship and networks. They become more hands-on and really focus on this capacity building rather than just training people with abstract concepts.

So that’s the first part of the book. I’m just trying to understand what works, what kind of things we’ve been doing. Then the last 2/3 of the book then take a different perspective and say, “What if instead of focusing on the people, either fixing their minds or fixing the traditionally disadvantaged groups, what if we fixed the organisations?”

Okay, and how do you develop this focus on the organisation?

So, broadly speaking, three brackets that we could look at. A first one that is a huge one is around talent management. Let me give you three examples of things that organisations could be thinking about. One is a particularly low hanging fruit, and that is our job advertisements. More specifically that’s the language that we use in our job advertisements.

What I’m saying is low hanging fruit is for two reasons. One is, it’s not really disruptive to the organisation, to the processes in an organisation to be a bit smarter about how they phrase their job advertisements. Secondly it’s low hanging fruit because we now have the technology available. A number of start-ups now will provide the software for companies and organisations and agencies in the public sector to be really independent where you are to just decode, quite literally, your de-bias, your language that you use in your job ads.

For example, not use gender specific adjectives in advertisements for teachers, which typically tend to have more feminine coded words, or male coded words when you look for an engineer. So that’s a very low-hanging fruit and that’s just something that every organisation should do, because every organisation must be interested in benefitting from a full set of talent and attracting the right kinds of people.

It’s also interesting that now some of the large companies such as SAP are developing some of these very same tools. So I think it’s very safe to predict that in five, and certainly in ten years from now, our HR will be completely revolutionised. So that’s on job advertisements. Then let me go into some thornier issues, and that is once people are applying for jobs, how to evaluate them.

That’s where a ton of biases happen, particularly because so many of us believe, I mean, including probably you and I believe that we are excellent interviewers. Most people think that, you know, by seeing somebody, by feeling somebody, they can really tell whether somebody is a good fit for an organisation. Of course that word ‘fit’ itself already exemplifies why this is a problem, because we tend to look for people who look like we do or who look like the workforce that we already have, because that’s one definition of ‘fit’.

That’s what interviewers, when interviewed about their interviews, would say. They would quite literally say, I have nothing else to go by so I’m using myself as a reference point. So we are replicating ourselves, which clearly can’t be good in terms of benefitting from diversity of thought and background. So what do we do? It turns out that these uncertain interviews are the worst predictor of future performance. I can safely say that based on lots of research. They are the worst predictor of future performance, so clearly we have to do something different.

The best predictor of future performance, and that won’t sound like rocket science, because it isn’t, is a work sample test. I mean, if I could test what I’m actually looking for in a job candidate, what kind of jobs, what kind of tasks the person’s going to have to do in the future, that would of course give me a good sense of whether that person is qualified or not. Maybe to be more specific, I recently hired an executive assistant for myself and we actually followed my own advice in that we created work sample tests.

I spent quite a bit of time thinking about, “What are the tasks that are really important to me that I really want to have a person be good at?” One of the tasks that’s been important for me is scheduling.  So we quite literally gave the person a scheduling problem off one of my recent trips, going to New York, speaking to women and travelling to London and doing a talk there, and kind of, asking a person to tell us, I mean online, so I hadn’t seen the person yet at all, you know, what kind of information does he need, does she need to make sure that I’m going to have a successful trip.

So work sample tests are maybe one of my most important messages: that we should really move into work sample tests. Having said this, I’m not naïve enough to believe that we are willing to give up the interviews soon. Most people feel quite strongly about this one on one interview of the finalists. So here is what people can do; they can at least move from unstructured interviews to structured interviews. The structured interview is defined by the following characteristics:

First of all, we have to think about the questions that we want to ask beforehand. So quite literally you would write down a list of five questions or ten questions beforehand, and you would use the very same five questions in all of your interviews of the different candidates that you talked to. Not only that, but you would in fact rate a person’s answers after each question.

That’s important because what we found was that your answer to question one might influence how I perceive your answer to question two. We can’t completely exclude those order effects, of course, because I will ask questions in some order, but we do want to at least give ourselves the chance to be slightly more objective with question two, three, four and five. That’s why we recommended rating each question right after..

Right after it was asked.

Yes. Then at the very end, obviously everyone will tell us that it’s very important to also have an unstructured part of the discussion where, for example, I can talk about the institution, I can talk about Harvard and a person can ask a question. So that is absolutely relevant, but it should be made very clear, that that’s not part of the interview, that’s not part of the formal interview.

So when I interview people now, I tell them, I have five prepared questions, and that’s for me to evaluate whether you could be great at the job that we’re interviewing for, and then at the end, I’m very happy to put this aside and have a conversation with my candidate.

There are a number of more things that people can do that I go through in my book in great detail of how we also use comparative evaluations in our interviewing processes where we have found – this is joint work with Max Bazerman at the Harvard Business School and Alexander van Green at Erasmus University in Rotterdam. We found that when people aren’t forced to compare a job candidate with others, but basically focus on this one person, what happens in our minds is that we use this internal reference, the stereotypes as a comparator.

Because it turns out that a lot of research in behavioural science strongly suggests that it is basically impossible for our minds to make absolute judgements. This has nothing to do with gender or people in particular, but just imagine the coffee that you drank this morning, or the bread that you ate this morning, how much did you like it? Whether or not you liked it or how much you liked has something to do with the kinds of coffees that you’re used to drinking or the kinds of breads that you’re used to eating, and it’s the same thing that’s happening for people.

So we have this urge to compare people with something that we are used to, the kinds of coffees that we’ve been drinking or the kinds of people that we have been confronted with in this kind of job. So in order to overcome that stereotype, what we show is after we have done our interviews, say, with ten different candidates, we compare candidates across questions, that would be much better in calibrating our judgements than just focussing on one candidate exclusively.

This question would still be designed to try to identify organisational fit?

So I don’t like the word ‘organisational fit’, because ‘fit’ again, when you ask people what it means to them, they will typically tell you, “Look for somebody who looks like me or looks like us.” So fit means going for homogeneity or the people who look like us.  That’s why I’m concerned about ‘fit’. But, yes, I mean, ideally of course what we would have to do is keep track of which questions that we use are in fact predictive of future performance. So big companies have done that.

Google, for example, has kept track of which of their questions are particularly predictive of future success at Google and then they exchange questions, right? So in future years they know question five, for example, might not have been useful and they try out a new question. So that’s the ideal scenario. I totally get it, not every organisation is as big as Google. Not every organisation has the analytical capabilities that Google does, but that would be the ideal scenario, because nobody can really tell you which questions are particularly great in your specific organisation.

That’s actually where your earlier question of, “What are the differences in organisations?” where that becomes relevant. I mean, for an engineering job, maybe a different question is relevant than for a managerial job. There is no question about that.

So there is a lot of learning by doing in this as well?

Oh there is learning by doing, but again, there are now start-ups who have developed the technology to make it easier for us to not have to reinvent the wheel. Again, I’m totally aware that a large majority of our companies have fewer than 50 employees. So they never have the data to do big data analytics the way Google does. So they are called Be Applied. That’s a UK company. It’s part of the behavioural insights team of the UK and they develop great software, which walks people through what I just described.

I mean, it’s more complex than that, but basically it very much builds on the insights in my book on anonymising applicants, something we haven’t talked about yet, but just taking out the demographic information, so I’ll get back to that in just a second, but take out demographic information completely from the job application and then thinking about these particular questions, evaluating those questions comparatively and also benefiting from what they call ‘the wisdom of the crowd’ in that we have different evaluators look at the different answers to the questions and submit their own scores independently of each other.

That’s another very important insight, because so many organisations do these panel interviews, which I urge people to discontinue tomorrow, because there is so much evidence suggesting that when we do this kind of work in a group, we will of course influence each other. So if we have five people on a panel, these are not five independent observations, but it’s basically a sample size of one, with these five people influencing each other.

So much better for these five people, if you want to take five, to have independent evaluation or independent interviews with the candidate. So Be Applied is one, that’s a UK one. In the US we have one, it is called Unitive, another great software platform doing some very similar things with some tweaks and differences, but also building on behavioural science, anonymising the applications and then walking people through a very structured process of interviewing, work sample tests and evaluations.

So yes, it is learning by doing, but maybe more importantly, either using a technology that is built on lots of insights, or if you want to learn by doing, don’t just learn by doing, but learn by measuring and really evaluating what’s happening.

What has been your experience? Are organisational leaders open to changing the way they recruit people? I mean, at the end of the day, the aim must be to create more diverse and as a result better performing organisations.

So I think we’re at the beginning of this journey, so I would say there about 20-ish start-ups in the space now, and most of them emerged in the last two years. So there is one that is a bit older that is called Edge, that is a Swiss company that helps companies self-evaluate, just measure how well they do in terms of gender equality, for example, in terms of pay or representation, but also just evaluate how well they do in terms of the kinds of input factors that are correlated with gender equality.

So it’s happening, but it’s happening now. That’s the reason we have to interview now. It really is happening now. So you might know that the UK government announced recently that they will move to blind evaluation procedures. So blind evaluation procedures, I want to come back to that, I think is something that is going to happen very soon. That’s that an organisation such as the UK government, which of course is the biggest employer in the UK, will not know whether a male or female is applying for the job or what nationality or what race or ethnicity the person has, because often of course that comes with the name that we see on our applications.

That’s super promising. Now in contrast to the UK, there are other countries, such as Germany and Switzerland, for example, Switzerland being my home country, where we still use photos with our CVs and we still add photos to our applications, which is a horrible practice. So again, lots of evidence that what somebody looks like is not a good predictor of future performance. So in terms of blind evaluations, I’m actually quite optimistic that more and more organisations will pick that up, and literally either work with one of these start-ups that blind themselves to the names of the applicants, or do it themselves.

Well, I mean, you say we are right at the beginning of this journey, and I hope our conversation today can help to popularise some of these steps that you suggest and have a proven impact. Iris, thank you very much for taking the time to talk to me today and I hope, you know, that a lot of leaders will take your advice.

Thank you very much for having me.