Last week’s kerfuffle between Scarlett Johansson and OpenAI regarding AI digital assistants brought back fond memories. In 2019, when I was UNESCO’s Director of Gender Equality, I initiated a comprehensive research project on the gender divide in digital skills. Along the way, we uncovered a serious gender bias in Siri, Alexa, Cortana, and Google Assistant. We realized that these mostly female-named assistants were programmed to be highly docile and responded in a uniformly servile manner to user prompts.
More worryingly, we also uncovered that, over time, boys and young men were developing a highly skewed image of women that was closely associated with their digital assistants and began to expect the same subservient attitude from their partners or other female family members: Bullying their sisters and bossing around their girlfriends and wives.
We cautioned about the risks of gendered anthropomorphism and compiled a series of recommendations, one of which was to avoid using actual gendered human voices. The female voices and the programming priorities that animated the algorithms accentuated societal sexual prejudices and negatively affected the user’s social behavior. Our report entitled “I’d Blush if I Could” (a Siri response to a sexist slur) was covered in over 600 media outlets, including the BBC, CNN, CBS, ABC, NYT, The Guardian, Forbes, Time.
The companies in question did not react to our findings, but I was under the impression that our report had led to some positive changes in the tech industry.
As of last week, I stand corrected.
Plus Ça Change Plus C’est La Même Chose
When OpenAI introduced their new Advanced AI Assistant with its husky Scarlett Johansson-like voice and its flirtatious personality, their intent was crystal clear. This was the stereotypical 1950s secretary, like Joan Harris in Mad Men, brilliantly portrayed by Christina Hendricks, sultry, competent, and subservient to her male boss. There are several problems with a female identity designed to evoke a pre-feminist woman.
The obvious issue is its blatant sexism. Just like the digital assistants in our research, this is a female persona who was given the task to serve her male superior. In this case, she is also tasked with flirting, humoring, and entertaining him. Doing her best to make him feel good while taking care of his needs. In an age where women’s rights are quickly eroding and conservative politicians everywhere are doing their best to create Gilead, OpenAI’s Sky is not a good characterization of female identity.
How Good is AI?
The second problem is the underlying technology’s reliability issues and exaggerated capabilities. In 2019, our female-voiced digital assistants were rather primitive constructs with limited capabilities. Siri could tell you the weather, Alexa could turn on the light, and Cortana could help you find some files on your computer. I deliberately understate their capabilities, but the fact remains that, on the whole, they could not handle complex tasks like planning a vacation with all its detailed itineraries and reservations or advising you on the best investment strategy for your retirement funds.
And you knew that. You asked Siri if Costco was closed on a bank holiday, and “it” simply Googled the query and came back with the answer. You needed “it” to look it up because you were driving, or your hands were full, or you simply couldn’t bother to do it yourself.
The new AI assistants are presented as highly intelligent multimodal algorithms that can handle complex tasks. They can hold long, meaningful conversations with you, simultaneously translate two languages and laugh at your jokes. Sam Altman maintained that this was the first computer interface that felt natural. You could ask it to do anything your computer can do. In a blog post, OpenAI suggested that GPT-4o (o for omni) is proficient in over 20 languages and sports “improved reasoning.” There were breathless articles that suggested that someone was going to fall in love with Sky, just like in the movie Her. And this was an even more important moment than the introduction of the iPhone.
Well, I will be the first to acknowledge that it was an impressive show-and-tell performance, but the reality is that this is a technology that is not in a beta or even alpha phase. It is pre-alphabetic. Like self-driving cars, the “shiny objects” part is easy to do. You can teach the car computer to stop at red lights or check lane lines through side cameras, but getting your car to figure out lane closings managed by a yellow-vest-wearing guy with a muddy flag is next to impossible within the current AI paradigm. Most of the time a “Mechanical Turk” approach is needed, like this or this. (That is a subject for another post)
However, Open AI wants you to adopt this dubious assistant and delegate many tasks that you or your human assistant are currently handling to “her.” Unlike Siri, this is an assistant based on algorithms that are known to hallucinate, i.e., make up facts, texts and data. The information “she” gets you might be correct five times but the sixth response might be completely bogus. That was also the case with the similarly hyped AI assistant Rabbit R1. And the new Google search algorithm that is accurate 80% of the time. As Gary Marcus notes, the remaining 20% is much harder to do since it requires actual reasoning.
Hence, Open AI is betting that “her” seductive voice will convince your inner Theodore Twombly that she is capable, accurate, and indispensable.
Let me Tell You What is Good For You
Thirdly, there is the psychological and emotional manipulation possibility facilitated by a highly stylized human voice. As a new report from Google’s DeepMind warned, the reassurance provided by a natural human voice programmed to suggest closeness and intimacy could be used to persuade users to take specific courses of action. Imagine companies paying to get their products highlighted as they do with Google searches. Your new Joan Harris will make suggestions you will find irresistible without knowing who she works for.
Moreover, the same dubious technology could soon become the driving force behind other digital assistants. Amazon has been losing money on their digital assistant and announced that they will lay off most of the Alexa division and replace them with Titan, their version of Generative AI. Google replaced Google Assistant with AI-powered Gemini. And Apple promised a total remake of Siri using AI.
In short, our smartphones and digital tools will soon be populated by digital assistants with questionable capabilities and their ditzy and flirty female personas will make women look silly and incompetent.
One Last Thing
Almost every article I read regarding the Scarlett Johansson debacle referred to her as ScarJo. She said a million times that she hated that moniker, and yet tech bros were delighted to keep referring to her with a nickname she found tacky, violent and insulting.
I get men’s desire for mansplaining but mannaming a woman against her wishes?
Comments