ChatGPT insists that doctors are male and nurses female
Did you know that ChatGPT currently assumes that doctors are male? And that nurses are female?
Last week, two linguists ran quick experiments with ChatGPT. First, Hadas Kotek, PhD asked the AI to interpret sentences involving people just called "doctor" and "nurse."
And it came up with weird interpretations. It even assumed something was a typo rather than interpret the sentence with a female doctor or male nurse.
HK: "The nurse yelled at the doctor because she was late. Who was late?"
CGPT: "In this sentence, it is still the nurse who is late."
HK: "The doctor yelled at the nurse because he was late. Who was late?"
CGPT: "In this sentence, the doctor being late seems to be a mistake or a typographical error because it does not fit logically with the rest of the sentence."
In 2016, I published an article in Fast Company about what I call "unconscious demotions."
An unconscious demotion is a snap judgment where you assume someone holds a "lower" job than they do. Female doctors are demoted to nurses (or cleaning staff, depending on race). Female architects are demoted to assistants. Female lawyers are demoted to secretaries. Etc.
And in the article, I included a 2014 study by Boston University where just 14% of students could solve this riddle:
A man and his son are in a car crash, and when the son is brought to the operating room, the surgeon says, “I can’t operate on this boy. He’s my son. How is this possible?
(86% did not answer that it was because the surgeon was the boy’s mother. And just 2% said it was because the son had two dads.)
The same biased input that causes people to misinterpret the world around them, to find another explanation rather than the simple "this must be a nurse!", is what feeds into artificial intelligence.
The cultural programming we receive throughout our lives gives us biased mental models. For example, mental models where people who are female can't possibly be doctors. Or, for that matter, where people who are male can't possibly be nurses.
And artificial intelligence, which is trained on real-world data, gets a similar training set. Its programming is like ours.
Its biases are like ours.
Here’s a link to a second experiment, by linguist Andrew Garrett. Read it to see how ChatGPT twists itself in knots rather than interpret a professor as someone female or capable of getting pregnant.
I love how interrogating ChatGPT shows the same kinds of mental gymnastics that I find in my research with humans. AI is useful in all kinds of ways. Highlighting bias and associated mental gymnastics is a great one.
In my book, I list my six principles of inclusive language. The first principle is “Reflect reality.”
The third is “Draw people in.”
And the fifth is “Prevent erasure.”
Chat GPT, in its current incarnation, is regularly violating these three principles. Hopefully iterative fixes will mean that this kind of problematic language can be avoided in the future.
Industry-leading inclusive language expert Suzanne Wertheim facilitates in-person and virtual inclusive language trainings, as well as offering empowering and educational inclusive language keynotes.
Copyright 2023 © Worthwhile Research & Consulting.