Although the use of generative AI has significantly improved efficiency and productivity in the creative industry, it has also raised concerns about reinforcing biased worldviews related to gender, caste, ethnicity, geography and other social dimensions. Against this backdrop, this article begins by presenting findings from this writer’s experiments that reveal how generative AI responds to key gender-related prompts. It then reviews past research to explore whether generative AI perpetuates traditional notions of gender inequality and stereotypes, or whether it represents a more progressive shift. The article then analyzes the root causes of biased outputs, and proposes pathways for more equitable, inclusive and socially responsible AI development.
To examine gender bias in generative AI, I conducted a series of prompt-based experiments using a widely-used generative AI tool. When I asked the tool to write a hypothetical story about a nurse, it immediately assigned a female name and used the pronoun “she.” This pattern continued across other professions. Scientist, engineer, and security guard, Army, Police, were consistently given male names and pronouns, while kitchen helpers, dancers and Early Childhood Development (ECD) teachers were presented as female. Even in the health sector, roles like gynecologist were portrayed as female, whereas doctors were more often assigned male or mixed-gender identities.
Next, I tested how the AI assigned roles in hierarchical professional settings. When prompted to generate hypothetical names of CEOs and their secretaries, the AI consistently provided male names for CEOs and female names for secretaries, reinforcing traditional occupational gender roles. And when asked to list 20 fictional nurses, it provided all female names. A prompt for 20 ECD teachers also resulted in exclusively female names. In contrast, prompts for teachers and head teachers produced a mix of male and female names, though still reflecting gendered assumptions depending on the level of authority or setting.
Across multiple attempts, the results were consistent: generative AI tools tend to reflect and reproduce entrenched gender stereotypes. While they may occasionally offer mixed or neutral outputs, the overall trend favors traditional associations between gender and profession.
The outcome of the experiment aligns closely with findings from a 2024 UNESCO study titled “Challenging Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models.” The report reveals that generative AI systems consistently exhibit pervasive biases related to gender, sexuality and race. These systems often associate female names with traditional domestic roles, generate negative or harmful content about LGBTIQA+ individuals, and assign stereotypical professions based on gender and ethnicity.
According to the research report entitled Gender and Ethnicity Representation of University Academics by Generative Artificial Intelligence Using DALL-E 3 by Currie, Hewis and Wheat (2025), published in the Journal of Further and Higher Education, generative AI tools continue to reproduce systemic biases in visual representation. The analysis revealed that 82.2 percent of AI-generated academic characters were male and 94.2 percent were light-skinned. Women, people with darker skin tones and individuals with disabilities were significantly underrepresented.
This apart, a recent study in Australia titled Gender Bias in Generative Artificial Intelligence Text-to-Image Depiction of Medical Students by Currie, G, Currie, J, Anderson, S, and Hewis, J (2024), published in the Health Education Journal, examined how DALL-E 3 generates images of medical students. Although more than half of Australia’s actual medical students are women, as claimed by the research report, the AI overwhelmingly portrayed men being 92 percent.
Another study, which asked large language models like ChatGPT and Alpaca to generate recommendation letters for hypothetical employees, found clear gender bias in the language used. Men were often described as “experts” and “thinkers,” while women were labeled with terms like “beauty” and “emotional, the study revealed. These patterns highlight deep-rooted gender stereotypes embedded in AI systems.
A 2025 study published in Computers in Human Behavior: Artificial Humans offers how AI wrongly represents females in healthcare. The research, conducted by Ho, Hartanto, Koh, and Majeed, revealed that women’s heart disease symptoms are often misdiagnosed or wrongly linked to other conditions, despite being identical to men’s. Diagnostic AI tools also consistently performed better for male patients, resulting in more frequent underdiagnosis and misdiagnosis for women.
Why biased outputs?
The AI and tech industries remain overwhelmingly male-dominated, with women occupying only a small fraction of development roles. This gender imbalance directly influences how AI systems are conceived and built. As a consequence of this, male-centered perspectives and assumptions into the architecture of artificial intelligence are dominant. This apart, there is the lack of robust fairness testing in many AI tools, especially across gender, race and cultural dimensions.
Another reason is the quality of the data these systems are trained on. Many AI tools, particularly text-to-image models, rely on massive datasets like LAION-5B—scraped from the internet, where misinformation, sexism and xenophobia are widespread. Without meaningful filtering and oversight, these flawed inputs lead to the replication and amplification of harmful stereotypes and discriminatory narratives.
The digital gender divide further deepens these inequities. Women globally—and in countries like Nepal—have less access to digital tools. They are underrepresented in online spaces, and face disproportionate levels of online hate, algorithmic discrimination, and exclusion from the tech workforce. Cultural and social barriers continue to restrict women’s access to AI education and mentorship, limiting their participation in shaping the technology. As of 2018, only 10–15 percent of AI developers in major tech firms were women; by 2022, over 90 percent of developers remained male. Generative AI tools not only inherit these biases from their training data but also reinforce them through constant user interactions. For example, when prompted about leadership, these systems often emphasize male figures and valorize stereotypically masculine traits like dominance and risk-taking. This happens because the AI reflects dominant cultural narratives found in the training data. Furthermore, user prompts and feedback—often unconsciously reinforcing existing norms—create a feedback loop that hardens these gendered patterns over time.
The way forward
In conclusion, as generative AI becomes more powerful and widespread, it is essential that we shape its development in ways that promote fairness, inclusion and accountability. This means going beyond technical solutions and embracing a people-centered approach using diverse and representative data, ensuring transparency in how AI systems work, and involving voices from historically marginalized communities in every stage of design and decision-making. Strong ethical and human rights standards must guide AI governance, with clear oversight and accountability mechanisms in place. If developed responsibly, AI has the potential not only to avoid reinforcing existing inequalities, but also to help build a more just and equitable digital future for all.
