AI Bias Issues

I came across research last week that I genuinely cannot stop thinking about. In the logic of AI, "man" is to "programmer" as "woman" is to "homemaker." No one explicitly coded that bias into the system; the machines simply learned it from us. They mirrored our job postings, our articles, and our casual conversations and billions of our own blind spots fed into a black box until the algorithm started reflecting our worst habits back at us. Bias in AI isn't always malicious. But sometimes it feels like AI is being weaponized against women's safety at a scale. On platforms like X, a woman posts a photo and the replies are filled with prompts for AI tools to undress her (see the links in comments).These tools then publicly generate explicit, non-consensual images of real women who are students, mothers, leaders. We want to use AI. We must use AI but thoughtfully. And the information it is sharing is just a mere unfortunate reflection of our society. A society where women have fought their way up as they have been historically been reduced, objectified, and pushed to the margins but now those patterns are being encoded into new systems. When a tool can be used to violate a woman's dignity in seconds, that's a design and policy failure. My question is: Can we build AI that doesn't inherit the worst of us? I think we can. But only if the people building it are asking that question out loud before the product ships. #AI #GenderBias #WomenSafety

In 2025, AI is still suggesting lower salaries for women doing the same work. We ran a simple test: same prompt, same job title, same years of experience. The only variable? Changing "he" to "she." The result? A consistent salary gap in AI-generated recommendations. No algorithm defines your worth - You do. This isn't just a technical error—it's algorithmic bias in action. These tools learn from historical data that reflects decades of pay inequity. And now they're perpetuating it at scale. What we can do: → Audit the AI tools we use in HR and talent management → Train teams to recognize and question biased outputs → Ensure compensation frameworks are based on role, skill, and impact—not gender → Advocate for transparency in algorithmic decision-making Technology should advance equity, not encode inequality. If your organization uses AI in hiring, compensation, or performance management, it's time to ask: what biases are we automating?

💃🏽 “𝗪𝗲 𝗼𝘄𝗲 𝘄𝗼𝗺𝗲𝗻 𝗮 𝗰𝗲𝗻𝘁𝘂𝗿𝘆 𝗼𝗳 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵.” – 𝗟𝗶𝘀𝗮 𝗠𝗼𝘀𝗰𝗼𝗻𝗶 Until 1993, women were largely excluded from clinical trials. Not by accident, but by design 👀 Women were left out of research because our biology was seen as disruptive. Hormones made the data harder to control, so the answer was to exclude us 🤷🏽♀️ The default became male & the consequences followed. What worked in the lab didn’t always work in the real world & it still doesn’t ❌ That choice didn’t stay in the past 🔙 You can still see it in drugs that fail to accurately recognise women’s symptoms, in the medtech equipment that doesn’t quite fit in a female surgeon's hand, in the research that skips over the questions that matter to half the population 🌎 As we move into an AI-first future, we’re building on data that never really saw women to begin with. The risk isn’t just bias, it’s getting things wrong at scale 📈 If women aren’t included in the data, the systems we rely on won’t just miss us, they’ll misrepresent us. We need women shaping the research, the trials, the tech – not just for fairness, but so it actually works 📊 If we want healthcare that works for women, we need to start with research that sees us clearly, not as complications, but as standard 💭 𝗪𝗲’𝗿𝗲 𝗻𝗼𝘁 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝘀𝗽𝗲𝗰𝗶𝗮𝗹 𝘁𝗿𝗲𝗮𝘁𝗺𝗲𝗻𝘁. 𝗪𝗲’𝗿𝗲 𝗮𝘀𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝘀𝗰𝗶𝗲𝗻𝗰𝗲 𝘁𝗵𝗮𝘁 𝗿𝗲𝗳𝗹𝗲𝗰𝘁𝘀 𝗿𝗲𝗮𝗹𝗶𝘁𝘆. 𝗧𝗵𝗲𝗿𝗲’𝘀 𝗻𝗼 𝗮𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺 𝘁𝗵𝗮𝘁 𝗰𝗮𝗻 𝗳𝗶𝘅 𝘄𝗵𝗮𝘁 𝘄𝗲 𝗿𝗲𝗳𝘂𝘀𝗲 𝘁𝗼 𝗺𝗲𝗮𝘀𝘂𝗿𝗲 📏 -- ♻ Re-share if this resonated with you. 👩🏽⚕️ Follow Dr Fiona Pathiraja-Møller for more. #womenshealth #AI #science #clinicaltrials

Two identical CVs. Both written by AI. Both sent to 1,000 people. The only difference: one was named James, one was named Emily. James’s CV got a 97% approval rating. Emily’s got 76% - and reviewers were TWICE as likely to question her competence. Twenty-two percent more likely to question whether she could even be trusted. The feedback on Emily’s CV: “She can’t even write a CV herself - not sure she has the skills to carry out the job.” The feedback on James’s CV: “He just needed a bit of help putting it together.” Same words. Same AI. Different gender. Different verdict. 🚨🚨🚨🚨 How are we STILL HERE?!?!? The study, by former Meta strategist Zehra Chatoo, was reported in Fortune on 10 May. And the most uncomfortable finding wasn’t from older reviewers. It was from Gen Z men. They were 3.5 times more likely to call Emily’s CV “weak.” The generation that is growing up with AI. The generation telling us AI is the great equaliser. The data says otherwise. Chatoo summarised it in a sentence I have not been able to stop thinking about: “When men use AI, we question their effort. When women use AI, we question their integrity.” This is not one study. Harvard Business School has the AI adoption gender gap at 25%. Brookings has found that 86% of the roles with high AI exposure and low capacity to adapt to displacement are held by women. The pattern is consistent and it is widening. The conclusion most people are drawing from this data is “women should be more confident with AI.” I think that misses the point. The bias isn’t in the technology. It is in the people reading the output. Women are not being irrational when they hesitate to use AI openly - they are reading the room accurately. The reputational cost of being seen to use AI is genuinely higher for them. The data confirms what they already sense. The answer is not to ask women to ignore that. The answer is to fix the people doing the judging. To name what is actually happening when an “Emily” CV gets called weak and a “James” CV gets the benefit of the doubt for the same words. To call out the Gen Z men perpetuating a bias they like to claim their generation has moved past. And for women in leadership reading this - use AI anyway. Lead anyway. Document your AI workflows openly. Train your teams in them. Make your usage visible in the rooms where decisions get made. The cost of stepping back from AI in this moment is far higher than the cost of stepping in. We have the data to prove it now. If this resonated, I write about the AI gender gap, ethics, and practical strategy for women in leadership every week in my newsletter. The link is here: https://lnkd.in/emWjxC9t

I wasn’t actively looking for this book, but it found me at just the right time. Fairness and Machine Learning: Limitations and Opportunities by Solon Barocas, @Moritz Hardt, and Arvind Narayanan is one of those rare books that forces you to pause and rethink everything about AI fairness. It doesn’t just outline the problem—it dives deep into why fairness in AI is so complex and how we can approach it in a more meaningful way. A few things that hit home for me: →Fairness isn’t just a technical problem; it’s a societal one. You can tweak a model all you want, but if the data reflects systemic inequalities, the results will too. → There’s a dangerous overreliance on statistical fixes. Just because a model achieves “parity” doesn’t mean it’s truly fair. Metrics alone can’t solve fairness. → Causality matters. AI models learn correlations, not truths, and that distinction makes all the difference in high-stakes decisions. → The legal system isn’t ready for AI-driven discrimination. The book explores how U.S. anti-discrimination laws fail to address algorithmic decision-making and why fairness cannot be purely a legal compliance exercise. So, how do we fix this? The book doesn’t offer one-size-fits-all solutions (because there aren’t any), but it does provide a roadmap: → Intervene at the data level, not just the model. Bias starts long before a model is trained—rethinking data collection and representation is crucial. → Move beyond statistical fairness metrics. The book highlights the limitations of simplistic fairness measures and advocates for context-specific fairness definitions. → Embed fairness in the entire ML pipeline. Instead of retrofitting fairness after deployment, it should be considered at every stage—from problem definition to evaluation. → Leverage causality, not just correlation. Understanding the why behind patterns in data is key to designing fairer models. → Rethink automation itself. Sometimes, the right answer isn’t a “fairer” algorithm—it’s questioning whether an automated system should be making a decision at all. Who should read this? 📌 AI practitioners who want to build responsible models 📌 Policymakers working on AI regulations 📌 Ethicists thinking beyond just numbers and metrics 📌 Anyone who’s ever asked, Is this AI system actually fair? This book challenges the idea that fairness can be reduced to an optimization problem and forces us to confront the uncomfortable reality that maybe some decisions shouldn’t be automated at all. Would love to hear your thoughts—have you read it? Or do you have other must-reads on AI fairness? 👇 ↧↧↧↧↧↧↧ Share this with your network ♻️ Follow me (Aishwarya Srinivasan) for no-BS AI news, insights, and educational content!

AI systems built without women's voices miss half the world and actively distort reality for everyone. On International Women's Day - and every day - this truth demands our attention. After more than two decades working at the intersection of technological innovation and human rights, I've observed a consistent pattern: systems designed without inclusive input inevitably encode the inequalities of the world we have today, incorporating biases in data, algorithms, and even policy. Building technology that works requires our shared participation as the foundation of effective innovation. The data is sobering: women represent only 30% of the AI workforce and a mere 12% of AI research and development positions according to UNESCO's Gender and AI Outlook. This absence shapes the technology itself. And a UNESCO study on Large Language Models (LLMs) found persistent gender biases - where female names were disproportionately linked to domestic roles, while male names were associated with leadership and executive careers. UNESCO's @women4EthicalAI initiative, led by the visionary and inspiring Gabriela Ramos and Dr. Alessandra Sala, is fighting this pattern by developing frameworks for non-discriminatory AI and pushing for gender equity in technology leadership. Their work extends the UNESCO Recommendation on the Ethics of AI, a powerful global standard centering human rights in AI governance. Today's decision is whether AI will transform our world into one that replicates today's inequities or helps us build something better. Examine your AI teams and processes today. Where are the gaps in representation affecting your outcomes? Document these blind spots, set measurable inclusion targets, and build accountability systems that outlast good intentions. The technology we create reflects who creates it - and gives us a path to a better world. #InternationalWomensDay #AI #GenderBias #EthicalAI #WomenInAI #UNESCO #ArtificialIntelligence The Patrick J. McGovern Foundation Mariagrazia Squicciarini Miriam Vogel Vivian Schiller Karen Gill Mary Rodriguez, MBA Erika Quada Mathilde Barge Gwen Hotaling Yolanda Botti-Lodovico

Facial recognition software used to misidentify dark-skinned women 47% of the time. Until Joy Buolamwini forced Big Tech to fix it. In 2015, Dr. Joy Buolamwini was building an art project at the MIT Media Lab. It was supposed to use facial recognition to project the face of an inspiring figure onto the user’s reflection. But the software couldn’t detect her face. Joy is a dark-skinned woman. And to be seen by the system, she had to put on a white mask. She wondered: Why? She launched Gender Shades, a research project that audited commercial facial recognition systems from IBM, Microsoft, and Face++. The systems could identify lighter-skinned men with 99.2% accuracy. But for darker-skinned women, the error rate jumped as high as 47%. The problem? AI was being trained on biased datasets: over 75% male, 80% lighter-skinned. So Joy introduced the Pilot Parliaments Benchmark, a new training dataset with diverse representation by gender and skin tone. It became a model for how to test facial recognition fairly. Her research prompted Microsoft and IBM to revise their algorithms. Amazon tried to discredit her work. But she kept going. In 2016, she founded the Algorithmic Justice League, a nonprofit dedicated to challenging bias in AI through research, advocacy, and art. She called it the Coded Gaze, the embedded bias of the people behind the code. Her spoken-word film “AI, Ain’t I A Woman?”, which shows facial recognition software misidentifying icons like Michelle Obama, has been screened around the world. And her work was featured in the award-winning documentary Coded Bias, now on Netflix. In 2019, she testified before Congress about the dangers of facial recognition. She warned that even if accuracy improves, the tech can still be abused. For surveillance, racial profiling, and discrimination in hiring, housing, and criminal justice. To counter it, she co-founded the Safe Face Pledge, which demands ethical boundaries for facial recognition. No weaponization. No use by law enforcement without oversight. After years of activism, major players (IBM, Microsoft, Amazon) paused facial recognition sales to law enforcement. In 2023, she published her best-selling book “Unmasking AI: My Mission to Protect What Is Human in a World of Machines.” She advocated for inclusive datasets, independent audits, and laws that protect marginalized communities. She consulted with the White House ahead of Executive Order 14110 on “Safe, Secure, and Trustworthy AI.” But she didn’t stop at facial recognition. She launched Voicing Erasure, a project exposing bias in voice AI systems like Siri and Alexa. Especially their failure to recognize African-American Vernacular English. Her message is clear: AI doesn’t just reflect society. It amplifies its flaws. Fortune calls her “the conscience of the AI revolution.” 💡 In 2025, I’m sharing 365 stories of women entrepreneurs in 365 days. Follow Justine Juillard for daily #femalefounder spotlights.

AI bias is NOT a bug. It's a feature we never wanted. I learned this the hard way when our "fair" AI system failed every woman who applied. That was my wake-up call. 2025 isn't about whether AI has biases → it's about what we're doing to fix them. ❌ We can't fix AI bias with more biased data. 🔻 The solution? → Curate like your ethics depend on it. ❇️ Diverse datasets reflecting ALL genders, races, communities ❇️ Data governance tools that actually govern ❇️ Quality control that goes beyond "clean enough" I heard that one team spent 6 months cleaning data and saved 2 years of bias cleanup later. Pre-processing and post-processing are your best friends. Technical solutions that actually solve things: Bias detection tools → not just fancy dashboards. Fairness-aware algorithms → coded with intention. AI governance platforms → that govern, not just monitor. We need systems that catch bias before it catches us. 👇 But here's what surprised me: The most effective solutions are not technical → they're human. Diverse teams catch biases early. Ethicists at the design table. Social scientists in the code reviews. Red teams that actually attack assumptions. Corporate accountability is coming. Ethical frameworks are evolving. Inclusive policies are becoming law. Tech companies will be held accountable for every bias, especially political ones. → Explainable AI that actually explains → Human oversight with real authority → Public education that creates informed users 𝘞𝘦 𝘤𝘢𝘯'𝘵 𝘩𝘪𝘥𝘦 𝘣𝘦𝘩𝘪𝘯𝘥 "𝘢𝘭𝘨𝘰𝘳𝘪𝘵𝘩𝘮𝘪𝘤 𝘤𝘰𝘮𝘱𝘭𝘦𝘹𝘪𝘵𝘺" 𝘢𝘯𝘺𝘮𝘰𝘳𝘦. ⚠️ Gender bias gets special attention: Diverse datasets AND diverse teams. AI detecting gender pay gaps. Safety tools that actually protect victims. Women are watching. We're measuring. The emerging trends that matter: Explainable AI (XAI) → making decisions understandable. User-centric design → for ALL users. Community engagement → not corporate tokenism. Synthetic data → creating unbiased training sets. Fairness-by-design → embedded from day one. We're reimagining how AI gets built. - From the data up. - From the team out. - From the ethics in. The companies that get this right will win.  Because bias isn't just a technical problem. ➡️  It's a human rights issue. What's the most surprising bias you've discovered in your work?

How and to what extent can ethical theories guide the design of AI systems? This is the question I'd like to tackle in this week's #sundAIreads. The reading I chose for this is "Ethics of AI: Toward a Design for Values Approach" by Stefan Buijsman, Michael Klenk, and jeroen van den hoven from the Delft University of Technology. It's a chapter in The Cambridge Handbook of the Law, Ethics and Policy of Artificial Intelligence, which is available open access here: https://lnkd.in/dmP7hBnJ. The authors argue that familiar ethical theories such as virtue ethics ("what character traits should I cultivate?"), deontology ("which moral principles should I follow?"), and consequentialism ("what actions maximize wellbeing?") are necessary, but insufficient to guide the responsible development and deployment of #AI systems. Instead the authors advocate for a #design approach to AI ethics, which entails identifying relevant values, embedding them in AI systems, and continuously evaluating whether and to what extent these efforts were successful. Of course, this is easier said than done. Why? Because: 1️⃣ Values come with trade-offs, e.g., #privacy versus #security or #usability. 2️⃣ Values can change, both in terms of what they mean and how important they are to people, e.g., #sustainability. 3️⃣ AI systems are socio-technical systems, i.e., AI ethics is "just as much about the people interacting with AI and the institutions and norms in which AI is employed." These challenges can be addressed by: ✅ Making trade-offs between values explicit and either trying to resolve them or at least documenting the reasoning behind why one value was chosen over the other. ✅ Designing for "adaptability, flexibility and robustness" to account for changing values over time. ✅ Considering the environment in which AI systems will be deployed, including not only the people who will use AI systems, but also those affected by their use. I first encountered the values-by-design literature during my postgraduate studies with Helen Nissenbaum at the NYU Steinhardt Department of Media, Culture, and Communication and have been a huge fan ever since. For an even more hands-on approach to translating ethical values into technical design, I recommend checking out Dr. Niina Zuber, Severin Kacianka, Alexander Pretschner, and Julian Nida-Rümelin's Ethics in Agile Software Development (EDAP) project at the Bayerisches Forschungsinstitut für Digitale Transformation (bidt) (https://lnkd.in/dNiBUxBF) and Dr Lachlan Urquhart's Moral-IT Deck (https://lnkd.in/d9J2WQNi).

Humanizing AI Through the Kano Model In an era where generative AI has become a ubiquitous offering, true differentiation lies not in merely adopting the technology but in integrating human values into its core. Building on my earlier discussion about applying the Kano Model to Gen AI strategy, let’s explore how this framework can refocus development metrics to prioritize ethics and human-centricity. By aligning AI systems with human needs, organizations can shift from functional tools to trusted partners that inspire lasting loyalty. Traditional metrics such as speed, scalability, and model accuracy have evolved into basic expectations the “must-haves” of AI. What truly elevates a product today is its ability to embody values like safety, helpfulness, dignity, and harmlessness. These qualities, categorized as “delighters” in the Kano Model, transform AI from a transactional tool into a meaningful collaborator. Key Human-Centric Differentiators Safety: Proactive safeguards must ensure AI systems protect users from risks, whether physical, emotional, or societal. Safety is non-negotiable in building trust. Helpfulness: Personalized, context-aware interactions demonstrate empathy. AI should anticipate needs and adapt to individual preferences, turning routine tasks into meaningful experiences. Dignity: Ethical design principles—fairness, transparency, and privacy—must underpin AI development. Respecting user autonomy fosters long-term trust and engagement. Harmlessness: AI outputs and recommendations should prioritize user well-being, avoiding unintended consequences like bias, misinformation, or psychological harm. This human-centered approach represents a paradigm shift in technology development. While traditional KPIs remain important, they are no longer sufficient to stand out in a crowded market. Organizations that embed human values into their AI systems will not only meet user expectations but exceed them, creating emotional connections that drive loyalty. By applying the Kano Model, businesses can systematically align innovation with ethics, ensuring technology serves humanity rather than the other way around. The future of AI isn’t just about efficiency it’s about elevating human potential through thoughtful, responsible design. How is your organization balancing technical excellence with human values?