AI and Equality

News

The Unseen Harms: How Biased Data Blinds AI to Gender Violence in Africa

Exploring Stage 3 of the AI LifecycleData Discovery  with Athandiwe Saba: Bringing to view crucial research by Code for Africa’s iLAB, documenting organized TFGBV campaigns across 11 African countries

Technology-Facilitated Gender-Based Violence (TFGBV) is not just a global issue; it is a critical threat to democracy and human rights across Africa. The latest research from Code for Africa’s iLAB is spotlighted as a case study in the African AI & Equality Toolbox – an initiative by Women At The Table and the African Center for Technology Studies. 

The research exposes a systematic failure in AI systems that leaves African women and LGBTQ+ communities profoundly vulnerable. The core of the problem is a combination of: 1). Biased and exclusionary training data, a blind spot rooted firmly in Stage 3: Data Discovery and Preparation of the AI life cycle.

The Toolbox breaks down the AI life cycle into six crucial stages. Stage 3  asks important questions:

  • Who collected the data and for which purpose?
  • What data pre-processing steps are required to create a “fair” model in this context?
  • What historical/present bias in the data might compromise human rights?


The Research: Mapping a Coordinated Threat

Athandiwe Saba, iLAB’s AI Newsroom Initiative Lead, has directed years of crucial research, documenting organized TFGBV campaigns across 11 African countries. The attacks are not random harassment; they are sophisticated, synchronized, and political.

Key Research Outcomes:

  • Systematic Blindness: Content moderation AI, primarily trained on English language data from North America and Europe, is systematically blind to African realities. It fails to detect hate speech and slurs in local languages like Swahili, Wolof, or Bambara.
  • Evasion Tactics: Bad actors deliberately use African languages, emojis, and modified spellings (e.g., using “@” instead of “a”) to easily circumvent automated moderation tools.
  • Coordinated Attacks: The research uncovered large-scale, coordinated campaigns, such as the targeting of Brenda Biya in Cameroon, where 34 identical harassment posts appeared simultaneously, accumulating close to 10 million views on Facebook alone. Platforms fail to recognize this organized activity as a collective threat.
  • The Weaponization of AI: The study highlights the chilling use of AI-generated deepfakes—like those targeting Ethiopia’s mayor, Adanech Abiebie—to destroy reputations and silence women in politics, demonstrating AI’s role in amplifying political violence.

 

The Insight: Data Gaps Equal Rights Gaps

One of the most profound insights from this work is that data gaps are not neutral; they are tools of exclusion. In the Data Discovery and Preparation stage, developers must collect and assess training data. When African languages, cultural slurs, and specific regional patterns of harassment are not included in the data sets, the resulting AI models are fundamentally incapable of recognizing harm.

This kind of failure isn’t technical; but rather a matter of consent and ethics. The research reveals that data sets often include harassment content that targeted women without their consent for its use in training moderation models. A truly human-rights-aligned approach, as Saba suggests, would require users to opt-in to data usage, not be forced to opt-out of an opaque process.

 

The Path Forward: From Extractive to Co-Creation

The critical lesson for policymakers and tech practitioners is that current AI governance is failing because it is built on an extractive, global-North-centric model. Human rights-aligned data practices in Africa require a fundamental shift:

  1. Prioritizing the Disadvantaged: Design tools and platforms with the most vulnerable communities in mind (the “ramp not stairs” principle).
  2. Localizing Data & Language: Invest in training data and models for the hundreds of African languages and local harassment taxonomies.
  3. Feminist Counter-Measures: The solution is not merely better filters, but building independent, feminist-led technical capacity—the creation of small, focused language models and forensic counter-measures by Africans for Africans.

 

For AI to be a force for equality, we must fix the foundations. The Data Discovery stage must become a conscious act of inclusion, ensuring African voices and lived experiences are central to the future of technology.

 

An African AI Toolbox: Integrating Human Rights considerations along the AI lifecycle

The Toolbox applies a Human Rights-based AI Lifecycle Framework, integrating reflective questions and the Human Rights Impact Assessment (HRIA) developed with the Alan Turing Institute.

It emphasizes participatory, multidisciplinary approaches and is rooted in feminist, decolonial, and Justice, Equity, Diversity, and Inclusion (JEDI) principles and incorporates lessons from emerging digital rights challenges, ensuring AI systems are designed with safety and dignity at their core.

Explore the  Webinar Series 

We’re creating a global community that brings together individuals passionate about inclusive, human rights-based AI.

Join our AI & Equality community of students, academics, data scientists and AI practitioners who believe in responsible and fair AI.