Governments employ artificial intelligence (AI) for various purposes, from data analysis to public service optimization and education. However, a significant concern with AI is its potential to amplify existing biases in the data on which it is trained. Therefore, relying on AI without considering its gendered implications could significantly compromise outcomes and policy decisions.

The Problem

In Guanajuato, Mexico, the Secretariat of Education faced high school dropout rates. In partnership with the World Bank via the Educational Trajectories initiative, the Secretariat created an AI-based early alert system, the Early Action System for School Permanence (SATPE), aimed at improving school retention and graduation rates by identifying and then supporting at-risk students.

As part of USAID's Equitable AI challenge, Athena Infonomics worked with PIT Policy Lab, Itad, and Women in Digital Transformation to guide the Secretariat of Education’s approach to AI integration, emphasizing the need to move beyond a narrow focus on privacy and data protection.  

Mitigating Gender Bias

The consortium supported raised awareness about the crucial need to incorporate a gender perspective in the upcoming phases of Educational Trajectories. This involved explicitly addressing and mitigating potential gender-based bias in the databases and interventions resulting from the development of the AI early alert system.

We used a three-pronged strategy to support the Educational Trajectories team.

• Leveraged IBM's AI Fairness 360 toolkit to detect bias

Utilizing IBM's open-source AI Fairness 360 Toolkit, the consortium uncovered a significant gender bias that could have resulted in misidentifying up to 4% of at-risk girls, potentially denying them the support they needed to continue their education.

• Developed an AI Ethics Guide for public officials to complement their work in AI

We developed an Ethical Guide for responsible AI implementation, covering an overview of AI, ethical concerns, case studies, reflective questions, a glossary, and a literature review.

• Created a Checklist to ensure responsible and equitable deployment of AI systems.

We also created a Checklist for AI Deployment, offering structured phases and comprehensive questions for ethical AI implementation. These tools address ethical concerns at different governmental levels, ensuring consistent practices.

Policy Recommendations And Way Forward

While SATPE serves as an impactful illustration of how government agencies can leverage AI to tackle urgent social challenges, AI-based tools are at risk of perpetuating existing biases and yielding discriminatory outcomes. The consortium offered practical policy recommendations covering self-assessment, decision criteria, inclusivity, transparency, system strengthening, and diagnostic precision, providing a broad framework for ethical and inclusive AI implementation across sectors.

Athena led a knowledge sharing event, discussing the relevance of this engagement with stakeholders in Indian state governments.

To learn more about this work, contact Shruti Viswanathan.