1a. Objective
Stage 1. Defining objective
The first essential step is to reflect on the objective and purpose for building a system. Often, the vision of what AI should look like, and who it should support only reflects the needs of the people in power as opposed to the needs of the communities it will serve and affect. To remedy this issue, early engagement and participatory development with the affected communities is essential.
Essential questions example:
- How can affected communities be included in the design process?
1b. Team
Stage 1. Team composition
Numerous people are involved in the creation and operation of an AI system – and more than just people writing code! Your objective will fundamentally inform the team composition required to create the system fulfilling the objective. This includes the skills and expertise needed, but also the diversity of its members regarding backgrounds, perspectives, and experiences with the environment for which your system is developed.
Essential questions example:
- Is the team diverse regarding lived experiences (culture & demographics)?
2. System requirements
Stage 2. System requirements
At the second stage, the system’s objective is formalized into a list of requirements. The list of system requirements should be developed in dialogue with various other roles to ensure the system is feasible (e.g. the developer team). Often, the dialogue with the various communities and roles includes no-code prototypes to communicate ideas, such as sketches, wireframes, or other prototypes.
The process of defining the system’s requirements should be iterative and fluid; it is very likely that the list of requirements will change as more details about the context and the needs of impacted communities become apparent.
Essential questions example:
3. Data discovery
Stage 3. Data discovery
In this step, developers reflect on whether your data is representative of your use case and context.The dataset can (and often is) the first way in which bias enters the system. To avoid this, it is essential that a good socio-cultural fit of the dataset is ensured with the intended application context, e.g. regarding demographics, culture, or environmental factors.
Domain experts should be consulted to ensure that the dataset captures the application context correctly. Their feedback on new or further data collection should be followed, as well as regarding mathematical methods to balance the dataset.
Essential questions example:
- Who collected the data and for which purpose?
- What data pre-processing steps are required to create a “fair” model in this context?
- What historical/present bias in the data might compromise human rights?
4. Model selection
Stage 4. Selecting and Developing a Model
Before the model can be developed, it is important to consider what type of AI model is the best to satisfy the system requirements. It is not always the most complicated deep-learning algorithm that is the most suitable for a context. Instead, it is about choosing the most suitable model for the required scope while managing trade-offs.
The model development itself should be seen as an iterative process.it is important at this stage to look back at data collection, to ensure there is a good fit between the data, what you are intending to model and the actual model chosen. This process is also seen in the testing stage, where the model is tested against the objective and then iteratively improved until it meets success metrics.
Essential questions example:
- Model ethical considerations:
- What type of model is most useful for this objective?
- Can this type of model fulfill fairness in a way that is suitable for the context?
- Explainability and transparency
- Can the model alert the user if it is confronted with an instance outside of the scope of its training data?
5. Test and Interpret
Stage 5.Testing and interpreting outcomes
After the model has been developed, we have to test whether it fulfills the system requirements defined in stage 3. For some metrics, this can be done via technical tests, others require stakeholder feedback, e.g. whether the intended level of explainability was achieved for the end-user.
It is important to place affected communities at the core of the assessment. At this stage, the involvement aims at assessing whether the requirements defined in stage 3 are fulfilled or respected appropriately. Since the affected parties were involved throughout the project, this stage can be understood as their final ‘sign-off’ of needs and risk mitigations identified and addressed throughout the development process.
Insights gained from the testing stage should inform a ‘manual’ handed to the future operators that defines the contexts in which the system is expected to operate well as well as situations in which the system is expected to perform poorly
Essential questions example:
- What measures of model performance are being tested, and why were those measures chosen?
- How much human oversight is required, and by a supervisor with what level of expertise?
6. Deploy and Audit
Stage 6. Deployment
The deployment step is something like the last sanity check, i.e. whether all harms, discriminatory impacts and consequences have been considered, communicated, and are accounted for. We recommend conducting a thorough Human Rights Impact Assessment) to ensure that the system was assessed for negative Human Rights impacts in its final form.
It’s crucial that operators and strongly affected communities are able to alert issues they experience in relation to the system. The decision as to whether the system is ready to be deployed is powerful. We recommend gathering the opinions from affected communities on the system’s readiness – after all, they have to bear the consequences of a faulty operation.
Post-deployment
Directly after the deployment, the system should be audited regularly in post-deployment audits, including opportunities for affected communities to provide feedback. The deployed system might expose previously unknown challenges or problems which have to be detected.
Lastly, ensure that the system complies with relevant regulations and guidelines, such as GDPR, CCPA, the new EU AI Act, or industry-specific standards.
Essential questions example:
- Deployment:
- Who decides the model is ready to be deployed?
- Have you conducted a Human Rights Impact Assessment where the full model operation is known?best practices available?
- Post-deployment:
- Are there processes that allow operators to alert suspected system inaccuracies?
- What alerts you to if the objective loses its purpose? How would you know if it was time to retire the system?
