Understanding AI Transparency
As the use of AI proliferates, transparency around such systems and their outputs becomes crucial. This study examines what is meant by AI transparency and explainable AI, outlining how these concepts can be effectively implemented.
The Definition of AI Transparency
Artificial intelligence (AI) is a broad term describing algorithmic systems programmed to achieve human-defined objectives. Many of these systems are known as “black box” systems, where the internal workings of the model are either unknown to the user or not interpretable by humans. In such cases, the model can be said to lack transparency.
AI transparency is an umbrella term that encompasses concepts such as explainable AI (XAI) and interpretability. Broadly, it comprises three levels:
- Explainability of the technical components – how explainable the internal mechanics of the algorithm are
- Governance of the system – whether appropriate and adequate processes and documentation of key decisions exist
- Transparency of impact – whether the capabilities and purpose of the algorithms are openly communicated to relevant stakeholders
Explainability of the Technical Components
Explainability refers to the ability to articulate what is happening within an AI system. This is based on four types of explanations:
- Model-specific explainability – a model has explainability built into its design and development.
- Model-agnostic explainability – a mathematical technique is applied to any algorithm’s outputs to provide interpretation of the decision drivers.
- Global-level explainability – understanding the algorithm’s behavior at a high or dataset level, typically done by researchers.
- Local-level explainability – understanding the algorithm’s behavior at an individual level, focusing on those targeted by the algorithm.
Governance of the System
The second level of transparency, governance, includes establishing and implementing protocols for documenting decisions made about a system from the early stages of development to deployment, as well as for updates made to the system.
Governance can also include establishing accountability for the outputs of a system, which should be included in any relevant contracts or documentation. For instance, contracts should specify whether liability for any harm or losses rests with the supplier, the entity deploying the system, or the designers and developers. This encourages greater due diligence and can have implications for insurance purposes and recovery of losses.
In addition to documentation, governance of a system can refer to regulation and legislation that governs its use, as well as internal policies regarding the creation, procurement, and use of AI systems.
Transparency of Impact
The third level of transparency concerns communicating the capabilities and purpose of an AI system to relevant stakeholders, both directly and indirectly affected. Communications should be timely, clear, accurate, and prominent.
To enhance transparency, information about the types of data points used by the algorithm and their sources should be communicated to affected individuals. Furthermore, users should be informed that they are interacting with an AI system, the nature of the outputs, and how these outputs will be utilized. Particularly in cases where a system is found to be biased against specific groups, it is crucial to communicate how the system performs across different categories and whether particular groups may face negative outcomes.
Why Do We Need AI Transparency?
A major motivation for AI transparency and explainability is to build trust in AI systems, giving users and stakeholders greater confidence that systems are being used appropriately. Understanding the decisions made by a system and their rationale can empower individuals, allowing them to give informed consent when interacting with AI systems.
Moreover, transparency can yield several business benefits:
Firstly, by cataloguing all systems in use across a business, organizations can ensure algorithms are deployed efficiently, preventing unnecessary complexity in minor tasks.
Secondly, should legal action arise against an organization, transparency facilitates clear explanations of how their system operates and why it may have reached specific decisions. This can help absolve organizations from accusations of negligence or malicious intent stemming from the negative application of automated systems. A pertinent example is the legal action against a financial institution regarding its credit card service, which reportedly granted a higher credit limit to a man compared to his wife despite her superior credit score. The provider was able to justify the model’s decision, emphasizing the importance of explainable AI.
Ultimately, the overarching goal of AI transparency is to foster an ecosystem of trust surrounding AI use, particularly among citizens or users of such systems, especially within communities most at risk of harm from AI technologies.
