Examining the Black Box: Tools for Assessing Algorithmic Systems

Examining the Black Box is a joint report from the Ada Lovelace Institute and DataKind UK that clarifies terms around algorithmic audits and impact assessments, and the current state of research and practice.

The report is primarily aimed at policymakers, to inform more accurate and focused policy conversations. It may also be helpful to anyone who creates, commissions or interacts with an algorithmic system and wants to know what methods or approaches exist to assess and evaluate that system.

Download the Examining the Black Box report

As algorithmic systems become more critical to decision making across many parts of society, there is increasing interest in how they can be scrutinised and assessed for societal impact, and regulatory and normative compliance.

Clarifying terms and approaches

Through literature review and conversations with experts from a range of disciplines, we’ve identified four prominent approaches to assessing algorithms that are often referred to by just two terms: algorithm audit and algorithmic impact assessment. But there is not always agreement on what these terms mean among different communities: social scientists, computer scientists, policymakers and the general public have different interpretations and frames of reference.

While there is broad enthusiasm among policymakers for algorithm audits and impact assessments, there is often lack of detail about the approaches being discussed. This stems both from the confusion of terms, but also from the different maturity of the approaches the terms describe.

Clarifying which approach we’re referring to, as well as where further research is needed, will help policymakers and practitioners to do the more vital work of building evidence and methodology to take these approaches forward.

We focus on algorithm audit and algorithmic impact assessment. For each, we identify two key approaches the terms can be interpreted as:

  • Algorithm audit
    • Bias audit: a targeted, non-comprehensive approach focused on assessing algorithmic systems for bias
    • Regulatory inspection: a broad approach, focused on an algorithmic system’s compliance with regulation or norms, necessitating a number of different tools and methods; typically performed by regulators or auditing professionals
  • Algorithmic impact assessment
    • Algorithmic risk assessment: assessing possible societal impacts of an algorithmic system before the system is in use (with ongoing monitoring often advised)
    • Algorithmic impact evaluation: assessing possible societal impacts of an algorithmic system on the users or population it affects after it is in use

Further research and practice priorities

For policymakers and practitioners, it may be disappointing to see that many of these approaches are not ‘ready to roll out’; that the evidence base and best-practice approaches are still being developed. However, this creates a valuable opportunity to contribute – through case studies, transparent reporting and further research – to the future of assessing algorithmic systems.

We look at the state of research and practice in each approach and make a series of recommendations, tailored to regulators, civil society, researchers, public and private sectors and data scientists on the ground.

Find out more:

-> Download Examining the Black Box
-> Read the launch blog for Examining the Black Box
-> Learn more about DataKind UK
-> Read about the wider Algorithmic State programme
-> Follow the conversation with @AdaLovelaceInst and @DataKindUK on Twitter (#ExaminingTheBlackBox)