Examining the black box: Tools for assessing algorithmic systems is aimed primarily at policymakers, to inform more accurate and focused policy conversations around algorithmic audits and impact assessments. It is also intended to help those creating, commissioning or interacting with an algorithmic system to understand what methods or approaches exist to assess and evaluate that system.
Examining the Black Box is a joint report from the Ada Lovelace Institute and DataKind UK that clarifies terms around algorithmic audits and impact assessments, and the current state of research and practice.
- Cited in joint letter to EU commission calling for regulatory inspection of algorithms
- Influenced thinking and terminology used (inspection over audit) in (confidential briefing) joint paper ‘Algorithm Inspection and Regulatory Action’ from Demos, Doteveryone, Global Partners Digital, Institute for Strategic Dialogue and Open Rights Group
- Cited in the CMA report published January 2021: Algorithms: how they can reduce competition and harm consumers
- Starting to see referencing in Academic conference papers e.g. at ICML
Report with recommendations and findings of a public deliberation on biometrics technology, policy and governance
A research partnership with NHS AI Lab exploring the potential for algorithmic impact assessments in an AI imaging case study
Examining how the commitment to responsible data in the UK's National Data Strategy could be realised and what it misses
Requirements that governments and developers will need to deliver in order for any vaccine passport system to deliver societal benefit