On 8 Dec, after over 36 hours of negotiations, a political deal has been reached on the EU’s Artificial Intelligence (AI) Act.
The European Parliament, Council and Commission have brokered a high-level compromise on the main rules for artificial intelligence in Europe. However, whilst EU negotiators celebrate the conclusion one of the most controversial digital legislations in EU history, the devil will be in the detail. The extent to which the AI Act protects people in Europe from the worst excesses of surveillance, discrimination and AI-based harms will require a fuller assessment of the technical drafts, which will surface over the next weeks. “EU governments have been forced to admit that AI systems are increasingly used for mass surveillance, racial profiling, and other harmful and invasive purposes. Whilst the deal contains some limited gains for human rights, it will be mostly a shell of the AI law Europe really needs.” - Sarah Chander, Senior Policy Adviser, European Digital Rights, sarah.chander@edri.org
So far, we know the deal includes: - A ban on live public facial recognition, but with several exceptions for the search for certain victims, suspects, and for the prevention of terror attacks. Whilst these conditions are narrower than EU member states had pushed for, they nevertheless create a path to use these systems in dangerous, discriminatory and mass surveillance ways;
- A limitation on post (retrospective) public facial recognition to the search for suspects of serious crime only, although the threshold for what constitutes a “serious” crime is currently undetermined;
- A partial prohibition on predictive policing including predictions as to likelihood of committing a criminal offence on the basis of "personality traits or characteristics", but not the majority of predictive policing systems;
- A full ban on emotion recognition systems, but only in workplaces and educational settings. This illogically omits the most harmful uses of all: those in policing and border and migration contexts;
- A purported ban on biometric categorisation systems, although whether or not this ban will be meaningful is still unclear;
- Limited rights-based safeguards, including a mandatory fundamental rights impact assessment, and some level of public transparency as to which high-risk AI systems are deployed in Europe;
- Very broad loopholes in the overall level of protection, including a wide discretion for AI developers to decide their systems are not ‘high-risk’, and also various exemptions to the rules for when law enforcement, migration and national security authorities deploy ‘high-risk’ AI. This is a major loss for public oversight of the most concerning AI systems.
“It’s hard to be excited about a law which has, for the first time in the EU, taken steps to legalise live public facial recognition across the bloc. Whilst the Parliament fought hard to limit the damage, the overall package on biometric surveillance and profiling is at best lukewarm. Our fight against biometric mass surveillance is set to continue.” - Ella Jakubowska, Senior Policy Advisor, European Digital Rights, ella.jakubowska@edri.org
|
