When AI transparency tools might endanger human rights

In July, I was in Geneva and attended a gathering hosted by WITNESS and the Forum on Information and Democracy at the headquarters of the International Federation of Red Cross and Red Crescent Societies. The event was focused on how to design AI to be trustworthy and, in particular, content authenticity and the future of reliable information. 

 If you haven't heard of WITNESS before, it is a global organisation that “helps people everywhere harness the power of video and audiovisual technology to tell their stories, document the truth, and defend human rights – today and in a future increasingly shaped by AI.”

At this event, WITNESS launched this really important report:

C2PA Content Credentials and the Surveillance Risk: Adversarial Scenarios and Governance Gaps in the Content Provenance Ecosystem - Library

We've recently been helping companies implement the new AI transparency rules in the EU AI Act that came into force at the beginning of August and the assumption was always that rules about digital watermarking or measures to help people know the provenance of content, who created it and whether AI was involved are inherently positive things. The rules help people know which content to trust, when they're interacting with AI and whether things like emotion recognition or biometric identification are being used.

However, WITNESS' report flagged some risks with technology designed to mark the provenance of content or creations from the perspective of journalists, human rights defenders or people like documentary filmmakers. Now that content credentials are being embedded into devices and the distribution ecosystem, whether that's cameras, mobile phones, AI tools, or distribution platforms, there is a risk that, if used in an adverse political environment, these provenance credentials could be used maliciously. WITNESS tells possible stories in its report about how a documentary maker or a social media content creator who speaks against a government, could be traced using these content credentials or how someone could be denied a role for verifying content online which a government didn't feel was ideologically aligned.

If you're thinking about your position on transparency and digital watermarking, this is a really important read. Thinking about how our implementation of tools designed to promote trust could be used adversarially is a really important stakeholder view many may have missed.

Subscribe to receive more articles like this here.