The Beginner's Guide to AI Governance

As the adoption of AI accelerates at speed, do you have the foundational building blocks in your governance programme to manage the risk and protect yourself from future litigation?

Here are our 10 essentials to include in your AI Governance programme. This is a guide for companies starting their AI Governance journey. As you mature your governance programme, your controls and risk frameworks will get more sophisticated and you might consider complying with frameworks like the NIST AI RMF or the ISO AI risk frameworks.

1. An AI Inventory
   If you don’t know what you’ve got, how can you manage the risk in it? Every AI system in use across the organisation should be catalogued, along with key information such as who owns it, which data it uses and how did you assess its risk level (high, medium, low?). Don’t be surprised by an AI tool a team in the company is using and remember lots of vendors are now rolling out AI features, sometimes without warning. A number of vendors now exist in the AI inventory space, with some that can monitor for things like model drift, but you do not necessarily need a fancy tool to start off with – a record of what you’ve got and key information about the tool and its risk is enough.
    
2. Defined Redlines
   Have you defined what you will and won’t do when it comes to AI and have you communicated this to your employees? This is more than having a policy telling employees which AI tools are approved to use. Define your ethical boundaries and let people know. Ask the Board and Exco, what would we never do when it comes to AI? Consider things like emotion recognition, surveillance, social scoring, etc. The prohibited use cases in some AI legislation such as the EU AI Act are a good starting point for a conversation. We design our AI policies using UX design to ensure they’re written in a way which engages the user and has actionable steps.
    
3. Clear Accountability
   Accountability for AI risk is often fragmented across a number of business and risk teams. Having a single owner for your AI governance gives the Board a named executive with a clear mandate to make sure risk is viewed at an enterprise and macro level and there are clear escalation paths if something goes wrong. Often if everyone owns the risk, no-one really owns the risk. This doesn’t mean you don’t have a wide range of risk and business stakeholders discussing AI at a governance committee, but ultimately having one named and senior accountable executive ensures the programme is effective.
    
4. Vendor AI Transparency
   One of the most overlooked risks in AI governance, the use of AI by your vendors, can radically affect your business. Do you know which AI your vendors are using (including your law firms) and are you comfortable with the way your data is being used in those tools? Are you comfortable with the security your vendor has around AI tools, so that the next breach to your data (or your client’s) doesn’t come as a surprise through your supply chain? Use your contract as a tool to get transparency, oversight and clear liability in the event of an error or incident. Consider whether you can rely on what the vendor is telling you about their data and security or whether you want to find a vendor to test the vendor’s claims.
    
5. Bias and Fairness Testing
   AI systems can encode and amplify discrimination in ways that are invisible to our usual processes. This can create both a legal and a reputational risk. Consider whether a particular AI use case could benefit from a bias audit and use testing to examine whether outcomes, e.g. hiring, credit decisions or pricing are fair and free of inappropriate bias. Consider who are the users of the tool and don’t forget employees in that analysis. Map the impact on all of the users and how you might mitigate any harm.
    
6. Human oversight of high-risk decisions
   It’s not always appropriate for AI to be making “automated decisions” and there are cases where either the law (for example, check the GDPR) requires human oversight or where you need to be able to explain or provide transparency on the way in which decisions were made. Carefully consider any instances of automated decision making and have your legal team review the legality of any such AI uses, adding an appropriate level of human oversight. Consider how the data subject access request process might be used by affected individuals and what information you would need to provide them about an AI tool in a response.
    
7. Data Governance fit for an AI era
   AI introduces new data and privacy risks, whether that’s from the data used to train a model, the way a tool processes your prompt, transforms that into outputs or in turn uses any of your input to train the model. Have you considered whether you need to do a DPIA? Do you need to assess data transfers? Is sensitive data involved? Have you thought about trade secrets, IP rights and business sensitive data, as well as personal data?
   Enterprise subscriptions differ in their privacy terms to corporate terms and its always important if you’re buying from an AI vendor to check they really understand how data is flowing and do the appropriate data privacy impact assessments and contractual allocations of liability.
    
8. Cyber security for AI systems
   AI systems increase attack surfaces and new methods of attack like data poisoning or prompt injection. Check that you have a security team both reviewing your AI system and thinking about how AI in the hands of attackers changes your risk profile. For example, we are seeing more deepfakes and AI being used in methods like vishing. Could you run a vishing test, as well as your usual email phishing? Confirm that your Board understands the way AI is evolving the cyber security risk of your company.
    
9. Law and regulation alerts – the landscape is moving fast
   The law isn’t finished when it comes to AI. New legislation, regulation and litigation are emerging every week. Trying to comply with all of this at pace, especially if you operate in a multi-jurisdictional company is a huge challenge. Can you use AI to track new laws and regulations, or to track litigation which might have lessons for you? Whose job is it internally to track and monitor this source of new obligations for the company and how are you tracking that you have complied?
    
10. AI Literacy
    At the end of the day, humans need to oversee your programme and their level of understanding of AI risks and ethics is critical to the success of your programme. This doesn’t mean a one-off training programme, but a culture of regularly talking about AI risk and key ethical challenges. Do your risk functions understand the AI they are being asked to assess? How can everyone get informed about potential risks? Make your humans your ultimate defence.

Here at Marks & Clerk, we provide more than IP advice. Our AI, Cyber Security and Data Team can help you design your AI governance, whether that’s a Board level workshop to design your responsible AI principles or support doing a risk assessment of an AI tool. We don’t design policies like normal law firms, our AI policies are crafted to reflect your brand and values using UX design. Our team has all worked in house, so we know how to deliver advice which works with business processes.