Stopping "Shadow AI by default": How to deal with vendors turning on AI features without your consent

The Problem

Increasingly, software vendors are shipping AI features as silent updates, rather than opt-in products. A CRM gets a “smart summarisation” feature turned on overnight. A note-taking app starts sending transcripts to a third party LLM to generate action items. Many of these new features do not require you to click “upgrade”, they just appear, maybe buried in a changelog. 

This creates exposure for companies who are valiantly trying to govern AI, but can't keep pace with the constant flow of new AI features and tools hitting their infrastructure. Is data now flowing in ways the company didn't anticipate? Are your vendor contracts adequate to determine liability for this new use case and is your AI inventory constantly out of date?

This is not to imply any malicious intent on behalf of the vendors, who are under enormous pressure to add AI features to their products, but how can companies evolve their IT governance to maintain control and oversight over what is hitting their stack.

Let's be honest, there's no perfect solution to this problem, especially in a world of AI vendors or providers with whom a purchaser may have a significant power imbalance. Here are some ideas to help companies manage their “shadow AI by default” risk.

Internal Governance 

Maintain a live AI inventory, not a point-in-time one. Consider how often you revisit your AI inventory and what the triggers are for a review. Ask yourself if “vendor release notes” could act as a review trigger point, rather than regular review dates. 

Assign ownership for monitoring vendor changes. Who is the designated team or person responsible for tracking vendor product announcements, changelogs and terms of service updates? Can this monitoring happen more frequently for your critical stack? 

Classify data as well as tools. As well as asking whether the vendor's tool is AI-enabled, have you already mapped what data each vendor tool already touches? If the tool touches sensitive or high risk data, any new AI feature should trigger review immediately, regardless of how the vendor frames the feature.

Ensure you have a rapid intake and kill-switch process. When a new AI feature appears, your team needs a fast path to a) evaluate it; b) disable it, if necessary; and c) communicate the decision internally.  

Require internal sign-off before AI features go live, not after. Where possible, configure vendor admin consoles so AI features default to “off” and require a deliberate internal decision to switch on.

Using your contracts

Contracts with your vendors might be a helpful tool to drive better discipline from vendors in AI feature shipping. Think about whether you can insert the following amendments into your existing vendor contracts and include them in any new negotiations:

Advance notice of new AI or automation features. Require the vendor to give you written notice, with a defined lead time, before introducing new AI tools, AI features or changing any AI or automated decision functionality.

Default “off”/opt-in requirement. Specify that new AI features must ship disabled by default for your organisation, until you have affirmatively opted in.

Right to audit and request documentation. Include the right to request information about how an AI feature works, what data it consumes and what safeguards exist. 

Change control and unilateral amendment limits. Consider any terms which allow the vendor to unilaterally change functionality or change with notice and if possible, require a positive step from your organisation to affirm acceptance.

Termination rights. Include a right for you to terminate the contract if the vendor rolls out an AI feature you cannot accept and make sure you have a right to disable it without breaching the contract. Consider AI specific indemnity rights.

Compliance representations. Put the onus on the vendor to assess a new feature and to inform you, for example, if the feature would be high risk under frameworks like the EU AI Act. 

(There are obviously many other clauses relevant to AI related and vendor contracts, not least related to data privacy, which you need to consider in addition to these clauses focused on “shadow AI by default”)

When the vendor won't negotiate

Sometimes you simply won't have any leverage to negotiate terms with the vendor. With major AI providers, standard terms will be in place. In these circumstances, you should at least ensure you are contracting on their enterprise or business version of their terms. Check whether the vendor offers an admin console setting, zero retention data option or opt-in feature flag that achieves the same outcome operationally. In some sectors industry groups may be able to negotiate collectively for additional AI addenda to meet their needs. If you are in a situation where a DPA needs to be negotiated, could this be the place to include your AI clauses? If you are not able to change the contractual position with the vendor, have you made a risk decision to accept that risk and documented it? Ask yourself what features might require you to exit the tool and can you operationally do that, at short notice. 

Contracts and operational controls can be two collaborators in your AI governance program to ensure you are aware of your risk and able to manage the AI you choose to implement. 

This article is provided for general informational purposes only and does not constitute legal advice. It should not be relied upon as a substitute for advice from a qualified lawyer or attorney. Consult legal counsel before relying upon or implementing any of the information contained here.

Subscribe to receive more articles like this here.