Commentary by Ivo Emanuilov, IP Lawyer / Researcher, KU Leuven Centre for IT & IP Law – imec

In April 2023, the Procurement of AI community, which supports public buyers in procuring AI-enabled solutions that are trustworthy, fair and secure, published a draft set of model contractual clauses for procurement of AI. This first draft was followed up by a final set of model clauses released on 5 October 2023.¹ The work of this open community has been supported by the Directorate-General for Communications Networks, Content and Technology (DG CNECT), the Directorate General for Internal Market, Industry, Entrepreneurship, and SMEs (DG GROW), Living-in.EU – an EU-funded project, supporting digital transformation of public authorities across Europe, and Pels Riicken, a Dutch law firm.

The model clauses are available for public organisations that want to procure an AI system developed by an external supplier. In light of the impending adoption of the AI Act later this year, these model contractual clauses offer a practical solution to public organisations that may need to procure AI systems now but would like to be sure that these systems would be aligned with the upcoming requirements. The authors underscore that these clauses need to be customised to the organisation’s specific procurement needs.

This brief analysis focuses on the clauses that interplay with intellectual property rights that may subsist in the different components of an AI System. It also explores to what extent these provisions are compatible with the procurement of AI Systems that include components, incl. machine learning models, that may be covered by open source licences.

A two-tier approach: high risk and non-high risk (light) AI procurement clauses

The clauses exist in two versions for high risk and for non-high risk AI systems, respectively, but the authors of the clauses highlight that they can also be applied to algorithmic systems more generally.

The two sets of clauses differ mostly in the obligations of the Supplier in relation to the AI system. For example, the Supplier is obliged to put in place a quality management system for high risk AI systems, to ensure the AI system has undergone a conformity assessment prior to delivery, to demonstrate compliance with the clauses, cooperate in an audit or other types of inspection, and to assist the Public Organisation in registering the AI system in any relevant register.

The model clauses are not a full contractual arrangement, and they do not include conditions on intellectual property, acceptance, payment, delivery times, applicable law or liability. All of these additional clauses need to be aligned with the organisation’s policy and have to be drafted by a legal counsel.

Intellectual property rights clauses: hidden in plain sight?

Even though the model clauses do not contain explicit clauses of intellectual property, there are relevant stipulations regarding disclosure of information about the functioning of the AI System and the rights to use the Data Sets.

Rights to use data sets

The clauses provide that the Supplier grants the Public Organisation the right to use, share

and disclose information required to explain how the AI System arrived at a particular decision or outcome, if and to the extent necessary to inform the persons or group of persons on which the AI System is (intended to be) used about the functioning of the AI System and/or in any legal proceedings. Specifically, this includes the following:

• Source code of the AI System
• Technical specifications
• Data Sets
• Technical information on how the Data Sets used in developing the AI System were obtained and edited
• Substantiation of the choice for a particular model (architecture) and its parameters, as well as information on the performance of the AI System.

The stipulations on the rights to use the Data Sets specify that all rights, including any intellectual property rights relating to either Publication Organisation Data Sets or Supplier and Third Party Data Sets accrue to the respective entity, ie, the Public Organisation, the Supplier or a Third Party.²

In the case of Supplier or Third Party Data Sets, the clauses stipulate that the Public Organisation must be granted a non-exclusive right to use these data sets in a way sufficient for performance of the provisions of the agreement.³ Optionally, the Public Organisation may also be granted a right to use the Supplier or Third Party Data Sets for further development of the AI System either by the Public Organisation or a third party.

Both sets of model clauses stipulate that on request of the Public Organisation, the Supplier must destroy Public Organisation Data Sets and provide evidence of the destruction.⁴

IP indemnification clauses

The indemnification clauses for high risk AI Systems concern specifically infringement of intellectual property rights.

The Supplier of a high risk AI system is under an obligation to indemnify the Public Organisation against all claims brought by third parties in respect of any breach of their intellectual property rights, privacy rights or equivalent claims relating to knowledge, unlawful competition etc. with respect to the Supplier Data Sets and Third Party Data Sets. Similarly, the Public Organisation that provides Public Organisation Data Sets is obliged to indemnify the Supplier against any such similar claim of intellectual property infringement.

Critical remarks on the model contractual AI clauses

The model contractual clauses seem to be built on the assumption that the procured AI Systems are distributed under a proprietary licence. This is evident from the clauses on the rights to use data sets which restrict the rights of using, sharing and disclosing information about the AI System to third parties.

The rights of use of the Public Organisation correspond to the Supplier’s obligation to provide information about how the AI System arrived at a particular decision or outcome (explainability). The Supplier’s disclosure obligation extends to virtually all essential components of an AI System. However, this ‘openness’ vis-à-vis the Public Organisation is conditional on the purpose, ie, the information needs to be disclosed only to the extent it is necessary to explain the decision or outcome. Another restriction is the personal scope of the use rights which is limited to persons on which the AI System is intended to be used or in cases where the data sets need to be disclosed in legal proceedings.

The obligation to disclose information about the functioning of the AI System is challenging not least because the described subject matter is vague. For example, there is an obligation to disclose the ‘source code of the AI System’, but it is notoriously difficult to even define what counts as ‘source code’ in AI. While the application layer (wrapper) around the models is clearly a computer program, this is not necessarily the case for the model. The closest analogy we could draw between a machine learning model’s ‘source code’ and computer programs’ source code would be to treat the neural network’s weights and biases as source code. However, it is well known that the weights and biases are not human-readable and are of little help in explaining the system’s functionality. That’s why these clauses should be adjusted accordingly in procurement contracts to avoid the impossibility of performance of the contract.

The model clauses also suggest that Public Organisations should only be granted rights of use in the data sets to the extent necessary for performance of the provisions of the agreement. Typically, contractual performance for the delivery of an AI System would not depend on continuous access to the data sets on which the system has been trained, validated, or tested. While the Public Organisation’s rights are explicitly limited to the purpose of explaining the functionality of the system, it is not clear whether the concerned persons should be informed pro-actively, or whether there should be a request for explanation of a particular decision or outcome for the stipulation to apply.

The mutual indemnification clauses extend to cover not only infringement of intellectual property rights but also privacy rights, personality rights, unlawful competition etc. These clauses read like an uncapped indemnity which leaves both parties open to potentially unlimited claims. It would have been helpful to offer Public Organisations different versions of the indemnification clauses to point their attention to their potential liability exposure.

The stipulations about further use of the Supplier or Third Party’s data sets for further development of the AI System are only optional. They are, however, essential to ensuring the delivered AI System can continue to be maintained over time, even if the Supplier or the Third Party ceases to exist or is, for example, acquired by another company that may want to restrict access to the original data sets.

Compatibility with open source licensed components

Our analysis shows that the default clauses applicable to the essential elements of an AI System impose significant limitations on the Public Organisation’s freedom to use, study and modify the system. AI Systems procured under these model contractual AI clauses cannot be delivered under an open source licence without modification of the access and use limitations imposed on the recipients of the AI System.

We recommend adding a clause that specifies that the Supplier must provide a list of all open source components upon request. This can be complemented by a stipulation to the effect that, if required by the open source licences covering these components, the terms of such licences apply instead of those of the model agreement. The same applies to any restrictions in the agreement that may be incompatible with the applicable open source licences in some of the components. This is a standard approach in transactional practice when including open source components in software distributed under a proprietary licence.

Conclusion

The model clauses in their current version would not work for procuring an AI System released under an open source licence. The language of the model clauses must be adapted to at least allow for the licence-compatible inclusion of open source components in the delivered AI System. This could be a boilerplate clause specifying that for components of the software covered by open source licences it is the terms of those licences that apply in lieu of the default stipulations.

In conclusion, we recommend that a future update of or add-on to the model contractual AI clauses should consider adding specific licence grant language and guidance on ensuring compatibility between open source components, incl. open source machine learning models, and other components that together make up the AI System.

---

1. New version of Procurement Clauses of AI available: supporting responsible use of AI in Public Authorities | Public Buyers Community ↩︎
2. Articles 11 and 12 Non-High Risk Clauses, Articles 14 and 15 High Risk Clauses ↩︎
3. Art 12.2 Non-High Risk Clauses, Art 15.2 High Risk Clauses ↩︎
4. Art 11.3 Non-High Risk Clauses, Art 14.3 High Risk Clauses ↩︎