With the vote in Coreper on 2 February, 2024 EU Member States formally and definitively approved the text of the AI ACT which had been agreed by the Trilogue on 6 December 2023. This is an extremely important event for the legislative path of the AI ACT since only now one can finally say that negotiations on the text are definitively closed. The next procedural steps, i.e. approval by the European Parliament (in committee and plenary, the latter scheduled in April 2024) and subsequent publication of the text in the Official Journal, appear to be obvious or purely formal steps, given that there is no doubt about the European Parliament’s desire to confirm the text as it is. The time for changes is therefore over.

It is now possible to make a preliminary assessment of how the European legislator intended to regulate the so called “foundation models”, such as for instance GPT-4 (from OpenAI) and LaMDA (from Google), that is to say the AI products that became well-known in the public since 2022 – when AI ACT negotiations had already started – thanks to emergence of very popular generative AI applications such Chat GPT and Bard for instance. The popularity of such applications and their potential impact upon society and democracy attracted attention of the EU legislators and resulted in he  agitated debate whether foundation models, which constituted the computing background of such applications, should be regulated or not. While some authors considered that regulation should apply only upon products put into the market (i.e. AI systems, applications and so on) other believed that the entire production chain should be regulated, including the foundation models which can be considered, by analogy with the telecom sector, a kind of wholesale component.

The debate sparked in the Trilogue, since the original proposal of the Commission (dated April 2021) did not precisely addressed the issue. Thus, how to regulate foundation models became very controversial with the consequence that the disagreement between the co-legislators risked derailing the entire Ai ACT until the last minute (and despite the political agreement found in the Trilogue). In fact, it is known that France and Germany would have preferred a much lighter regulatory framework on foundation models than the one proposed by Parliament and the Commission.

After various leaks (in particular: Euractiv) the official agreed text has been published by the Council.

1. New name and definition

Foundation Models are now defined by the AI ACT as “General Purpose AI Models” (“GPAI Models“). In summary, they can be defined as computer models which, through training on a vast amount of data, can be used for a variety of tasks, individually or inserted as components into an AI system. The AI ACT legal definition, however, is more complex and will certainly not fail to raise objections (if not even jurisdictional appeals before the EU Court), as it can be interpreted in such a way as to potentially include a vast variety of technological cases:

“GPAI model means an AI model, including when trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable to competently perform a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications. This does not cover AI models that are used before release on the market for research, development and prototyping activities“.

Needless to say, we are faced with an industrial sector where representing technological realities through simple legal definitions looks very difficult, due to the rapid developments in the market.  This is the main reason why the EU legislator decided for a very wide definition. Instead, choosing a narrower definition would have been problematic and challengeable from others point of view, because it would have excluded relevant business cases from the ambit of application of the AI Act. Over-reaching has ben preferred to a limited legislative scope.  Next years will show whether this has been a wise choice or, instead, we are going into a season of permanent litigations.

2. Categories of GPAI models

The AI ACT identifies two categories of GPAI Models: generic GPAI models and “systemic” GPAI models. The latter are models which, by virtue of the “systemic risks” they can cause at a European level, are subject to more pervasive regulation than the generic ones. It should be noted that France and Germany, with a non-paper presented in November 2023, had proposed a single category of foundation model to be  subject to the self-regulation by operators themselves. The Parliament, somehow supported by the Commission, had proposed binding obligations. The Commission responded with a “scalar” regulatory system, based on a distinction between generic and systemic operators probably borrowed from the Digital Service Act (“DSA”), a discipline where larger platforms (so-called VLOPs) are in fact subject to stricter rules than generic platforms, due to the systemic risks associated with larger operators.

However, while in the case of the DSA the setting of the parameters (turnover, number of users) for the identification of VLOPs was not too problematic due to the fact that the online platform market is  sufficiently known and consolidated, the same operation appeared more problematic in the AI ACT, which is addressing a sector where even main technological players and their impact on the market are still a fairly new phenomenon.

Therefore, it was not easy at all to define the condition under which a foundation model may be considered that “large” to imply excessive risks with respect to the rest of players. In the end, the AI ACT entrusts the designation of systemic GPAI models to a procedure managed by the Commission alone, which acts on the basis of quite vague criteria indicated in the regulation and which can be adapted by the Commission itself over time.

Indeed, what is a systemic risk at European level? The AI ACT explains it with a somewhat tautological definition:

“‘systemic risk at Union level’ means a risk that is specific to the high-impact capabilities of general-purpose AI models, having a significant impact on the internal market due to its reach, and with actual or reasonably foreseeable negative effects on public health, safety, public security, fundamental rights, or the society as a whole, that can be propagated at scale across the value chain“.

That this is a tautology can also be seen from the concept of “high impact capabilities“, i.e. the most significant characteristic for evaluating whether a GPAI is systemic or not:

‘high-impact capabilities’ in general purpose AI models means capabilities that match or exceed the capabilities recorded in the most advanced general purpose AI models; 

In other words, we are in a field in which the Commission, which has the task of identifying, through the AI Office, the systemic GPAI models, will have highly discretionary and therefore preponderant power. It will therefore be able to conduct a real industrial policy, simply deciding which GPAIs can be designated as systemic and which cannot.

Furthermore, the AI ACT also indicates a quantitative criterion, based on the computational capacity of the model, to identify systemic risks:

“when the cumulative amount of compute used for its training measured in floating point operations (FLOPs) is greater than 10^25″.

However, this is a simple presumption, which can be overcome both positively and negatively, according to the discretion of the Commission itself. On the other hand, there is a widespread belief that in the future the power of GPAI models will not necessarily depend only on computing power.

3. Regulation of basic GPAI models

During the legislative process Council and Parliament showed different approaches as how to regulate foundation models: while the Council preferred to start with a lighter framework (although allowing stricter rules following an analysis of the Commission), the Parliament immediately advocated for stricter rules from the first moment. The EU legislators used different legal definitions respectively, a situation which contributed to add uncertainty in the debate since it made difficult to measure the real distance between the two regulatory approaches.

The final text of the AI ACT is now more clear and precise. Generic GPAI models are subject to mere transparency obligations, consisting in guaranteeing the availability of technical documentation that makes their functioning understandable (also in relation to the data training process) to the AI Office as well as to third parties who intend to integrate the model into their AI systems. This is a reasonable regulation which in itself should not constitute a significant obstacle to the development of the models.

There is also an obligation to provide a policy aimed at respecting copyright legislation. A final decision has therefore not been made whether the use of copyrighted data could lead to an obligation to remunerate rights holders, as some of them have been clamoring for. This decision will have to be taken in the future as part of a possible reflection or revision of EU copyright law. At the moment, however, it can be said that the obligation for GPAI models to establish a policy for this purpose indicates that the topic is worthy of relevance.

Note that lighter regulation is expected in the case of GPAI models with a free and open license, unless they are “systemic”. Therefore, if obligations are applicable to open source cases, the problem arises of identifying the obliged subject, given that sometimes we are faced with a community rather than a specific provider.

4. Regulation of systemic GPAI models

Systemic GPAI models are subject to the same obligations as basic GPAI models, plus additional ones which give rise, overall, to more pervasive regulation. In fact, they must: (a) carry out the evaluation of the model in accordance with standardized protocols and tools that reflect the state of the art, including the conduct and documentation of “adversarial tests” in order to identify and mitigate systemic risk; (b) assess and mitigate possible systemic risks at Union level, including their sources, that could arise from the development, placing on the market or use of AI models of a general nature with systemic risk; (c) track, document and report without undue delay to the AI Office and, where appropriate, to the competent national authorities, relevant information on serious incidents and possible corrective measures to address them; (d) ensure an adequate level of cybersecurity protection relating to the model and its physical infrastructure.

The question is whether such regulation is so pervasive to concretise the dangerous obstacle to the development of european Foundation Models that had been feared by some governments, in particular France. Of course, how the Commission evaluates these obligations will matter. For example, in the case of “adversarial testing”, the AI Office’s discretion in considering the testing process sufficient will be relevant. As regards accidents, it will be necessary to understand whether the GPAI model should be able to report all the information concerning the AI systems based on it (since they are normally different companies).

5. Codes of practice    

AI ACT provides that, pending the publication of harmonized European standards, both categories of GPAI models, generic and systemic, can rely on “codes of practice” to demonstrate compliance with their obligations. By “codes of practice” we mean technical documents that report the standards of a technological sector.

Compliance with the codes creates a mere presumption of compliance with the obligations of the AI Act.

The development of codes by companies is encouraged and supervised by the AI Office, which also makes use of the collaboration of the AI Board (the latter is the body where the representatives of the Member States sit). National authorities can be involved and the “support” of stakeholders and experts is also expected.

A formal agreement from the AI Office on the text of the code is not necessary, although the system suggests that, in the event of an objection from this office, the value of the code would effectively be diminished, since it provides a mere presumption of conformity to the obligations of the regulation. Therefore, for the code to have the effects desired by the industry, it is actually necessary for it to be supported by the AI Office. The ability of Member States to share this power with the AI Office is delegated to the Board and its ability to collaborate with that office.

However, formal approval by the AI Office is possible to make the code valid throughout the European Union. In this case, an implementation act by the Commission is envisaged, to be adopted with the approval of the member states (AI ACT refers to art. 5 of European regulation 182/2011 on comitology procedures).

Violation of the codes of practices is equivalent to violation of the obligations of the AI Act and as such involves the imposition of sanctions according to a mixed system (sanctions partly defined by the regulation itself, partly delegated to the Member State).

6. Vertically-integrated GPAI models

Remarkably, the AI ACT specifically regulates cases of vertical integration, i.e. when there is identity between the GPAI provider and the deployer of the relevant AI system. In this case, the AI Office operates as a market surveillance authority, replacing the competence of national authorities. This is the first evident case to apply specific competition rules to the AI sector, where numerous voice already have alerted about the risks of concentration. It’s worth to remind that the European Commission has recently started to investigate the relations between OpenAI and Microsoft.

7. Conclusions

In conclusion, there is no doubt that the Commission has taken a leading role in the treatment of Foundation Models in the AI ACT, having been recognised, directly or through the AI Office, formidable power not only in regulation and enforcement, but also in real industrial policy of the sector. In particular, the Commission enjoys exclusive competence, and broad discretion, in the “executive” phase of the system, i.e. in the identification of systemic GPAI models and subsequent application of regulation.

Greater control by the Member States instead seems to take place with regard to the adaptation of both the obligations incumbent on models in general and on the parameters for the designation of systemic models. The AI Act in fact refers to acts, both implementing and delegated, which the Commission can adopt on the basis of procedures which should involve a certain involvement of Parliament and Council.