What Counts as AI?

The Material Scope of the AI Act, or: What Counts as AI?

A question of paramount importance in applying the AI Act is what, precisely, qualifies as artificial intelligence under the regulation.

This article examines and explains what is meant by the concept of an AI system or model under the AI Act, and how an AI system can be distinguished from a traditional (or classical) automated decision-making procedure. In the case of the latter, the only expectation is that the same input events always produce the same output results. The entire decision-making process is known and traceable. By contrast, in the case of AI models and systems as defined by the AI Act, the decision-making process is far less clear-cut: the “black-box” effect may arise in the operation between input and output — that is, a situation where an AI system provides useful information to the user but the system’s decision-making process is not necessarily fully transparent or traceable.

The Regulation defines an AI system as follows: “a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments”

Since only a system that meets this definition falls within the material scope of the AI Act, it is worth examining the substantive elements of the definition in greater detail.

What Does the Definition Mean Precisely?

The first conceptual element — that the AI system or model is machine-based — is obviously necessary to describe how it operates, but it is not suitable for distinguishing an AI system from other IT products. According to the Preamble of the Regulation, “the term ‘machine-based’ refers to the fact that AI systems run on machines.”

The second element of the concept — operating with varying levels of autonomy — is, however, a particularly significant element of the definition, as it clearly points to a characteristic specific to AI systems compared to any other IT system. It is important that the mere existence of this capability is a sufficient condition for a given technical solution to qualify as AI; this does not, however, affect the assessment of its live operation, since as a risk-reduction and safeguard mechanism this very capability is often deliberately restricted.

The autonomy of an AI system indicates the extent to which a system is capable of learning or acting without human intervention. Autonomy essentially implies a certain degree of human intervention — or its complete absence. In practice, it is important to distinguish between autonomous and automated operation, since the latter can also function without human intervention, but in automated operation the system operates according to pre-defined steps to achieve a given result, whereas at the various levels of autonomous operation the final result cannot be fully predicted or determined in advance.

The capacity for inference is also an important distinguishing element. The Preamble states the following on this point: “The notion of inference refers, on the one hand, to the process of generating outputs such as predictions, content, recommendations, or decisions, which can influence physical and virtual environments, and, on the other hand, to the AI system’s capacity to derive models or algorithms, or both, from inputs or data. The inferential capacity of an AI system exceeds basic data processing by enabling learning, reasoning, or modelling.”

In practice, the inferential capacity of AI systems and models means that they are capable of modifying their own operation by independently analysing and evaluating the effect of their previous steps or decisions. In other words, in pursuit of a specific goal, the system can plan its next optimal step by processing data it has itself collected or generated, and can also respond to that data. By contrast, classical models and systems that do not yet qualify as AI are not capable of such independent evaluation.

The definition treats adaptiveness as a non-mandatory element. That is, if a system possesses the capability of adaptiveness, this reinforces the conclusion that it is an AI system, but the absence of adaptiveness does not in itself mean that the system is not an AI system. The same is true of the final conceptual element — that the system’s “outputs” can influence physical or virtual environments. According to the Preamble, adaptiveness refers to the self-learning capabilities of an AI system that allow it to change during use.

It is also important to note that there are AI systems which, while meeting the definition, are nonetheless excluded from the scope of the AI Act by the Act itself. There are self-evident exceptions, such as AI systems that are deployed entirely outside the European Union and do not affect EU citizens at any point during their operation. Another excluded category consists of AI systems designed for military, defence, and national security purposes, to which the regulation likewise does not apply, given that these are typically areas falling within the competence of Member States rather than the EU.

Two further types of exception may be of considerably greater relevance to typical operators, including financial institutions in particular. The rules of the Regulation do not apply to AI systems developed and put into service for the exclusive purpose of scientific research and development, and — potentially even more importantly — neither do they apply to any research, testing, or development activity relating to AI systems or AI models prior to their being placed on the market or put into service.

In other words, during the research and development phase, compliance with the strict rules of the AI Act is not yet required; however, the regulation itself specifies that these activities must still be carried out in accordance with all applicable EU legislation, and that this exception does not apply to testing in real-world conditions.

The next article in this series examines the prohibition of discrimination in the context of AI systems.

Reviewed by: dr. Laura Bikki Kovácsné, dr. Bernadett Bocsi, and dr. András Bencsik

Related articles:

The AI Act and the Financial Sector