Wählen Sie Ihre Sprache:

Australia

Germany

Hong Kong

India

Ireland

Netherlands

New Zealand

Singapore

Sweden

Switzerland

United Kingdom

United States

  1. Startseite

You’re Already Deploying High-Risk AI Systems: You Just Don't Know It

 

If you’re using AI tools in your organisation, you’re probably already aware of the big change on the horizon; the EU AI Act.

| By Ramesh Koovelimadhom

This new wave of AI law – the first of its kind to be rolled out – aims to regulate the use of AI and minimise the risk it can pose to individuals. It sits alongside guidance on AI already issued by the Information Commissioner in the UK.

In its mission to ensure safe, transparent AI use, the Act categorises AI-powered applications into four risk categories; minimal, limited, high and unacceptable. Each tier comes with its own set of compliance obligations that must be fulfilled if a company wants to keep using AI tools without incurring a hefty fine of up to 7% of annual global turnover.

Under the Act, organisations that use off-the-shelf AI are considered ‘deployers’. If employees within your organisation are using AI assistants, for instance, you’re a deployer. (It’s important to note, however, that if you’re using in-house AI systems, you could also be considered a provider, and therefore subject to a whole additional set of obligations.)

As a deployer of AI systems, you’ll be expected to adhere to guidelines that cover areas from transparency and data governance to ethical standards, ongoing monitoring and user literacy.

You’re Already a High-Risk Deployer

There’s a lot to prepare for, and the go-live date for the bulk of the Act’s requirements is approaching fast. Getting ahead of these new compliance considerations before December 2027 is a great place to start. But in all likelihood, you’re already deploying high-risk AI solutions – and there’s work you’ll need to do to be ready for the Act.

Many organisations will be tripped up by the different approach the EU AI Act takes compared to the regulations they’re used to dealing with. Under GDPR, legal teams could usually draw clean boundaries between personal and non-personal data; the AI Act takes a broader approach. Risk depends on the role AI plays in decision-making inside your organisation, not on the name of the tool, who supplies it or the type of data you’re working with.

  • GDPR: Definitions are specific enough that legal teams could draw the lines. Supervisory authorities, case law and precedent filled the gaps.
  • The EU AI Act: The same hiring tool can be high-risk in one organisation and not in another, depending on whether it meaningfully influences the outcome.

That’s why high‑risk status often emerges quietly rather than through a single, obvious launch. When an AI system starts influencing decisions about hiring, access to services, or financial outcomes, it moves into high-risk territory. In some organisations that threshold was crossed months, or even years, ago.

This exposure doesn’t announce itself. It builds gradually as systems become more embedded in everyday workflows. Without actively revisiting how AI is used and what influence it has; many teams end up running high-risk AI systems without ever labelling them as such.

If you’re using AI for things like HR, credit, or procurement scoring tasks, your use of AI qualifies as high-risk. That means your compliance obligations are continuously developing behind your back. And as soon as those regulations kick in, you’ll be on the hook.

What is a High-Risk AI System?

According to the Act’s definitions, an AI system can be classified as high-risk based on the context in which it’s used. A lot of contextual use cases are fairly common sense; any AI tool used to manage things like biometrics, critical infrastructure and law enforcement is going to have a serious and significant impact if something goes awry, hence their classification.

But there are plenty of other high-risk use cases that may not be so obvious. Use in employment and workplace management, for example, falls into this category too. If you use AI for things like recruitment, training, candidate screening, or performance evaluation you’ll be subject to these high-risk governance rules due to the danger of bias and discrimination being introduced by AI algorithms.

What Does Governance of a High-Risk AI System Look Like Under the EU AI Act?

As a deployer of a high-risk AI system, you have several duties to carry out if you want to stay on the right side of the Act. 

  • Registration: Any new high-risk system must be registered in the EU database before it’s launched for use. If your system is already in use before 2 December 2027 you don’t need to register it unless it undergoes significant modification.
  • Proper Use: You must use the tool according to the provider’s instructions; that means no altering the system, and no using it in unauthorised or unintended ways.
  • Proper Use: You must use the tool according to the provider’s instructions; that means no altering the system, and no using it in unauthorised or unintended ways.
  • Human Oversight: You must appoint humans to oversee the operation of the AI system, and these humans should have the competence, training, authority and resources to do the job properly.
  • Data Quality: If you have control over what data is fed into the AI system, you need to make sure it’s relevant and appropriate for the task at hand.
  • Transparency: If you’re using AI to make decisions that impact individuals (like parsing their CVs or evaluating their work) then you must tell those affected people that AI is being used in the process. You also need to inform those working with the system internally, and their representatives, if a high-risk AI system is being used in your organisation.
  • User Literacy: All staff using the high-risk AI system must be trained to do so and have a sufficient level of AI literacy to harness it safely.
  • Ongoing Monitoring: The system must be monitored continuously, with any issues reported to the provider.
  • Log Retention: All system-generated logs must be retained for at least six months.

You also have existing responsibilities under Article 22 of GDPR to make sure that consumers aren’t subject to a wholly automated decision; adhering to these rules will likely involve conducting a Data Protection Impact Assessment before you use AI tools.

The key takeaway around how use cases are classified is that it depends heavily on context. Two companies can deploy the same system and face different obligations because the system plays a different role in each organisation. One uses AI output as a reference point; another lets it shape final decisions. The difference matters.

That flexibility is intentional, but it also introduces uncertainty. Guidance will evolve over time, but organisations won’t get neat, universal answers before enforcement starts. Decisions about how systems are classified will often need to be made with incomplete information.

That’s why defining high‑risk use cases ends up sitting with leadership. The question isn’t purely technical or legal. It’s about understanding where AI meaningfully affects outcomes and being prepared to explain why systems were assessed the way they were as they change over time.

What the Summaries Don’t Tell You

With a piece of legislation as weighty and complex as the EU AI Act, there will always be more to the situation than meets the eye. Here are four less obvious things to think about when putting together your new AI governance strategy.

1. Personal Director Liability Is Real

The Act doesn’t just target companies over lax AI governance; directors can also face personal liability if their oversight of AI is found wanting.

Boards are now expected to demonstrate credible, documented oversight of AI systems, including understanding where AI is used, ensuring classification and risk controls match the law and overseeing compliance, data governance, human oversight and incident reporting.

2. Your AI Vendor Doesn't Protect You

With more companies using widely available AI models to build their own tools or agents, the line between vendor and deployer responsibility can become blurred. But the Act makes it clear; deployers are still responsible for their AI systems, no matter whose foundation they’re built on.

3. The Brussels Effect Is a Global Strategy Decision

The EU AI Act is doing for AI governance what GDPR did for privacy: setting the most stringent, auditable and globally influential regulatory benchmark possible. We know from countless precedents that many non-EU jurisdictions eventually adopt comparable rules so that they can remain interoperable. And not only that, but global customers (especially enterprises) gravitate toward the most transparent, auditable and risk-controlled systems because it reduces their own liability.

4. "Human Oversight" Will Restructure Your Organisation

The Act's human oversight requirements are deceptively in-depth. When the Act says human oversight, it doesn’t just mean someone needs to check the outputs occasionally.

It means having dedicated, competent and properly trained humans who are accountable for AI-driven decisions in defined categories.

Firstly, let’s look at how the Act defines oversight. To comply with this substantial aspect of the new regulations, human oversight must involve:

  • Active, ongoing monitoring of system performance
  • Intervention capability when risks emerge
  • Accountability for decisions influenced or made by AI
  • Responsibility for ensuring that instructions for use and risk controls are followed
  • Monitoring for any bias, affecting people’s wellbeing, that occurs from the AI

This kind of obligation has direct implications when it comes to how you hire, how you operate and how you procure.

The Sovereignty Spectrum

Data sovereignty sits quietly underneath many of the EU AI Act’s requirements, but it shapes almost all governance decisions organisations make.

When you deploy an AI system, you’re also deciding where data lives, who can access it, under which legal jurisdiction it operates and how easily you could change or exit that setup if circumstances shift.

The Act wants to know who is accountable, and sovereignty wants to know who has control.

  • Data sovereignty: Where data lives, who can compel access, how flows are controlled
  • Technical sovereignty: Open standards, portability, avoidance of lock-in across the stack
  • Operational sovereignty: Who runs it, who can access it, under whose jurisdiction

Those choices affect your ability to meet obligations around oversight, incident reporting, documentation and accountability. In other words, governance only works if you still have control. 

The challenge is that most organisations are already making these sovereignty choices implicitly through cloud platforms, vendors and architectural defaults, without ever naming them as decisions. In the context of the EU AI Act, that creates risk. 

If you don’t understand who ultimately has control over your data, models and operations, it becomes much harder to demonstrate compliant oversight or respond when things go wrong. Every AI deployment therefore carries a sovereignty posture with it, whether it’s been deliberately designed or inherited by default.

The sovereignty issue transforms the introduction of the Act from a compliance issue to a leadership one. It’s not about legal reviews and quarterly attestations, but the flow of data, operational jurisdiction and the ability to make good decisions under uncertainty.

Imagine you’re a financial services firm deploying an AI risk-scoring tool through a cloud vendor. On paper, you’ve completed the necessary legal and compliance checks, but key sovereignty decisions have already been made by default. Customer data is stored in specific jurisdictions, the AI model is tied to the vendor’s platform, and operational control sits partly outside the organisation. 

When a regulator later asks the firm to explain decisions, demonstrate oversight, or intervene in the system, the issue is no longer whether you followed the right process; it’s whether you still have enough control over the data to respond effectively.

This is where sovereignty shifts the conversation from compliance to leadership. The EU AI Act focuses on accountability, but sovereignty determines who can actually act when something goes wrong. If leadership can’t move the model, interrogate its behaviour, or adapt operations under pressure, governance breaks down in practice. 

In light of data sovereignty, the real question becomes whether decision-makers are comfortable operating under those constraints. Because effective AI governance depends on control over data, operations and outcomes, not just documented compliance.

So, What’s Next?

In light of data sovereignty, the real question becomes whether decision-makers are comfortable operating under those constraints. Because effective AI governance depends on control over data, operations and outcomes, not just documented compliance.

If a regulator walked into your building tomorrow and asked to see evidence of human oversight of your AI systems, what would you show them?

If you can’t pull anything out, then you need to start preparing. Your operating model for AI governance should cover these four key pillars:

Your 12-Month Governance Plan

  • Now: Map all AI use cases against Annex III high-risk categories (hiring, credit, education, critical infrastructure, law enforcement).
  • In 90 days: Review all AI vendor contracts against AI Act obligations separately from GDPR DPAs.
  • In six months: Assign named human oversight as outlined in Article 14 and redesign affected workflows; issuing a written policy is not enough.
  • In 12 months: Get your living register live, establish a board reporting cadence and test your escalation paths before an incident forces them into play

The EU AI Act forces a strategic choice: organisations can either view it as a regulatory burden and risk slipping further behind or leverage its clear rules and the EUwide regulatory sandbox ecosystem to position themselves as global leaders in trustworthy enterprise AI.

Those that build truly transparent AI capabilities will earn the confidence of customers that are increasingly looking for accountability, trust and high performance in the fast-moving and too often risk-laden world of AI.

With over 25 years of experience in the industry, Ramesh is responsible for driving growth with Google technologies. Aligned closely with our practice and sales leadership, he interfaces with multiple layers of our clients’ technology and business management to identify, position and deliver business outcomes. He has successfully led several digital transformation engagements and helped clients in bridging the strategy-execution gap and resetting the culture in the IT organisation, translating the strategy to everyday plans and reorganising costs to grow stronger.

Related Articles

If you need support to harness AI without the risk, TEKsystems Global Services can help

Our specialists work with your organisation to implement secure, reliable AI solutions, clarify accountability and design governance that works in practice, not just on paper.

Talk to the Experts