Andres Andreu, CISSP, ISSAP, QTE

Why AI Governance Is Now a Critical Leadership Responsibility

Unfortunately, many organizations are on the path to repeat one of the most consequential mistakes that we have made in this industry. AI Governance is now a critical leadership responsibility.

For years, executive leaders treated cybersecurity as a technical issue. It was convenient to tuck it away under Information Technology (IT) and it became someone else’s problem. They delegated it to specialists, discussed it only when budgets or incidents demanded attention, and assumed that technical teams could contain the risk.

Then the breaches became business disruptions. Regulatory consequences reached the CFO as well as the boardroom. Trust degraded, especially from customers. Operations stopped. Executives discovered that although they could delegate security work, they could not delegate accountability for the outcome. Tucking it conveniently inside of IT was no longer an option.

Artificial Intelligence (AI) is now following an eerily similar path, only much faster.

Many organizations still treat AI governance as a collection of technical controls, acceptable-use policies, legal reviews, and model assessments. They assign it to IT, data science, security, privacy, or compliance and assume those functions can govern the technology on behalf of the enterprise.

Simply put, they cannot.

Those teams can implement controls, evaluate models, monitor systems, and advise the business. They cannot independently decide which risks the organization should accept, which decisions should be influenced by AI or automation, where humans must retain authority, or who remains accountable when an AI-enabled processes have a negative impact.

Those are leadership decisions.

AI governance is not a technical specialization that executives can delegate. It is a leadership capability that executives must develop.

Leadership Cannot Outsource Accountability

I have spent much of my career moving between deeply technical responsibilities and executive leadership. I have worked in federal law enforcement technology, application architecture, offensive security, cybersecurity, the CISO function, the CTO function, and the CEO role.

Those experiences repeatedly reinforced the same lesson: technology may create the mechanism, but leadership creates the consequence.

For me, it took a while but that had to sink in as I lived my professional journey.

An algorithm does not determine whether an organization should use AI to evaluate employees, prioritize customers, detect fraud, approve transactions, recommend medical actions, or automate security responses. Leaders make those decisions.

The system may generate a recommendation, classification, or action. It does not absorb responsibility for the result. It simply generates an output.

A model cannot accept enterprise risk.

A chatbot is likely to not be able to explain a decision to an auditor.

An autonomous agent cannot appear before the board and defend the authority it was granted.

The human signature may become less visible as the footprints of AI and automation increase, but it does not disappear. It moves upward through the organization until it reaches the leaders who authorized these systems, established their boundaries (hopefully), funded their deployments, and accepted the risks at hand (again, hopefully).

AI Means More Than Generative AI

One reason organizations misunderstand AI governance is that most current conversations concentrate so heavily on Generative AI (GenAI). And the notion of “AI” in those conversations is incorrectly used to mean “GenAI”.

Large Language Models (LLMs), copilots, image generators, and conversational interfaces have made a subset of AI (GenAI) visible to almost everyone. They have also narrowed the discussion.

AI as a field extends far beyond generated text and images. Some organizations already use AI to:

Detect financial fraud and account takeover.
Score credit and insurance risk.
Identify cyber threats and automate containment.
Rank candidates and evaluate employee performance.
Recognize faces, objects, behaviors, and anomalies.
Predict equipment failures and optimize industrial processes.
Recommend products, services, prices, and content.
Route vehicles, shipments, and supply-chain resources.
Support medical diagnosis and clinical decisions.
Operate robots, sensors, and autonomous systems.

These systems may never generate a paragraph, but they can still shape someone’s employment, financial access, safety, privacy, or treatment.

Leaders who define AI governance as a policy for using ChatGPT will govern only the most visible layer of a much larger technology landscape.

Every system that predicts, accepts, rejects, classifies, recommends, prioritizes, optimizes, or acts should fall within the governance conversation. Yet, the limited understanding of where AI actually exists within organizations does not make that proper conversation possible.

AI Governance Begins With Ownership

Every material AI system needs an accountable owner. It doesn’t need a committee or some vague reference to “the business.”

A named leader must own the business purpose, risk, performance, and consequences of the system.

Technical ownership also matters, but it is not the same as business accountability. A data science team may build a model. A cloud team may host it. Security may monitor it. Legal may review it. None of those activities answers this central question:

Who has the authority to decide that this system should operate?

Ownership must extend across the AI lifecycle:

Who approved the use case?
Who authorized the data?
Who selected or developed the model?
Who defined acceptable performance?
Who approved production deployment?
Who monitors changes in behavior?
Who can suspend the system?
Who is accountable for the outcome?

When organizations cannot answer those questions, they do not have governance. They have the illusion of governance via distributed activity and no focused accountability.

Leaders Must Establish AI Risk Appetite

Many organizations speak about AI principles. Fewer define their AI risk appetite.

Principles describe what an organization values. Risk appetite determines what it will permit.

Effective leadership demands decisions around where AI may operate autonomously, where it may only recommend, and where it should not participate at all.

That requires decisions about:

Which data AI systems may access.
Which decisions may be automated.
Which decisions require human approval.
How much uncertainty the organization will tolerate.
What level of explainability a use case requires.
How much authority and/or autonomy an AI agent may receive.
Which failures require immediate shutdown.
When efficiency cannot outweigh safety, fairness, privacy, or trust.

For example, a fraud-detection model and an autonomous industrial controller should not operate under identical tolerance levels. Neither should a marketing assistant and a system that affects employment or access to employee resources.

Optimally, governance reflects potential consequence.

That judgment cannot come exclusively from a technical scoring system. It requires leaders who understand the organization’s culture, strategy, customers, obligations, operations, and values.

Human Oversight Must Be Real

“Human in the loop” has become a very overused phrase in AI governance.

Organizations often point to human review as evidence that a system remains under control. But placing a person near an automated decision does not guarantee meaningful oversight. Nor does it even reflect reality in some cases. The sheer volume of what AI powered systems can generate make human intervention questionable.

The human may lack sufficient time and/or information to challenge the system. The interface may encourage automatic approval. Time pressure may make careful review impossible. Employees may assume that the model is more accurate than they are. Responsibility may become so distributed that nobody feels empowered to intervene.

Realistically, human oversight requires more than a final approval button.

The reviewer must have:

Enough context to understand the decision.
Enough authority to reject or override it.
Enough time to exercise independent judgment.
Enough technical literacy to recognize uncertainty.
Enough organizational protection to challenge the system.

Effectively, leaders must also consider automation bias. This is the natural tendency for people to trust the output of a system that appears objective, complex, or authoritative.

Ultimately, the human factor does not disappear when AI enters a workflow. It becomes more complicated.

Identity and Authority Form the AI Control Plane

Oddly, many AI governance discussions often focus on models and data while overlooking identity.

That is a serious mistake.

People build AI systems. Service accounts train them. Pipelines deploy them. Applications invoke them. Administrators change them. Agents increasingly act through them.

Every step involves an identity exercising authority.

An organization must know:

Who or what is acting.
Which identity the actor represents.
What authority that identity possesses.
What constraints exist on that authority.
Who granted that authority.
Whether the authority remains appropriate.
Whether the identity remains trustworthy.

This becomes especially important with autonomous agents. An agent may retrieve information, call APIs, create accounts, modify configurations, communicate with customers, or initiate actions.

An agent should not receive unrestricted access simply because an authenticated employee launched it.

It needs its own identity, constrained privileges, defined purpose, limited duration, attributable owner, and immediate revocation path.

The organization should preserve the full chain of authority:

Human initiator
Agent identity
Delegated permission
Tool invocation
Impacted resource

Without that chain, the organization cannot distinguish legitimate automation from compromised autonomy.

The Adversary Gets a Vote

AI governance cannot operate only under the assumption that people and systems will behave as intended.

Adversaries couldn’t care less about the rules. They will manipulate models, compromise identities, poison data, steal credentials, exploit integrations, and misuse legitimate functionality.

They will search for the gap between what leaders think the system does and how it actually behaves under pressure.

This is where an adversarial mindset becomes essential.

Leaders should not ask only, “Does the system work?” In a headspace where there are no limits, they should also ask:

How could someone intentionally misuse it?
What happens if its data becomes untrustworthy?
Could a compromised identity change its behavior?
Can an attacker manipulate the human reviewer?
What authority could the system silently accumulate over time?
How would we detect subtle rather than catastrophic failure?
Can we stop it before we fully understand the incident?

Governance that assumes normal behavior is policy. And look at how effective policies are at stopping nefarious actors.

Governance that anticipates manipulation is a healthy step towards resilience.

AI Governance Must Become an Operating Rhythm

Organizations will not govern AI effectively through a policy, its annual review, or a one-time model assessment.

AI systems change. Their data changes. As do their users. Their integrations expand while authority grows. Their behavior may also shift as the environment around them changes. All of this is also happening at a rate of speed many organizations are not prepared for.

Governance must therefore become part of the organization’s operating rhythm.

Executive teams should receive recurring visibility into:

The inventory of blindly discovered (approved and unapproved) AI systems.
High-consequence use cases.
Detected changes in model behavior or authority.
Exceptions to established guardrails.
Third-party and supply-chain dependencies.
Identity exposure affecting AI environments.
Evidence that human oversight remains effective.

The objective is not to force leaders to review algorithms. One is to ensure that leadership understands where the organization has transferred decision-making power to machines and what could happen if that transfer fails. Another is to build a cadence of readiness preparation so that negative surprises are minimized.

Five Questions Executive Leaders Should Ask Now

Every executive team should be able to answer five questions:

Where is AI already influencing decisions or actions across the organization?
Who owns each material AI system and remains accountable for its outcomes?
Which decisions may AI make autonomously, recommend to a human, or never influence?
Can we trace every important AI action to a human or non-human identity and its delegated authority?
Can we suspend the system quickly when its behavior, data, identity, or operating environment becomes untrustworthy?

If leadership cannot answer those questions, the organization is not ready to deploy and/or scale AI responsibly.

Leadership Is the Ultimate AI Control

Technical teams will remain essential to AI governance. Organizations need skilled architects, data scientists, security professionals, privacy experts, engineers, and legal counsel.

But expertise does not replace executive accountability.

AI will compress the distance between a leadership decision and its technological consequences. A policy choice can become an automated workflow. A risk tolerance can become a model threshold. A poorly governed identity can become an autonomous actor.

The organizations that succeed will not necessarily be those that adopt AI fastest.

They will be the organizations whose leaders understand where AI should have authority, establish clear boundaries around that authority, demand attributable ownership, anticipate adversarial behavior, and retain the ability to intervene.

We eventually learned that cybersecurity was not merely a technology problem.

We should not need another decade of incidents to learn the same lesson about AI.

AI Governance is now a critical leadership responsibility, it is also a leadership test.

The outcomes will reveal who studied and prepared for that test.

Compliance does not equate to security, or protected

Compliance does not equate to security, or protected. They are not the same thing and the dividing line has become somewhat blurred. Parts of the Cybersecurity industry are in crisis due to an over reliance on compliance frameworks. The unfortunate reality is that many cybersecurity leaders rely on these frameworks because they just don’t know any better. Maybe, they were never practitioners and approach cybersecurity from a purely business perspective. Yes, we cybersecurity leaders need to have a business perspective. But, we also need to know our craft and be able to drive true protective initiatives. Over-focusing on compliance efforts may actually hurt security. Even worse, it may give an organization a false sense of being protected.

Compliance has sort of lost its way

My understanding is that compliance initiatives never intended to give a false sense of security. The intentions were to:

provide leaders and practitioners guided mechanisms that could promote better security
provide a maturity gauge
possibly expose areas of deficiency

One of the areas where they intended to help is that of introducing industry standards. The well intended thought process was that adherence to some well defined standards could make organizations “more secure”.

While I commend the intent, the end result, our current state, could not have been foreseen. Earlier generations of cybersecurity leaders, those who came up the practitioner ranks, treated these as mere guiding tools. Today, there are some with great sounding titles that rely on these as evidence of their brilliance and fine work. The problem is that those unfortunate organizations have been led to believe that they are secure and protected.

This creates a misleading culture of compliance=security. And those who know no better push this rhetoric, leave a trail of insecure environments, but move their careers forward. The funniest part here is that often those leaders bring in an external entity to assess if the organization in question is compliant. This is supposed to somehow add credibility to this weak approach. And this box checking exercise is often pursued in lieu of encouraging a real focus on security and protective controls.

Secure companies might be compliant

A secure company may or may not be compliant. I have encountered environments that are well protected and have not been through any compliance exercise. I have also seen compliant organizations go through the unpleasant experience of one or more negatively impacting incidents. Look no further than Uber. According to this they have received, and maintain, the highly coveted, and difficult to get, ISO-27001 certification. Yet, here is an incomplete list of incidents showing that this does not equate to security: 2014, 2016, 2022.

How they differ

Objectives and target audience

The objective of a typical compliance exercise is to measure an organization against a model, or set of recommendations. These might be industry standards. Audiences interested in these types of results can range from internal audit to the C-Suite. Generally speaking, technologists are not interested in these data points. But, some industries place great weight on these types of results and will not even consider business opportunities with organizations that do not have them.

Security, on the other hand either strives to reach certain levels of Confidentiality, Integrity, or Availability (CIA) and/or being Distributed, Immutable or Ephemeral (DIE). The objectives here revolve around threats, risks and building a resilient organization. The audience are those who see security by way of actual protection and resilience.

Rules of engagement

In security, there are none. Nefarious actors hardly ever play fair.

Compliance, however, has clear rules of engagement. In some cases the compliance exercise is even based on the honor system, think of the “self attestation” documents we have put together over the years. There is also a cadence where organizations know when relevant cycles begin and end. These are structured practices attempting to measure environments where the real battlefields have very little structure.

External motivations

When it comes to compliance, the external entities (auditors, assessors, etc) involved are motivated to generate a report or certificate. In some cases there are degrees of freedom for cutting corners or adjusting the wording of controls. This opens up many possibilities that aren’t all positive. Things can get manipulated so that a specific outcome (leaning towards the positive) is achieved. These folks are running a business and looking for repeat business.

The external entities on the security side are motivated towards results (destruction, your data, etc).

Focal areas

Security is usually focused on protection against threats. This focus could very well reach very granular levels. Granularity levels aside, protection could be both active and reactive in nature. Prioritization plays a big role here. This is especially so in organizations of larger sizes. A cybersecurity leader ultimately directs protective dollars to focal areas of priority.

Conversely, compliance takes a broad and high level look at things, treating all areas equally and so there is very little by way of focus. These exercises are more about checking boxes against a list of requirements that are all of equal importance.

Rate of change

Compliance is generally static and operates on long cycles. Moreover, the process requirements don’t change often at all. The challenge here is that internal changes are not factored in until subsequent cycles, and that could in one years time. In the world of security that feels like an eternity.

Security needs to keep up with constant changes. Simply factor in automated deployments and/or ephemeral cloud entities and it’s easy to see the impact of rate of change. Externally, nefarious entities are always changing, improving, adapting, learning. This means that an organization’s attack surface, and risk profile, are constantly changing. This alone negates the usefulness of a point in time compliance validation.

Organizational culture and the mindset

Mindset

Organizational culture drives the mindset people adopt at work. This could be a subtle dynamic but it is powerful nonetheless. A healthy relationship between compliance and security is the right mindset, and approach as each is important. Compliance is necessary for modern day business, without it an organization may not even be able to bid for certain business opportunities. Different from this modern day construct is security. Security is necessary to actually protect an organization, its assets, users, and customers. The nebulous tier that exists between these two is that compliance represents the only set of data points for external entities (e.g. potential business partners, consumers, etc) to consume and construct a maturity picture.

A company can have the best product on the planet, but if they cannot prove to a potential business partner that it is safe to use the opportunity may not flourish. Here are a few simple example:

SOC-2 reports represent proof of security maturity in the United States of America (USA)
in the United Kingdom there is generally the requirement of a Cyberessentials certification
to interact with credit card providers, an organizations needs to prove (at varying degrees) adherence to the Payment Card Industry Data Security Standard (PCI-DSS) standard.

Culture

The organizational culture point here is that there are security-first and compliance-first mindsets. One actually protects while the other aims to prove that protection exists and is effective. Security protects the organization, while compliance proves it.

This space has evolved into an erroneous approach. A compliance-first mindset is treated as actually meaning an organization is somehow secure. Sadly this mindset merely provides protections against auditors and not attackers. There is no framework I am aware of that actually leads to anything protective. To expect security, or protection, from some compliance initiative, or framework, is foolish and amateurish. Worst off, it can lead to a false sense of security for an organization.

Does a compliance-first mindset hurt security?

Yes, compliance leads to a false sense of security

Security hurts. It adds time to external projects (such as software development), costs money and takes up human resources. Moreover, it is never a static exercise as the entire space is always in flux (new sophisticated attacks, evolving internal changes, etc). Falling back to compliance as a way to alleviate this pain is just a mistake, but it happens. Compliance provides executive leadership a convenient way to create the illusion of security. This attitude can actually hurt a security program and minimize the effectiveness and/or support of certain projects that can add real protective value.

Yes. Compliance often wins a prioritization battle

Theoretically, compliance and security should synchronized in the goal of improving an organizations ability to conduct business. More often than not the two become competing entities for priority. At that point the synchronized goal ceases to exit. In so many organizations, when this prioritization conflict arises, compliance will win. Some executive will compare cost, effort, ultimate benefit and will fall for the allure of a compliance report or certificate. The perception at that executive level will come down to that person seeing that certificate or report as adding more value to the top/bottom line.

Yes – Top security talent see compliance as soul-draining

Compliance work is not exciting to talented security practitioners, especially younger ones. Let’s face it, that is boring work in comparison to blue/red teaming for instance. These folks come to the table ready to protect an organization’s environment. They absolutely do not come to an organization to check boxes on a document while adding screenshots for evidential purposes. From this perspective compliance hurts security by becoming an obstacle in the way of real protective work. Worst off, it creates an environment that is not pleasant to talented security team members.

Is a compliance-first mindset shocking?

Absolutely not. It is fair to say that entities (humans, companies, etc) will generally pick the path of least resistance and pain in order to reach some goal. Contextually, compliance represents that path towards what the security uneducated consider to be something positive. We cannot blame these people for confusing compliance and security. To the security uneducated, the two can actually look identical. After all, they sound alike, talk the same game, and are sometimes spoken of interchangeably by some who should know better. Some executives mistakenly perceive a strong correlation between compliance and security. Because of this, a checkbox exercise can easily seem like the right thing to do.

Those of us in the security educated space cannot take a judgemental approach to this unfortunate wave of development. It is entirely on us to educate these people and do our part to clear up the confusion that is permeating this subset of our domains. Educating these people selfishly benefits us. It future proofs us from executives getting excited after reading the marketing campaign that promises to secure their business with an ISO-27001 certificate.

Security-first mindset and making compliance work for you

Pursuing an organization-wide security-first mindset is a must in today’s world. Nothing is digitally safe and you should trust nothing. In order to foster a pervasive cultural change towards a security-first mindset you have to be very much in sync with the organizations culture. More importantly, you need to understand the business itself so that your work enhances it by adding safety. The last thing you want to do is push a security-first mindset incorrectly where it hinders business operations.

Internally, an understanding of the business factors in an understanding of an organizations crown jewels and holistic (inside out and outside in) attack surface. The key question is what do we want to protect here in this organization. The mindset discussed here then has a subliminal bias towards always having the organization as a whole leaning in that direction.

Measuring the effectiveness of this mindset and cultural change is very important. This has to be continuous and happen over time. The results may not be linear as they can be impacted by many external factors (M&A, etc). But the key here is you have a deep connection to, and understanding of, the business and its culture.

Compliance frameworks, processes, and standards cannot provide answers to the questions that lead to the necessary, and change impacting, connections discussed here. So it becomes a tool you manipulate and control it in order to make it beneficial. Make it a value add to your security program and the organization as a whole. Make those processes become sources of valuable data points that, for example, can point you at deficient areas. There is inherent value if you learn anything from one of these exercises. This is certainly more beneficial than a checkbox exercise that becomes pass/fail journey.

Final thoughts

It is understandable that many leaders see compliance frameworks, processes, and standards as something useful. They add structure to a space they may not truly understand. Don’t forget that a percentage of cyber and information security leaders did not come up the ranks as practitioners. They, understandably so, perceive compliance exercises as valuable. For those in the know, the value add from the compliance space comes in the form of posture improvements based on provided data points. However, when compliance efforts are prioritized, and treated as a source of security, harmful situations will arise.

I ask my peers to ponder this question: Who exactly is your adversary? You are security-first in nature if your adversary is an actual nefarious entity. Contrarily, you are not if your adversary is some auditor whom you need a passing grade from. We should all be striving to implement a security-first mindset, regardless of the state of compliance within an organization because compliance does not equate to security, or protected.