Board Memo · Systemic Risk Series
The Evolution of Governance
AI, Corporate Trusteeship and the Legitimacy of Corporate Power
Executive summary
Artificial intelligence is consciously uncoupling economic and technological power from organisational size. Today, a company operated by one or two people may command autonomous systems, reach millions and impact the lives of stakeholders and beyond. The original and conventional board and non-executive director model was created to enable an operative human hierarchy, navigate information asymmetry and cushion periodic oversight. In an environment of an always-on, AI-enabled infrastructure, the traditional governance's functions are increasingly absorbed. However, accountability's importance increases, as power becomes easier to exercise and harder to locate. This memo argues that instead of introducing AI ethics and Ethics Boards, the oversight of corporate management should evolve into a trusteeship of corporate power: an enforceable obligation to protect human interests, grounded in the social legitimacy of the corporation itself.
Author's note
I write as a founder and non-executive director whose board memoranda have repeatedly examined human behaviour, the incentives that shape it, and the consequences institutions permit. Following a debate on AI and the future of corporate governance, I found myself questioning the future of my role as a NED. If organisations can operate without extensive human workforces, what becomes of the architecture designed to supervise those workforces? And if the architecture changes, who safeguards the people affected by the organisation?
This memo belongs to my Board Memos series: papers intended to bring systemic risks, often absent from conventional board packs, into the discipline of governance. It is a thesis and an invitation to challenge its assumptions, not a claim that existing boards or statutory duties have already disappeared.
Opening note to the Board
Remember, remember the fifth of November. The Gunpowder Plot of 1605 attempted to destroy Parliament and kill the monarch; its failure, rather than a popular uprising, is what Bonfire Night originally commemorated. Yet the anniversary invites a question that transcends its original politics: where does power reside, and what makes its exercise legitimate?
For generations, corporate power was recognisable as it was resident in big buildings, employed people, assembled management hierarchies and convened directors around a table. Corporate oversight followed the institution, and now AI is disturbing that arrangement. As the Cadbury Report defined it in 1992, corporate governance is the system by which companies are directed and controlled.
The institution may become smaller than the power it exercises.
Today, there are organisations that employ just two people and serve hundreds of thousands of customers. Medvi, a telehealth company selling weight-loss medication, reported $401 million in sales and around 250,000 customers in 2025, its first full year, with two employees: its founder and his brother (The New York Times, 2 April 2026). Their decisions may be made or executed continuously by systems and agents that do not require supervision in the familiar managerial sense. The human capital inside the corporation may or may not be able to explain the external impact, and that is the real danger here. The visible structure of governance is slowly fading before we have established where the accountability lives. The same company had received an FDA warning letter two months before that profile was published, and has since faced allegations of fabricated testimonials in its advertising. Scale arrived first; scrutiny followed.
The age of visible governance may be ending, and it gives way to a new age of accountability.
1. The corporation as an institutional solution to human limitations
Modern corporate governance was developed around separate ownership and control, delegated authority, fiduciary responsibility, information asymmetry and the need for durable legal organisations; the complexity of those depended on the number of employees, processes and the environmental and societal impact of the company's product. As corporations grew, managerial hierarchies and specialist committees made the allocation and supervision of authority more elaborate.
Boards were designed to approve direction, scrutinise management, review financial resilience, oversee risk and challenge assumptions. The non-executive director became a distinctive instrument of independent judgement: close enough to understand the business, separate enough to question those running it.
Yet much of the work is constrained by the cadence and economics of human attention. The Board must review reports, while committees convene to interpret historical results. AI can compress those cycles through continuous analysis, automated testing, anomaly detection and strategic simulation.
If oversight could become continuous, would it make sense to keep its institutional form periodic?
A company cannot lawfully exist or operate without a director; the director is legally responsible for running the company, managing risks, filing annual accounts and confirmation statements with Companies House, and ensuring the business pays its taxes on time. But if all of these can be managed by AI, what is the true role of a director? Do they become just a business’s façade?
2. AI-native companies and the slimmer human organisation
The emerging AI-native enterprise is designed intelligently from inception around software agents that perform administrative, analytical, commercial and operational tasks that in a traditional organisation would be allocated to teams. A founder can coordinate capabilities without reproducing a traditional chain of command.
This does not mean a two-person company can effortlessly perform every activity of a multinational organisation, as physical constraints such as infrastructure, capital, safety testing, licensing and access to markets still exist. But in some domains, especially in the digital service sector, the distance between a small team and substantial economic influence is narrowing.
The consequences for governance are asymmetric and depend more heavily on the founders. The founders may be conscientious, and in that case the risk is minimised. But if the founders prove to be careless, conflicted or indifferent, then the company can bring significant risk exposure to its customers, workers elsewhere in its supply chain, communities and the public.
Human headcount is becoming a poor proxy for the scale of corporate power.
A governance model calibrated principally to organisational complexity risks missing the most consequential feature of the AI-native corporation: its external reach and impact. This now becomes a matter of quality over quantity and substance over form.
3. The NED portfolio career: profession or transitional arrangement?
The path to a NED portfolio was once predictable: climb the corporate ladder, then join boards. AI-native businesses now automate much of the reporting, analysis and research that fills a director's time. Founders keep control without the politics of the boardroom.
Yet judgement cannot be delegated wholesale to a model. AI systems do not carry a director's legal duties; technical accuracy confers neither legitimacy nor independence. The economic value of a job description may shrink while the purpose it served grows.
If the board's work has always centred on human behaviour, incentives and impact, what remains when the company no longer manages a large workforce? The people outside it. Stewardship moves from how people inside the corporation behave to how the corporation behaves towards those beyond its walls. For the developers of frontier AI and their partners, those people number in the billions.
Why would a founder, shareholder or chair invest in such stewardship when its returns arrive late and are shared with society? Aristotle addressed the premise of this question more than two thousand years ago. In the Nicomachean Ethics, he observes that wealth is not the good we seek; it is useful only for the sake of something else. In the Politics, he separates the management of a household, which knows its limit, from the art of acquisition that pursues money without end. A company that accumulates wealth while eroding the society that sustains it has mistaken the means for the end. No society, no market. No market, no company.
Aristotle was also a realist. He recognised that most people respond to necessity and punishment rather than to argument or a sense of what is noble; for that reason, law must shape habit. The same holds for corporate power. Eudaimonia may be the destination, but those who act from fear, insecurity or an appetite for power will not be persuaded by it. They will respond to levers: regulation, licensing conditions for high-impact activities, the requirements of investors and insurers, and the terms of public procurement.
Some companies have chosen this path without compulsion. Bosch placed most of its ownership in a charitable foundation; Patagonia transferred its ownership to a purpose trust in 2022; Johnson & Johnson's Tylenol recall in 1982 put patients before shareholders, and the company endured. None is flawless. Each shows that a purpose beyond profit can coexist with commercial longevity.
The NED’s role, then, may evolve into something more demanding: a trustee not only of the P&L, but of the civilisation that makes the P&L possible.
4. Beyond the boardroom: capability without institutional scale
Looking into highly regulated sectors, the same pattern seems to emerge. More companies use AI tools to facilitate their governance, and the same tools can widen access to satellite design, computational research and early-stage drug discovery. Access to knowledge is democratised and the key force is simple curiosity. Individuals and small teams can explore technical possibilities once concentrated in universities, industrial laboratories and large firms.
However, designing a satellite is not the same as launching and operating one. Identifying a candidate molecule is not producing an approved medicine. Regulation, testing, safety and physical execution remain essential. Nevertheless, capability can spread more quickly than the institutional structures traditionally used to supervise it.
Historically, access to knowledge and the authority to exercise it were often entwined. The history of women's healing practices and accusations of witchcraft is complex, but it reminds us that societies have repeatedly contested who may possess expertise and who may legitimise its use.
AI does not remove the need for standards. It makes institutional membership an increasingly incomplete indicator of who can act, and with what consequences. We used to have chartered accountants, lawyers' privilege and even doctors taking an oath to serve humanity… We relied (and still do) on these, but somehow with AI, we have become 'clever as clever'… We shortcut our way to the end result, but who gets to be blamed when things go wrong?
When powerful capability no longer requires a large institution, governance cannot rely exclusively on the institution as its point of control.
5. Disclosure does not mean discharge: the frontier AI test
Frontier AI developers have publicly discussed the possibility that advanced systems could cause severe or catastrophic harm. Anthropic's published constitutional and safety materials are one illustration of the industry's acknowledgement of difficult control and deployment questions. Such disclosure may be useful, but it also shows that risks were foreseeable.
Disclosure is not prevention, and publication of a risk statement does not automatically discharge legal responsibility. The applicable duties, evidence of reasonable safeguards, causation and jurisdiction all matter. Nor should we equate every speculative scenario with a demonstrated present danger.
The more fundamental governance question is who gets to determine the acceptable risk imposed on people who are not customers, shareholders or employees. A developer's internal board cannot, by its own declaration, acquire a mandate to decide on behalf of humanity.
A company's acknowledgement of risk is not society's acceptance of that risk.
This is the point where a company's societal impact is not a matter of corporate self-governance, as the effects of advanced systems go beyond the perimeter of the company that deploys them. An example that I constantly like to revisit is the clock: a product created to measure time and facilitate our lives, that eventually evolved into dictating our lifestyle, our work patterns, our log of milestones and aspirations, motivations and even ethics. When a company with a product as powerful as AI voluntarily discloses the risk and impact on society, simple assurance may complement independent oversight, but it cannot substitute for legitimate, enforceable public safeguards where public interests are at stake.
6. The paradox of AI governing AI
When we use tech as a tool to help us substitute for conventional oversight, then the tool is an object of those oversights and someone needs to control that tool. The technology that may render conventional oversight less central is itself an object of oversight. An organisation may rely on AI to flag fraud, test compliance and monitor autonomous agents, while the same ecosystem contains errors, hidden assumptions, adversarial vulnerabilities and incentives to optimise the wrong outcomes.
Governance embedded in technology could involve permissions, traceable decisions, independent evaluations, audit trails, human escalation and the ability to interrupt harmful operations. It could be continuous rather than episodic. But its architecture must be independently testable, and responsibility for setting objectives and accepting risks must remain attributable to a human.
Invisible governance must never mean invisible responsibility.
The issue is not whether a machine can generate a better board paper. It is who authorises the system, who can challenge it, especially the information it provides, who can stop it and who answers when it fails.
7. From NED to Corporate Trustee
I do not believe the future is simply the disappearance of the non-executive director. There will always be a need for trusteeship in the broad sense: independent stewardship of power exercised on behalf of others. I would propose that commercial NEDs are transformed into Corporate Trustees, with the ethos and framework within which Trustees operate.
Under UK law, company directors already owe fiduciary duties, and trusteeship carries specific legal meanings in trusts and charities. The model is not about a change in title, but a defined mandate, enforceable powers, independence and accountability. This would bring a significant change to the portrait of the current NED: where past CEO and CFO roles and responsibilities are expected today, stewardship would become a more expansive matter of character, experience and understanding of justice, balance, humanity, sustainability and human rights.
The Corporate Trustee would not exist primarily to review the executive's monthly performance pack. The trustee would test whether the company's exercise of power is consistent with the legitimate interests of those affected by it, particularly where the reach of its technologies vastly exceeds the scale of its workforce.
That remit might include independent access to system-level assurance, the ability to demand remediation, escalation of material risks, protected routes for affected parties to raise concerns and, for specified high-impact activities, powers to halt or refer deployment. The exact design would need legislative and institutional legitimacy rather than private self-appointment.
The board is a mechanism. Accountability is the principle. Trusteeship may be the next institutional expression of that principle.
8. Logic of corporate legitimacy over AI ethics
A corporation is a legal creation of human society. Society recognises its separate personality, enables it to own property and enter contracts, and often grants its investors limited liability. These arrangements facilitate investment, innovation and economic coordination. They are not natural entitlements existing independently of law.
If society confers such powers, on what grounds should their exercise be justified? Profit is an important commercial measure. It cannot be the complete answer to whether corporate conduct is legitimate when its costs are imposed on people who did not choose to bear them.
This argument rests on an explicit normative premise: legal institutions created by societies should ultimately be justified by their compatibility with human welfare, rights and collective interests. From that premise, enforceable constraints on corporate harm follow as a matter of institutional logic, not simply as an optional ethics programme.
Existing law already constrains corporations through criminal, civil, employment, safety, environmental and sector-specific obligations. Section 172 of the UK Companies Act 2006 requires directors to promote the success of the company for its members as a whole while having regard to specified stakeholder and long-term considerations. It does not create a free-standing overriding duty to advance humanity.
I am proposing to take this a step further: corporate success should not be treated as sufficient justification for exercising power where the resulting harms to people are unacceptable. The minimum is a legally enforceable obligation not to impose unjustifiable harm. A broader positive obligation to advance human progress would require more careful democratic definition, because humanity does not have a single uncontested conception of progress.
Why should corporate success be considered legitimate if it is achieved at an unacceptable cost to society?
This is the point at which the memo ceases to be principally about the future of the NED profession. It becomes a question about the social contract that legitimises the corporation. And it also tries to answer a fundamental question: what is the cost or value of human life, or any life on the planet?
9. Designing accountability for impact and consequences
A forward-looking framework of governance should be proportionate to the power exercised and the consequences that may follow, not simply to revenue, employee count or legal form. Different sectors will need different thresholds; a small software firm and a manufacturer of safety-critical equipment do not pose identical risks, but then again the context is essential: a few decades ago, we would have considered an aerosol a bigger risk to the environment and human life than a microchip.
The following principles offer a starting point for debate:
- Capability and reach: assess what systems can do, whom they affect and whether effects can propagate beyond the firm.
- Severity and reversibility: distinguish minor, remediable errors from irreversible or systemic harm.
- Attributable control: identify the people and entities that set objectives, control deployment and can interrupt operations.
- Independent assurance: provide protected access to relevant data, testing and audit trails rather than relying only on management narratives.
- Effective challenge and redress: give affected people meaningful avenues to contest decisions and seek remedies.
- Proportionate intervention: reserve stronger trustee, licensing or supervisory powers for activities with more consequential external effects.
The governance mechanism may become embedded, distributed and largely invisible in daily operation. However, the accountability architecture must remain visible to regulators, affected people and the public where legitimate interests require it.
10. Two transitions: the birth of AI-native organisations and the transformation of the role of NEDs
I have begun to think of my own role as a transition doula. One transition concerns companies being born into an AI-native world: how to design responsible authority, assurance and safeguards from inception without simply importing the bureaucracy of the twentieth-century corporation. The other concerns NEDs whose careers have been built around institutional arrangements that may progressively lose their economic rationale.
These are different forms of work. Founders need to understand the power they are acquiring and the responsibilities that accompany it. Established directors need to distinguish the principles worth preserving from the professional forms they may need to relinquish.
The role of a transition guide is not to protect the old institution or romanticise the new one, but to ensure that what is valuable survives the passage between them.
One generation must learn how to exercise new power responsibly. Another must learn how to release old forms of power responsibly.
11. What good governance looks like: a framework for action
When planning for transition, we map the current point, design the end game and then work backwards to create a map that will guide us there. Direction is more important than speed, and usually auditing the existing situation is what takes most of our time and effort.
The end game is usually an inspirational idea. A few can see it, and even they would have a blurry vision. While progressing towards it, the image will become clearer. It is these visionaries who can guide us and pace us, and the most difficult thing to do is to follow them, because they do not promise financial success or a positive P&L impact within six months (despite most of the time delivering it). They advocate for the hope of a new era.
For boards and NEDs today
- Map AI-enabled decision rights, including decisions made or executed without direct human review.
- Identify material externalities: who can be harmed beyond the company's employees, customers and shareholders?
- Test the independence and reliability of automated assurance, including the ability to challenge model outputs.
- Examine whether board agendas overemphasise retrospective performance at the expense of legitimacy, impact and intervention.
- Develop a transition plan for director capabilities: from periodic scrutiny towards stewardship, system assurance and accountability design.
For AI-native founders and leadership teams
- Design accountability into the operating architecture before scale makes remediation difficult.
- Assign named human responsibility for objectives, permissions, escalation and shutdown authority.
- Build contestability and redress for people affected by consequential automated decisions.
- Seek independent assurance proportionate to the potential severity and reach of the systems deployed.
For legislators and regulators
- Review whether thresholds based on company size or traditional sector boundaries capture small, high-impact operators.
- Define duties and enforcement mechanisms proportionate to foreseeable harm and actual control.
- Explore independently mandated trusteeship or equivalent oversight for activities whose external consequences warrant it.
- Ensure that legal duties do not become symbolic: specify access rights, intervention powers, remedies and accountable decision-makers.
Final observation
The immediate temptation is to treat AI as another technology that boards must understand. That framing is too narrow. AI is changing the scale at which individuals can exercise corporate power, the speed at which consequences can travel, and the institutional arrangements through which responsibility is traditionally assigned.
The profession of the non-executive director may change, but the need for stewardship will remain. The relevant question is no longer merely whether a company is properly managed, but whether its exercise of power is justified by the human interests that make corporate existence legitimate in the first place.
The challenge for governance is not to preserve every board, committee or title, but the capacity of society to identify, constrain and hold accountable those who exercise consequential power. Even when that power is wielded by a company of two people and a thousand autonomous systems.
If governance becomes invisible in its operation, accountability must become unmistakable in its attribution.
That is the transition before us and the test of whether the corporate form remains fit for the world AI is creating.