Governance-by-Design for Experience Infrastructure
SCS Brief · Baseline principles for privacy-native systems, auditability and human agency, written for public and civic partners · Oct 4, 2026
In civic infrastructure, the person is the principal. An AI agent that coordinates on someone's behalf should act only on a mandate that person gave, leave a ledger they and an oversight body can read, and stop when anyone affected pulls the brake. Personal data should never sit in the same agent as untrusted input and a way out.
This brief sets out those four baseline principles for public and civic partners: municipalities, regions, cultural institutions and the companies that serve them. Each principle comes with what a partner can require in procurement and how to check it. The evidence behind them is set out in a forthcoming companion paper from Deriss Research, Rogue by Consensus.
1. Why now
AI agents are moving from answering questions to acting: booking, messaging, matching people and coordinating events across calendars, inboxes and public services. Once an agent holds keys to other systems, a breach no longer needs a hacker inside the network. It needs one poisoned input, or two agents whose combined actions add up to something neither was meant to do.
The 2026 record shows this is no longer theoretical:
Civic systems raise the stakes. They hold data about residents who never chose the vendor, and public trust lost to one incident takes years to rebuild. Governance therefore has to be designed in from the start, not added after a breach.
2. Four baseline principles
Each principle turns a security mechanism from the research into a right the person can see and a requirement a partner can write into a contract.
- Principle: Mandate (agency); What it means: An agent acts only within a consent the person gave: task, scope, data, tools and an expiry; What a partner can require: Every agent action traces to a readable mandate; credentials are issued per mandate and expire with it; the person can revoke it at any time; How to check it: Ask for a sample mandate and a test revocation; actions outside a mandate are blocked, not just logged
- Principle: Ledger (auditability); What it means: Every agent action is recorded in structured logs on one governed channel, with no side doors; What a partner can require: The person can see what was done in their name; a named oversight body can audit the full record; logs cannot be edited by the agents; How to check it: Request a person-level activity export and an auditor's read-only view
- Principle: Brake (human agency); What it means: An independent watcher can stop agents but cannot act itself; people decide what happens next; What a partner can require: A pause control for the person and the operator; a named human who decides after containment; consequential actions (payments, deletion, contacting outsiders) need human or independent approval; How to check it: Run a pause drill; check the watcher has no tools except stopping
- Principle: Privacy-native (privacy); What it means: No single agent holds personal data, reads untrusted content and can send data out at once; What a partner can require: Roles split so personal data never leaves through an agent that reads outside content; data stays local or federated by default; outbound destinations are allow-listed; How to check it: Ask for the agent role map and the outbound allow-list
The test for all four is the same: an honest agent loses almost nothing, because its actions match its mandate. A rogue one has to break a visible rule to act, and breaking the rule is the signal.
3. Alignment with GDPR and the EU AI Act
The four principles put into practice duties that already exist in European law. Governance-by-Design is the AI-agent counterpart of GDPR's data protection by design.
- Principle: Mandate; GDPR: Art. 6 lawful basis; Art. 7(3) consent can be withdrawn at any time; EU AI Act: Art. 50(1): people must be told they are dealing with an AI system (applies from 2 Aug 2026)
- Principle: Ledger; GDPR: Art. 5(2) accountability; Art. 30 records of processing; EU AI Act: Art. 12 automatic record-keeping
- Principle: Brake; GDPR: Art. 22 right to human intervention in automated decisions; EU AI Act: Art. 14 human oversight
- Principle: Privacy-native; GDPR: Art. 5(1)(c) data minimisation; Art. 25 data protection by design and by default; EU AI Act: Art. 15 robustness and cybersecurity
Timing matters for public partners. The Digital Omnibus, in force since 27 July 2026, moved most high-risk duties to 2 December 2027 (Annex III) and 2 August 2028 (Annex I) (Gibson Dunn https://www.gibsondunn.com/eu-ai-act-omnibus-agreement-postponed-high-risk-deadlines-and-other-key-changes/). Public bodies that deploy high-risk systems will also owe a fundamental-rights impact assessment (Art. 27). Whether a given civic coordination tool counts as high-risk depends on its use; building to these principles now means the answer does not change the design. This is not legal advice; partners should confirm classification with their own counsel.
4. How SCS applies it
Experience OS is being built to these Governance-by-Design principles, and Stockholm is where they are tested. It is a voice-first, privacy-native coordination layer, currently in private alpha. It works with people's calendars and messages on their behalf, which makes it exactly the kind of agent these principles exist for. We hold ourselves to the same test we ask partners to apply.
- Principle: Mandate; How Experience OS is designed to apply it: Each request is captured as a structured intent (when, who, why, how) that sets the agent's scope
- Principle: Ledger; How Experience OS is designed to apply it: A person-readable record of every action taken in their name
- Principle: Brake; How Experience OS is designed to apply it: A pause and revoke control for the person; consequential actions confirmed before they run
- Principle: Privacy-native; How Experience OS is designed to apply it: Federated learning with Scaleout Systems' FEDn; privacy-risk testing with LeakPro
Nothing is described as live until it can be shown to a partner. European by design; Stockholm is where it is proven.
5. For partners: what to ask any AI vendor
Partners can use these ten questions in procurement, whoever the supplier is, SCS included:
- Can you show us the mandate behind any agent action, and how a resident revokes it?
- Are credentials issued per task, and how long do they live?
- Is all agent-to-agent traffic on one logged channel? Which other surfaces can agents write to?
- Can a resident export a record of what was done in their name?
- Can our oversight body read the full log, and can the agents edit it?
- Who can pause the system, and who decides what happens next?
- Which actions need human or independent approval before they run?
- Does any single agent hold personal data, read outside content and send data out?
- Where is personal data stored and processed, and which outbound destinations are allowed?
- When did you last test your monitoring against an agent that tries to evade it?
Next step. SCS invites public and civic partners in Stockholm to test these principles with us https://www.socialcapitalstockholm.com/contact on a real coordination use case, and to tell us where they fall short.
Sources
- Gibson Dunn (2026). EU AI Act omnibus agreement: postponed high-risk deadlines https://www.gibsondunn.com/eu-ai-act-omnibus-agreement-postponed-high-risk-deadlines-and-other-key-changes/.
- Help Net Security (2026, 5 Aug). AI agent deception moves from theory to reality in UK cyber tests https://www.helpnetsecurity.com/2026/08/05/ai-agent-deception-in-cyber-tests/.
- Hugging Face (2026). Anatomy of a frontier lab agent intrusion https://huggingface.co/blog/agent-intrusion-technical-timeline.
- TechCrunch (2026, 30 Jul). Anthropic says its own AI models breached three companies during security tests https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/.
- VentureBeat (2026). Agents identifying as OpenAI systems wrote 17,000 posts to a wiki no one was supposed to write to https://venturebeat.com/security/agents-identifying-as-openai-systems-wrote-17-000-posts-to-a-wiki-no-one-was-supposed-to-write-to.
- Regulation (EU) 2016/679 (GDPR), Arts. 5, 6, 7, 22, 25, 30. Regulation (EU) 2024/1689 (AI Act), Arts. 12, 14, 15, 27, 50.
References and links are citations, not affiliations or endorsements.