AI-Native Software Architecture and Bounded Semantic Sovereignty — Research Brief

Full research memo: AI-Native Architecture and Bounded Semantic Sovereignty

1. Domain question

How should an AI-native system divide semantic judgment, context, authority, and execution so that language models contribute real intelligence without receiving unrestricted control; what current evidence supports typed judgments, specialized cognitive contexts, deterministic effect boundaries, local–frontier model complementarity, and person-retained authority; and which larger AIOS implications remain conditional rather than established?

2. Executive answer

The research supports an architecture in which a language model receives genuine but bounded discretion over a semantic question, while trusted software separately controls identity, authorization, scope, and effects. This is more precise than “the model reasons and code executes.” A robust boundary must distinguish syntactic validity, semantic validity, identity validity, authorization validity, and effect validity. Structured output can secure the first of these and help express the others, but it does not prove them. Planetarium’s central result is decisive: GPT-4o produced PDDL descriptions that were 96.1% parseable and 94.4% solvable, yet only 24.8% semantically correct.

The evidence therefore rejects two extremes. Deterministic scraping and premature schemas can flatten the semantic intelligence a model was introduced to provide; peer-reviewed and preprint evidence shows that strict formatting can reduce reasoning performance and increase wrong-but-valid answers. Broad autonomy creates the opposite error by allowing one probabilistic component to interpret the request, choose tools, manage permissions, and cause effects. Tool-agent benchmarks show large prompt-injection surfaces, low repeated reliability, and weak long-horizon performance. Security work such as CaMeL and Progent instead places capabilities, information-flow rules, and privilege enforcement outside the model.

Evidence for separate cognitive modes is promising but not yet universal. Agentless demonstrates that an explicit localization–repair–validation pipeline can outperform more autonomous approaches in a bounded software task. NoLiMa shows that nominal context capacity does not ensure usable semantic access. SWE-Edit provides early evidence for clean viewer and editor contexts. These results support specialization according to purpose, admissible context, output contract, and permission—not a claim that every named faculty requires its own agent.

The research also sharpens AIOS’s system-intelligence thesis. Individual studies show that interfaces, context composition, staged workflows, schemas, validation, and model routing can independently affect quality or cost. OpenJarvis provides preliminary evidence that decomposed scaffolding can narrow a local–cloud gap that naive model substitution leaves wide. The cumulative claim—that purpose, composed context, layered memory, relational metadata, provenance, planning, verification, and integration jointly reduce inference burden and errors—remains untested. If it holds, durable intelligence and authority can increasingly reside in a person-owned local system while local and frontier models are invoked selectively. That implication complements rather than replaces frontier training and inference.

3. Essential findings

Finding 1 — Schema validity is not semantic validity

• Finding: A typed response can guarantee form while remaining wrong in meaning, target, authorization, or effect.
• Evidence: Planetarium evaluated 145,918 text-to-PDDL pairs across 73 state combinations with a semantic-equivalence algorithm; GPT-4o outputs were 96.1% parseable and 94.4% solvable but only 24.8% semantically correct.
• Relationship to AIOS: Direct mechanism. It validates AIOS’s separation between a model’s typed proposal and trusted resolution, policy, and execution.
• Implication: AIOS should validate canonical identities, versions, permissions, domain invariants, and observed post-state independently of schema acceptance.
• Limits or counterevidence: PDDL has unusually explicit semantics, so the exact gap will not transfer to every workflow; where semantic correctness is independently computable, deterministic systems may do more than merely mediate effects.
• Exact descent: Full memo sections “Executive assessment”, “2.4 A better response contract”, “4.1 Typed outputs solve an interface problem, not the whole control problem”, and “6.2 Planetarium — syntax, solvability, and semantics diverge”; Planetarium.

Finding 2 — Over-constraining the model can destroy useful reasoning

• Finding: Deterministic structure protects integration only when it does not prematurely compress an open semantic task.
• Evidence: Let Me Speak Freely? compared natural-language, instructed formats, constrained decoding, and natural-language-then-format generation across reasoning and knowledge tasks; stricter constraints generally caused larger reasoning declines. The Constraint Tax preprint found hard schemas raised validity to 100% for tested small models while reducing answer accuracy from 19.7% to 11.0% and raising wrong-valid outputs to 88.9%.
• Relationship to AIOS: Complicating evidence. It warns that typed judgments must preserve semantic freedom, uncertainty, and abstention rather than turn the model into a brittle classifier.
• Implication: “Reason in a non-authoritative stage, constrain at the boundary” should be a tested default for rich tasks; compact menus remain appropriate when the ontology is stable and complete.
• Limits or counterevidence: Constraint costs depend on model, task, schema, and decoding method. Structured output remains valuable, and some simple decisions may improve when the choice space is explicit.
• Exact descent: Full memo sections “2.2 Safe deterministic extraction”, “2.3 Destructive deterministic extraction”, “4.1 Typed outputs solve an interface problem, not the whole control problem”, and “6.3 Let Me Speak Freely? — constraints can tax reasoning”; Let Me Speak Freely?; The Constraint Tax.

Finding 3 — Semantic judges are useful but cannot serve as their own control plane

• Finding: Language models can supply semantic review signals, but difficult, perturbed, or repeatedly presented cases reveal unstable and biased judgment.
• Evidence: JudgeBench used 350 hard response pairs with objective ground truth; reported overall results included 50.86% for a vanilla GPT-4o judge and 56.57% for an arena-style configuration. CodeJudgeBench found every tested code judge sensitive to ordering and code-specific perturbations, especially misleading comments, variable renaming, and statement reordering.
• Relationship to AIOS: Boundary condition. It supports bounded review roles while excluding unverified model judgment as the sole authority for irreversible action.
• Implication: AIOS can use specialized reviewers with purpose-specific evidence, abstention, and presentation-bias tests, while reserving deterministic checks and consequential human judgment for properties the reviewer cannot establish.
• Limits or counterevidence: Earlier preference benchmarks found strong agreement with humans in open-ended assistant evaluation. Hard objective correctness and editorial preference are different tasks; local calibration matters more than a universal verdict on model judging.
• Exact descent: Full memo sections “2.1 Semantic judgment is the model’s comparative advantage”, “3.1 Semantic judges and critics”, and “6.1 JudgeBench — semantic judgment under objective ground truth”; JudgeBench; CodeJudgeBench.

Finding 4 — Authority must be enforced outside the model

• Finding: The strongest security pattern is to assume the model may be wrong or compromised while an external reference monitor still prevents unauthorized effects and data flows.
• Evidence: AgentDojo introduced 97 tasks and 629 prompt-injection security cases; Agent Security Bench covered ten scenarios, more than 400 tools, and 27 attack or defense methods, recording a highest average attack-success rate of 84.30%. CaMeL reports 67% of AgentDojo tasks completed with security guaranteed under its formal assumptions. Progent reports attack-success reductions from 39.9% to 1.0% on AgentDojo and 70.3% to 3.9% on ASB through deterministically enforced privilege policies.
• Relationship to AIOS: Direct mechanism. This is the closest external analogue to bounded semantic sovereignty and exact file operations.
• Implication: Tools, resource identities, arguments, destinations, credentials, and policy expansion should be capability-bound below the prompt. Model alignment remains defense in depth.
• Limits or counterevidence: These defenses rely on threat-model assumptions and cannot prevent harmful behavior inside legitimately granted privilege. Policy generation, trusted-query extraction, and implementation can themselves fail.
• Exact descent: Full memo sections “4.2 Capability security and least privilege”, “4.3 Authorization is not model alignment”, “5.1 Prompt injection is a control/data problem”, “6.7 AgentDojo and Agent Security Bench”, and “6.8 CaMeL, Progent, and Fides”; AgentDojo; Agent Security Bench; CaMeL; Progent.

Finding 5 — Context composition and staged cognition can outperform undifferentiated autonomy

• Finding: More context and more agent freedom do not reliably produce better work; purpose-specific context and explicit stages can improve performance or cost.
• Evidence: NoLiMa tested 13 long-context models with minimal lexical overlap; at 32K tokens, 11 fell below half their short-context baseline and GPT-4o fell from 99.3% to 69.7%. Agentless used fixed localization, repair, and validation phases and resolved 32.67% of SWE-bench Lite at a reported $0.68 average cost, leading compared open systems at submission time. SWE-Edit reports a 2.1-point gain and 17.9% cost reduction from clean viewer/editor contexts.
• Relationship to AIOS: Supporting evidence. It supports distinct cognitive contracts and composed context, but not a fixed universal faculty ontology.
• Implication: AIOS should separate activities when their purpose, admissible evidence, output contract, evaluator, or permission differs, and test handoffs against a single-context control.
• Limits or counterevidence: Fixed stages can block useful interleaving; specialization adds latency and handoff loss. SWE-Edit is a preprint, and its benchmark has known contamination and test-quality concerns.
• Exact descent: Full memo sections “3.3 Editors and software agents”, “3.4 Context specialists and cognitive modes”, “6.5 Agentless”, and “6.9 NoLiMa”; NoLiMa; Agentless; SWE-Edit.

Finding 6 — Broad agents remain variable and weak on long-horizon work

• Finding: Plausible trajectories and high bug-fix scores do not establish reliable autonomous execution across repeated or feature-scale tasks.
• Evidence: In τ-bench, GPT-4o completed fewer than half of retail and airline tasks, with retail pass^8 below 25%. FeatureBench constructed 200 feature tasks and 3,825 executable environments; its strongest full-set baselines resolved 11.0% and 12.5%, with average tasks changing about 790 lines across 15.7 files.
• Relationship to AIOS: Supporting evidence. It favors persistent plans, bounded effects, independent end-state checks, and human integration over global autonomous control.
• Implication: AIOS should evaluate repeated reliability and exact artifact state, not only successful-looking responses, and treat long-horizon work as durable staged coordination.
• Limits or counterevidence: Benchmarks simplify production work and age rapidly. Agent systems and models continue to improve; bounded architecture must still demonstrate comparable task utility rather than relying on current agent weakness.
• Exact descent: Full memo sections “3.2 Planners, routers, and tool selectors”, “3.3 Editors and software agents”, “6.4 τ-bench”, and “6.6 FeatureBench”; τ-bench; FeatureBench.

Finding 7 — Human approval must be sparse, trusted, and bound to execution

• Finding: A confirmation dialog preserves authority only when trusted software renders the exact boundary action and the approved action is the one that executes.
• Evidence: Anthropic’s operational report states that users approved roughly 93% of permission prompts and that sandboxing reduced prompts by 84%, indicating approval fatigue. The Consent Integrity preprint formalizes trusted rendering and bind-to-execution; its prototype also exposes a severe usability frontier, marking 87% of a normal-command corpus uninspectable.
• Relationship to AIOS: Direct mechanism. It sharpens person-retained authority from a general principle into a concrete trusted-path requirement.
• Implication: AIOS approvals should be generated from canonical targets, versions, parameters, and computed diffs—not model narration—and should appear mainly at meaningful authority boundaries.
• Limits or counterevidence: The telemetry is vendor-authored and the consent mechanism is an early preprint. Trusted rendering does not make the underlying choice wise or eliminate operator error.
• Exact descent: Full memo sections “4.4 Human approval is necessary in some paths and insufficient as a general defense”, “4.5 Recommended boundary invariants”, and “5.2 Production systems converge on bounded mechanics”; Anthropic containment report; Consent Integrity.

Finding 8 — System scaffolding may narrow the local–frontier gap, but naive substitution fails

• Finding: Local models are credible components of person-owned intelligence, yet architecture and selective frontier use—not simple replacement—currently determine whether they can approach cloud performance.
• Evidence: OpenJarvis found that replacing Claude Opus 4.6 with Qwen3.5-9B inside an unchanged stack reduced accuracy by 25–39 percentage points. Its typed five-primitive stack and frontier-guided configuration search produced local inference that matched or exceeded the cloud baseline on four of eight benchmarks and was within 3.2 points on average. Apple separately documents an approximately 3B on-device model supporting multilingual, multimodal, and tool-use tasks.
• Relationship to AIOS: Supporting evidence. It supports AIOS’s complementary local–frontier architecture and the proposition that scaffolding reduces model burden.
• Implication: If AIOS’s durable context and workflow layers generalize, routine private cognition may increasingly run locally while frontier models remain selective resources for difficult synthesis and system improvement.
• Limits or counterevidence: OpenJarvis is a recent preprint with author-controlled benchmarks and frontier assistance during configuration. Local execution does not guarantee privacy, accessibility, or lower total cost, and current capability gaps remain substantial.
• Exact descent: Full memo sections “3.5 Intelligence as a cumulative system outcome”, “6.10 OpenJarvis”, “6.11 Apple on-device models and Gemma 3”, and “8.2 Reduced application and inference dependence”; OpenJarvis; Apple Foundation Models Tech Report 2025.

4. How the evidence refines the AIOS account

1. A five-part validity model: syntax, semantics, identity, authorization, and effects must be evaluated separately; a typed response addresses only part of the chain.
2. A trusted reference-monitor contract: model outputs are untrusted semantic proposals whose targets, permissions, preconditions, approvals, execution, and postconditions are independently mediated.
3. A constraint-placement rule: preserve open reasoning where it creates value, then package the result at the non-authoritative boundary; measure wrong-valid outputs separately from schema validity.
4. An operational definition of human authority: consequential consent should be rendered from the exact boundary event and bound to what executes, with sandboxed low-risk work reducing approval fatigue.
5. A comparative evidence standard: AIOS architecture is evaluated for Pareto improvement over both deterministic flattening and broad autonomy across semantic success, repeated reliability, unsafe effects, disclosure, human effort, and total cost.

5. Architectural boundaries preserved

1. Distributed intelligence: External research supplies no basis for relocating AIOS intelligence into one privileged model or agent.
2. Local-first, file-native ownership: Evidence complicates local-model capability and privacy claims without justifying replacement of person-owned durable artifacts with provider-owned conversational state.
3. Distinct cognitive activities: Research supports careful testing without a universal faculty map and supplies no basis for collapsing thinking, writing, editing, planning, review, and integration into one scratchpad.
4. Person-retained consequential authority: More exact approval design strengthens rather than transfers human authority when confirmation is imperfect.
5. The Fractal Seed: External work neither validates nor invalidates Why–How–What; it remains an AIOS organizing hypothesis until directly compared with alternative grammars.

6. Where the evidence connects to AIOS

7. Relationships across research programs

1. Context and memory: Purpose-composed context and layered-memory risks connect to provenance and promotion rules; shared language must not obscure differences among context, memory, metadata, and retrieval mechanisms.
2. Faculties, roles, and workflow structure: Bounded-context evidence supports separation when information and output contracts differ, not merely when role names or additional calls are introduced.
3. Model routing and local inference: OpenJarvis and the disclosure boundary establish capability, privacy, sensitivity, latency, and cost as routing variables without settling the routing policy.
4. Security, authorization, and human control: Reference-monitor and consent-integrity mechanisms connect runtime containment to governance while keeping model alignment, authorization, and human judgment distinct.
5. Large-scale implications and infrastructure dependence: Local, private, and organizational consequences may follow from AIOS architecture, hardware, model markets, regulation, or social adoption; their causal contribution remains separable in evaluation.

8. Priority source set

1. April 2025 — peer-reviewed benchmark: Planetarium supports the exact claim that parseability and solvability can greatly exceed semantic correctness in typed planning output.
2. November 2024 — peer-reviewed industry-track study: Let Me Speak Freely? supports the claim that stricter output constraints can degrade reasoning and that delayed packaging can mitigate the loss.
3. April 2025 — peer-reviewed benchmark: JudgeBench supports the claim that leading model judges can perform near chance on difficult objective response pairs.
4. April 2025 — peer-reviewed benchmark: τ-bench supports the claim that tool agents remain inconsistent across repeated trials even when individual trajectories appear competent.
5. June 2025 — peer-reviewed software-engineering paper: Agentless supports the claim that explicit localization–repair–validation stages can outperform contemporary autonomous open systems on a bounded benchmark.
6. ICLR 2026 — peer-reviewed, vendor-coauthored benchmark: FeatureBench supports the claim that frontier coding agents remain weak on multi-file feature development despite strong bug-repair scores.
7. December 2024 — peer-reviewed security benchmark: AgentDojo supports the claim that tool-returned data creates a systemic indirect-prompt-injection problem.
8. April 2025 — peer-reviewed security benchmark: Agent Security Bench supports the claim that vulnerabilities span system prompts, user input, tools, and memory across diverse agents.
9. March 2025 — preprint: CaMeL supports the mechanism claim that control/data separation and capabilities can preserve security properties outside a vulnerable model.
10. April 2025 — preprint: Progent supports the mechanism claim that fine-grained, deterministically enforced tool policies can sharply reduce out-of-scope attack success.
11. July 2025 — peer-reviewed long-context benchmark: NoLiMa supports the claim that admitting more tokens does not ensure retrieval of semantically relevant evidence.
12. May 2026 — preprint: OpenJarvis supports both the local–cloud capability-gap claim and the preliminary claim that decomposed scaffolding can narrow it.
13. June 2026 — preprint: Consent Integrity supports the claim that human approval must be rendered from and bound to the exact action at the execution boundary.

9. Open research and design questions

1. Which tasks and failure measures distinguish bounded semantic sovereignty from bounded judgment alone?
2. What evidence and implementation invariants are necessary for the five validity layers and reference-monitor sequence to function as one complete control boundary?
3. Under what conditions can local–frontier complementarity reduce application and inference dependence when direct local-model substitution remains weak?
4. Which cognitive movements are stable architectural distinctions, and which remain task-dependent examples pending controlled comparison?
5. Is a three-way comparison—deterministic pipeline, bounded judgment, and broad agent—the strongest primary falsification test of the architecture?