Local-first, on-device, private, and sovereign AI

Abstract

On-device generative AI is now technically credible for a meaningful but bounded class of work. Peer-reviewed measurements show that quantized small language models can run interactively on recent phones and embedded accelerators, while current vendor systems extend that envelope with tightly integrated neural processors, multimodal encoders, weight streaming, sparsity, and task-specific adapters. The resulting capability is not equivalent to unrestricted frontier-model reasoning. Device memory, prefill cost, thermal throttling, context growth, runtime maturity, and model-task fit remain decisive. A nominal context window or parameter count is especially poor shorthand for usable local capability.

Privacy is likewise a system property, not a synonym for deployment location. Keeping computation local removes a major class of routine data transfers, but it does not prevent malicious applications, prompt injection from local files, poisoned memory, unsafe tool use, model extraction, insecure backups, output leakage, or compromised system software. Confidential-cloud designs can reduce operator access through trusted execution environments, attestation, stateless processing, and release transparency. They still depend on hardware, firmware, client, code, and governance assumptions. Recent attacks against confidential-computing hardware show why such guarantees must be stated as threat-model-bounded rather than absolute. FHE and MPC research reduces hardware trust through cryptographic computation, but current results are still bounded to smaller transformers and relative gains over costly private-inference baselines.

Hybrid local/cloud routing has solid evidence as a cost–quality optimization and emerging evidence as an edge-performance technique. The evidence does not establish privacy-preserving routing by itself. A privacy-respecting system needs a policy gate before any learned capability router: sensitivity, authorization, jurisdiction, network state, provenance, and allowed disclosure should determine whether cloud execution is permissible; only then should expected quality, latency, energy, and cost influence placement.

The architectural proposition that a bounded knowledge system may use ordinary files, exact read/write/edit/search operations, and derived metadata relationships without a canonical database, vector store, or embedding pipeline is defensible as a contingent engineering choice. Research comparing long-context use with retrieval, measuring effective context, and evaluating sparse, dense, hybrid, and graph retrieval supports no universal winner. For bounded, well-curated, human-navigable corpora, exact selection and lexical search may be simpler, more auditable, and sufficient. As query volume, corpus scale, vocabulary mismatch, multi-hop relationship depth, update concurrency, or latency requirements rise, derived lexical, vector, relational, or graph indexes become increasingly useful. The key distinction is between the canonical file representation and an operational acceleration layer: an index may be disposable, rebuildable, and versioned while ordinary files remain authoritative.

The AIOS theses are therefore examined as consequential architectural hypotheses whose direct system-level validation remains open. Current evidence establishes useful bounded local capability and shows that context selection, retrieval, tool exposure, memory design, validation, and model routing can each change performance or cost. AIOS adds the stronger thesis that useful intelligence is an outcome of their coordination with purpose, provenance, exact operations, post-response integration, and human authority—not merely a property of model weights. If substantially correct, the cumulative system could let smaller local models satisfy a growing share of recurring personal and organizational work, reduce retries and unnecessary remote inference, preserve private local cognition, and make frontier models selectively invoked complements rather than custodians of durable knowledge. The combined magnitude, interaction effects, and generality remain unmeasured; model limitations, integration overhead, poisoned context, stale memory, and external dependencies are credible counterforces. The memo preserves both the implication and the conditions required to test it.

1. Terms and evidentiary boundaries

Four terms are often collapsed even though they answer different questions:

• On-device AI says where inference executes. It does not by itself establish privacy, user control, portability, or independence from a platform vendor.
• Local-first AI says the user can perform core work from a local copy, normally including offline operation and later synchronization. It is an architectural priority, not a guarantee that every computation is local.
• Private AI says personal or confidential information is protected across collection, inference, storage, logging, synchronization, output, and deletion. This is an end-to-end claim requiring an explicit adversary model.
• Sovereign AI, as used here, means that a person or accountable organization retains meaningful control over canonical data, keys, retention, execution policy, models or model substitutes, portability, and human decision authority. The term has no single controlling technical definition, so this memo decomposes it rather than assigning it a binary label.

The evidence supports several distinctions that matter for this research scope:

1. Location is not governance. A proprietary device model may be local but difficult to inspect, migrate, or replace. A confidential remote service may provide strong protection from the operator but still depend on one provider.
2. Open weights are not automatically open source. The Open Source Initiative's Open Source AI Definition 1.0 requires the freedoms and preferred form needed to use, study, modify, and share a system, not merely a downloadable weight file [S20].
3. Canonical data and performance infrastructure are separable. A vector or graph index need not become a canonical source or system of record. It can be a versioned projection of canonical files.
4. Benchmark capability is not operational reliability. Most current studies test accuracy, latency, or energy under bounded workloads. Few jointly test security, long-running memory, edit fidelity, provenance, recovery, and human oversight.

1.1 Research method and source selection

The review used primary conference proceedings and paper repositories for systems, NLP, machine learning, and security research; final or draft publications from NIST and the European Data Protection Board; current project documentation for portability; and first-party technical reports where proprietary on-device or confidential-cloud systems have no equivalent independent description. Sources were selected for methodological relevance rather than count. Product announcements without meaningful architectural or evaluation detail were excluded.

The main review window is August 2024 through August 10, 2026. Older work appears only in the separately labeled foundational-lineage section. Quantitative findings are reported as results of the named study, not universal device facts. Vendor reports and previews are used to describe current architectures, not as independent confirmation of quality, security, or energy. Preprints and accepted-but-not-yet-presented work are visibly labeled. The final interpretation is a synthesis by this memo and should be tested against the actual target workload.

2. What current devices can actually do

2.1 Evidence-calibrated capability envelope

The table below is a synthesis, not a compatibility guarantee. “Interactive” depends on prompt length, output length, accelerator support, thermal state, runtime, quantization, and the task's acceptable latency.

These categories deliberately avoid a universal “largest model that runs.” Loading is a weak test. A useful deployment must meet time-to-first-token, sustained generation, peak memory, temperature, battery or wall energy, response quality, and failure-rate requirements on its actual workload.

Direct tablet-specific research is notably thin. Tablets often share phone or laptop-class system-on-chip families and may have more thermal or memory headroom, but extrapolating from phones or vendor compatibility claims is not an independent tablet benchmark. This is an evidence gap rather than evidence of equivalence.

2.2 Major evidence profile: PalmBench

Status: Peer-reviewed, ICLR 2025 [S01].

• Research question. How do compressed language models behave on real mobile and edge platforms when evaluated across latency, throughput, memory, power, temperature, output similarity, toxicity, and hallucination rather than accuracy alone?
• Method and sample. PalmBench tested multiple Llama, Phi, Qwen, Mistral, Gemma, and related models at 2–8-bit compression through mobile runtimes on three iPhones, several Android phones, two Orange Pi configurations, and NVIDIA Jetson devices. The hardware included older and then-current consumer devices, from a Pixel 4 and iPhone 12 Pro to an iPhone 15 Pro and Jetson Orin Nano.
• Principal finding. Four-bit quantization was usually a strong size–quality compromise; the paper reports model storage near one quarter of the uncompressed baseline with broadly preserved output behavior. Hardware and runtime mattered greatly. In the tested configurations, iPhones often delivered better latency, throughput, power, and thermal results than the Android devices, while embedded accelerators varied substantially.
• Limitations. The comparison is inseparable from the selected frameworks, kernels, prompts, device generations, and operating conditions. The device set does not represent every modern NPU, and output-similarity or toxicity probes do not establish correctness or safety. The paper predates important 2025–2026 system-model releases.
• Relevance. PalmBench strongly supports measuring the complete device path and treating quantization as workload-dependent. It does not support the claim that a given parameter class will be equally usable across devices or that local execution is inherently private.

2.3 Major evidence profile: Demystifying Small Language Models for Edge Deployment

Status: Peer-reviewed, ACL 2025 [S02].

• Research question. How capable are public 100M–5B small language models, and how do model size, quantization, prefill, decoding, and hardware affect edge deployment?
• Method and sample. The authors evaluated 68 models from 24 organizations on ten language benchmarks, then measured representative inference on a Jetson Orin NX, a Meizu 18 Pro, and—for one Qwen model—three Android phones.
• Principal finding. Strong recent small models sometimes matched or exceeded older 7B models on general benchmarks, but remained weak at in-context learning. Quantization helped most in memory-bandwidth-bound decoding and less in compute-bound prefill. In one configuration, the GPU was about 40 times faster than CPU for prefill but only 1.84 times faster for decoding. The phone experiments also showed thermal throttling and decreasing quantization advantage as context grew.
• Limitations. Model selection ended in September 2024, hardware coverage was narrow, and benchmark accuracy does not establish reliable tool use, long-context synthesis, or safe editing. The results should not be extrapolated directly to newer NPUs or fused vendor runtimes.
• Relevance. The study supports task–model matching, quantization, and accelerator-aware scheduling. It argues against using parameter count alone as the routing signal.

2.4 Major evidence profile: SlimLM

Status: Peer-reviewed system demonstration, ACL 2025 [S03].

• Research question. What model sizes and specializations are practical for document assistance on a phone?
• Method and sample. SlimLM profiled 125M–8B 4-bit models on a Samsung Galaxy S24, then trained 125M–1B document-assistance models on approximately 83,000 documents and 414,000 examples for summarization, question answering, and suggestion tasks. Inputs were capped at about 1,000 tokens.
• Principal finding. The small models were dramatically more responsive. On a short prompt, a 135M model had a reported 0.46-second time to first token, while tested 7B and 8B models required roughly 128 and 262 seconds. Task-specific small models performed well on the paper's intent-focused evaluations.
• Limitations. One phone, one deployment framework, short truncated documents, teacher-generated data, and specialized tasks sharply bound the result. The paper does not show that a sub-billion model can replace a general model for open-ended reasoning.
• Relevance. It provides unusually concrete evidence for a local “small specialist first” architecture and warns against equating loadability with usability.

2.5 Current vendor systems: useful evidence with narrower epistemic weight

Apple's 2025 foundation-model report describes an approximately 3B on-device multilingual and multimodal model, two-bit quantization-aware training for most decoder weights, eight-bit KV caching, and KV-cache sharing reported to reduce cache memory by 37.5% [S04]. Its developer framework exposes guided generation, tool calls, streaming, and adapters. These are significant implementation disclosures, but the evaluation is vendor-authored and the hardware/software stack is proprietary.

Apple's June 2026 third-generation preview goes further: its “Core Advanced” model is described as a 20B sparse system that activates 1–4B parameters, using prompt-level selection and periodic reselection while keeping the full weight set in device storage rather than swapping experts token by token [S05]. This is evidence of an important systems direction—coarse-grained sparsity plus flash-resident weights—but not yet independent evidence of sustained energy, latency, or quality across applications. At the cutoff, Apple stated that a fuller technical report would follow; the page is therefore labeled vendor preview, not a peer-reviewed result.

Gemma 3n's 2025 model card describes open-weight E2B and E4B models designed for phones, tablets, and laptops, with a 32K context window, text/image/video/audio inputs, more than 140 languages, nested MatFormer submodels, parameter skipping, and optional modality loading [S07]. These features show how effective active parameter count and conditional loading can matter more than total stored weights. The claims remain vendor-authored; the model card is not an independent cross-device energy benchmark, and the Gemma license should not be conflated with the OSI definition of open-source AI.

2.6 Major evidence profile: instruction-following pruning

Status: Peer-reviewed, ICML 2025; Apple-affiliated authors [S06].

• Research question. Can a model select a structured subnetwork from the user's instruction and retain much of a larger model's quality at smaller-model latency?
• Method and sample. The method dynamically prunes a 9B model to an approximately 3B active subnetwork at instruction level and evaluates mathematical, coding, and language tasks.
• Principal finding. The reported 3B-active system exceeded a dense 3B baseline by roughly five to eight absolute points on several math and coding evaluations, approached the 9B model on selected tasks, and ran near dense-3B inference latency.
• Limitations. Results are model- and benchmark-specific, come from an affiliated research team, and do not directly measure a production phone's storage traffic, battery, thermals, or long-running behavior.
• Relevance. The work supports elastic local inference and explains one route to the 2026 vendor architecture. It does not establish that sparse local models have frontier-model reliability.

3. Performance, energy, and context are coupled

3.1 The useful performance model

Local inference has at least four distinct resource phases:

1. Model load and initialization move weights from storage into usable memory and compile or select kernels.
2. Prefill processes the prompt in parallel. Its cost grows quickly with input length and is often compute-bound.
3. Decode generates tokens sequentially and is frequently memory-bandwidth-bound, which is why low-bit weights can help disproportionately.
4. Post-processing and tools parse output, retrieve files, render results, execute actions, and persist state. These costs are routinely omitted from model-only benchmarks.

Weight storage has a rough lower bound of parameter count × bits per weight / 8, but real memory also includes embeddings, scale metadata, temporary activations, runtime buffers, loaded modalities, and the KV cache. KV memory grows with context length and model structure. Consequently, a model advertised for a long context may fit at short context but fail a device budget—or become thermally and interactively unacceptable—at the maximum.

This produces several evidence-supported design rules:

• Benchmark prefill and decode separately. A backend that excels at matrix-heavy prefill may offer a smaller decode advantage [S02].
• Benchmark warm and cold starts. A background assistant may pay load costs repeatedly unless the platform can retain state.
• Report system energy, not accelerator power alone. Memory, CPU coordination, networking, display, storage, and cooling are part of the deployment.
• Test sustained sessions. Short bursts conceal thermal throttling [S01, S02].
• Measure at the real context distribution, including retrieved passages, metadata, tool schemas, and image/audio tokens.
• Treat tokens per second as only one service metric. Time to first useful answer, grounded correctness, edit success, battery drain, and recovery from failure are often more important.

3.2 Major evidence profile: end-to-end RAG on a mobile NPU

Status: Preprint—not peer-reviewed, June 2026 [S08].

• Research question. Can an entire retrieval-augmented generation path—embedding, retrieval, reranking, prefill, and decode—run efficiently on a laptop-class mobile NPU?
• Method and sample. The authors implemented the pipeline on one Dell XPS 13 with a Snapdragon X Elite/Hexagon NPU and evaluated indexing plus 120 Wikipedia-passage questions against CPU and GPU paths.
• Principal finding. The paper reports 9.1-times higher embedding throughput and 12.3-times lower system energy for indexing, plus 18.1-times faster prefill and about four-times lower end-to-end latency and system energy for the question-answering path compared with CPU. An LLM judge rated the answer quality as comparable.
• Limitations. One laptop, one SoC, one corpus, 120 questions, framework-specific implementation, and LLM-judged output make this an important but non-replicated result. It does not establish equivalent gains on phones, other NPUs, or arbitrary retrieval workloads.
• Relevance. It is direct evidence that derived indexes need not imply cloud dependence and that retrieval can be compatible with local energy goals. It should motivate replication, not a universal NPU claim.

3.3 Energy accounting beyond the device

Local processing often saves network transfer and remote inference for small, frequent, personal tasks. It can also avoid sending raw content to a provider. It does not follow that local execution always minimizes total energy or carbon. A small device may run a poorly optimized model much longer than a shared datacenter accelerator; conversely, network transport, datacenter idle capacity, and repeated remote prompts can dominate some workloads. Current studies rarely use a common lifecycle boundary.

The appropriate comparison is task-specific:

useful task success / joule, measured across device computation, networking, remote computation, failed attempts, retries, and index maintenance. Carbon claims additionally require time- and location-specific electricity data and embodied-hardware assumptions. None of the reviewed papers supports a blanket “local is greener” conclusion.

4. Hybrid local/cloud routing

4.1 Routing patterns

For a privacy-oriented system, a learned router should not be the first decision-maker. A safer decision order is:

1. Policy eligibility: Is remote processing allowed for this domain, file, person, jurisdiction, and task?
2. Disclosure minimization: What is the smallest sufficient context? Can identifiers or irrelevant passages be removed without making the task unsafe?
3. Execution guarantees: Does the selected remote path meet required retention, logging, attestation, key, region, and model-use constraints?
4. Capability routing: Among eligible paths, which is most likely to meet correctness, latency, energy, availability, and cost goals?
5. Authority and integration: Is the result advisory, or may it change canonical files? What validation or human confirmation is required?

This ordering prevents a quality or confidence score from silently overriding a non-disclosure rule.

4.2 Major evidence profile: RouteLLM

Status: Peer-reviewed, ICLR 2025 [S09].

• Research question. Can a learned router choose between a strong expensive model and a weaker cheaper model while maintaining target quality?
• Method and sample. RouteLLM trained several binary routers from human preference data, augmented that data, and tested GPT-4 versus Mixtral 8×7B on MMLU, MT-Bench, and GSM8K, including generalization to unseen model pairs.
• Principal finding. The routers reduced model cost by more than twofold at comparable benchmark quality in reported settings, with small routing overhead and some transfer to new model pairs.
• Limitations. This is cloud-to-cloud cost–quality routing under dated model prices and benchmark distributions. It does not measure privacy, local energy, offline availability, context disclosure, or the consequences of routing error. Binary routing is much simpler than heterogeneous device/cloud placement.
• Relevance. RouteLLM supports learned capability routing as a component. It does not validate privacy-aware local/cloud routing; a privacy-oriented system would need hard policy gates and more objectives.

4.3 Major evidence profile: CITER

Status: Peer-reviewed, COLM 2025 [S10]; related implementation is a peer-reviewed system demonstration, ACL 2025 [S30].

• Research question. Can a smaller model consult a larger model at selected token positions rather than routing the whole request?
• Method and sample. CITER used reinforcement learning to decide token-level collaboration between smaller and larger Qwen-family models over five benchmarks. A related edge demonstration tested a 0.5B local model on an M1 laptop with a server-side 32B model and measured network overhead.
• Principal finding. CITER reports up to 30% fewer large-model calls at comparable accuracy or up to 25% higher accuracy at a similar number of calls than a prior collaboration baseline. The edge demonstration reports a 60% CommonsenseQA performance gain while uploading under 7% of generated tokens. It also exposed practical costs: roughly 170 ms of network latency and about 0.9 seconds for a cloud request in one setup, plus re-prefill when the remote side lacked the local KV cache.
• Limitations. Compatible tokenization and state exchange are strong assumptions. Repeated offload increases latency variance and can expose a richer interaction trace. Benchmarks do not model sensitive personal files, adversarial content, or disclosure policy.
• Relevance. CITER demonstrates that hybrid placement can operate below the request level. For private knowledge work, it is more useful as a systems boundary case than an immediately desirable architecture.

5. Confidential computation and secure enclaves

5.1 What current designs add

Apple's Private Cloud Compute (PCC) security guide specifies stateless request processing, no privileged runtime access to plaintext, anonymous routing, hardware-rooted secure boot, and device-side verification that a node's complete software measurement appears in a public append-only transparency log [S11]. Apple also publishes binaries and a research environment intended to allow inspection. In June 2026, Apple described extending PCC to selected Google Cloud and NVIDIA infrastructure while retaining the same attestation and release-transparency requirements. These are unusually concrete, vendor-authored security design claims; they are not proof that every implementation layer is flaw-free.

Google’s 2025 Private AI Compute brief describes encrypted, attested processing using Titanium Intelligence Enclaves, an AMD CPU trusted execution environment, hardened TPU paths, and peer attestation for multi-node workloads [S12]. Google’s 2026 private-compute infrastructure documentation describes attestation-linked mutual TLS and restricted egress from protected workloads [S27]. This is also vendor-authored. Public documentation explains the intended architecture but is not equivalent to independent verification of every production release.

NIST IR 8320E, an authoritative initial public draft published May 29, 2026, provides a useful neutral vocabulary and example implementation for confidential-computing platforms, isolation, attestation, and cloud AI workloads [S13]. Because it was not final at the cutoff, it should not be cited as settled normative guidance.

On-device secure hardware usually plays a narrower but essential role. Apple documents its Secure Enclave as the root for protected key generation/storage, biometric authorization, secure boot relationships, and data-at-rest encryption on recent phones, tablets, and Apple-silicon computers [S28]. Android’s current Keystore/KeyMint documentation describes application-scoped, hardware-backed keys in a trusted execution environment or StrongBox and exposes attestation of hardware-enforced properties [S29]. These facilities can protect domain-encryption keys, gate sensitive actions on user authentication, sign audit records, and authenticate an approved remote workload. They normally do not execute the whole language model or make plaintext invisible to the authorized application process that composes the prompt. The NPU should not be assumed to be a secure enclave merely because it is on the same chip.

5.2 Guarantees must be scoped

Confidential computing can protect data in use from a cloud operator, hypervisor, or neighboring workload—if the hardware and attested software behave as specified. It does not inherently:

• make the model's answer correct or aligned with the user's intent;
• prevent prompt injection or memory poisoning inside authorized content;
• prevent authorized workload code from retaining or emitting data;
• protect a compromised client before encryption or after decryption;
• hide every network or timing metadata signal;
• guarantee deletion from client logs, backups, telemetry, or downstream tools;
• eliminate hardware, firmware, cryptographic, supply-chain, or attestation-service vulnerabilities;
• confer model/provider portability or legal sovereignty.

5.3 Major evidence profile: Fabricked

Status: Accepted prepublication, USENIX Security 2026; accepted and public before the cutoff, conference presentation scheduled after it [S14].

• Research question. Can a malicious hypervisor defeat AMD SEV-SNP by manipulating Infinity Fabric configuration despite memory-encryption and attestation protections?
• Method and sample. The researchers developed a software-only attack that reconfigured memory address routing on affected AMD systems and demonstrated unauthorized reads and writes against protected virtual machines.
• Principal finding. The attack bypassed core SEV-SNP confidentiality and integrity assumptions and undermined attestation's meaning for affected configurations. It received CVE-2025-54510; mitigations were issued for relevant processor generations.
• Limitations. This is a specific implementation vulnerability, not evidence that all trusted execution environments fail. The final conference presentation had not occurred by the research cutoff, and patched systems differ from the evaluated state.
• Relevance. Fabricked is concrete evidence that “hardware protected” must remain a qualified, patch-dependent statement. Attestation identifies a measured configuration; it cannot prove that underlying hardware has no exploitable defect.

5.4 Cryptographic privacy beyond trusted hardware

Fully homomorphic encryption (FHE) permits a server to compute on ciphertext, while secure multi-party computation (MPC) distributes computation so that no designated participant learns the protected input under the protocol's assumptions. These approaches reduce reliance on a confidential-computing processor, but transformer linear algebra, nonlinear activations, autoregressive state, ciphertext expansion, and communication remain expensive.

Major evidence profile: EncryptedLLM

Status: Peer-reviewed, ICML 2025 [S31].

• Research question. Can GPU acceleration make a fully homomorphic transformer forward pass materially faster while keeping the client's input and output encrypted from the server?
• Method and sample. EncryptedLLM implemented GPU-accelerated FHE, approximated activation functions for encrypted arithmetic, and benchmarked an encrypted GPT-2 forward pass against its CPU FHE implementation.
• Principal finding. The paper reports more than a 200-fold runtime improvement over the CPU FHE baseline while studying the accuracy effect of activation approximations.
• Limitations. The comparison is against a cryptographic CPU baseline, not ordinary plaintext inference. It covers GPT-2 and a forward pass rather than a current multi-billion-parameter assistant completing a long autoregressive session. Relative speedup therefore should not be read as interactive parity.
• Relevance. The work demonstrates meaningful progress toward cloud inference where the provider never receives plaintext. At the cutoff, it is a research alternative for exceptional confidentiality requirements rather than a general replacement for local inference or attested cloud execution.

Major evidence profile: Breaking the Layer Barrier

Status: Peer-reviewed, USENIX Security 2025 [S32].

• Research question. Can a hybrid of CKKS homomorphic encryption and MPC reduce the costly protocol conversions in private transformer inference?
• Method and sample. The BLB system fused adjacent linear operators, introduced a secure CKKS/MPC conversion, and evaluated BERT-base, BERT-large, and GPT-2-base with GPU acceleration.
• Principal finding. Relative to BOLT and BumbleBee, the authors report communication reductions of 21-fold and twofold and latency reductions of 13-fold and 1.8-fold, respectively.
• Limitations. These are gains over prior private-inference systems, not over plaintext serving. The evaluated models are much smaller than current frontier assistants, and network, adversary, setup, and numerical assumptions condition the result.
• Relevance. BLB shows rapid systems progress and the importance of communication-aware protocol design. It does not show that general LLM inference under MPC/FHE has reached routine consumer latency or energy budgets.

6. Threat model for a file-native local reasoning system

6.1 Assets and adversaries

Relevant assets include canonical files; encryption keys; derived metadata and indexes; prompts and intermediate context; model weights and adapters; tool credentials; action history; personal preferences; outputs; and human decisions. Plausible adversaries include a remote service provider, compromised dependency or model artifact, malicious application, local account or thief, untrusted collaborator, poisoned document or web content, compromised synchronization service, and an authorized insider exceeding purpose.

Local execution substantially reduces routine provider access to raw data. It does not remove most endpoint and content-layer adversaries.

6.2 Major evidence profile: AgentDojo

Status: Peer-reviewed dataset and benchmark, NeurIPS 2024 [S15].

• Research question. How robust are tool-using language-model agents when untrusted data contains prompt injection, and how well do proposed defenses preserve utility?
• Method and sample. AgentDojo built 97 realistic tasks and 629 security test cases across simulated applications with tools and attacker-controlled content, then evaluated attacks and defenses on multiple agent configurations.
• Principal finding. Contemporary agents struggled both with benign task completion and with indirect prompt injection; no evaluated defense produced a general solution without important utility or coverage limitations.
• Limitations. The environments are simulated, the model snapshot is from 2024, and any aggregate score depends on task and attack construction. It does not reproduce every local filesystem or operating-system boundary.
• Relevance. A local file can be trusted as data provenance yet still contain adversarial instructions. Exact file operations and action authorization therefore need to be security boundaries, not merely convenience functions.

6.3 Major evidence profile: Agent Security Bench

Status: Peer-reviewed, ICLR 2025 [S16].

• Research question. How vulnerable are LLM agents to a broad set of prompt injection, memory poisoning, backdoor, and mixed attacks, and how effective are current defenses?
• Method and sample. The benchmark spans ten scenarios, ten agents, more than 400 tools, 27 attack/defense categories, seven metrics, and 13 model backbones.
• Principal finding. The strongest tested attacks reached an 84.3% average attack-success rate in the paper's setting, while the 11 tested defenses offered limited and uneven protection.
• Limitations. The maximum reported success rate is not a universal production risk estimate. Benchmark scaffolding, agent implementation, and model generation strongly affect results, and security benchmarks themselves can omit realistic constraints.
• Relevance. Memory, companion metadata, annotations, and tool calls all need provenance, least privilege, and poisoning controls. More memory can increase usefulness and attack persistence simultaneously.

6.4 Authoritative attack taxonomy

NIST AI 100-2e2025 is a final authoritative report, not a performance benchmark [S17]. It organizes adversarial machine-learning risk across evasion and prompt injection, poisoning, privacy attacks, misuse, and the data/model/deployment lifecycle. Its value here is completeness and shared terminology. It does not prescribe a foolproof control set, and NIST explicitly treats mitigations as limited and context-dependent.

6.5 Threat-to-control map

The controls are design candidates. Their effectiveness must be tested in the actual operating system, synchronization, parser, model, and tool environment.

7. Data minimization, sovereignty, and provider independence

7.1 Data minimization is a lifecycle discipline

The European Data Protection Board's Opinion 28/2024 is a regulatory opinion addressing AI models and personal data [S18]. It states that model anonymity is case-specific and requires a very low likelihood both of identifying people and of extracting their personal data. It also stresses that unlawfully processed upstream data can affect later deployment. The opinion is European legal guidance rather than a systems benchmark, but it usefully rebuts the assumption that model training automatically anonymizes source data.

For a local-first knowledge system, minimization should occur at each boundary:

• Collection: acquire only files and fields needed for declared domains.
• Selection: retrieve the smallest sufficient passages or structured facts, not entire folders by default.
• Context composition: exclude unrelated metadata, secrets, hidden text, and old memory.
• Inference: prefer local execution when it satisfies the task; where remote execution is authorized, disclose a manifest of what leaves.
• Logging: avoid raw prompt and document retention unless necessary; separate operational metrics from content.
• Derived data: treat embeddings, summaries, entity graphs, and caches as potentially sensitive; version, encrypt, expire, and delete them with their sources.
• Output and actions: minimize copied personal information and apply purpose-specific retention.
• Backups and synchronization: apply the same deletion and access policy to replicas.

Local storage helps mainly at the inference-transfer boundary. Duplicate files, embeddings, caches, logs, and backups can expand the local privacy surface if unmanaged.

7.2 A decomposed sovereignty test

Sovereignty is therefore graduated. A system may be private but provider-dependent, local but non-portable, or open-weight but remotely operated.

7.3 Portability and model/provider independence

ExecuTorch 1.0, released in October 2025, is a project/vendor release that provides a common PyTorch-oriented edge runtime across CPU, GPU, and NPU backends from multiple hardware vendors [S19]. Portable model representations and other cross-platform runtimes serve a similar architectural purpose. These projects reduce integration duplication, but a common file format does not guarantee operator coverage, numerical parity, tokenizer equivalence, performance, or access to every proprietary accelerator.

A provider-independent deployment would need more than interchangeable API endpoints:

1. Canonical files and accepted ground in documented, exportable formats.
2. Model-independent task specifications, tool schemas, and evaluation sets.
3. Explicit tokenizer, prompt, adapter, and structured-output dependencies.
4. At least two viable runtimes or model families for important tasks, with a degraded offline mode.
5. Versioned quality, latency, energy, and safety tests run before substitution.
6. No provider-only hidden state as the sole copy of memory or provenance.
7. Licenses that permit the intended use, preservation, modification, and redistribution.
8. A migration path for derived data: embeddings and indexes either rebuild under a new model or retain their old runtime and version metadata.

The last point is often missed. An embedding index couples a corpus to a model, preprocessing pipeline, dimension, distance function, and chunking policy. It can remain non-canonical, but only if the source-to-index transformation is recorded and reproducible.

8. The file-native, no-canonical-database proposition

8.1 The claim under examination

The proposition is not that databases, vector stores, or embeddings are unnecessary. It is narrower:

For a bounded, self-contained domain, ordinary files plus exact read/write/edit/search and derived metadata relationships may be sufficient as the canonical file-and-metadata substrate. Specialized indexes can be added as disposable acceleration or discovery layers rather than made authoritative by default.

That proposition is consistent with available retrieval and long-context evidence, but the evidence does not identify a universal boundary at which it succeeds or fails. The correct boundary depends more on task distribution and service requirements than raw corpus size.

8.2 What ordinary files do unusually well

• Human inspection and editing do not require a database client or migration.
• Version control, content hashes, backups, diff, and recovery can operate at the canonical representation.
• Exact paths and exact text matches provide deterministic, explainable retrieval.
• File and companion-metadata boundaries can mirror human domains and permissions.
• A damaged or obsolete index can be discarded and rebuilt without losing the knowledge itself.
• Provider and model substitution does not require exporting knowledge from an embedding-specific schema.

These are engineering properties, not empirical proof of higher answer quality.

8.3 Conditions under which files plus exact operations are likely sufficient

Files are a plausible complete canonical substrate when most of the following hold:

• the working domain is bounded and has a stable, comprehensible folder and naming structure;
• common questions map to known files, exact terms, metadata fields, or shallow relationships;
• an explicit selector can assemble the relevant material within the model's effective—not advertised—context;
• search and opening latency meet the foreground response objective on target hardware;
• update concurrency is low, usually one person or a small coordinated group;
• cross-file atomic invariants are rare;
• provenance, inspectability, and faithful editing matter more than global analytical queries;
• omissions can be detected by a human or deterministic check;
• the system can fall back to broader search without violating a latency or energy budget.

This describes many personal projects, case files, research notebooks, policy domains, and small expert corpora. It does not describe every personal archive, enterprise repository, or high-rate sensor system.

8.4 Major evidence profile: long context versus RAG

Status: Peer-reviewed industry paper, EMNLP 2024 [S21].

• Research question. When should a system use long-context prompting, retrieval-augmented generation, or a router between them?
• Method and sample. The study compared Gemini 1.5 Pro, GPT-4o, and GPT-3.5 across nine datasets using long context, RAG, and a “Self-Route” method that predicted whether retrieval was sufficient.
• Principal finding. Long context generally achieved higher average performance when computation was available; RAG used fewer input tokens and was cheaper. Self-Route approached long-context quality while using substantially fewer tokens in the reported configurations.
• Limitations. Proprietary 2024 models, benchmark corpora, one retrieval design, and then-current API prices constrain generalization. It does not test local files, exact editing, privacy, or NPU energy.
• Relevance. The study rejects a universal need for retrieval when a bounded corpus fits an effective context, while showing why retrieval becomes economically and operationally useful as context grows. It does not establish that raw full-context stuffing is reliable.

8.5 Major evidence profile: LongBench v2

Status: Peer-reviewed, ACL 2025 [S23].

• Research question. Can current long-context models solve realistic problems requiring understanding of very long documents, repositories, conversations, and structured data?
• Method and sample. LongBench v2 contains 503 human-curated multiple-choice questions across six task categories, with contexts from 8,000 to 2 million words. Nearly 100 educated annotators contributed; an expert baseline was measured under a 15-minute limit.
• Principal finding. The best directly evaluated model scored 50.1%, while the reported o1-preview result reached 57.7%; the time-limited expert baseline was 53.7%. Long advertised windows did not translate into uniformly strong deep-context reasoning.
• Limitations. Multiple-choice evaluation, a small number of questions per category, a time-constrained human comparison, and late-2024 model snapshots limit interpretation. It does not isolate retrieval from reasoning or represent every file-native workflow.
• Relevance. LongBench v2 supports treating effective context as a measured capability. A system should not assume that placing all selected files inside the nominal window eliminates retrieval or decomposition needs.

8.6 Major evidence profile: RULER

Status: Peer-reviewed, COLM 2024 [S24].

• Research question. Do models retain reliable retrieval, multi-hop, and aggregation ability throughout their advertised context windows?
• Method and sample. RULER constructs configurable synthetic tasks, varies input length and task complexity, and evaluates long-context models beyond simple single-needle retrieval.
• Principal finding. Many evaluated models had an effective context substantially shorter than the length they nominally accepted, particularly as task complexity increased.
• Limitations. Synthetic tasks do not reproduce all natural-document ambiguity, and rapidly changing model generations can shift the measured frontier.
• Relevance. RULER supplies a controlled reason to benchmark selection accuracy throughout the intended context distribution. It does not prove that retrieval is always better than long context.

8.7 Major evidence profile: RAG pipeline best practices

Status: Peer-reviewed, EMNLP 2024 [S22].

• Research question. Which combinations of query classification, rewriting, chunking, retrieval, reranking, repacking, and generation produce effective RAG systems?
• Method and sample. The authors ran a modular comparative evaluation across representative datasets and assembled recommended combinations from component-level experiments.
• Principal finding. Retrieval quality depended on interacting pipeline choices; no isolated “add RAG” step captured the tradeoff, and stronger configurations introduced additional latency and complexity.
• Limitations. Results depend on the selected datasets, models, retrieval components, and evaluation metrics. The study does not test canonical file editing, local energy, or index recovery.
• Relevance. RAG is an optional maintained subsystem, not neutral infrastructure. A file-native system should add it only where measured retrieval gains exceed its operational and epistemic costs.

8.8 When an index or database becomes useful

The following escalation ladder keeps canonical files and accepted ground conceptually separate from access infrastructure:

No research-backed document-count threshold cleanly separates these layers. An engineering heuristic is to instrument the actual workload and escalate when one or more measurable conditions persists:

• p95 foreground search or context-selection latency exceeds the target;
• recall on a representative query set falls below the accepted level;
• exact/lexical queries cannot bridge recurring vocabulary mismatch;
• the selected context regularly exceeds the model's empirically effective window;
• relationship queries require repeated full-corpus scans or complex ad hoc parsing;
• index energy and maintenance are lower than the cost of repeated scans;
• multiple writers or invariants make file-level conflict resolution unsafe;
• rebuild time no longer meets recovery requirements.

For rough planning only—not as a research law—teams often encounter lexical-index pressure in the tens of thousands of documents or hundreds of megabytes to a few gigabytes, especially with repeated interactive queries. A well-named million-file corpus may favor an index much earlier due to filesystem enumeration, while a few hundred enormous heterogeneous documents may require semantic or structural indexing sooner. Query entropy and update behavior dominate document count.

8.9 Major evidence profile: HybGRAG

Status: Peer-reviewed, ACL 2025 [S25].

• Research question. How should retrieval answer questions that require both textual evidence and relationships in a semi-structured knowledge base?
• Method and sample. HybGRAG combines a bank of textual and graph-oriented retrievers with a critic that selects or revises retrieval paths, evaluated on the STaRK benchmark.
• Principal finding. The paper reports an average relative Hit@1 improvement of 51% over its selected baselines on these hybrid questions.
• Limitations. STaRK is a curated semi-structured benchmark; the agentic retriever adds latency and implementation complexity, and the result does not compare ordinary-file editing or canonical data models.
• Relevance. The result shows that relational and multi-hop tasks can justify graph-aware retrieval. It does not imply that the graph database must be canonical: a graph can be derived from files and companion metadata if provenance and rebuild are reliable.

8.10 Sparse, dense, and hybrid retrieval are workload choices

A 2026 financial-domain study of 23,088 queries and 7,318 text-and-table documents reports that BM25 remained a strong or superior baseline for precise financial retrieval, while hybrid retrieval plus reranking produced the best aggregate Recall@5 and MRR@3 [S26]. This is a preprint—not peer-reviewed and not independently replicated. It is included because the scale and table-heavy domain are methodologically relevant, not as settled evidence. Its result reinforces a broader conclusion already visible in peer-reviewed RAG studies: dense retrieval is not universally superior, and exact identifiers, numbers, and domain terms often favor sparse methods.

For a file-native system, an evaluation should always include an exact/lexical baseline before adopting embeddings. The right question is not “Does semantic search feel intelligent?” but “What unique errors does it remove, what new errors does it introduce, and can the index be rebuilt and audited?”

8.11 A decisive file-native experiment

The evidence gap is direct evaluation on real file-native domains. A useful study would freeze representative corpora and task sets, then compare:

1. explicit file selection plus exact search;
2. a local lexical index;
3. dense retrieval;
4. hybrid lexical+dense retrieval with reranking;
5. metadata/graph-aware retrieval for relationship tasks;
6. long-context composition without retrieval where the corpus fits.

Measure at least source recall, grounded-answer correctness, unsupported-claim rate, exact edit success, provenance fidelity, p50/p95 latency, time to first useful response, peak memory, system energy, index size, update/rebuild cost, corruption recovery, portability to a second model/runtime, and human ability to understand a miss. Stratify by corpus size, file type, query specificity, paraphrase distance, relationship depth, and model context. This would test the contingent design choice; adjacent literature cannot substitute for it.

9. Four-layer examination of the AIOS theses relevant to this topic

This section preserves the ambition of the AIOS architectural thesis while separating four analytical layers:

1. what current evidence establishes;
2. the AIOS architectural thesis;
3. first- and second-order implications if the thesis is substantially correct;
4. conditions, uncertainties, counterforces, and tests.

Absence of direct validation limits the confidence assigned to an implication; it does not make the implication analytically unimportant. AIOS is examined here as a complement to frontier models and centralized frontier training. The thesis is that durable context, expertise, memory, workflow, and authority can increasingly reside in person- or organization-controlled local systems, which may consult local or remote frontier models selectively.

9.1 Complementary local intelligence rather than model replacement

Layer 1 — Current evidence. Quantized local models already perform bounded document, language, retrieval, and multimodal tasks on phones, laptops, and edge computers [S01–S08]. Hybrid routers can select between model tiers, although the reviewed routing research optimizes cost and benchmark quality rather than sovereignty [S09, S10, S30]. Long-context and retrieval evidence shows that model calls can operate over external, updateable knowledge rather than requiring all durable knowledge to reside in model parameters [S21–S25].

Layer 2 — AIOS architectural thesis. A model is a reasoning faculty within a larger local intelligence environment, not the location of the person's durable knowledge or authority. Canonical files, annotations, relationships, policies, memory, task history, and exact operations remain locally controlled. Small local models handle suitable foreground work; stronger local or remote models are selected when their marginal capability justifies the latency, energy, disclosure, and dependency cost. Centralized frontier training remains valuable because better base models expand what every compatible local system can do.

Layer 3 — If the thesis is substantially correct.

• First order: people and organizations can upgrade or replace models without surrendering accumulated context; routine tasks can continue offline; remote calls can be smaller, less frequent, and more deliberate; model competition occurs against a stable person-owned knowledge layer.
• Second order: durable intelligence becomes less synonymous with an account held by one application provider. Frontier models become selectively invoked faculties, while continuity, identity, memory, and authority persist across model generations. The value of a local system can compound through better context and workflow even when the underlying model is unchanged.
• Systemic implication: capability may be distributed across many bounded domain systems rather than concentrated in a single universal agent. The resulting architecture could be more plural, replaceable, and resilient than a design in which one provider holds both the model and the only durable copy of the user's cognitive context.

Layer 4 — Conditions and tests. This implication requires portable file and metadata formats, model-independent task interfaces, reliable local runtimes, policy-gated routing, affordable hardware, secure updates, and evidence that retained context continues to help after model substitution. It also requires measuring whether local context truly reduces remote tokens and dependence rather than merely duplicating cloud state. A longitudinal study should replace the model and runtime while holding the local domain constant, then measure task continuity, migration loss, remote-call frequency, and human trust.

9.2 Bounded model judgment, exact operations, and human authority

Layer 1 — Current evidence. ContextualJudgeBench shows that contextual semantic evaluation remains difficult even for strong judge models; its best reported model reached about 55% consistent accuracy over 2,000 challenging response pairs [S34]. τ-bench shows that policy instructions and API tools do not make transactional agents reliable: the strongest tested agent succeeded on fewer than half the tasks and had retail pass^8 below 25% [S33]. AgentDojo, Agent Security Bench, and NIST's adversarial taxonomy show why model text cannot safely serve as its own authorization boundary [S15–S17].

Major evidence profile: ContextualJudgeBench

Status: Peer-reviewed, ACL 2025 [S34].

• Research question. How reliably can specialized judge models and general-purpose models compare responses when evaluation depends on supplied context and conditional criteria such as faithfulness, completeness, refusal, or concision?
• Method and sample. The benchmark contains 2,000 difficult response pairs across eight contextual splits, constructed from human annotations and model-based perturbations. The paper evaluates 11 judge models and nine general-purpose models.
• Principal finding. Context and conditional evaluation criteria remained difficult; the best reported model, o1, reached only about 55% consistent accuracy.
• Limitations. Pairwise benchmark judgments are not equivalent to production decisions. Construction includes synthetic perturbations, model generations have advanced, and the selected criteria do not span all human values.
• Relevance. Models can contribute semantic judgment, but bounded contextual evaluation still benefits from calibration, abstention, deterministic checks where available, and human review for consequential ambiguity.

Major evidence profile: τ-bench

Status: Peer-reviewed, ICLR 2025 [S33].

• Research question. Can tool-using agents interact with users, follow domain policy, and leave a transactional system in the correct state consistently?
• Method and sample. τ-bench simulates user conversations in retail and airline domains. Agents receive domain-specific API tools and policy guidelines; evaluation compares final database state with an annotated goal state and uses pass^k to measure repeatability.
• Principal finding. The paper's strongest tested function-calling agent succeeded on fewer than half the tasks, and retail pass^8 was below 25%, showing severe run-to-run inconsistency.
• Limitations. There are only two simulated domains, users are model-simulated, and the model snapshot is from 2024. Newer systems may perform better, and end-state equality can miss qualitative interaction differences.
• Relevance. The benchmark supports deterministic state validation and bounded tools while illustrating why exact effects and model intent must be evaluated separately.

Layer 2 — AIOS architectural thesis. Models make bounded semantic judgments—classifying, comparing, planning, drafting, interpreting, or proposing an edit. Typed interfaces carry those proposals. Deterministic code establishes identity, resolves exact paths, enforces scope and authorization, validates schemas and preconditions, and applies exact effects. Humans set purpose and retain authority over consequences that cannot be safely delegated. No central agent has to embody all three forms of responsibility.

Layer 3 — If the thesis is substantially correct.

• First order: systems can use the semantic flexibility of language models without converting model prose directly into unchecked effects. Invalid or unauthorized actions become rejectable at the operation boundary; source context and human decisions can remain linked to every consequential change.
• Second order: application logic can shift from large collections of brittle intent-specific workflows toward a smaller set of exact, reusable operations governed by local policy. New reasoning models can enter the system without receiving greater authority merely because they are more capable.
• Human implication: preserving purpose and consequential authority may counter a common failure mode in which fluent automation gradually becomes the de facto decision-maker. The human role changes from manually performing every step to defining purpose, adjudicating ambiguity, and governing durable change.

Layer 4 — Conditions and tests. Deterministic code guarantees only specified invariants. It cannot prove that a semantic judgment is fair, complete, strategically wise, or faithful to an ambiguous purpose. Some apparently semantic work—dates, identities, arithmetic, policy lookup—should move into exact checks; some authorization decisions require semantic and human interpretation. Human review must avoid becoming a rubber stamp. Test free-form parsing, typed proposals, deterministic-only rules, validated proposals, and human-approved execution on identical tasks, measuring semantic correctness, invalid-action rejection, authorization error, edit fidelity, abstention, and review burden.

9.3 Scaffolding, distinct cognitive modes, and bounded menus

Layer 1 — Current evidence. Scaffolding can both expose and obscure model capability. AgentIF's 707 instructions from 50 agentic applications average 1,723 words and 11.9 constraints; models struggled especially with complex constraints and tool specifications [S37]. Tool-description edits produced more than tenfold changes in use for some competing tools, demonstrating interface-induced selection bias [S40]. ToolScope reports tool-selection improvements ranging from 8.38% to 38.6% by merging redundant tools and exposing a context-relevant subset [S38]. MultiAgentBench finds coordination topology affects performance and reports a 3% milestone improvement from cognitive planning in its setting [S39]. Debate or Vote, however, finds that majority voting explains most gains attributed to multi-agent debate across seven benchmarks [S36].

Major evidence profile: ToolScope

Status: Peer-reviewed, ACL 2026 [S38].

• Research question. Can tool merging and context-aware filtering reduce ambiguity and improve tool selection when an agent faces large or redundant catalogs?
• Method and sample. ToolScope evaluates a merge–retrieve–rerank pipeline on three tool benchmarks using GPT-4o, Llama 3.3 70B, and Command R, with top-k candidate menus varying by benchmark.
• Principal finding. The reported correct-selection gains are large on Seal-Tools and UltraTool and smaller but positive on BFCL; reranking helps most when only a few tools are shown.
• Limitations. Correct tool selection is not end-to-end task success. The system relies on embeddings, a cross-encoder, model-assisted merging, curated ground truth, and preprocessing. It does not measure prompt injection, permission safety, or local-device energy.
• Relevance. Bounded menus can improve action selection, while their own retrieval layer must remain versioned, testable, and rebuildable.

Major evidence profile: Debate or Vote

Status: Peer-reviewed, NeurIPS 2025 [S36].

• Research question. Are gains attributed to multi-agent debate caused by inter-agent deliberation or by sampling multiple answers and voting?
• Method and sample. The authors disentangled majority voting from debate across seven NLP benchmarks and modeled debate as a stochastic belief-update process.
• Principal finding. Majority voting alone explained most gains usually attributed to debate. Under the paper's assumptions, debate did not improve expected correctness without a targeted bias toward correction; simple ensembles were often stronger and more reliable.
• Limitations. Static NLP benchmarks do not represent long-running file edits, heterogeneous specialist models, private evidence, or human-mediated workflows. Targeted debate designs can behave differently.
• Relevance. Role benefits must be separated from sampling diversity, extra test-time compute, and aggregation.

Layer 2 — AIOS architectural thesis. Thinking, writing, editing, structuring, planning, and review are treated as different modes with different context, tools, permissions, and success criteria. Bounded menus expose only the actions relevant to the current mode. This seeks a middle path between rigid deterministic pipelines and an agent with unrestricted context and authority. The “role” may be a separate process, a phase-specific context, a local model specialization, or a human handoff; it need not imply a society of autonomous agents.

Layer 3 — If the thesis is substantially correct.

• First order: smaller contexts and relevant tools reduce distraction and token cost; phase-specific validators become easier to design; a local model can perform a narrow faculty well even when it is not a strong general agent.
• Second order: workflow expertise becomes an inspectable local asset rather than an opaque property of a single provider's agent. Different faculties can be upgraded independently, assigned different privacy routes, and evaluated against task-specific criteria.
• Capability implication: some apparent model deficits may be harness deficits. As scaffolding matures, the amount of ordinary work that fits within local-model capability may grow faster than model benchmarks alone suggest because the environment supplies structure, memory, exact operations, and error containment.

Layer 4 — Conditions and counterforces. Phase separation can create handoff loss, duplicated inference, correlated errors, longer total latency, and false confidence in “independent” review by the same model. A bounded menu can hide a necessary action. The exact AIOS mode taxonomy has not been directly tested. Equal-budget experiments must compare phase-specific roles with a monolithic context, independent samples, majority voting, self-review, and heterogeneous specialists while holding model calls, tokens, tools, and latency constant.

9.4 Local domain systems, personal intelligence, and private cognition

Layer 1 — Current evidence. Current devices support useful bounded local work; specialized small models can be responsive on phones, and end-to-end local retrieval and generation is technically plausible on a laptop NPU [S01–S08]. Long-context versus RAG evidence shows that bounded corpora can sometimes be used directly, while retrieval lowers token and compute cost as corpus scale grows [S21–S24]. File-native architecture can keep source knowledge inspectable while using derived indexes only where the workload justifies them.

Layer 2 — AIOS architectural thesis. A self-contained domain contains the durable files, metadata, relationships, annotations, memory, policies, and workflows needed for a recurring area of life or work. Increasingly capable local models, supported by mature context composition and exact operations, may satisfy a large share of ordinary personal and organizational needs inside these domains. Frontier models remain available for exceptional difficulty, external knowledge, or tasks whose quality benefit outweighs disclosure and dependency cost.

Layer 3 — If the thesis is substantially correct.

• Personal intelligence: a person gains a persistent local reasoning environment that remembers across applications, remains available offline, preserves private drafts and deliberation, and survives model-provider changes. “Private local cognition” would mean not only local inference, but also locally held working memory, annotations, unresolved questions, and the history of how judgments evolved.
• Organizational intelligence: teams can maintain bounded systems for legal matters, research programs, operations, products, archives, or professional practice. Expertise can remain attached to source files and decisions rather than dispersed across application silos or lost when staff and vendors change.
• Second-order institutional effect: the unit of intelligence shifts from an application account or central agent toward many governed domain systems that can cooperate. The organization can decide which domains remain entirely local, which share selected artifacts with peers, and which escalate to remote frontier models.
• Market implication: model providers compete to serve person-owned and organization-owned knowledge systems rather than using custody of context as the principal switching barrier.

Layer 4 — Conditions and counterforces. The “large share” is a population-level empirical question, not a fixed claim. Knowledge may not be closed: live law, prices, schedules, scientific evidence, counterparties, or organizational state may require external access. Action may not be closed: real effects often involve institutions and shared systems. High-stakes evaluation may require independent expertise. Organizational work needs concurrency, identity, synchronization, conflict resolution, retention, and audit. A coverage study should sample real personal and organizational tasks over time, compare local, indexed-local, policy-gated hybrid, and cloud-first conditions, and classify failures by missing knowledge, model capacity, collaboration, external action, latency, or policy.

9.5 Reduced dependence on centralized application infrastructure

Layer 1 — Current evidence. Local canonical files can remove a central content database from some bounded workflows; local models and retrieval can remove routine inference calls; portable runtimes and model artifacts can reduce dependence on one provider [S08, S19–S22]. The reviewed literature does not directly measure whole-application infrastructure displacement, so the evidence establishes technical ingredients rather than aggregate effect.

Layer 2 — AIOS architectural thesis. When canonical files, accepted ground, memory, workflow state, and authority reside locally, centralized services need not own the durable intelligence layer. They may provide optional synchronization, distribution, collaboration, external data, confidential computation, or frontier reasoning. AIOS therefore predicts reduced dependence on centralized application-layer infrastructure and remote inference, not the disappearance of centralized training or all network services.

Layer 3 — If the thesis is substantially correct.

• First order: fewer raw documents and prompts need to reside in provider systems; routine retrieval and generation can occur without a network round trip; outage and provider-exit risk decline; remote inference can receive smaller, policy-selected context bundles.
• Second order: some applications may become thin interfaces and protocols around person-owned domain state rather than custodians of that state. Central services can specialize in coordination, discovery, updates, or scarce computation instead of holding the only canonical copy of knowledge and memory.
• Economic and governance effect: switching costs may fall, provider bargaining power may weaken, and organizations can negotiate remote computation separately from knowledge custody. Infrastructure becomes more federated: substantial device-side computation plus narrower shared services.
• Resilience effect: local continuity can reduce dependence on constant connectivity and single-provider availability, while selective cloud access preserves the benefit of frontier capability.

Layer 4 — Conditions and counterforces. Synchronization, identity, backup, recovery, collaboration, model distribution, security updates, and organizational policy remain substantial. Heterogeneous device fleets increase testing and support. Shared services may be cheaper or more energy efficient for some loads; duplicated local computation can be wasteful. Network effects and regulated recordkeeping can favor central systems. The correct empirical question is which named services, costs, data flows, and trust dependencies are removed, retained, or relocated. Equivalent reference deployments should measure server compute, storage, transfer, identity, synchronization, backup, update distribution, support labor, device energy, recovery, and remote-inference frequency.

9.6 Regulated use, peer collaboration, and global accessibility

Layer 1 — Current evidence. Local processing can minimize routine transfer; hardware-backed keys and device encryption can protect local domains; confidential computation can provide a bounded remote path; EDPB guidance reinforces case-specific anonymity, upstream lawfulness, and data minimization; portable runtimes and licensing definitions clarify conditions for model substitution [S11–S20, S27–S32]. Current local models are multilingual and multimodal to varying degrees, but vendor coverage claims are not equivalent to uniform quality across languages, devices, or accessibility needs [S04, S05, S07].

Layer 2 — AIOS architectural thesis. Person- and organization-controlled domains can attach purpose, provenance, retention, authorization, and disclosure policy to the knowledge itself. Collaboration exchanges selected files, diffs, annotations, relationship metadata, or derived artifacts rather than requiring every participant to place their full domain inside one provider. Regulated and sensitive domains may perform most processing locally and use attested or cryptographic remote computation only where authorized. Ordinary reasoning can remain available in low-connectivity settings.

Layer 3 — If the thesis is substantially correct.

• Regulated use: local-first domains could make data minimization and purpose limitation architectural defaults rather than optional provider settings. Source-linked decisions, exact operations, and local audit history may make human review and record production more tractable. Selective frontier access could occur through minimized context and an explicitly governed route.
• Peer collaboration: people and organizations could share bounded knowledge packages while retaining separate canonical domains and authority. Collaboration becomes a negotiated merge of evidence, annotations, decisions, and permissions rather than automatic central custody. Federations of domain systems could build collective intelligence without requiring one central agent to possess all private context.
• Global accessibility: capable small models and offline operation could bring reasoning tools to places with intermittent connectivity, high network cost, or limited access to remote services. Locally maintained language, cultural, agricultural, educational, or professional domains could preserve knowledge that is poorly represented in general models.
• Second-order global effect: a plural ecosystem of interoperable local domains may allow communities to participate in AI without contributing all durable context to a small number of global providers. Sovereignty becomes practical at personal, institutional, and community scales while frontier research remains globally shared.

Layer 4 — Conditions and counterforces. Locality does not itself confer legal compliance; policy must match jurisdiction, purpose, retention, and human-accountability requirements. Peer collaboration needs identity, signatures, provenance, conflict resolution, revocation, and protection against poisoned contributions. Global accessibility depends on affordable capable hardware, energy, repair, secure model distribution, inclusive interfaces, high-quality local-language models, and training rights. A local system that is difficult to update or only effective in high-resource languages could widen rather than reduce inequality. Tests should include regulated-domain pilots with independent governance review, adversarial peer-sync trials, low-bandwidth field studies, accessibility evaluation, and per-language task quality rather than aggregate multilingual claims.

9.7 Context is a force multiplier, not a complete failure theory

Layer 1 — Current evidence. Better-selected context and tool menus can improve performance [S21–S24, S38], but agent benchmarks identify independent failures in semantic judgment, policy adherence, security, state management, GUI grounding, operational knowledge, and tool use [S15, S16, S33–S35, S37].

Major evidence profile: OSWorld

Status: Peer-reviewed dataset and benchmark, NeurIPS 2024 [S35].

• Research question. Can multimodal agents complete open-ended tasks in real computer environments across applications and operating systems?
• Method and sample. OSWorld provides 369 tasks with reproducible initial states and execution-based evaluation, spanning web and desktop applications, local file operations, and multi-application workflows.
• Principal finding. Humans completed more than 72.36% of tasks, while the best tested model achieved 12.24%. The paper attributes major failures to GUI grounding and operational knowledge.
• Limitations. The baseline agents and models are from 2024, interface operation differs from direct exact file APIs, and benchmark task coverage is finite. Current computer-use agents may be materially better.
• Relevance. Exact operations and better context may remove some interface errors, but planning, state, policy, and semantic correctness remain separate capability dimensions.

Layer 2 — AIOS architectural thesis. Much model capability is wasted when the environment supplies undifferentiated context, ambiguous tools, unstable memory, or no exact connection between judgment and effect. A composed local environment can make relevant context, identity, relationships, and permitted operations available at the right resolution and time.

Layer 3 — If the thesis is substantially correct. Better context composition and exact operations could yield capability gains that raw model benchmarks understate, especially for local models operating repeatedly in a known domain. The combination of persistent domain knowledge, role-specific context, and deterministic execution may make a smaller model economically equivalent to a larger remote model for many recurring tasks. Second-order, progress in scaffolding could expand practical local capability even between major model releases.

Layer 4 — Conditions and counterforces. Context cannot repair missing world knowledge, inadequate reasoning, malicious instructions, perception errors, or an incomplete action interface. More context can increase latency, energy, prompt injection exposure, and instruction interference. Causal experiments must vary context relevance, length, provenance, perception fidelity, tool reliability, and state observability independently.

9.8 Useful intelligence as a cumulative system outcome

Layer 1 — Current evidence. The reviewed literature supports component-level effects rather than an integrated AIOS effect. Context and retrieval strategy change answer quality and compute [S21–S26]; tool descriptions and menu filtering change model behavior [S38, S40]; deterministic end-state checks expose failures that fluent interaction can conceal [S33]; routing can trade model cost against measured quality [S09, S10, S30]; and persistent context introduces both continuity and poisoning risk [S15, S16]. These results establish that the environment around a model matters. They do not establish that many small improvements will add linearly, produce a particular aggregate gain, or always outperform using a stronger model.

Layer 2 — AIOS architectural thesis. Useful intelligence arises from the coordinated system: purpose framing, composed context, distinct cognitive modes, multiple memory resolutions, relational metadata, provenance, bounded semantic judgment, planning, verification, exact effects, and post-response integration. Each element is intended to reduce a different source of wasted inference or error. Routing can occur at each cognitive movement rather than once per conversation, selecting among small local, larger local, ordinary remote, or confidential-remote models according to task capability, privacy policy, sensitivity, latency, energy, cost, and permitted disclosure. Frontier models remain part of the system where their marginal value is material.

Layer 3 — If the thesis is substantially correct. Several modest improvements could compound: better source selection reduces irrelevant tokens; durable memory avoids rediscovery; provenance enables targeted verification; exact operations reduce recovery from malformed effects; mode-specific tools reduce selection ambiguity; and post-response integration prevents useful work from vanishing into chat history. The first-order result could be fewer retries, shorter prompts, lower remote-token use, and lower unsupported-claim rates for recurring domain work. The second-order result could be a shift in capability comparisons: the relevant unit would be the whole person-owned system over a longitudinal workload, not one model answering an isolated prompt. A well-composed local system might therefore outperform a stronger but context-poor remote call on familiar work while still escalating genuinely difficult movements to the frontier.

Layer 4 — Conditions, interactions, and tests. Benefits may overlap, cancel, or introduce overhead. More metadata can become stale; more modes can multiply calls; planning can add latency without improving execution; verification can repeat correlated model errors; memory can propagate poison; routing itself can misclassify difficulty; and an apparently lower inference burden can be offset by indexing, synchronization, or device energy. “Hallucination risk” must be measured as source-supported factuality, edit correctness, calibrated abstention, and downstream error—not inferred from architectural neatness. A limited factorial and sequential-ablation study should measure task success, unsupported claims, retries, total local and remote tokens, calls by route, latency, energy, disclosure volume, integration failures, and human correction time. Because a separate benchmark and routing investigation owns the full question, this memo treats those measurements as required validation rather than specifying a universal routing algorithm.

9.9 Research program for the larger implications

1. Model–code–human boundary study. Compare free-form model output parsed by code, typed proposals, deterministic-only rules, validated proposals, and human-approved execution. Measure semantic correctness, invalid-action rejection, authorization error, edit fidelity, abstention, and review burden.
2. Equal-budget cumulative-system factorial. Hold model, sampling budget, tools, and task set constant while varying purpose framing, monolithic versus mode-specific contexts, full versus retrieved menus, memory layers, provenance, planner/reviewer roles, deterministic validation, and post-response integration. Report both individual and interaction effects, including retries, unsupported claims, human correction time, local and remote tokens, latency, energy, and disclosure volume.
3. Role and faculty portability test. Compare named roles with independent samples, majority voting, self-review, and heterogeneous specialist models. Then replace the underlying model while retaining local workflow and domain state to measure continuity.
4. Local-domain coverage study. Sample real personal and organizational tasks for several weeks. Compare offline local, indexed local, policy-gated hybrid, and cloud-first conditions; record task coverage, failure cause, remote context sent, energy, latency, and user preference.
5. Infrastructure displacement accounting. Build equivalent reference deployments and measure every retained or relocated service: server compute, storage, network, identity, sync, backup, update delivery, support, device energy, recovery, and remote inference.
6. Regulated-domain pilot. In a legally and ethically approved setting, test data minimization, local auditability, human authority, retention, model substitution, and a governed confidential-cloud fallback. Independent domain and governance experts should assess whether local architecture changes—not merely documents—improve control.
7. Peer-domain collaboration trial. Exchange signed files, annotations, relationship metadata, and proposed edits between separately governed domains. Measure provenance preservation, merge conflicts, revocation, poisoning resistance, and whether collaboration requires central custody.
8. Global-access field study. Evaluate low-bandwidth and intermittent-connectivity use on affordable hardware, stratified by language, disability access, energy availability, and local support capacity. Compare local benefit with the cost of model distribution and updates.
9. Longitudinal integrity test. Run months of edits and memory updates with benign and poisoned documents. Measure provenance loss, stale summaries, semantic drift, rollback, and whether exact controls contain unsafe effects.
10. Frontier-complementarity study. Route the same workload among small local, larger local, ordinary cloud, and confidential-cloud models at task and cognitive-movement granularity. Test whether a stable local knowledge layer can reduce remote disclosure, retries, and switching cost while preserving quality on difficult tasks; analyze full routing behavior in the dedicated benchmark/routing study.

This program can establish the range in which the large implications hold, identify where centralized or frontier capability remains essential, and reveal whether local person-owned intelligence systems change the economics and governance of AI as the thesis predicts.

10. Architectural interpretation for AIOS

This section is explicitly interpretive. It identifies possible connections; it does not report validated AIOS properties.

10.1 A plausible local-first execution order

1. Purpose and policy define the domain, task, authority, and disclosure boundary.
2. Exact local operations identify candidate files, metadata, annotations, and relationship edges.
3. Derived retrieval, if enabled, proposes additional candidates but does not override access rules or provenance.
4. Context composition records which source slices and transformations were included.
5. A bounded model judgment produces a draft, decision aid, or tool proposal.
6. Deterministic post-processing validates paths, schemas, citations, and requested edit scope.
7. Human authority controls consequential changes, cloud escalation, and disputed interpretations.
8. Memory integration preserves provenance, distinguishes source from inference, and keeps derived state reversible.

This order fits the evidence better than a design in which an opaque model decides both what may be disclosed and what may be changed.

10.2 Policy-gated routing

A suitable AIOS router would be closer to a constrained policy engine than a model leaderboard. It could make a new decision for research, planning, drafting, editing, review, or verification rather than binding the entire interaction to one model. Eligibility should be determined first by domain policy, file sensitivity and provenance, person authorization, jurisdiction, permitted disclosure, network state, and whether the movement can affect canonical files or accepted ground. Among eligible routes, selection could then consider measured task capability, expected risk, latency, device energy and thermal state, monetary cost, and confidence or escalation signals. The route and the exact disclosed context should be inspectable. Remote output should return as evidence or a proposal, not silently become canonical state.

This design would permit provider- and model-independent composition: a private local model for sensitive source selection, a specialist local model for a narrow edit, and a frontier or confidential-remote model for a difficult synthesis or independent review. It also creates new failure modes—misrouting, gaming of confidence, route thrashing, inconsistent judgments, hidden state transfer, and coordination overhead. No reviewed routing paper tests this complete architecture. This is a possible connection point, not a validated design, and the full benchmark question belongs to the dedicated routing investigation.

10.3 Multiple-resolution memory

Short summaries, working notes, durable domain files, annotations, and relationship metadata can reduce repeated context and support fast foreground responses. Security evidence also shows that every persistent layer can carry poisoning forward [S15, S16]. Each memory item should therefore record its source, creation method, timestamp, confidence or review state, and invalidation relationship. Machine-generated summaries should not silently replace their source material.

10.4 Fractal grammar as interface discipline

The recurring Why–How–What grammar may help structure tasks, outputs, metadata, and review because it makes intent, method, and artifact explicit at multiple scales. None of the reviewed literature evaluates that grammar. Its value remains a design hypothesis to test for comprehension, edit quality, retrieval precision, and human oversight.

10.5 Short foreground responses and non-blocking work

Separating an immediate bounded response from deeper authorized background work may lower perceived latency and limit the first-pass context. It can also create stale integration, unnoticed cloud disclosure, or action races. Background work needs a visible job record: purpose, selected sources, route, remote disclosures, model/version, permissions, state changes, cancellation, and final integration decision.

11. Foundational lineage (pre–August 2024)

The following older works are separated because the charter permits them only when indispensable to the lineage.

Local-first software principles (2019)

Kleppmann and colleagues' Ink & Switch essay is a perspective, not a peer-reviewed AI study [F01]. It articulated local copies, offline capability, user ownership, longevity, privacy, and collaboration as design priorities. It supplies the software-architecture lineage for “local-first,” but it does not establish the safety or capability of generative models.

Retrieval-augmented generation (2020)

Lewis et al.'s RAG paper is a peer-reviewed NeurIPS 2020 paper [F02]. It combined parametric generation with retrieved passages from a non-parametric corpus and showed gains on knowledge-intensive tasks. It established retrieval as a way to update and ground model outputs; it did not establish that dense retrieval is necessary for every bounded corpus or that a vector store should be canonical.

BEIR retrieval benchmark (2021)

BEIR is a peer-reviewed NeurIPS 2021 benchmark covering heterogeneous retrieval tasks [F03]. It showed that lexical BM25 remained a robust zero-shot baseline and that dense retrievers could suffer under domain shift, although later methods improved. Its enduring relevance is methodological: retrieval approaches must be evaluated out of domain and against strong sparse baselines.

12. Closing assessment

Strongest supported conclusions

1. Bounded on-device generative AI is operationally real. Quantized small models can provide useful latency and quality for constrained language tasks on current phones, laptops, and edge accelerators. The reliable envelope is smaller than “any model that loads,” and task specialization often matters more than parameter count [S01–S08].
2. Quantization, sparsity, and accelerator integration are central, but not free. Four-bit weights are a common practical compromise; prefill, decode, KV cache, storage traffic, and thermal state respond differently. Current prompt-level sparsity and flash-resident designs are promising but still lack broad independent replication [S01, S02, S05, S06].
3. Privacy is end-to-end. Local inference minimizes routine content transfer, but endpoint compromise, malicious files, memory poisoning, unsafe actions, logs, synchronization, and outputs remain important [S15–S18].
4. Confidential cloud can narrow provider trust. Hardware isolation, remote attestation, stateless processing, egress restrictions, and release transparency create meaningful protection under specified assumptions. They do not eliminate implementation, hardware, client, correctness, or governance risk [S11–S14, S27].
5. Cryptographic private inference is advancing but remains workload-bounded. Current FHE and MPC systems report large improvements over earlier private-inference baselines, yet their transformer scales and end-to-end latencies do not establish parity with plaintext serving [S31, S32].
6. Hybrid routing is viable but privacy-aware routing is not solved by current cost–quality routers. Policy and disclosure eligibility should precede learned capability selection [S09, S10, S30].
7. No universal retrieval architecture wins. Long context can outperform RAG when the relevant corpus fits and resources permit; RAG can be much cheaper; lexical, dense, hybrid, and graph retrieval each win under different query and corpus conditions [S21–S26].
8. Files can remain canonical while indexes evolve. The literature is compatible with—though does not validate—a design in which exact files are authoritative and lexical, vector, or graph structures are derived, versioned, and rebuildable.
9. Sovereignty requires portability and authority, not only locality. Canonical data formats, model/runtime substitutes, explicit licenses, rebuildable derived state, offline recovery, provenance, and human control are separate requirements [S19, S20].
10. Bounded semantic judgment plus deterministic enforcement is a supportable pattern, not a correctness proof. Models can propose contextual interpretations; exact code can enforce typed interfaces, scope, authorization, and state invariants; humans remain necessary for purpose and consequential ambiguity. Contextual-judge and transactional-agent results show that none of those layers can safely stand in for the others [S33, S34].
11. Harness design matters, but no single scaffolding theory explains agent performance. Long constraint sets, tool descriptions, candidate menus, perception, state, and coordination topology all change outcomes. Selective tool exposure can help, while debate benefits can be confounded by voting and extra computation [S35–S40].
12. Useful intelligence must be evaluated at system as well as model level. Current studies show that context, retrieval, interfaces, memory, validation, and routing can each change observed quality, cost, or risk. They support treating model weights as one causal component; they do not yet establish the aggregate AIOS effect or how its components interact [S09, S10, S15, S16, S21–S26, S33, S38, S40].

Unresolved or contradictory evidence

• Independent, cross-platform benchmarks for 2025–2026 system models and NPUs lag vendor releases. Vendor quality and energy numbers are difficult to compare because tasks, runtimes, thermal conditions, and measurement boundaries differ.
• Tablet-specific measurements are particularly sparse; most tablet capability claims are inferred from shared silicon or vendor support rather than tested as a distinct sustained workload.
• Small models can match older larger models on selected benchmarks yet remain weak in-context learners and unreliable on open-ended agent tasks. The boundary between useful specialist and inadequate generalist is workload-specific.
• Long-context studies find quality advantages over some RAG configurations, while effective-context benchmarks show severe degradation inside advertised windows. Better context handling may shift but not remove the retrieval boundary.
• Confidential-computing architectures offer stronger verifiability than conventional cloud promises, yet hardware attacks demonstrate that attestation and isolation are patch- and implementation-dependent.
• FHE and MPC papers report rapid relative improvement, but there is little evidence of interactive, end-to-end encrypted inference for current multi-billion-parameter generative models under realistic network and energy constraints.
• Token-level local/cloud collaboration can improve cost–quality tradeoffs, but its privacy cost, network variance, and state-transfer complexity are under-evaluated.
• NPU energy results are promising but sparse, often single-device, and not yet normalized across full application lifecycles.
• There is little evidence on long-running personal AI systems that jointly measures retrieval quality, exact editing, memory poisoning, provenance, energy, portability, and human trust.
• The necessity of separate thinking, writing, editing, planning, and review contexts has not been tested under equal model, token, tool, and latency budgets. Existing multi-agent results point in conflicting directions [S36, S39].
• Current contextual judges remain inconsistent, but it is unresolved how much typed task narrowing, domain fine-tuning, abstention, or human calibration can improve them [S34].
• The security–utility frontier for dynamically bounded tool menus is under-measured: filtering can reduce ambiguity and blast radius while also hiding a necessary action [S38].
• No longitudinal study found in this review quantifies which centralized services a mature local-first reasoning environment actually removes, retains, or relocates.
• No reviewed experiment measures the cumulative effect of purpose framing, composed context, multiple memory layers, relational metadata, provenance, distinct modes, planning, verification, exact effects, and integration under a common budget. Component benefits may compound, overlap, or be offset by coordination costs.
• The share of real personal and organizational workloads that a mature bounded local system can satisfy—and how that share varies by hardware, language, stakes, domain closure, and access to frontier escalation—remains unknown.
• It is unresolved whether movement-level dynamic routing produces a stable quality–privacy–latency–energy advantage after including routing error, context-transfer cost, retries, and operational complexity.
• There is little longitudinal evidence on federated peer-domain collaboration, regulated local reasoning with governed frontier fallback, or accessibility in low-bandwidth and low-resource-language settings.

Possible AIOS connection points for later consideration

• Keep ordinary files and documented companion metadata as the durable, human-inspectable knowledge layer; treat indexes as rebuildable projections with model and transformation provenance.
• Begin retrieval with exact paths, metadata filters, and strong lexical search. Add dense or graph retrieval only after a real task set demonstrates material recall or latency gains.
• Use a two-stage local/cloud decision: a deterministic disclosure-policy gate followed by a measured capability/latency/energy router among eligible paths.
• Record a context manifest for important responses: purpose, source slices, derived memory, route, model/version, transformations, and tool permissions.
• Separate broad read/search authority from narrow write/edit/action authority; require deterministic validation and reviewable diffs for canonical changes.
• Mark untrusted document content as data rather than instructions, and propagate trust/provenance labels through summaries, annotations, relationships, and memory.
• Preserve multiple-resolution memory without allowing machine summaries to erase sources; provide expiry, invalidation, rollback, and rebuild.
• Maintain model-independent task and safety evaluations so local and remote providers can be substituted on evidence rather than assumed equivalence.
• Evaluate the Why–How–What grammar and short-foreground/background pattern as hypotheses, with human-comprehension, task-success, provenance, and failure-recovery measures.
• Express model decisions as typed proposals rather than executable prose; validate identity, path, authorization, schema, preconditions, and exact effects outside the model.
• Issue short-lived, purpose-bound capabilities for writes and external actions, with a visible preview and an attributable human decision where consequences are material.
• Run equal-budget ablations before adopting permanent cognitive roles; compare roles against independent samples, majority voting, self-review, and simpler phase-specific prompts.
• Track failure causes separately—missing context, faulty judgment, tool selection, perception, state, authorization, external dependency, or poisoned memory—so context is not used as a catch-all explanation.
• Measure infrastructure displacement component by component, including synchronization, identity, backup, update delivery, remote escalation, device energy, and support burden.
• Evaluate AIOS as a longitudinal system, not only as a wrapper around one prompt: include purpose, context composition, memory reuse, provenance, exact execution, verification, and the quality of post-response integration.
• Measure whether those components jointly reduce total inference burden, retries, unsupported claims, human correction time, and remote disclosure, while accounting for indexing, coordination, synchronization, and device energy.
• Route at the cognitive-movement level only after applying disclosure and authorization policy; permit different eligible models for research, planning, drafting, editing, and review, with each route and disclosed context inspectable.
• Preserve the frontier-model complement explicitly: test whether routine local execution and stable local knowledge improve continuity while selective remote escalation retains quality on genuinely difficult work.
• Treat private local cognition as broader than local inference by protecting working notes, unresolved questions, annotations, relationship metadata, memory, and decision history across storage, backup, sync, and collaboration.
• Explore organization-level federation through separately governed domain systems, signed exchanges, reviewable merges, revocation, and provenance-preserving peer collaboration rather than assuming either total isolation or central custody.
• For regulated use, evaluate whether locally attached purpose, retention, authorization, provenance, and a governed confidential-compute fallback improve actual control under independent domain review.
• For global accessibility, test affordable hardware, intermittent connectivity, update distribution, energy constraints, disability access, and per-language quality; offline availability alone is not sufficient.

Claims that would be unsafe to make

• “On-device AI is private” without specifying storage, logging, synchronization, tools, outputs, endpoint security, and adversaries.
• “Secure enclaves make cloud inference zero trust” or eliminate the need to trust hardware, firmware, attested code, client software, or governance.
• “FHE or MPC already provides plaintext-equivalent performance for general current LLM assistants.”
• “A 3B model is sufficient for personal intelligence” or any other capability claim based on parameter count alone.
• “Local inference always uses less energy or has lower carbon emissions than cloud inference.”
• “Long context eliminates retrieval,” or conversely, “every knowledge system needs a vector database.”
• “Embeddings are objective metadata.” They are model- and preprocessing-dependent statistical representations.
• “Ordinary files scale indefinitely” or “a canonical database is never necessary.” Concurrency, invariants, retrieval load, and relationship depth can make a database operationally appropriate.
• “Open weights guarantee sovereignty, auditability, safety, or legal freedom.”
• “Current routing research validates private local/cloud orchestration.” Most reviewed routing work optimizes cost and benchmark quality, not disclosure or human authority.
• “Frontier models exercise universally superhuman judgment.” Current contextual-judgment and agent benchmarks directly contradict such a general claim [S33–S35].
• “Insufficient context is the sole cause of agent failure,” or that removing current scaffolding will reliably reveal latent general competence.
• “Thinking, writing, editing, planning, and review universally require separate model roles.” This remains an AIOS-specific design hypothesis, not an established cognitive architecture.
• “A bounded local domain has already been shown to satisfy a large share of personal or organizational reasoning across populations.” The consequential hypothesis remains worth testing, but it requires a defined task population, hardware and language strata, external dependencies, stakes, success thresholds, and a frontier-escalation policy.
• “A universal, precise reduction in application-layer infrastructure has already been established.” The thesis predicts material reduction under some conditions; the magnitude requires component-level reference deployments and measured baselines.
• “Centralized compute and frontier services become unnecessary.” The relevant thesis is selective complementarity: training, model distribution, collaboration, backup, and difficult or confidential computation remain plausible centralized workloads.
• “Many small architectural improvements necessarily compound” or “a scaffolded small local model will always outperform a stronger context-poor model.” Interaction effects, overhead, task fit, and the quality of the stronger model must be measured.
• “Dynamic model routing guarantees privacy, lower energy, lower cost, or higher quality.” Routing inherits policy, classifier, confidence, context-transfer, network, and model errors, and can optimize one objective at the expense of another.
• “Deterministic code guarantees that an outcome is correct.” It can guarantee specified identity, scope, structure, authorization, and state effects; it cannot prove that the governing semantic judgment or purpose is sound.
• “Adjacent research validates AIOS.” Direct comparative evaluation on AIOS tasks and corpora has not been performed.

Source table

Dates identify publication, release, or material update; all links are direct.