Building the Atlas Systems Control Plane

Overview

Atlas Systems began as a collection of useful systems rather than one planned platform. Cloudflare Workers exposed public APIs and observability data. GitHub Actions deployed services and checked repository policy. Local services on SPECULAR-CORE provided telemetry, search, AI inference, and voice control. Smaller tools handled dependency findings, quota checks, synthetic journeys, deployment outcomes, incident evidence, and notifications.

Each repository solved a real problem. The estate around them developed a different problem: important facts existed in more than one place, and those copies did not all carry the same authority.

Lifecycle appeared in manifests, repository metadata, README language, policy files, and memory. Public and private systems were sometimes described in the same planning material. A service could be reachable without having an approved reliability objective. A fixture could prove that a backup adapter worked without proving that a current backup existed. A merged workflow change could be mistaken for a live deployment. A detector could identify a repair without having any legitimate right to apply it.

I treated those as control-plane failures. The applications were not necessarily broken. The failure was that declaration, observation, evidence, proposal, publication, and live verification could be collapsed into one plausible-looking status.

The July 2026 Waves 0 to 3 programme rebuilt that boundary in stages. Wave 0 established who owns each kind of fact. Wave 1 defined operational evidence without inventing certainty. Wave 2 made repository presentation and GitHub state executable policy. Wave 3 connected those policies to deployable artefacts, evidence graphs, retirement records, model-promotion decisions, and bounded remediation.

Control-Plane Boundary

The control plane is not one service. It is a set of authority boundaries distributed across repositories.

I modelled the lifecycle of a fact as five separate states:

declared
  -> observed
  -> proposed
  -> published
  -> live verified

A declaration says what should exist. Observation says what a collector saw. A proposal says what should change. Publication says an approved projection or artefact was written. Live verification says the deployed or provider-side state was checked after the write.

The system is designed to preserve the gaps between those states. Missing observation does not invalidate a declaration, but it prevents an observed-health claim. A successful source merge does not imply publication. A successful publication does not imply the expected endpoint is live until that endpoint is checked. A valid remediation plan does not imply the plan is safe to merge.

Wave 0: Public and Private Authority

Wave 0 answered the question every later automation depended on: which source is allowed to say what is true?

The first correction was the public/private estate boundary. Atlas Systems contains intentionally public portfolio infrastructure and private owner-operated systems. Repository visibility and estate membership are different concerns. A private component can require dependency checks, CI, monitoring, backups, metadata validation, and aggregate engineering metrics without becoming public documentation.

The accepted model uses two governance planes with a one-way publication boundary.

Public runtime classification lives in atlas-infra/policy/estate-registry.json. Public non-runtime repositories use a separate assurance declaration. A generated public-repository-classifications.json projection combines those non-overlapping authorities for consumers. Private repositories own .atlas/governance.json inside their authenticated source repository and validate it through reusable source-local workflows. No central public policy file lists private repository identities.

Cloudflare discovery follows the same rule. Account-level observation may see more Workers than the public estate declares. atlas-api-index publishes only an explicit public allowlist, and atlas-api-public independently filters its output against the approved manifest. Unknown repositories and Workers are private by default.

The public manifest remains useful, but its responsibility is narrower. It maps topology, surfaces, feeds, and dependencies for things Atlas Systems intentionally publishes. It does not own lifecycle, scope, provenance, or private membership. That distinction matters because topology answers how approved systems connect, while classification answers what those systems are and how they should be governed.

Wave 0 also removed volatility from the public repository inventory. Activity timestamps had made the generated file and its fingerprint change even when membership did not. I replaced that with sorted canonical classification material and a SHA-256 fingerprint. A consumer can now distinguish a real classification change from ordinary repository activity.

The result was a smaller set of facts with named owners:

• Atlas Infra owns public classification;
• public manifests own public topology and presentation;
• private governance stays private and source-owned;
• generated projections are evidence, not editing surfaces;
• unknown or conflicting classification fails closed;
• public consumers never depend directly on raw account-wide discovery.

Phase I — Duplicate Classification Authority

Before the authority split, the same repository could carry lifecycle, scope, or provenance in more than one source. A downstream manifest could remain internally valid after the canonical registry changed. Two files could disagree without either one looking obviously malformed.

The drift was not theoretical. Public topology still carried stale lifecycle values for repositories whose authoritative classification had changed. A consumer reading the manifest alone would receive a plausible answer, just not the current one.

I moved runtime classification into the estate registry, added an explicit source for public non-runtime repositories, made the two authority sets non-overlapping, and generated one deterministic consumer projection. Schema validation rejects overlap, missing classifications, duplicate repositories, unsorted output, stale fingerprints, and attempts to treat the generated projection as an independent source.

The manifest then consumed classification rather than restating it. Private repositories retained equivalent source-local governance without entering the public projection.

The important part was not reducing the number of JSON files. It was making every remaining file answer one bounded question.

Wave 1: Operational Proof

Once authority was clear, Wave 1 addressed evidence. The goal was to distinguish desired state, observed state, fixture proof, and unknown state rather than flattening them into one green result.

Provider resource ownership

Atlas Infra gained an explicit declaration for approved public Cloudflare KV, D1, and R2 resources. Version one records nine public KV namespaces by provider identity, exactly one owner for each resource, and explicit consumers where sharing is intentional. TELEMETRY_KV, for example, is owned by specular-edge and consumed read-only by specular-sonify.

The declaration is deliberately not an account inventory. A live audit may observe private or undeclared resources, but the public result can expose only approved names and relationships. Unknown resources contribute redacted drift counts by kind. Their names, IDs, metadata, and inferred owners do not cross the publication membrane.

declared public resource
  -> identity and ownership may be checked publicly

observed but undeclared resource
  -> counted as drift
  -> identity withheld

Backup and recovery intent

Every runtime received either an explicit recovery target or an explicit no-backup decision. atlas-notify gained a KV recovery target with six-hour RPO intent, four-hour RTO intent, and 30-day retention intent. ramone-memory gained a Chroma recovery target with 24-hour RPO intent, eight-hour RTO intent, and 30-day retention intent.

Derived, stateless, cache-like, and bounded-relearnable systems received reviewed no-backup rationales instead of remaining unclassified. That distinction prevents a missing backup from becoming an accidental policy decision.

The first phase remained fixture-only. Synthetic data and temporary restore destinations proved that schemas, adapters, checksums, and restore logic were internally consistent. They did not prove that a current provider backup had been observed, that retention was active, or that production data had been restored.

Reliability without invented certainty

Reachability was separated from reliability. An HTTP response can prove that one route answered at one moment. It does not automatically prove an availability objective, scheduled delivery, data freshness, inference readiness, or correctness.

Eight active runtime services remained explicitly unmeasured after owner review. Each has a service-specific reason. atlas-api-public cannot independently prove itself by self-probing. atlas-daily-digest can be reachable while scheduled delivery fails. atlas-dora can serve a health route while producing incorrect metrics. ramone-edge intentionally treats a sleeping local machine as healthy but has no approved availability target. Other services lacked an independent probe or approved objective.

The policy therefore uses three meaningful states:

Unknown is preserved instead of being translated into healthy.

Phase II — The Stale Reliability Fingerprint

The explicit unmeasured policy exposed a second control-plane problem. The status projection contained a fingerprint from an older runtime set. It predated the Wave 0 narrowing of the public registry, when three services still appeared in runtime reliability coverage even though they were no longer part of the current public runtime authority.

The status file was valid JSON. Its embedded fingerprint was well formed. The problem was provenance: it represented a historical policy shape rather than the current one.

I reconstructed the previous objective and unmeasured sets and regenerated the historical canonical material. That produced the exact fingerprint already embedded in status:

a279dca1767367f78d91d185f6902dcbdd77b3d64c6b7c3ddbf4e671834e5b3e

That result proved the drift was pre-existing rather than introduced by the new policy. It also demonstrated why a fingerprint is useful only when its source material and publication path are explicit.

I made active runtime coverage exact: services with objective files must be measured, active services without objectives must appear once in the reviewed unmeasured declaration, and deprecated or archived services are excluded by lifecycle. The publication workflow remains the only write path for the public reliability policy, and the status projection must carry the same resulting fingerprint.

The lesson was not that fingerprints prevent drift. They make drift diagnosable. Preventing it requires a gate that compares the fingerprinted source and every published consumer.

Wave 2: Repository Policy

Wave 2 moved public repository presentation and GitHub configuration out of informal convention. Its original rollout cohort covered 31 intentionally public repositories. By the 26 July editorial snapshot, the authoritative public projection had grown to 33 repositories; the same three independent contracts still applied rather than one oversized compliance score.

The controlled topic vocabulary requires the atlas-systems topic and allows one to eight approved topics. Archive state must match lifecycle. Descriptions and homepages are bounded so portfolio links remain intentional rather than accumulating arbitrary provider metadata.

The five Atlas-specific labels are:

• status:blocked;
• status:live-verified;
• status:owner-review;
• status:rollout-pending;
• status:superseded.

Draft state, CI state, approval, merge state, and deployment checks remain native GitHub facts. Recreating them as labels would introduce a second status system that could drift from the platform.

Public audits are allowlist-driven from the public classification projection. They do not enumerate account membership. Private repositories call reusable validators from their own authenticated context, where .atlas/governance.json must match github.repository, declare private visibility, and keep public projection disabled. Evidence stays in the private caller repository.

The rollout also separated read-only audit from provider remediation. Adding the policy did not silently rewrite README files, descriptions, topics, labels, archive state, or default branches. Source fixes moved through ordinary pull requests. Provider metadata changes remained explicit owner actions followed by enforcement evidence.

Wave 3: Deployable Evidence and Bounded Remediation

Wave 3 connected policy to built artefacts, evidence relationships, removal planning, model decisions, and controlled repository writes.

Validate the deploy bundle, not the source claim

The reusable Worker deployment workflow gained optional require_meta and require_openapi gates. Validation runs after Wrangler produces the exact dry-run bundle and before the deployment step.

require_meta checks that the bundle retains /_meta and the fixed Atlas metadata shape: name, description, version, endpoints, status, and source. require_openapi checks for /v1/openapi.json, an OpenAPI 3.0 or 3.1 marker, paths, and the expected route/document material.

This moves the contract boundary from source convention to deployable evidence. Source code can contain a route that tree shaking, bundling, or a stale build path removes. The deploy gate checks the artefact Cloudflare would actually receive.

Atlas Trace

Atlas Trace introduced versioned node, edge, and graph contracts with deterministic identities and canonical bytes. The offline assembler consumes saved JSON evidence only. Every relationship requires explicit evidence references and a named basis.

Timestamp proximity is not a valid basis. CORRELATED_WITH may use correlation-with-explicit-evidence; stronger relationships require exact identity, verified runtime identity, release-watch identity, signed provenance, declared governance, or manual attestation. This prevents a release and an incident occurring near each other from being presented as causally related without evidence.

Visibility is part of the contract. Nodes and edges may be public, internal, or restricted metadata, and a graph cannot expose evidence more restrictive than its own visibility. Public Trace and Proof Chain surfaces consume bounded projections rather than a raw graph of every system I operate.

Retirement and model promotion

Retirement became a reviewable evidence state rather than a deletion. A tombstone preserves previous identity, reason, replacement, references, and expected downstream effects. Final archived evidence fails closed unless dependency, route, binding, manifest, Worker allowlist, production pull request, historical evidence, and recovery implications are verified or explicitly not applicable.

The model-promotion contract follows the same pattern. It binds one capability to exact model and configuration identity, exact evaluation-suite revision, evidence fingerprint, threshold result, human approval, and optional supersession. The contract cannot pull or delete a model, change Open WebUI, alter Home Assistant, restart services, or change routing by itself.

Atlas Gardener

Gardener consumes schema-valid findings and emits deterministic remediation proposals. The general engine hard-codes a narrow envelope:

• one allowlisted fixer type;
• at most five files;
• at most 200 changed lines;
• repository-relative UTF-8 text changes;
• exact preimage and postimage hashes;
• no arbitrary shell or HTTP capability;
• no deprecated, archived, external-derived, unknown, dirty, or stale target;
• no direct main, merge, deploy, secret, billing, environment, branch-protection, or Cloudflare change.

The GitHub App adapter binds one reviewed proposal to the exact target classification fingerprint, base commit, blob identities, file modes, patch digest, plan digest, risk class, expiry, rollback, and validation evidence. It rechecks the remote base before writing, creates reviewed blobs, creates one tree and commit, creates one deterministic branch, opens one draft pull request, and stops.

Version one refuses workflow files rather than silently requesting wider App permissions. The App has no endpoint for approval, merge, deployment, workflow dispatch, repository settings, secrets, environments, branch deletion, or provider mutation.

Phase III — The Corrupted Receipt

The first real Gardener canary produced an awkward result. The GitHub App transaction succeeded: it created the commit, branch, and draft pull request. The local wrapper then reported failure because jq could not parse the recorded receipt.

Repeating the provider action would have been the wrong response. The write had already happened. The failed component was the evidence path used to describe it.

I separated operator output from machine evidence and added an explicit --result-output path. The provider adapter now writes a dedicated JSON receipt that can be schema-checked and fingerprinted independently from prompts and progress text.

The recovery procedure preserved the successful transaction and repaired only the receipt path. That distinction matters for any non-idempotent provider action. A failed report is not evidence that the underlying action failed.

The canary exposed a failure after the write boundary, which is exactly where evidence quality matters most. Before a provider mutation, a bad receipt is inconvenient. After a provider mutation, it can cause duplicate branches, repeated writes, or incorrect rollback decisions.

Validation, Limits, and Evidence Map

Every wave used repository-native CI, strict schemas, deterministic fixtures, canonical fingerprints, fail-closed negative tests, pinned actions, bounded workflow permissions, and explicit timeouts. Public audits consumed approved allowlists. Private checks executed in authenticated source-local context. Provider writes remained separate from source merges.

The evidence model intentionally distinguishes what was proved:

The programme did not claim every automation possibility was complete. The real-model benchmark for the model-promotion contract remained deferred until SPECULAR-CORE was available. I kept the contract and evaluation path without substituting synthetic evidence for a reviewed GPU-backed run.

Gardener also remained owner-triggered at the Waves 0 to 3 boundary. It could prepare a bounded repair across an eligible repository, but it was not yet an unattended finding-to-merge controller. Automatic execution requires its own scheduling, deduplication, expiry, notification, approval, and failure-recovery design.

The programme spans many commits, but the article does not need to become a pull-request index. Current evidence is grouped by repository responsibility:

• atlas-infra owns policy, ADRs, reusable workflows, validation, evidence contracts, and public projections;
• atlas-gardener owns deterministic remediation planning and the bounded GitHub App transport;
• atlas-api-public owns the declared public estate map and bounded public API projection.

Current repository files and accepted ADRs are the authority for the system described here. Publication and deployment claims require separate scheduler, provider, and live-endpoint evidence.

Outcomes

The control plane replaced implied trust with explicit contracts. Important facts now have named authorities, deterministic projections, bounded public views, and separate declaration, observation, proposal, publication, and verification states. The remediation path can produce one exact draft change without inheriting authority to decide that the change is correct or safe to ship.

The transferable principle is that automation should gain authority only when the evidence, contract, and rollback boundary for that authority are explicit. A system becomes safer when each new capability arrives with a narrower statement of what it can do, what it cannot infer, and where a human or independent verifier must take over.