View source for De-Risk: Sovereign Training Data v0.1

= De-Risk: Sovereign Training Data v0.1 =
== Building Model Intelligence Without Frontier Tokens in the Corpus ==

<blockquote>'''Evaluator:''' Justin Aquino (CEO / CISO-DPO), authored as JCA</blockquote>
<blockquote>'''Evaluation Date:''' 2026-07-15</blockquote>
<blockquote>'''Methodology:''' Strategy document — synthesis of the Fable 5 export-control incident, Anthropic/OpenAI commercial-terms analysis, and the Directed Execution Process economics. Companion to <code>2026-07-15-fable-ban-sovereign-ai-and-local-model-strategy.md</code> (source report, veracity-reviewed) and <code>2026-07-15-fable-fact-check-and-csama-legal-qa-conversation.md</code> (full Q&A transcript).</blockquote>

'''Owner:''' JCNA (CEO / CISO-DPO) · '''Status:''' Draft for review · '''Date:''' 2026-07-15
'''Companion documents:''' Directed Execution Process v0.1 · Fable Ban & Local AI (veracity-reviewed 2026-07)

----

== 1. Context — What the Fable Episode Actually Proved ==

Three facts from June–July 2026 matter to us. Everything else in the source document was dramatization.

'''1.1 Frontier access is now a policy variable, not an infrastructure constant.''' A Mythos-class model went from launch to worldwide suspension in three days on a single Commerce directive, and stayed dark for 19 days. The trigger was external (a reported jailbreak escalated to Treasury), the scope was blunt (all foreign nationals, unverifiable, therefore everyone), and the reversal was equally abrupt. No SLA, no contract, no self-hosted deployment would have protected a workflow hard-wired to that model. The lesson is not "Anthropic is unreliable" — the lesson is that ''any'' frontier vendor now sits downstream of national-security discretion, and OpenAI's government-coordinated GPT-5.6 rollout confirms the pattern applies industry-wide.

'''1.2 The naive response — distill frontier outputs into your own weights — is a legal trap.''' Anthropic's and OpenAI's terms broadly prohibit using model outputs to train competing models. A pipeline that harvests Fable or Sol completions into a fine-tuning corpus trades a ''regulatory'' dependency for a ''contractual'' liability, and does so in writing, with provenance logs that would be discoverable. For an organization whose CISO-DPO signs the compliance posture, this is not a gray area we can occupy. The source document's "Distilabel ensures ToS compliance" claim was its weakest section precisely because tooling cannot launder provenance.

'''1.3 The clean asset class already exists in our practice.''' Skills, specs, checklists, and QA-derived process knowledge — instructions authored by our people, versioned in our Forgejo — transferred capability across the blackout at zero legal risk. The Directed Execution Process doc formalized this. What it did ''not'' yet do is close the loop from process knowledge to '''model weights we own'''. This document closes that loop, cleanly.

----

== 2. The De-Risk Principle: Frontier-Token Quarantine ==

One rule, enforced architecturally, resolves the entire ToS exposure:

<blockquote>'''No frontier-model-generated token ever enters the training corpus. Frontier models may shape the process; they may never author the data.'''</blockquote>

This splits our AI estate into two zones:

{| class="wikitable"
|-
! Zone !! Contents !! Frontier models (Claude/Fable, Sol)... !! Legal character
|-
| '''Process zone''' || Plans, specs, QA verdicts, RCAs, skill drafts, test designs || ...operate freely here || Ordinary licensed use — Claude helping our team work
|-
| '''Corpus zone''' || Prompts, completions, error traces, corrections, preference pairs destined for fine-tuning || ...are architecturally excluded || 100% owned: our people, our machines, open-weight models, permissive licenses
|}

The subtle failure mode to guard against: a Claude-written QA verdict or RCA is still a frontier token. If that text is pasted verbatim into a training sample, the quarantine is breached. Therefore:

* **Claude's judgments become ''signals'', not ''text''.** Pass/fail, severity, "principle violated" tags — structured fields a human confirms — may drive ''selection and labeling'' of corpus samples. Claude's prose stays in the process zone.
* '''The strongest labels don't come from Claude at all.''' They come from deterministic verifiers — tests pass, lint clean, build succeeds, output matches oracle. Claude helps ''design'' the tests; the CI system ''issues'' the labels. A label emitted by pytest has no author to have a ToS with.
* '''Skills remain process-zone artifacts.''' Instructions and context engineering are not model training; the Directed Execution loop continues unchanged. This document adds a parallel track, it does not replace that one.

----

== 3. The Error Foundry — Why the Error Is the Gold ==

Commodity training data is worthless: every lab has scraped the same internet, and open models are already trained on it. What no one else has — what cannot be scraped, bought, or distilled — is '''the distribution of failures our specific models make on our specific work, paired with verified corrections.'''

A failure trajectory is the single most valuable sample type because it encodes three things at once:

# '''Where the model's capability boundary actually is''' (not where benchmarks claim it is — recall that ~30% of SWE-bench Pro tasks are broken; our CI is not).
# '''A hard negative''' — a plausible-looking wrong answer, which is exactly what preference-optimization methods (DPO/KTO) need and exactly what synthetic "textbook" data lacks.
# '''A verified positive twin''' — once the correction lands and the verifier goes green, we hold a (failed attempt, working attempt) pair on identical context. That pair is a gradient pointing from "what our model does" to "what our work requires."

This also sidesteps the model-collapse problem the source document correctly cited: collapse comes from training on unfiltered synthetic output. Our corpus is synthetic ''attempts'' filtered by ''non-synthetic ground truth'' (executing code, real acceptance criteria, human confirmation). The verifier anchors the distribution to reality.

We already run this pattern. '''Team C at Cabuyao logs negative information from AGV runs for model training''' — this document generalizes Team C's discipline to the entire software and knowledge-work estate.

----

== 4. Pipeline Architecture ==

=== 4.1 Roles ===

{| class="wikitable"
|-
! Layer !! Actor !! Produces !! Zone
|-
| '''Direction''' || Claude (Director session) || FRD-style spec, acceptance criteria, test design, ≤2K-token plan || Process
|-
| '''Execution''' || DeepSeek / Kimi / local Qwen & DeepSeek on OpenCode (vLLM, dual RX 7900 XT) || All code, drafts, attempts — every token of it || '''Corpus'''
|-
| '''Verification''' || CI: pytest/lint/type-check/build via n8n or pre-commit || Deterministic pass/fail labels || '''Corpus''' (labels)
|-
| '''QA''' || Claude (Checker session, separate from Director) || Verdict as structured fields; prose stays out of corpus || Process
|-
| '''RCA''' || Claude + human || Root-cause analysis → feeds ''skill drafts'' and ''new test cases'', never sample text || Process
|-
| '''Correction''' || Open model (re-attempt with hint) or human dev || The corrected trajectory — corpus-eligible because its author is ours || '''Corpus'''
|-
| '''Curation''' || Interns + devs || Provenance manifest, dedup, pair assembly, PR to dataset repo || Corpus
|}

The critical asymmetry: '''Claude reads everything and authors nothing that trains.''' Its intelligence is spent on specs, test oracles, verdict signals, and RCA insight — all of which make the ''open'' models' next attempts better, and make the ''corpus'' labels sharper, without ever contributing a training token.

=== 4.2 The loop, end to end ===

# '''Objective in.''' Real ticket or deliberately challenging intern objective. Director-Claude produces spec + machine-checkable acceptance criteria. Spec is stored in Forgejo (process repo).
# '''Open model executes.''' Full trajectory logged: prompt, context, every attempt, tool calls, outputs. This logging is non-negotiable — the exhaust ''is'' the product.
# '''Verifier fires.''' Fail → the failure trace is tagged and banked (this is the gold, not discarded). Pass → candidate positive.
# '''On failure:''' RCA (Claude + human) identifies the principle violated → two outputs: (a) a ''skill draft'' or new test case (process zone), (b) a ''hint'' given to the open model for a re-attempt. The re-attempt that passes becomes the positive twin.
# '''Pair assembly.''' (context, failed, passed, verifier evidence, provenance manifest) → PR into the dataset repo. Intern authors it; dev reviews it; Forgejo history is the audit trail and the intern's portfolio.
# '''Periodic fine-tune.''' When a domain accumulates enough pairs (target: 500–1,000 verified pairs before first run), fine-tune Qwen/DeepSeek variants on the 7900 XT rig (LoRA/QLoRA first — full fine-tunes are not needed to capture correction patterns). Evaluate against a held-out set of ''our own'' past failures.
# '''Promotion.''' A tuned model earns executor duty only by beating the base model on the held-out failure set and on live first-pass acceptance rate — the same metric already in the Directed Execution scoreboard.

=== 4.3 Provenance manifest (per sample, mandatory) ===

<syntaxhighlight lang="text">
sample_id: uuid
generator: {model: qwen2.5-coder-32b, host: PC00-vllm, license: apache-2.0}
context_source: {ticket: FORGEJO-####, spec_hash: ..., spec_author: human|claude-process-zone}
label: {verifier: pytest@ci, result: fail|pass, evidence: run_url}
frontier_tokens_present: false        # hard gate; CI rejects true
human_touch: {curated_by: intern_id, reviewed_by: dev_id}
correction_lineage: failed_sample_id  # for positive twins
</syntaxhighlight>

<code>frontier_tokens_present: false</code> is enforced by a pre-merge check on the dataset repo: samples whose context includes Claude-session exports are rejected mechanically, not by policy memo. Same philosophy as Synopsis — deterministic rules first.

=== 4.4 Storage & plumbing ===

* '''Trajectory log:''' SQLite first (Synopsis pattern), one DB per domain, schema versioned in Forgejo. Graduate to ERPNext DocTypes only when the team needs commission-model-style visibility.
* '''Dataset repo:''' dedicated Forgejo org, one repo per domain (<code>ds-coding</code>, <code>ds-erpnext</code>, <code>ds-docs</code>), PR-gated, LFS for traces.
* '''Orchestration:''' n8n handles verifier dispatch and label writes; no human copies test results by hand.
* '''Compute:''' existing dual RX 7900 XT / vLLM stack for both inference and LoRA training; no new capex required for Phase 1–2.

----

== 5. Risk Register — What This De-Risks and What It Doesn't ==

{| class="wikitable"
|-
! Risk !! Naive distillation !! This architecture
|-
| Frontier ToS breach (training competing models) || High — corpus is frontier tokens || '''Eliminated''' — corpus is 100% own/open provenance, mechanically enforced
|-
| Frontier blackout (Fable-type event) || Capability frozen at last distill || Skills + tuned weights both survive; Claude roles degrade gracefully to human plan/QA (slower, not broken)
|-
| Model collapse from synthetic data || High — unfiltered frontier outputs || Low — every sample anchored by deterministic verifier; failures kept as hard negatives
|-
| Vendor lock-in of process knowledge || N/A || Skills in model-portable SKILL.md format; dataset in open formats; weights are Apache/MIT-base
|-
| Data privacy (RA 10173) || Frontier API sees raw work product || Secrets/prod data → '''local agents only''' (existing hard rule, unchanged); corpus lives on-prem; CISO-DPO sign-off on any sample class containing personal data
|-
| Benchmark self-deception || Marketing benchmarks || Held-out set of our own historical failures — unfakeable, directly relevant
|}

'''Residual risks, stated honestly:'''
* ''Judge-influence gray zone.'' Even using Claude's pass/fail ''signals'' to select samples arguably "assists" model development. Our mitigation is that the load-bearing labels are deterministic-verifier labels, humans confirm every Claude-derived tag, and no frontier text enters the corpus. This is a defensible posture, not a court-tested one; counsel (BBLAW) should sanity-check the framing once, briefly.
* ''LoRA ceiling.'' Correction-pattern LoRAs will not make a 32B model a Mythos-class reasoner. That is fine — the Directed Execution economics never asked it to. The 5% burst path to frontier APIs remains for genuinely hard problems, in the process zone where it belongs.
* ''Curation labor.'' The pipeline lives or dies on logging discipline and intern throughput. Hence §6.

----

== 6. Call to Action ==

'''This week'''
# '''CEO/CISO-DPO:''' ratify the Frontier-Token Quarantine as policy (one page, references the transcription-tool memo as precedent for "approved-tool" governance style).
# '''Sysadmin + Carl:''' stand up the dataset Forgejo org, SQLite trajectory schema, and the <code>frontier_tokens_present</code> pre-merge check. Half a day of work; it makes the policy mechanical.
# '''Devs:''' switch OpenCode local-agent sessions to full trajectory logging ''now'', even before curation starts. Every unlogged failure this month is discarded gold.

'''Weeks 2–4'''
# '''Interns (all, not just dev interns):''' first curation objectives issued — assemble 50 verified pairs each from live failure logs, PR'd to the dataset repo. The curriculum ''is'' the loop: they learn spec-reading, verifier evidence, and provenance hygiene simultaneously. Scoreboard: pairs merged, review-pass rate.
# '''Team C liaison:''' map the AGV negative-information logging schema onto the manifest in §4.3 so robotics and software share one provenance convention.
# '''BBLAW (existing engagement):''' 30-minute review of the quarantine framing and the judge-signal question. Not a new engagement — an agenda item.

'''Month 2–3'''
# First LoRA run on the coding domain once ≥500 pairs verified. Evaluate on held-out failure set. Publish results internally — win or lose, the eval harness itself is a durable asset.
# '''Decision gate:''' if tuned executor beats base on first-pass acceptance, promote it in the Directed Execution rotation and begin domain #2 (ERPNext customization is the obvious candidate — our failure data there is uniquely ours).

----

== 7. The Strategic Frame ==

Two tracks, different clock speeds, one direction:

* '''Skills (fast, instant transfer):''' capability as ''instructions'', portable across any executor, already running. Survives blackouts by construction.
* '''Weights (slow, compounding):''' capability as ''parameters we own'', grown exclusively from our own error distribution, legally unencumbered because the corpus never contained a frontier token.

Claude's highest-value role in both is the same: not as the source of answers to memorize, but as the intelligence that designs the tests, reads the failures, and sharpens the questions — while our models, our people, and our verifiers author every byte we will ever train on.

The Fable ban gave us the precedent. The error logs give us the data. The quarantine gives us the clean title.

''The error is the gold. Log everything.''