Justinaquino: Add Directed Execution: Token-Efficient Agent Development Process v0.1

2026-07-15T11:02:54Z

Add Directed Execution: Token-Efficient Agent Development Process v0.1

New page

= Directed Execution: Token-Efficient Agent Development Process =
'''CGG IT — Process & Skillset Definition v0.1'''
Owner: JCA · Status: Draft for team review · Date: 2026-07-15

----

== 1. Problem Statement ==

Developers are consuming Claude Sonnet and Opus tokens for raw code generation. This inverts the economics of the model market. Frontier-model output tokens are the single most expensive resource in the stack (Opus 4.8 output: ₱1,450/M; Fable 5: ₱2,900/M), while commodity-model output is nearly free by comparison (DeepSeek V4 Flash: ₱16/M — a 90× to 180× spread). Meanwhile, frontier-model ''input'' tokens are roughly 5× cheaper than their own output tokens, and cache hits cheaper still.

The consequence is a simple design rule that the current workflow violates:

<blockquote>'''Expensive models should read a lot and write a little. Cheap models should write a lot and be corrected.'''</blockquote>

Nobody has directed the developers to work this way, and there is no shared mechanism for making the cheap models better over time. Both gaps are addressed here: a role-separated loop for day-to-day work, and a skill system that converts QA corrections into reusable capability for the cheap agents.

== 2. Economic Rationale (worked example) ==

Assume a typical feature task: ~30K tokens of relevant context, ~50K tokens of generated code and tests.

'''Current pattern — Opus does everything:'''
30K in (₱8.70) + 50K out (₱72.50), plus one revision cycle of similar size ≈ '''₱150–160 per task''', with no independent QA gate.

'''Directed pattern:'''

{| class="wikitable"
|-
! Step !! Model !! Tokens !! Cost
|-
| Plan + FRD || Opus 4.8 || 30K in / 2K out || ₱11.60
|-
| Execute || DS V4 Flash || 35K in / 50K out || ₱1.08
|-
| QA review || Opus 4.8 || 60K in / 1K out || ₱18.85
|-
| Correction || DS V4 Flash || 40K in / 20K out || ₱0.64
|-
| Re-check || Opus 4.8 (cached ctx) || 25K in / 0.5K out || ~₱8
|-
| '''Total''' || '''≈ ₱40'''
|}

Roughly a 70–75% cost reduction ''while adding a QA gate that did not previously exist''. The QA read is the largest line item, which is correct — that is where the intelligence is actually needed. Substituting Sonnet as the checker for routine tasks pushes the total under ₱20. As skills accumulate and first-pass acceptance rises, the correction cycle amortizes toward zero.

The point is not merely savings. It is that '''judgment and volume are different products''', and we have been buying volume at judgment prices.

== 3. Principles ==

# '''Direction is the deliverable of intelligence.''' The frontier model's job is decomposition, constraints, acceptance criteria, and verdicts — all short, dense outputs. If Claude is emitting more than ~2K tokens on a task, something is misrouted.
# '''Deterministic-first, same as Synopsis.''' Anything checkable by a linter, test suite, type checker, or script never reaches a paid model. Machine checks run before Claude sees the diff.
# '''Every correction is an asset.''' A QA correction consumed once is an expense; distilled into a skill, it is capital. The loop is not complete until the correction is either distilled or explicitly judged one-off.
# '''Cheap agents are pushed, not babysat.''' Executors get hard objectives with clear acceptance criteria and their accumulated skill library — not hand-holding. Failures are information for the skill system.
# '''The loop is the curriculum.''' Interns learn the same process the agents run: plan, execute, check, correct, distill. Team-wide token efficiency is a literacy goal, not a dev-only optimization.

== 4. Roles ==

{| class="wikitable"
|-
! Role !! Model tier !! Reads !! Writes !! Output budget
|-
| '''Director''' || Claude Opus / Fable || Objective, codebase context, skill index || FRD: decomposition, interfaces, acceptance criteria, risks || ≤ 2K tokens
|-
| '''Executor''' || DS V4 Flash, Kimi K2.6; later local Qwen/DeepSeek via OpenCode || FRD + relevant skills + code context || Code, tests, migration scripts — full volume || Unbounded
|-
| '''Checker''' || Claude Opus (critical) / Sonnet (routine) || Full diff + FRD + machine-check results || Verdict: PASS/FAIL per acceptance criterion, itemized corrections anchored to lines || ≤ 1K tokens
|-
| '''Skill Librarian''' || Human (rotating dev/intern) + cheap model drafting || QA correction logs || Skill drafts, skill index maintenance || —
|}

Director and Checker should be '''separate sessions''' even when the same model — the Checker must evaluate against the FRD, not defend its own plan.

Model routing within tiers: DeepSeek V4 Flash is the default executor. Kimi K2.6 when the task needs multimodal input (screenshots, UI review) or its larger instant-mode quality helps. Local agents take over as the vLLM/Ollama stack proves out on benchmarked task classes — the skill library transfers with them, which is the strategic payoff: skills are model-portable capability, not vendor lock-in.

== 5. The Loop ==

'''Plan → Execute → Machine-check → QA → Correct → Distill → Reuse'''

'''Plan.''' Director receives the objective and produces an FRD-lite (template below). No code beyond interface signatures and test-case ''descriptions''. The FRD is the contract; ambiguity discovered later is a planning defect and goes back to the Director, not patched silently by the Executor.

'''Execute.''' Executor works in OpenCode with the FRD and the relevant skills loaded (skills prepended as a stable prefix — this also maximizes cache hits). It generates code and tests to satisfy the acceptance criteria. It may burn tokens freely; that is what it is for.

'''Machine-check.''' Lint, type-check, test run, formatter. Automated (n8n or a pre-commit script). Failures loop back to the Executor without any paid QA read. Claude never reviews code that fails machine checks.

'''QA.''' Checker reads FRD + diff + machine-check results and returns a structured verdict. It does not rewrite code. Each FAIL item states ''what is wrong, why, and the principle violated'' — the "why" is raw material for distillation.

'''Correct.''' Executor applies corrections. Two correction cycles maximum; a third failure escalates to a human plus a mandatory skill-gap review ("what skill would have prevented this?").

'''Distill.''' Weekly, the Skill Librarian reviews the QA correction log. Any correction pattern appearing '''twice or more''' becomes a skill candidate. A cheap model drafts the skill from the correction transcripts; a human reviews and merges it into the Forgejo skill repo.

'''Reuse.''' Skills are indexed by trigger. The Director's FRD names which skills apply; the Executor loads them. Cheap agents therefore climb: yesterday's correction is today's default behavior.

== 6. Deliverable Contracts ==

=== FRD-lite (Director output) ===

<syntaxhighlight lang="text">
FRD-<seq>: <title>
Objective: one sentence.
Scope in / Scope out: explicit exclusions.
Interfaces: function/API signatures the Executor must implement.
Acceptance criteria: numbered, each machine- or QA-checkable.
Constraints: security, style, dependency, and performance rules that apply.
Skills to load: skill IDs from the index.
Risk notes: where the Executor is most likely to go wrong.
</syntaxhighlight>

=== QA Verdict (Checker output) ===

<syntaxhighlight lang="text">
VERDICT: PASS | FAIL
AC-1: PASS/FAIL — <one line>
AC-n: ...
Corrections (if FAIL):
C1 [file:line] What / Why / Principle violated
Distill flags: corrections likely to recur → candidate skill name
</syntaxhighlight>

The "Principle violated" field is deliberately mandatory. It is what makes the verdict distillable rather than a one-off patch instruction.

== 7. The Skill System ==

A skill is a short markdown file teaching a cheap model a specific competence, in the same shape Anthropic uses for Claude skills — which means skills built here also work when the executor ''is'' Claude, or a local model, or whatever comes next.

'''Format''' (<code>SKILL.md</code> per skill, one directory per skill in the Forgejo repo, e.g. <code>skills/erpnext-patch-migrations/</code>):

<syntaxhighlight lang="text">
name: <kebab-case>
trigger: when this skill applies (used by Director for routing)
procedure: numbered steps / rules
examples: one good, one bad (drawn from real QA corrections)
anti-patterns: the specific failures that birthed this skill
provenance: FRD/QA references, author, date
status: draft | reviewed | adopted | deprecated
</syntaxhighlight>

'''Lifecycle.''' Draft (from correction log, cheap-model-written) → Reviewed (human, PR in Forgejo) → Adopted (in the index, loadable) → Deprecated (superseded or model-obsoleted). Skills are versioned like code because they are code — for the agents.

'''Ownership.''' Devs are consumers and reviewers, not the bottleneck. Interns, working with the local DS agents in OpenCode, author most drafts as part of their objectives. The repo's PR history doubles as an intern portfolio.

== 8. Escalation Rules ==

{| class="wikitable"
|-
! Situation !! Route
|-
| Routine feature/bugfix, skills exist || Flash executes, Sonnet checks
|-
| Novel architecture, security-sensitive, cross-system design || Opus/Fable plans and checks
|-
| Executor fails 2 correction cycles || Human review + skill-gap analysis
|-
| Executor requests clarification || Back to Director (cheap→expensive calls are Director-mediated, never ad hoc)
|-
| Anything touching secrets, credentials, prod data || Local agents only; no external API
|}

The last row is a hard rule and aligns with the transcription-tool memo posture: sensitive context stays on the ZeroTier overlay.

== 9. Metrics & Token Ledger ==

Per task, logged (SQLite to start, consistent with the Synopsis MVP philosophy; ERPNext later if it earns its keep):

* Tokens and ₱ by role (Director / Executor / Checker), per task and per week.
* '''First-pass acceptance rate''' — the headline metric. Rising FPA means skills are working.
* '''Correction cycles per task''' and '''escalation rate'''.
* '''Skill reuse count''' and '''skills authored''' per person — the intern scoreboard.
* '''Cost per merged deliverable''', trended. Target: sustained <30% of the all-frontier baseline within one quarter.

The ledger is not surveillance; it is the feedback signal that tells us when a task class is ready to move from Flash to a local model, and when a checker can be downgraded from Opus to Sonnet.

== 10. Rollout ==

'''Week 1 — Instrument and declare.''' Stand up the skill repo in Forgejo, the token ledger, and the machine-check gate. Short directive to the team: no frontier-model code generation for tasks with an FRD path; frontier models plan and check.

'''Week 2 — Seed skills.''' Mine the last month of Claude chat history for recurring correction patterns; interns draft the first 5–10 skills (likely candidates: ERPNext patch conventions, Frappe API idioms, our lint/commit standards, test scaffolding).

'''Week 3 — First directed sprints.''' Each intern gets one challenging objective run fully through the loop with a local DS agent in OpenCode. Devs act as Skill Librarian reviewers. Every completed objective must ship at least one skill PR.

'''Week 4 — Review.''' FPA, cost per deliverable, and skill count reviewed against baseline. Decide the first task class to pilot on fully local models.

'''Steady state.''' The loop is the standard operating rhythm. Quarterly, revisit model routing (the pricing table this document was built from will be stale in months) and prune deprecated skills.

----

''Reasoning summary: the spread between frontier output pricing and commodity output pricing (90×+) makes "smart model writes code" the single most expensive habit in the workflow, while the input/output asymmetry (5×) makes "smart model reads everything, writes verdicts" nearly free by comparison. The skill system converts the residual cost — QA corrections — into permanent, model-portable capability, so the cheap tier's quality compounds and the expensive tier's involvement shrinks over time. Interns are the labor force and the beneficiaries of the same loop.''

Directed Execution: Token-Efficient Agent Development Process - Revision history

Justinaquino: Add Directed Execution: Token-Efficient Agent Development Process v0.1