MediawikiCIT - Recent changes [en]

Main Page

Justinaquino — Sun, 02 Aug 2026 06:05:55 GMT

Add New Articles feed (10 most recent, pruned) + auto Newest Pages section, per the hub-and-spoke publishing protocol

@@ Line 3: / Line 3: @@
 Welcome to the '''Comfac IT Knowledge Base'''. Use the sections below to find guides, SOPs, and references organized by topic.
 </div>
 == ERPNext / Frappe ==
@@ Line 60: / Line 81: @@
 * [[Directed Execution: Token-Efficient Agent Development Process]] — Director/Executor/Checker loop and skill distillation for token-efficient agent development (v0.1, 2026-07-15)
 * [[De-Risk: Sovereign Training Data v0.1]] — Frontier-Token Quarantine and the ToS-clean path to owned model weights, prompted by the June 2026 Fable 5 export-control suspension (v0.1, 2026-07-15)
 </div>

Hugging Face Practical Guide 260802

Justinaquino — Sun, 02 Aug 2026 05:03:29 GMT

New article: Hugging Face practical guide (Spaces, local builds, Model Hub/datasets, enterprise + hf jobs) - lae/HF backlog

New page

= Hugging Face — Practical Guide =

<blockquote>'''Compiled:''' 2026-08-02 · '''Status:''' Living page — open for deeper research</blockquote>
<blockquote>'''Why it is here:''' Hugging Face is currently used at Comfac as a download endpoint for stock model files. It is four things, and the other three are the ones we under-use.</blockquote>

The goal behind this page: access to free open-source models that are '''better than the stock ones''' and more tailored to a specific job, the '''harnesses''' built around them, and the '''training resources''' that take a corpus of a few hundred owned data points and work it up toward thousands.

That last one is the unclosed loop in [[De-Risk: Sovereign Training Data v0.1]] — we have clean, owned process knowledge and no mechanism yet for turning it into '''weights we own'''.

Each section below ends with '''search keywords''' for going deeper on that one bucket.

== Source videos ==

<blockquote>'''Note:''' this wiki does not have the <code>EmbedVideo</code> extension installed (verified 2026-08-02 via <code>api.php ... siprop=extensions</code>), so videos are listed as preview cards rather than embedded players. The [https://wiki.gi7b.org/index.php/Hugging_Face_Practical_Guide_260802 gwiki copy of this page] has EmbedVideo and shows them inline. Installing the extension here is an administrator action.</blockquote>

{| class="wikitable" style="width:100%;"
! style="width:34%;" | Video !! What it covers !! Use it for
|-
| '''[https://www.youtube.com/watch?v=3kRB2TXewus What Is Hugging Face and How To Use It]'''
| The general tour — Spaces, the Model Hub, the Datasets section, and how they relate.
| Sections 1–3 below. Start here if Hugging Face has only ever been a download link.
|-
| '''[https://www.youtube.com/watch?v=wUzfRBi9wZ0 The Hugging Face Hub for Enterprise & Academia]'''
| Private workspaces, SSO, access control, separated compute/storage billing, SSH onto Spaces hardware, and remote training with <code>hf jobs</code>.
| Section 4. This is the one that matters for training on our own data.
|}

----

== 1. Exploring and using pre-built AI (Spaces) ==

'''Spaces''' are ready-to-use, browser-based applications built by the AI community — image and video generators, voice synthesis, transcription, live demos of new papers. All runnable without writing code.

If a Space is useful, click '''Duplicate Space'''. That forks it into your own account as your own copy, public or private, on hardware you select.

Two things worth internalising:

* A Space is a '''real repository''', not a demo sandbox. Duplicating gives you the source.
* Most Spaces are '''Gradio''' apps. Learn Gradio once and the whole catalogue becomes editable.

'''Search keywords:''' ''"Hugging Face Spaces tutorial"'' · ''"How to build and deploy Gradio apps on Hugging Face"'' · ''"Duplicating Hugging Face Spaces"''

== 2. Building custom apps locally ==

Because Spaces are open source, the code comes to your machine. Clone the repository, open it in an AI-assisted IDE, run it locally, and connect your own API keys instead of the demo's. From there the IDE can help you change colours, fonts, or actual behaviour.

This is the cheapest route to '''a harness we control''' — instead of writing scaffolding from scratch, start from a working app and cut it down.

Practical notes:
* Expect the first local run to fail on dependencies or a missing key. That failure, written down, is the reusable knowledge — not the happy path.
* Check the licence before adapting anything. '''Open-source and permissively-licensed are not the same thing''', and licence direction is one-way once code is borrowed.

'''Search keywords:''' ''"Clone Hugging Face Space locally"'' · ''"Build AI apps with Cursor and Hugging Face"'' · ''"Integrate LLM API keys in Hugging Face local apps"''

== 3. Finding models and datasets ==

The '''Model Hub''' is the core: over 1.5 million open-source models categorised '''by task''' — computer vision, NLP, audio, robotics. The '''Datasets''' section holds the data that trained them.

'''Navigate by task tag, not by keyword.''' Searching an acronym returns papers; filtering by task returns weights:

: <code>https://huggingface.co/models?pipeline_tag=robotics</code>
: <code>https://huggingface.co/api/models?pipeline_tag=robotics</code> ''(API form, for scripting)''

The advanced feature almost nobody touches is the '''dataset viewer''': chat with a dataset through an AI agent and '''run SQL queries in the browser''' to analyse it ''before'' downloading a single gigabyte. For evaluating training data this is the highest-value habit on the page.

'''Our standing admission rule applies here:''' a model card is a '''claim, not a measurement'''. Nothing enters the local model roster on a card alone — it enters on a real API call, the same gate used for the Synopsis model-selection slate. See [[Comfac GPU Scaling and AI Research Goals]] §Applied Model Evaluation and [[🧠 Process: Selecting and Installing the Right Ollama Model for Your Hardware]].

'''Search keywords:''' ''"How to choose and run Hugging Face models"'' · ''"Hugging Face dataset viewer SQL tutorial"'' · ''"Fine-tuning open source models Hugging Face"''

== 4. Scaling for teams and enterprise ==

For organisations and academic teams the paid tiers add private workspaces, single sign-on (SSO), and granular access control.

The structural benefit is that '''compute and storage are billed separately''' — no paying for idle hardware. Two working modes follow:

* '''SSH directly onto Spaces hardware''' and develop there, as if it were a remote box.
* '''<code>hf jobs</code>''' — fire a heavy training workload at remote hardware and collect the result.

The second is what makes a fine-tune possible without owning the GPU it would need.

'''Search keywords:''' ''"Hugging Face Enterprise setup guide"'' · ''"Running hf jobs on zero GPU Hugging Face"'' · ''"Hugging Face private workspaces and security"''

----

== Open problem — where this is going ==

Three goals, none closed. Tracked in <code>work/comfac-operations/BACKLOG-huggingface-model-sourcing-and-training.md</code>.

# '''Better than stock.''' The models we run locally today are stock bases. Tuned community models in the same size band may follow instructions better for a specific task, at no cost — but only a run against a gate written '''beforehand''' counts as evidence.
# '''Harnesses instead of rebuilds.''' Identify the Spaces that overlap tooling we would otherwise write ourselves.
# '''Training on owned data.''' Start at a few hundred data points and work toward thousands.

=== The constraint on goal 3 ===

'''Frontier-Token Quarantine is binding.''' Per [[De-Risk: Sovereign Training Data v0.1]], distilling commercial frontier-model outputs into our own weights trades a regulatory dependency for a '''contractual liability''' — in writing, with discoverable provenance logs. Any training corpus must be '''clean by construction'''. Tooling cannot launder provenance; that was named as the weakest claim in the source report.

=== Hardware reality check ===

Anything meant to run on a normal workstation has to fit the VRAM actually available, '''alongside whatever else is already resident''' — a vision model kept loaded for a document pipeline is not free. Size band and quantisation are a filter applied '''first''', not a compromise made afterwards. A related finding already measured here: 4-bit quantisation degrades digit reading, so anything touching numbers on a form runs at 8-bit.

== See also ==

* [[De-Risk: Sovereign Training Data v0.1]] — why owned weights matter and what may not go in the corpus
* [[Comfac GPU Scaling and AI Research Goals]] — GPU capacity and the applied model-evaluation method
* [[🧠 Process: Selecting and Installing the Right Ollama Model for Your Hardware]]
* [[Project OpenCoder: AI Independence Initiative]]
* [[Directed Execution: Token-Efficient Agent Development Process]]

[[Category:AI]]
[[Category:Research]]
[[Category:Open Source]]
[[Category:Comfac]]

Audio Learning

Justinaquino — Sat, 01 Aug 2026 06:15:16 GMT

Created: Audio Learning research-to-TTS tool article with link to citfj repo

New page

{{New article banner|date=2026-08-01}}
'''Audio Learning''' is the practice of converting research, articles, notes, and study material into scripts written to be '''listened to''' rather than read. It is a shared Comfac tool maintained as a private Forgejo repository and designed for staff who consume technical material hands-free — commuting, walking, between machines on the plant floor.

== What it is ==

Written prose and spoken prose are different mediums. A reader can re-read a sentence, skim a heading, and hold structure visually. A listener gets one linear pass with no backtracking. Audio Learning provides the discipline to re-compose research for the ear.

The repository contains a '''skill for AI agents''' ('''research-to-tts''') that converts knowledge into one of two house formats:

; Lecture format (Great Courses style)
: For technical study — networking, protocols, CISM/CISSP, security architecture, governance frameworks, systems engineering, ISO standards. The listener walks away understanding a mechanism they did not understand before. Useful for Comfac training material: Network+ basics, pfSense operations, BIR compliance, ERPNext workflows.

; Analyst format (thesis-and-evidence style)
: For contested material — energy markets, housing policy, supply-chain economics, regulatory analysis, industry studies. The listener walks away able to make or defend a decision. Useful for bid analysis, feasibility studies, vendor evaluations, and competitive intelligence.

== What it contains ==

* Concrete prose mechanics — how to speak numbers, acronyms, symbols, citations, code, and homographs so a text-to-speech engine renders them correctly
* Format templates — cold-open structures, spine metaphors, prerequisite ordering, decision frames, thesis stacking, approach ladders, and tripwire indicators
* Visual description protocols — a two-pass method for turning charts, diagrams, tables, and maps into something a listener can build a mental model of
* A pre-delivery audit — ten checks every script must pass before it is spoken

== Where it lives ==

The source repository is private on the Comfac Forgejo:

[https://git.comfac-it.net/cgg/audio-learning cgg/audio-learning]

== How to use it ==

An AI agent loads the skill, reads the reference files, picks a format, plans an outline from the source material, drafts a script with spoken prose only (no markdown, no symbols, no parentheticals), and audits the result before delivering it. The output is a plain-text script with a non-spoken production header and a clean spoken body — ready to be fed into any text-to-speech engine.

== Related ==

* [[GPU Scaling and AI Research|GPU Scaling and AI Research Goals]] — Comfac's model-evaluation infrastructure
* [[Open Source Software Analysis]] — tool evaluation methodology
* [[Resume / CV Renamer Pipeline]] — OCR-to-searchable pipeline (same ingestion-to-structured-output pattern)

[[Category:Tools]]
[[Category:AI Agents]]
[[Category:Training]]

Main Page

Justinaquino — Fri, 31 Jul 2026 14:25:42 GMT

Tools & Productivity: add Resume / CV Renamer Pipeline

← Older revision		Revision as of 14:25, 31 July 2026
Line 104:		Line 104:
	<div style="column-count:2; column-gap:2em;">		<div style="column-count:2; column-gap:2em;">
	* [[Excel Description Filler Tool 📝]]		* [[Excel Description Filler Tool 📝]]
			* [[Resume CV Renamer Pipeline\|Resume / CV Renamer Pipeline]] — CV intake → OCR / vision-OCR → AI field extraction → date-stamped rename + VCF export
	* [https://github.com/Comfac-Global-Group/comfac-studies Comfac Studies - Spaced Repetiton Active Recall]		* [https://github.com/Comfac-Global-Group/comfac-studies Comfac Studies - Spaced Repetiton Active Recall]
	* [[LibreOffice / Nextcloud Office – "Due Tasks" Conditional Formatting]]		* [[LibreOffice / Nextcloud Office – "Due Tasks" Conditional Formatting]]

Resume CV Renamer Pipeline

Justinaquino — Fri, 31 Jul 2026 14:25:40 GMT

Add Resume / CV Renamer Pipeline tool article (intake, OCR/vision-OCR, AI extraction, rename, VCF export)

New page

= Resume / CV Renamer Pipeline =

<div style="background:#f8f9fa; border:1px solid #ddd; border-radius:4px; padding:10px 16px; margin-bottom:16px;">
Automated resume ingestion, OCR, AI extraction, file renaming, and VCF contact export for recruitment workflows.
</div>

'''Location:''' <code>/path/to/resume-renamer/</code> (configure to your local HR/resume folder)

'''Upstream docs:''' [[Frappe HRMS - Ch01 Recruitment]]

----

== What It Does ==

# Drop a resume file (PDF, DOCX, JPG, PNG) into the folder.
# The scanner detects the new file and extracts text with <code>pdftotext</code>.
# Image files or image-based PDFs are converted with <code>img2pdf</code> and OCR'd with <code>ocrmypdf</code> if available.
# If OCR fails or is unavailable, the scanner falls back to '''vision OCR''' using a local VLM (e.g. <code>glm-ocr:bf16</code>).
# A local LLM extracts structured candidate data.
# The file is renamed to: <code>YYMMDD Full Name Degree.pdf</code>
# A <code>YYMMDD-HHMMSS_Bulk_Import.vcf</code> file is generated for contact import.

The single source of truth is the SQLite database <code>resumes.db</code>.

----

== Supported Inputs ==

{| class="wikitable"
! Format !! Handling
|-
| PDF with text || Direct extraction
|-
| Image-based PDF || <code>ocrmypdf</code> → vision OCR fallback
|-
| JPG / PNG || <code>img2pdf</code> → vision OCR fallback
|-
| DOCX || Detected; convert to PDF manually if needed
|}

----

== Quick Start ==

=== Run once manually ===

<syntaxhighlight lang="bash">
cd /path/to/resume-renamer
source ./venv/bin/activate
python cv_auto_scanner.py --once
</syntaxhighlight>

=== Dry-run preview ===

<syntaxhighlight lang="bash">
python cv_auto_scanner.py --once --dry-run
</syntaxhighlight>

=== Daemon mode ===

<syntaxhighlight lang="bash">
python cv_auto_scanner.py --daemon --interval 900 # 15 minutes
</syntaxhighlight>

=== Force a full re-scan ===

<syntaxhighlight lang="bash">
rm -f .scanner_state.json
python cv_auto_scanner.py --once
</syntaxhighlight>

----

== Common Issues ==

=== Degree shows "General" or is empty ===

The scanner now explicitly accepts diplomas and certificates in the <code>degree</code> field. If an older file still shows "General", reset it and reprocess:

<syntaxhighlight lang="bash">
source ./venv/bin/activate
python3 -c "
import resume_db
resume_db.init_db()
resume_db.upsert_candidate('ORIGINAL_FILENAME.pdf', status='pending', retry_count=0, last_error=None)
"
rm -f .scanner_state.json
python cv_auto_scanner.py --once
</syntaxhighlight>

=== "Text too short / unreadable" ===

The scanner automatically tries OCR and vision OCR. If it still fails:

<syntaxhighlight lang="bash">
source ./venv/bin/activate
python /path/to/rpdf-skills/vision_ocr.py FAILED_FILE.pdf --model glm-ocr:bf16 --format text -o extracted.txt
</syntaxhighlight>

Then rebuild a searchable PDF and re-run the scanner.

=== venv broken after Python upgrade ===

<syntaxhighlight lang="bash">
sudo apt install python3.14-venv python3-pip
cd /path/to/resume-renamer
rm -rf venv
python3.14 -m venv venv
source ./venv/bin/activate
pip install --upgrade pip
pip install img2pdf requests pdfplumber psycopg2-binary fpdf2
</syntaxhighlight>

----

== VCF Import ==

New <code>*_Bulk_Import.vcf</code> files are generated automatically. Import them into your contacts app:

* '''Google Contacts:''' contacts.google.com → Import → Select VCF
* '''Android:''' Copy VCF to phone → Contacts → Settings → Import
* '''Thunderbird:''' <code>thunderbird -compose "to=hr@example.com,subject=Contact Card,attachment=/path/to/YYMMDD-HHMMSS_Bulk_Import.vcf"</code>

The VCF uses a non-standard layout for multi-part names:

{| class="wikitable"
! vCard Field !! Value
|-
| <code>N</code> (Last Name) || Role context, e.g. <code>Electrical Applicant 260616</code>
|-
| <code>N</code> (First Name) || Candidate's full name
|-
| <code>FN</code> || <code>{Full Name} - {Context}</code>
|}

To specify the context for a batch:

<syntaxhighlight lang="bash">
python cv_auto_scanner.py --vcf-context "Electrical Applicant 260616"
</syntaxhighlight>

----

== Related Pages ==

* [[Frappe HRMS]]
* [[Frappe HRMS - Ch01 Recruitment]]
* [[Steps to Repair and OCR a Scanned or Corrupted PDF in Ubuntu]]
* [[🧠 Process: Selecting and Installing the Right Ollama Model for Your Hardware]]

[[Category:HR]]
[[Category:Recruitment]]
[[Category:Automation]]
[[Category:Comfac]]
[[Category:IT Operations]]

Comfac GPU Scaling and AI Research Goals

Justinaquino — Fri, 31 Jul 2026 13:02:32 GMT

Add Applied Model Evaluation - Synopsis mail-agent model-selection test (2026-07-31)

@@ Line 34: / Line 34: @@
 * Study and participate in open-source projects that allow community-based compute contributions (similar to Folding@home).
 * Learn and experiment with decentralized compute-sharing models that enable contributors to sell '''tokens or compute time''' securely and transparently.
 ----

Main Page

Justinaquino — Wed, 15 Jul 2026 15:21:51 GMT

Add De-Risk article link

← Older revision		Revision as of 15:21, 15 July 2026
Line 59:		Line 59:
	* [[PAD-Driven ERPNext Strategy 260303]] — Procedural Agentic Development for Frappe/ERPNext configuration and deployment		* [[PAD-Driven ERPNext Strategy 260303]] — Procedural Agentic Development for Frappe/ERPNext configuration and deployment
	* [[Directed Execution: Token-Efficient Agent Development Process]] — Director/Executor/Checker loop and skill distillation for token-efficient agent development (v0.1, 2026-07-15)		* [[Directed Execution: Token-Efficient Agent Development Process]] — Director/Executor/Checker loop and skill distillation for token-efficient agent development (v0.1, 2026-07-15)
			* [[De-Risk: Sovereign Training Data v0.1]] — Frontier-Token Quarantine and the ToS-clean path to owned model weights, prompted by the June 2026 Fable 5 export-control suspension (v0.1, 2026-07-15)
	</div>		</div>

De-Risk: Sovereign Training Data v0.1

Justinaquino — Wed, 15 Jul 2026 15:21:51 GMT

Add De-Risk: Sovereign Training Data v0.1

New page

= De-Risk: Sovereign Training Data v0.1 =
== Building Model Intelligence Without Frontier Tokens in the Corpus ==

<blockquote>'''Evaluator:''' Justin Aquino (CEO / CISO-DPO), authored as JCA</blockquote>
<blockquote>'''Evaluation Date:''' 2026-07-15</blockquote>
<blockquote>'''Methodology:''' Strategy document — synthesis of the Fable 5 export-control incident, Anthropic/OpenAI commercial-terms analysis, and the Directed Execution Process economics. Companion to <code>2026-07-15-fable-ban-sovereign-ai-and-local-model-strategy.md</code> (source report, veracity-reviewed) and <code>2026-07-15-fable-fact-check-and-csama-legal-qa-conversation.md</code> (full Q&A transcript).</blockquote>

'''Owner:''' JCNA (CEO / CISO-DPO) · '''Status:''' Draft for review · '''Date:''' 2026-07-15
'''Companion documents:''' Directed Execution Process v0.1 · Fable Ban & Local AI (veracity-reviewed 2026-07)

----

== 1. Context — What the Fable Episode Actually Proved ==

Three facts from June–July 2026 matter to us. Everything else in the source document was dramatization.

'''1.1 Frontier access is now a policy variable, not an infrastructure constant.''' A Mythos-class model went from launch to worldwide suspension in three days on a single Commerce directive, and stayed dark for 19 days. The trigger was external (a reported jailbreak escalated to Treasury), the scope was blunt (all foreign nationals, unverifiable, therefore everyone), and the reversal was equally abrupt. No SLA, no contract, no self-hosted deployment would have protected a workflow hard-wired to that model. The lesson is not "Anthropic is unreliable" — the lesson is that ''any'' frontier vendor now sits downstream of national-security discretion, and OpenAI's government-coordinated GPT-5.6 rollout confirms the pattern applies industry-wide.

'''1.2 The naive response — distill frontier outputs into your own weights — is a legal trap.''' Anthropic's and OpenAI's terms broadly prohibit using model outputs to train competing models. A pipeline that harvests Fable or Sol completions into a fine-tuning corpus trades a ''regulatory'' dependency for a ''contractual'' liability, and does so in writing, with provenance logs that would be discoverable. For an organization whose CISO-DPO signs the compliance posture, this is not a gray area we can occupy. The source document's "Distilabel ensures ToS compliance" claim was its weakest section precisely because tooling cannot launder provenance.

'''1.3 The clean asset class already exists in our practice.''' Skills, specs, checklists, and QA-derived process knowledge — instructions authored by our people, versioned in our Forgejo — transferred capability across the blackout at zero legal risk. The Directed Execution Process doc formalized this. What it did ''not'' yet do is close the loop from process knowledge to '''model weights we own'''. This document closes that loop, cleanly.

----

== 2. The De-Risk Principle: Frontier-Token Quarantine ==

One rule, enforced architecturally, resolves the entire ToS exposure:

<blockquote>'''No frontier-model-generated token ever enters the training corpus. Frontier models may shape the process; they may never author the data.'''</blockquote>

This splits our AI estate into two zones:

{| class="wikitable"
|-
! Zone !! Contents !! Frontier models (Claude/Fable, Sol)... !! Legal character
|-
| '''Process zone''' || Plans, specs, QA verdicts, RCAs, skill drafts, test designs || ...operate freely here || Ordinary licensed use — Claude helping our team work
|-
| '''Corpus zone''' || Prompts, completions, error traces, corrections, preference pairs destined for fine-tuning || ...are architecturally excluded || 100% owned: our people, our machines, open-weight models, permissive licenses
|}

The subtle failure mode to guard against: a Claude-written QA verdict or RCA is still a frontier token. If that text is pasted verbatim into a training sample, the quarantine is breached. Therefore:

* **Claude's judgments become ''signals'', not ''text''.** Pass/fail, severity, "principle violated" tags — structured fields a human confirms — may drive ''selection and labeling'' of corpus samples. Claude's prose stays in the process zone.
* '''The strongest labels don't come from Claude at all.''' They come from deterministic verifiers — tests pass, lint clean, build succeeds, output matches oracle. Claude helps ''design'' the tests; the CI system ''issues'' the labels. A label emitted by pytest has no author to have a ToS with.
* '''Skills remain process-zone artifacts.''' Instructions and context engineering are not model training; the Directed Execution loop continues unchanged. This document adds a parallel track, it does not replace that one.

----

== 3. The Error Foundry — Why the Error Is the Gold ==

Commodity training data is worthless: every lab has scraped the same internet, and open models are already trained on it. What no one else has — what cannot be scraped, bought, or distilled — is '''the distribution of failures our specific models make on our specific work, paired with verified corrections.'''

A failure trajectory is the single most valuable sample type because it encodes three things at once:

# '''Where the model's capability boundary actually is''' (not where benchmarks claim it is — recall that ~30% of SWE-bench Pro tasks are broken; our CI is not).
# '''A hard negative''' — a plausible-looking wrong answer, which is exactly what preference-optimization methods (DPO/KTO) need and exactly what synthetic "textbook" data lacks.
# '''A verified positive twin''' — once the correction lands and the verifier goes green, we hold a (failed attempt, working attempt) pair on identical context. That pair is a gradient pointing from "what our model does" to "what our work requires."

This also sidesteps the model-collapse problem the source document correctly cited: collapse comes from training on unfiltered synthetic output. Our corpus is synthetic ''attempts'' filtered by ''non-synthetic ground truth'' (executing code, real acceptance criteria, human confirmation). The verifier anchors the distribution to reality.

We already run this pattern. '''Team C at Cabuyao logs negative information from AGV runs for model training''' — this document generalizes Team C's discipline to the entire software and knowledge-work estate.

----

== 4. Pipeline Architecture ==

=== 4.1 Roles ===

{| class="wikitable"
|-
! Layer !! Actor !! Produces !! Zone
|-
| '''Direction''' || Claude (Director session) || FRD-style spec, acceptance criteria, test design, ≤2K-token plan || Process
|-
| '''Execution''' || DeepSeek / Kimi / local Qwen & DeepSeek on OpenCode (vLLM, dual RX 7900 XT) || All code, drafts, attempts — every token of it || '''Corpus'''
|-
| '''Verification''' || CI: pytest/lint/type-check/build via n8n or pre-commit || Deterministic pass/fail labels || '''Corpus''' (labels)
|-
| '''QA''' || Claude (Checker session, separate from Director) || Verdict as structured fields; prose stays out of corpus || Process
|-
| '''RCA''' || Claude + human || Root-cause analysis → feeds ''skill drafts'' and ''new test cases'', never sample text || Process
|-
| '''Correction''' || Open model (re-attempt with hint) or human dev || The corrected trajectory — corpus-eligible because its author is ours || '''Corpus'''
|-
| '''Curation''' || Interns + devs || Provenance manifest, dedup, pair assembly, PR to dataset repo || Corpus
|}

The critical asymmetry: '''Claude reads everything and authors nothing that trains.''' Its intelligence is spent on specs, test oracles, verdict signals, and RCA insight — all of which make the ''open'' models' next attempts better, and make the ''corpus'' labels sharper, without ever contributing a training token.

=== 4.2 The loop, end to end ===

# '''Objective in.''' Real ticket or deliberately challenging intern objective. Director-Claude produces spec + machine-checkable acceptance criteria. Spec is stored in Forgejo (process repo).
# '''Open model executes.''' Full trajectory logged: prompt, context, every attempt, tool calls, outputs. This logging is non-negotiable — the exhaust ''is'' the product.
# '''Verifier fires.''' Fail → the failure trace is tagged and banked (this is the gold, not discarded). Pass → candidate positive.
# '''On failure:''' RCA (Claude + human) identifies the principle violated → two outputs: (a) a ''skill draft'' or new test case (process zone), (b) a ''hint'' given to the open model for a re-attempt. The re-attempt that passes becomes the positive twin.
# '''Pair assembly.''' (context, failed, passed, verifier evidence, provenance manifest) → PR into the dataset repo. Intern authors it; dev reviews it; Forgejo history is the audit trail and the intern's portfolio.
# '''Periodic fine-tune.''' When a domain accumulates enough pairs (target: 500–1,000 verified pairs before first run), fine-tune Qwen/DeepSeek variants on the 7900 XT rig (LoRA/QLoRA first — full fine-tunes are not needed to capture correction patterns). Evaluate against a held-out set of ''our own'' past failures.
# '''Promotion.''' A tuned model earns executor duty only by beating the base model on the held-out failure set and on live first-pass acceptance rate — the same metric already in the Directed Execution scoreboard.

=== 4.3 Provenance manifest (per sample, mandatory) ===

<syntaxhighlight lang="text">
sample_id: uuid
generator: {model: qwen2.5-coder-32b, host: PC00-vllm, license: apache-2.0}
context_source: {ticket: FORGEJO-####, spec_hash: ..., spec_author: human|claude-process-zone}
label: {verifier: pytest@ci, result: fail|pass, evidence: run_url}
frontier_tokens_present: false # hard gate; CI rejects true
human_touch: {curated_by: intern_id, reviewed_by: dev_id}
correction_lineage: failed_sample_id # for positive twins
</syntaxhighlight>

<code>frontier_tokens_present: false</code> is enforced by a pre-merge check on the dataset repo: samples whose context includes Claude-session exports are rejected mechanically, not by policy memo. Same philosophy as Synopsis — deterministic rules first.

=== 4.4 Storage & plumbing ===

* '''Trajectory log:''' SQLite first (Synopsis pattern), one DB per domain, schema versioned in Forgejo. Graduate to ERPNext DocTypes only when the team needs commission-model-style visibility.
* '''Dataset repo:''' dedicated Forgejo org, one repo per domain (<code>ds-coding</code>, <code>ds-erpnext</code>, <code>ds-docs</code>), PR-gated, LFS for traces.
* '''Orchestration:''' n8n handles verifier dispatch and label writes; no human copies test results by hand.
* '''Compute:''' existing dual RX 7900 XT / vLLM stack for both inference and LoRA training; no new capex required for Phase 1–2.

----

== 5. Risk Register — What This De-Risks and What It Doesn't ==

{| class="wikitable"
|-
! Risk !! Naive distillation !! This architecture
|-
| Frontier ToS breach (training competing models) || High — corpus is frontier tokens || '''Eliminated''' — corpus is 100% own/open provenance, mechanically enforced
|-
| Frontier blackout (Fable-type event) || Capability frozen at last distill || Skills + tuned weights both survive; Claude roles degrade gracefully to human plan/QA (slower, not broken)
|-
| Model collapse from synthetic data || High — unfiltered frontier outputs || Low — every sample anchored by deterministic verifier; failures kept as hard negatives
|-
| Vendor lock-in of process knowledge || N/A || Skills in model-portable SKILL.md format; dataset in open formats; weights are Apache/MIT-base
|-
| Data privacy (RA 10173) || Frontier API sees raw work product || Secrets/prod data → '''local agents only''' (existing hard rule, unchanged); corpus lives on-prem; CISO-DPO sign-off on any sample class containing personal data
|-
| Benchmark self-deception || Marketing benchmarks || Held-out set of our own historical failures — unfakeable, directly relevant
|}

'''Residual risks, stated honestly:'''
* ''Judge-influence gray zone.'' Even using Claude's pass/fail ''signals'' to select samples arguably "assists" model development. Our mitigation is that the load-bearing labels are deterministic-verifier labels, humans confirm every Claude-derived tag, and no frontier text enters the corpus. This is a defensible posture, not a court-tested one; counsel (BBLAW) should sanity-check the framing once, briefly.
* ''LoRA ceiling.'' Correction-pattern LoRAs will not make a 32B model a Mythos-class reasoner. That is fine — the Directed Execution economics never asked it to. The 5% burst path to frontier APIs remains for genuinely hard problems, in the process zone where it belongs.
* ''Curation labor.'' The pipeline lives or dies on logging discipline and intern throughput. Hence §6.

----

== 6. Call to Action ==

'''This week'''
# '''CEO/CISO-DPO:''' ratify the Frontier-Token Quarantine as policy (one page, references the transcription-tool memo as precedent for "approved-tool" governance style).
# '''Sysadmin + Carl:''' stand up the dataset Forgejo org, SQLite trajectory schema, and the <code>frontier_tokens_present</code> pre-merge check. Half a day of work; it makes the policy mechanical.
# '''Devs:''' switch OpenCode local-agent sessions to full trajectory logging ''now'', even before curation starts. Every unlogged failure this month is discarded gold.

'''Weeks 2–4'''
# '''Interns (all, not just dev interns):''' first curation objectives issued — assemble 50 verified pairs each from live failure logs, PR'd to the dataset repo. The curriculum ''is'' the loop: they learn spec-reading, verifier evidence, and provenance hygiene simultaneously. Scoreboard: pairs merged, review-pass rate.
# '''Team C liaison:''' map the AGV negative-information logging schema onto the manifest in §4.3 so robotics and software share one provenance convention.
# '''BBLAW (existing engagement):''' 30-minute review of the quarantine framing and the judge-signal question. Not a new engagement — an agenda item.

'''Month 2–3'''
# First LoRA run on the coding domain once ≥500 pairs verified. Evaluate on held-out failure set. Publish results internally — win or lose, the eval harness itself is a durable asset.
# '''Decision gate:''' if tuned executor beats base on first-pass acceptance, promote it in the Directed Execution rotation and begin domain #2 (ERPNext customization is the obvious candidate — our failure data there is uniquely ours).

----

== 7. The Strategic Frame ==

Two tracks, different clock speeds, one direction:

* '''Skills (fast, instant transfer):''' capability as ''instructions'', portable across any executor, already running. Survives blackouts by construction.
* '''Weights (slow, compounding):''' capability as ''parameters we own'', grown exclusively from our own error distribution, legally unencumbered because the corpus never contained a frontier token.

Claude's highest-value role in both is the same: not as the source of answers to memorize, but as the intelligence that designs the tests, reads the failures, and sharpens the questions — while our models, our people, and our verifiers author every byte we will ever train on.

The Fable ban gave us the precedent. The error logs give us the data. The quarantine gives us the clean title.

''The error is the gold. Log everything.''

Main Page

Justinaquino — Wed, 15 Jul 2026 11:02:55 GMT

Add Directed Execution article link

← Older revision		Revision as of 11:02, 15 July 2026
Line 58:		Line 58:
	* [[OpenCode Multi-Provider Configuration Guide 260401]] — Dual-mode AI coding setup with Kimi cloud + Ollama local models via Distrobox		* [[OpenCode Multi-Provider Configuration Guide 260401]] — Dual-mode AI coding setup with Kimi cloud + Ollama local models via Distrobox
	* [[PAD-Driven ERPNext Strategy 260303]] — Procedural Agentic Development for Frappe/ERPNext configuration and deployment		* [[PAD-Driven ERPNext Strategy 260303]] — Procedural Agentic Development for Frappe/ERPNext configuration and deployment
			* [[Directed Execution: Token-Efficient Agent Development Process]] — Director/Executor/Checker loop and skill distillation for token-efficient agent development (v0.1, 2026-07-15)
	</div>		</div>

Directed Execution: Token-Efficient Agent Development Process

Justinaquino — Wed, 15 Jul 2026 11:02:54 GMT

Add Directed Execution: Token-Efficient Agent Development Process v0.1

New page

= Directed Execution: Token-Efficient Agent Development Process =
'''CGG IT — Process & Skillset Definition v0.1'''
Owner: JCA · Status: Draft for team review · Date: 2026-07-15

----

== 1. Problem Statement ==

Developers are consuming Claude Sonnet and Opus tokens for raw code generation. This inverts the economics of the model market. Frontier-model output tokens are the single most expensive resource in the stack (Opus 4.8 output: ₱1,450/M; Fable 5: ₱2,900/M), while commodity-model output is nearly free by comparison (DeepSeek V4 Flash: ₱16/M — a 90× to 180× spread). Meanwhile, frontier-model ''input'' tokens are roughly 5× cheaper than their own output tokens, and cache hits cheaper still.

The consequence is a simple design rule that the current workflow violates:

<blockquote>'''Expensive models should read a lot and write a little. Cheap models should write a lot and be corrected.'''</blockquote>

Nobody has directed the developers to work this way, and there is no shared mechanism for making the cheap models better over time. Both gaps are addressed here: a role-separated loop for day-to-day work, and a skill system that converts QA corrections into reusable capability for the cheap agents.

== 2. Economic Rationale (worked example) ==

Assume a typical feature task: ~30K tokens of relevant context, ~50K tokens of generated code and tests.

'''Current pattern — Opus does everything:'''
30K in (₱8.70) + 50K out (₱72.50), plus one revision cycle of similar size ≈ '''₱150–160 per task''', with no independent QA gate.

'''Directed pattern:'''

{| class="wikitable"
|-
! Step !! Model !! Tokens !! Cost
|-
| Plan + FRD || Opus 4.8 || 30K in / 2K out || ₱11.60
|-
| Execute || DS V4 Flash || 35K in / 50K out || ₱1.08
|-
| QA review || Opus 4.8 || 60K in / 1K out || ₱18.85
|-
| Correction || DS V4 Flash || 40K in / 20K out || ₱0.64
|-
| Re-check || Opus 4.8 (cached ctx) || 25K in / 0.5K out || ~₱8
|-
| '''Total''' || '''≈ ₱40'''
|}

Roughly a 70–75% cost reduction ''while adding a QA gate that did not previously exist''. The QA read is the largest line item, which is correct — that is where the intelligence is actually needed. Substituting Sonnet as the checker for routine tasks pushes the total under ₱20. As skills accumulate and first-pass acceptance rises, the correction cycle amortizes toward zero.

The point is not merely savings. It is that '''judgment and volume are different products''', and we have been buying volume at judgment prices.

== 3. Principles ==

# '''Direction is the deliverable of intelligence.''' The frontier model's job is decomposition, constraints, acceptance criteria, and verdicts — all short, dense outputs. If Claude is emitting more than ~2K tokens on a task, something is misrouted.
# '''Deterministic-first, same as Synopsis.''' Anything checkable by a linter, test suite, type checker, or script never reaches a paid model. Machine checks run before Claude sees the diff.
# '''Every correction is an asset.''' A QA correction consumed once is an expense; distilled into a skill, it is capital. The loop is not complete until the correction is either distilled or explicitly judged one-off.
# '''Cheap agents are pushed, not babysat.''' Executors get hard objectives with clear acceptance criteria and their accumulated skill library — not hand-holding. Failures are information for the skill system.
# '''The loop is the curriculum.''' Interns learn the same process the agents run: plan, execute, check, correct, distill. Team-wide token efficiency is a literacy goal, not a dev-only optimization.

== 4. Roles ==

{| class="wikitable"
|-
! Role !! Model tier !! Reads !! Writes !! Output budget
|-
| '''Director''' || Claude Opus / Fable || Objective, codebase context, skill index || FRD: decomposition, interfaces, acceptance criteria, risks || ≤ 2K tokens
|-
| '''Executor''' || DS V4 Flash, Kimi K2.6; later local Qwen/DeepSeek via OpenCode || FRD + relevant skills + code context || Code, tests, migration scripts — full volume || Unbounded
|-
| '''Checker''' || Claude Opus (critical) / Sonnet (routine) || Full diff + FRD + machine-check results || Verdict: PASS/FAIL per acceptance criterion, itemized corrections anchored to lines || ≤ 1K tokens
|-
| '''Skill Librarian''' || Human (rotating dev/intern) + cheap model drafting || QA correction logs || Skill drafts, skill index maintenance || —
|}

Director and Checker should be '''separate sessions''' even when the same model — the Checker must evaluate against the FRD, not defend its own plan.

Model routing within tiers: DeepSeek V4 Flash is the default executor. Kimi K2.6 when the task needs multimodal input (screenshots, UI review) or its larger instant-mode quality helps. Local agents take over as the vLLM/Ollama stack proves out on benchmarked task classes — the skill library transfers with them, which is the strategic payoff: skills are model-portable capability, not vendor lock-in.

== 5. The Loop ==

'''Plan → Execute → Machine-check → QA → Correct → Distill → Reuse'''

'''Plan.''' Director receives the objective and produces an FRD-lite (template below). No code beyond interface signatures and test-case ''descriptions''. The FRD is the contract; ambiguity discovered later is a planning defect and goes back to the Director, not patched silently by the Executor.

'''Execute.''' Executor works in OpenCode with the FRD and the relevant skills loaded (skills prepended as a stable prefix — this also maximizes cache hits). It generates code and tests to satisfy the acceptance criteria. It may burn tokens freely; that is what it is for.

'''Machine-check.''' Lint, type-check, test run, formatter. Automated (n8n or a pre-commit script). Failures loop back to the Executor without any paid QA read. Claude never reviews code that fails machine checks.

'''QA.''' Checker reads FRD + diff + machine-check results and returns a structured verdict. It does not rewrite code. Each FAIL item states ''what is wrong, why, and the principle violated'' — the "why" is raw material for distillation.

'''Correct.''' Executor applies corrections. Two correction cycles maximum; a third failure escalates to a human plus a mandatory skill-gap review ("what skill would have prevented this?").

'''Distill.''' Weekly, the Skill Librarian reviews the QA correction log. Any correction pattern appearing '''twice or more''' becomes a skill candidate. A cheap model drafts the skill from the correction transcripts; a human reviews and merges it into the Forgejo skill repo.

'''Reuse.''' Skills are indexed by trigger. The Director's FRD names which skills apply; the Executor loads them. Cheap agents therefore climb: yesterday's correction is today's default behavior.

== 6. Deliverable Contracts ==

=== FRD-lite (Director output) ===

<syntaxhighlight lang="text">
FRD-<seq>: <title>
Objective: one sentence.
Scope in / Scope out: explicit exclusions.
Interfaces: function/API signatures the Executor must implement.
Acceptance criteria: numbered, each machine- or QA-checkable.
Constraints: security, style, dependency, and performance rules that apply.
Skills to load: skill IDs from the index.
Risk notes: where the Executor is most likely to go wrong.
</syntaxhighlight>

=== QA Verdict (Checker output) ===

<syntaxhighlight lang="text">
VERDICT: PASS | FAIL
AC-1: PASS/FAIL — <one line>
AC-n: ...
Corrections (if FAIL):
C1 [file:line] What / Why / Principle violated
Distill flags: corrections likely to recur → candidate skill name
</syntaxhighlight>

The "Principle violated" field is deliberately mandatory. It is what makes the verdict distillable rather than a one-off patch instruction.

== 7. The Skill System ==

A skill is a short markdown file teaching a cheap model a specific competence, in the same shape Anthropic uses for Claude skills — which means skills built here also work when the executor ''is'' Claude, or a local model, or whatever comes next.

'''Format''' (<code>SKILL.md</code> per skill, one directory per skill in the Forgejo repo, e.g. <code>skills/erpnext-patch-migrations/</code>):

<syntaxhighlight lang="text">
name: <kebab-case>
trigger: when this skill applies (used by Director for routing)
procedure: numbered steps / rules
examples: one good, one bad (drawn from real QA corrections)
anti-patterns: the specific failures that birthed this skill
provenance: FRD/QA references, author, date
status: draft | reviewed | adopted | deprecated
</syntaxhighlight>

'''Lifecycle.''' Draft (from correction log, cheap-model-written) → Reviewed (human, PR in Forgejo) → Adopted (in the index, loadable) → Deprecated (superseded or model-obsoleted). Skills are versioned like code because they are code — for the agents.

'''Ownership.''' Devs are consumers and reviewers, not the bottleneck. Interns, working with the local DS agents in OpenCode, author most drafts as part of their objectives. The repo's PR history doubles as an intern portfolio.

== 8. Escalation Rules ==

{| class="wikitable"
|-
! Situation !! Route
|-
| Routine feature/bugfix, skills exist || Flash executes, Sonnet checks
|-
| Novel architecture, security-sensitive, cross-system design || Opus/Fable plans and checks
|-
| Executor fails 2 correction cycles || Human review + skill-gap analysis
|-
| Executor requests clarification || Back to Director (cheap→expensive calls are Director-mediated, never ad hoc)
|-
| Anything touching secrets, credentials, prod data || Local agents only; no external API
|}

The last row is a hard rule and aligns with the transcription-tool memo posture: sensitive context stays on the ZeroTier overlay.

== 9. Metrics & Token Ledger ==

Per task, logged (SQLite to start, consistent with the Synopsis MVP philosophy; ERPNext later if it earns its keep):

* Tokens and ₱ by role (Director / Executor / Checker), per task and per week.
* '''First-pass acceptance rate''' — the headline metric. Rising FPA means skills are working.
* '''Correction cycles per task''' and '''escalation rate'''.
* '''Skill reuse count''' and '''skills authored''' per person — the intern scoreboard.
* '''Cost per merged deliverable''', trended. Target: sustained <30% of the all-frontier baseline within one quarter.

The ledger is not surveillance; it is the feedback signal that tells us when a task class is ready to move from Flash to a local model, and when a checker can be downgraded from Opus to Sonnet.

== 10. Rollout ==

'''Week 1 — Instrument and declare.''' Stand up the skill repo in Forgejo, the token ledger, and the machine-check gate. Short directive to the team: no frontier-model code generation for tasks with an FRD path; frontier models plan and check.

'''Week 2 — Seed skills.''' Mine the last month of Claude chat history for recurring correction patterns; interns draft the first 5–10 skills (likely candidates: ERPNext patch conventions, Frappe API idioms, our lint/commit standards, test scaffolding).

'''Week 3 — First directed sprints.''' Each intern gets one challenging objective run fully through the loop with a local DS agent in OpenCode. Devs act as Skill Librarian reviewers. Every completed objective must ship at least one skill PR.

'''Week 4 — Review.''' FPA, cost per deliverable, and skill count reviewed against baseline. Decide the first task class to pilot on fully local models.

'''Steady state.''' The loop is the standard operating rhythm. Quarterly, revisit model routing (the pricing table this document was built from will be stale in months) and prune deprecated skills.

----

''Reasoning summary: the spread between frontier output pricing and commodity output pricing (90×+) makes "smart model writes code" the single most expensive habit in the workflow, while the input/output asymmetry (5×) makes "smart model reads everything, writes verdicts" nearly free by comparison. The skill system converts the residual cost — QA corrections — into permanent, model-portable capability, so the cheap tier's quality compounds and the expensive tier's involvement shrinks over time. Interns are the labor force and the beneficiaries of the same loop.''

Tutorial: Exposing a Dockerized Web App via NPM Direct DB Config

Justinaquino — Sat, 11 Jul 2026 12:56:50 GMT

Uploaded tutorial

New page

__NOTOC__

<div style="background-color: #e7f3fe; border-left: 6px solid #2196F3; padding: 15px; margin-bottom: 20px;">
<strong>Tutorial:</strong> Exposing a Dockerized Web App via NPM Direct DB Config<br>
<strong>Audience:</strong> Self-hosters who need to publish a Dockerized service through Nginx Proxy Manager without admin-panel credentials.<br>
<strong>Last updated:</strong> 2026-07-11
</div>

= My Dockerized Web App — NPM Proxy Host Setup (Direct DB Method) =

'''Scope:''' Expose a Dockerized internal web application through Nginx Proxy Manager (NPM) on <docker-host> without using the NPM Admin UI, because NPM admin-panel credentials are not stored in the BRC credential store.

'''Outcome:''' <code>http://app.example.org</code> resolves to the analyzer on <docker-host> (<code><npm-host-zerotier-ip>:<app-port></code>) via NPM proxy host <code><proxy-host-id></code>. HTTPS is pending the manual SSL-certificate step in the NPM UI.

----

== What You Need ==

{| class="wikitable"
|-
! Item
! Value / Location
|-
| Target domain
| <code>app.example.org</code>
|-
| Backing service
| <code><npm-host-zerotier-ip>:<app-port></code> (<docker-host> ZeroTier, <code>myapp</code> container)
|-
| NPM host
| <docker-host> (<code><npm-host-local-ip></code> local / <code><npm-host-zerotier-ip></code> ZeroTier)
|-
| NPM DB container
| <code>nginx-proxy-manager-db</code>
|-
| NPM app container
| <code>nginx-proxy-manager-app</code>
|-
| NPM DB credentials
| <code>npm</code> / <code>npm</code> (hardcoded in NPM stack)
|-
| <docker-host> SSH credentials
| <code><credential-store>/<docker-host></code>
|-
| Tooling on the agent side
| <code>sshpass</code> (<code>command -v sshpass</code> to verify)
|}

----

== Step 1 — Deploy the Application ==

The analyzer runs as a Docker Compose stack on <docker-host>:

<pre>
# On <docker-host>
/opt/myapp/docker compose up -d
</pre>

It exposes host port <code><app-port></code> and has a one-minute cron sync that pulls <code>main</code> and restarts the container when the repo changes.

Verify it responds locally:

<pre>
curl http://127.0.0.1:<app-port>/api/health
# expected: {"status":"ok","authenticated":false}
</pre>

----

== Step 2 — Set Up the SSH Helper ==

From the workspace, load <docker-host> credentials from BRC and define a one-line SSH wrapper:

<pre>
WORKDIR=$(git rev-parse --show-toplevel 2>/dev/null || pwd)
BRC="$WORKDIR/<credential-store>/<docker-host>"
PASS=$(grep 'password:' "$BRC" | awk '{print $NF}')
HOST=$(grep 'zerotier_ip:' "$BRC" | awk '{print $NF}')
<docker-host>_USER=$(grep -m1 'user:' "$BRC" | awk '{print $NF}')

<docker-host>_SSH() {
sshpass -p "$PASS" ssh \
-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
"$<docker-host>_USER@$HOST" "$@"
}
</pre>

If <code>sshpass</code> is missing, install it first (<code>apt-get install -y sshpass</code> or equivalent). Do '''not''' treat a missing <code>sshpass</code> as "no SSH access."

----

== Step 3 — Check for Existing Hosts ==

List any existing proxy host that uses the target domain:

<pre>
<docker-host>_SSH "docker exec nginx-proxy-manager-db mysql -unpm -pnpm -h127.0.0.1 npm -e \\"
SELECT id, domain_names, forward_host, forward_port, enabled
FROM proxy_host
WHERE domain_names LIKE '%app.example.org%' AND is_deleted=0;
\\""
</pre>

If a row exists but points to the wrong target, update it or delete it before inserting a duplicate.

----

== Step 4 — Back Up the <code>proxy_host</code> Table ==

Always backup before writing:

<pre>
<docker-host>_SSH "docker exec nginx-proxy-manager-db mysqldump -unpm -pnpm -h127.0.0.1 npm proxy_host" \
> "/tmp/npm-proxy-host-backup-$(date +%Y%m%d-%H%M%S).sql"
</pre>

<blockquote>'''Note:''' Use <code>-h127.0.0.1</code>; the default socket connection may reject the <code>npm</code> user.</blockquote>

----

== Step 5 — Insert the Proxy Host Row ==

Insert an HTTP-only row (<code>certificate_id=0</code>, <code>ssl_forced=0</code>). SSL is requested later from the NPM UI.

<pre>
DOMAIN="app.example.org"
TARGET_IP="<npm-host-zerotier-ip>"
TARGET_PORT="<app-port>"

<docker-host>_SSH "docker exec nginx-proxy-manager-db mysql -unpm -pnpm -h127.0.0.1 npm -e \\"
INSERT INTO proxy_host
(created_on, modified_on, owner_user_id, domain_names, forward_host, forward_port,
access_list_id, certificate_id, ssl_forced, caching_enabled, block_exploits,
advanced_config, meta, allow_websocket_upgrade, http2_support, forward_scheme, enabled, locations,
hsts_enabled, hsts_subdomains, trust_forwarded_proto)
VALUES
(NOW(), NOW(), 1, '[\\\\\"${DOMAIN}\\\\\"]', '${TARGET_IP}', ${TARGET_PORT},
0, 0, 0, 0, 0,
'',
'{\\\\\"nginx_online\\\\\":true,\\\\\"nginx_err\\\\\":null}',
0, 0, 'http', 1, '[]',
0, 0, 0);
\\""
</pre>

Record the <code>id</code> returned by the next SELECT; for this host it is <code><proxy-host-id></code>.

----

== Step 6 — Archive Any Manual Config File ==

If a previous workaround created a hand-edited <code>.conf</code> file in <code>/data/compose/3/data/nginx/proxy_host/</code>, move it out of nginx's include path so it does not conflict with the DB-generated config. The proxy_host directory is owned by root, so use a privileged throwaway container:

<pre>
<docker-host>_SSH "docker run --rm \
-v /data/compose/3/data/nginx/proxy_host:/proxy \
alpine sh -c 'test -f /proxy/<manual-config>.conf && mv /proxy/<manual-config>.conf /proxy/<manual-config>.conf.manual-backup-$(date +%Y%m%d-%H%M%S) || true'"
</pre>

----

== Step 7 — Restart the NPM App Container ==

Restarting regenerates the nginx configs from the DB:

<pre>
<docker-host>_SSH "docker restart nginx-proxy-manager-app"
sleep 5
</pre>

----

== Step 8 — Force Config Generation If Needed ==

Direct DB inserts create the row, but NPM's internal config generator may not write <code>proxy_host/<id>.conf</code> until the host is created or updated through the app. If the config file is missing, force it from inside the app container:

<pre>
HOST_ID=<proxy-host-id>

<docker-host>_SSH "docker exec nginx-proxy-manager-app sh -c 'cat > /tmp/regen-proxy-host.mjs <<EOF
import proxyHostModel from \\"/app/models/proxy_host.js\\";
import internalNginx from \\"/app/internal/nginx.js\\";

const host = await proxyHostModel.query().where(\\"id\\", ${HOST_ID}).first();
if (!host) {
console.error(\\"proxy_host ${HOST_ID} not found\\");
process.exit(1);
}
await internalNginx.configure(proxyHostModel, \\"proxy_host\\", host);
console.log(\\"Regenerated config for\\", host.domain_names);
EOF
node /tmp/regen-proxy-host.mjs'"
</pre>

This writes the config file, tests nginx, updates the row's <code>meta</code>, and reloads nginx.

----

== Step 9 — Verify ==

Check the generated config:

<pre>
<docker-host>_SSH "ls -la /data/compose/3/data/nginx/proxy_host/<proxy-host-id>.conf"
<docker-host>_SSH "cat /data/compose/3/data/nginx/proxy_host/<proxy-host-id>.conf"
</pre>

Test nginx syntax:

<pre>
<docker-host>_SSH "docker exec nginx-proxy-manager-app nginx -t"
</pre>

Test end-to-end HTTP (from <docker-host>, because public DNS may not be live yet):

<pre>
<docker-host>_SSH "curl -fsS -H 'Host: app.example.org' http://127.0.0.1:80/api/health"
# expected: {"status":"ok","authenticated":false}
</pre>

Once Cloudflare DNS points <code>app.example.org</code> at the <vps-relay> VPS relay, <code>http://app.example.org/</code> will work publicly.

----

== Step 10 — Request the SSL Certificate (Manual NPM UI Step) ==

A DB insert cannot complete an ACME DNS-01 challenge. The final certificate step must be done in the NPM Admin UI:

# Open <code>http://<npm-host-zerotier-ip>:81</code> (<docker-host> ZeroTier) or your NPM admin URL.
# '''Hosts → Proxy Hosts''' → click <code>app.example.org</code>.
# Go to the '''SSL''' tab.
# '''SSL Certificate → Request a new SSL Certificate'''.
# Enable '''Force SSL''', '''HTTP/2 Support''', and '''HSTS''' as desired.
# Agree to the Let's Encrypt ToS and request the certificate.

After the certificate issues, NPM updates <code>proxy_host.certificate_id</code> and <code>ssl_forced</code> automatically and regenerates <code><proxy-host-id>.conf</code> with HTTPS listeners.

----

== Troubleshooting ==

{| class="wikitable"
|-
! Symptom
! Cause
! Fix
|-
| <code>Access denied for user 'npm'@'localhost'</code> during backup
| <code>mysqldump</code> used socket instead of TCP
| Add <code>-h127.0.0.1</code>
|-
| Row exists but no <code><id>.conf</code> after restart
| Direct DB insert did not trigger config generator
| Run the Node force-regeneration script in Step 8
|-
| <code>nginx: [emerg] duplicate server_name</code>
| Old manual <code>.conf</code> still in <code>proxy_host/</code>
| Archive/remove it with a privileged container (Step 6)
|-
| <code>curl</code> returns 404 or no response
| DNS not pointing at NPM yet, or wrong <code>forward_host</code> / <code>forward_port</code>
| Verify DNS A record and the <code>proxy_host</code> row
|-
| Cert request fails
| Let's Encrypt rate limit or Cloudflare API token missing
| Check NPM SSL logs; use the unpause link if rate-limited
|}

----

== References ==

* Reusable NPM direct-DB skill: <code>tools/IT-knowledge/skills/npm-direct-db-modification/SKILL.md</code>
* Analyzer repo: <code><your-app-repo></code>
* NPM Proxy Map in orientation: <code><orientation-doc>.md</code> § Published Sites
* Cloudflare/VPS relay notes: <code>tools/IT-knowledge/networking/NPM_Migration_to_Homelab_VPS_Relay_260401.mw</code>

[[Category:Tutorial]]
[[Category:Nginx Proxy Manager]]
[[Category:Docker]]
[[Category:DevOps]]