Field logRFCs from production engagements
Memos, post-mortems, and methodology drops written by the engineers who shipped the work — eval suites, drift dashboards, red-team probes, and routing economics from legal, medical, financial, and insurance deployments. Nothing is published unless it cleared an adversarial second-engineer review.
02 / logrecent entries
Each entry below was authored during a live engagement, signed by the engineer who held the pager, and reviewed by a second engineer who tried to break the claim. Customer-identifying detail is redacted; the methodology and the numbers are not.
Routing layer that defers to the larger model only when first-pass confidence drops below 0.78. We share the eval matrix, the cost-per-decision math, and the two failure modes we hit before stabilising on a token-budget guard.
When a redaction agent fabricated a Bates range that did not exist, JSON-mode was the gate that caught it. We document the strict-schema harness we wrap around every legal-domain agent and the three test families it exposes.
Two leakage paths surfaced when intake summaries were pasted from upstream systems containing adversarial markdown. We publish the probe pack, the regression cases, and the policy-layer mitigation that closes both.
Perplexity flapped on holiday traffic; embedding KL did not. We share the drift dashboard, the alert thresholds we settled on, and the playbook when KL exceeds 0.05 against the production reference window.
How we propagate a single trace through router, retriever, validator, summariser, classifier, escalation, and audit agents — and how the trace becomes the contract that the eval suite, the SOC 2 auditor, and the on-call engineer all read.
We rebuilt our control evidence so that the same eval-run record satisfies the model-risk auditor, the privacy officer, and the security reviewer. The artefact, the retention policy, and the gotchas we hit are documented here.
03 / topicswhat we write about
We publish where we have shipped — and only where we have shipped. If a topic is missing, it is missing on purpose: we have not yet done the work in production to claim it.
04 / processhow a memo is born
Notes follow the same gate that production code follows at TestML: artefacts, adversarial review, redaction, and an engineer's name on the byline. No ghostwriting, no marketing rewrites, no claims without a captured eval run behind them.
Every memo begins as a production engagement. We write nothing we have not shipped behind a SOC 2-controlled boundary on a real customer workload.
Eval runs, drift snapshots, red-team probes, and routing economics are captured as artefacts during the engagement — not reconstructed afterwards from memory.
A second engineer who was not on the engagement attempts to break the claim. If the claim survives, the note moves to draft. If it does not, the note is rewritten or shelved.
We redact customer-identifying detail, keep the methodology and the numbers, and publish under the engineer who ran the work. No ghostwriting, no marketing rewrites.
We are working through a 12-month dataset of underwriting decisions, isolating the deferral threshold that holds faithfulness above 0.97 while clawing back roughly 40% of model spend. Draft is in adversarial review; numbers are final. Expected publication this quarter.
Methodology in detailA 30-minute production review with one of the engineers behind these notes. No sales pitch. We read your eval setup, your guardrails, and your incident log, and we tell you which of the surfaces above is most likely to fail you in the next ninety days.
Book a production review// engagement.start
Thirty minutes, one of our platform engineers, your real workload. You leave with a written readiness audit and the eval-run artefacts to back every line of it.
Book your readiness audit