R&D · All hands
Harnesses, not prompts.
Nobody grades their own homework.
Gilad Hoch
Part one · the harness · what has already changed
“It is already clear that:
Andrew Ng — on the future of software engineering, 2026 · x.com/AndrewYNg
- (i) As AI makes coding easier, a lot more people will be doing it.
- (ii) Writing code by hand and even reading (generated) code is not that important, because we can ask an LLM about the code and operate at a higher level than the raw syntax (although how high we can or should go is rapidly changing).
- (iii) There will be a lot more custom applications, because now it's economical to write software for smaller and smaller audiences.
- (iv) Deciding what to build, more than the actual building, is becoming a bottleneck.
- (v) The cost of paying down technical debt is decreasing (since AI can refactor for you).”
What has already changed · the cost
If nobody reads the generated code, reading is no longer the safety net. Something else has to be — or there isn't one.
“Human review is being bypassed. When developers accept AI-generated code with minimal or no review, the traditional human safeguard against intent violations disappears. […] The intent gap — the distance between what a user means and what a program does — is the central reliability bottleneck.”
Shuvendu K. Lahiri, Microsoft Research — “Intent Formalization: A Grand Challenge for Reliable Coding in the Age of AI Agents”, Mar 2026 · arXiv:2603.17150
This talk rebuilds that safeguard out of something other than human attention.
What has already changed · the upside
(iv) deciding what to build
The spec becomes the scarce artifact — the thing that carries intent rather than machinery. And intent is something a human can review efficiently.
(v) cheap refactoring
Cheap refactoring makes a complexity budget enforceable. Paying down debt is no longer a year-long project — so we can finally insist the repo stays simple and clean.
What has already changed · slop
Plausible-looking. Compiles. Even passes a test or two. Wrong — and the next agent inherits it.
“I think code is not cheap. In fact, bad code is the most expensive it’s ever been. Because if you have a codebase that’s hard to change, you’re not able to take all of the bounty that AI can offer.”
Matt Pocock — “It Ain’t Broke: Why Software Fundamentals Matter More Than Ever”, AI Engineer Europe, Apr 2026 · video · 3:55
Introducing the harness
The loop · /spec-to-ship · five phases
The loop · one packet, end to end
The loop · what is actually in the repo
click · open the command
Principle 1 · the whole point
“…some of them have to do with how you keep the AI on the leash.” — Andrej Karpathy, “Software Is Changing (Again)” · video
The leash is made of things that are not us — and none of them get tired, bored, or generous on a Friday.
CRAP ≤ 30 · complexity ≤ 12Principle 2 · nobody grades their own homework
An agent whose job is code to green, and that decides whether a surviving mutant or a lint rule may be waived, is deciding inside a goal that is biased towards allowing it. A separate, read-only agent has no such goal — so it judges honestly. That is the whole of “nobody grades their own homework.”
We had it collapsed
Phase 3 wrote the code and triaged its own mutation survivors — which is how you get a 100% score that measures nothing. The judgment moved out, into a read-only phase 5.
The rule
A suppression is legitimate only when the invariant that makes the mutant equivalent is named and proved — never by what the tests happen to do. Every disable is audited: apply the mutant by hand, run the suite, restore.
Principle 3 · the panel · swiss-cheese review
Every gate has holes. Stack enough of them and the holes stop lining up. Hover a slice.
Triage every comment: fix it, or reply. Never leave one silently unresolved.
Part two
Part one was how we keep the agents honest.
This half is what it costs — and the one habit that moves it.
Mostly what and how. The why is a number, and it is coming up next.
Part two · cost · the bill
June → August 2026 · dollars from Finance, tokens from my machine.
Cost
Tokens
Models
AS BILLED · FINANCE
$2,006.01
Jun
$645.91
Jul
$210.45
Aug
| Jun | Jul | Aug | |
|---|---|---|---|
| Cursor seat | $40.00 | $40.00 | $120.00 |
| Cursor on-demand | $1,941.01 | $387.17 | $0.00 |
| Claude seat | $25.00 | $25.00 | $25.00 |
| Claude on-demand | $0.00 | $193.74 | $65.45 |
Finance · as billed
TOKENS · ALL TOOLS
2.0B
Jun
1.7B
Jul
4.5B
Aug
| Jun | Jul | Aug | |
|---|---|---|---|
| Cursor | 1.99B | 1.34B | 2.63B |
| Claude Code | 0.02B | 0.26B | 1.58B |
| Other | — | 0.09B | 0.28B |
Local telemetry · ai-usage collector
CURSOR MODEL MIX · SHARE OF TOKENS
Jun
Jul
Aug
| Jun | Jul | Aug | |
|---|---|---|---|
| sonnet + opus + fable | 92% | 30% | 1% |
| grok + composer | 6% | 46% | 99% |
| other | 2% | 23% | 0% |
Cursor export · share of tokens
Cost per billion tokens
$999 → $47
21× better — spend fell 9.5× while the work went up 2.2×.
Nothing about how I work changed. I changed which model does the grind.
click →
Part two · cost · how that was possible
Two meters run behind our tools — and they fail in opposite directions.
Seat quota · Claude Code, Cursor
Inside, free at the margin 😀.
Past it, list price 💰💰💰 — our most expensive tokens anywhere.
The wall is silent.
Metered · what we run ourselves · coming up
Small and real, every request 💵.
There is no wall at all.
↯ Anthropic models inside Cursor drain the same pool grok falls back on. Run each where it is cheap.
Cost mitigation
Three moves. Only one of them is expensive.
✗ Keep going 💰💰💰
The default — it needs no decision, so nobody makes one.
✓ Don't hop — flip 💵
One command. Same session, same context — swap what's behind it.
✓ Hop, but hand off 💵
Burn the seats you already pay for first: Claude Code → Cursor → OpenCode / pi. The artifacts are the handoff.
Decide before you're out — not after.
Valar · arm one · the wrapper
valar claude on # flip to Valar valar claude off # hand back to your seat
It re-points the tool you are
already in. Same repo, same /spec-to-ship, same guardrails.
Nothing about your workflow changes — which is the entire point.
“We are releasing next week automatic Valar Code integration with existing subscriptions, such as only extra usage goes through Valar.”
Tom Amsterdam, Valar · internal Slack, 20 Aug 2026
That is seat first, overflow metered — the right answer, enforced by plumbing instead of discipline. Chased again on 31 Aug: “the upcoming 2–3 days.”
Until it lands,
a timer is a fine substitute:
valar claude on && sleep 120m && valar claude off
One catch: the wrapper runs everything at the Now tier — the most expensive one Valar sells.
Valar · arm two · the API
An OpenAI-compatible endpoint — so it names a tier, which the wrapper never does.
Tier ladder · deepseek-v4-pro
· valar model page, 2 Sep 26
| Tier | Latency | $/M in·out | vs Now |
|---|---|---|---|
| Now | immediate | 1.056 · 3.168 | — |
| Priority | ~10 s | 0.792 · 2.376 | 1.3× |
| Standard | ≤ 1 min | 0.528 · 1.584 | 2.0× |
| Flex | ≤ 5 min | 0.370 · 1.109 | 2.9× |
| Batch | ≤ 12 h — off-gateway | 0.317 · 0.950 | 3.3× |
Same ladder on the best open weights going: GLM-5.3, Qwen-3.8-max, Kimi-K3, DeepSeek-V4.
Why this is the arm that matters
The model is identical. You are buying patience, not quality.
A reviewer that answers in five minutes instead of a second costs a third. Nobody is watching it type.
And it speaks OpenAI, so anything can sit in front of it — which is the next slide.
Wrap to save your session. Choose the tier to save money.
The plumbing · click a box
Click a box
OpenCode · where you work
A coding agent, like Cursor or Claude Code — except it talks to any inference provider out of the box, so it is not tied to one vendor's models.
LiteLLM · the gateway
Acts as our own inference provider. Behind it: other providers, proxied — or models we host ourselves. We run it centrally, so you don't have to.
Bedrock · behind it, on credits
An inference provider — AWS's. What we spend here comes out of credits we already hold.
Anthropic · behind it, metered
An inference provider — the same models your Claude Code seat runs on. Through the gateway they are metered per token, no seat required.
Valar · behind it, negotiated
An inference provider with cheaper-than-list tiers on strong open-weight models. Wait a little, pay a lot less. Both arms you just saw live behind this box.
The cheapest good-enough model changes every few weeks. You should not have to notice.
The lever · cost-adapted models, per phase
● Seat-included capacity — costs nothing marginal.
↔ Click any alias — or press R — to see what LiteLLM resolves it to right now.
One file per repo —
.harness/models.json — the only place a model may be named.
Retargeting, in practice
Model Group Alias Settings
litellm.internal.stg.versatile.ai → Models + Endpoints
| Alias | Target Model Group | Actions |
|---|---|---|
| vn-spec | mantle-gpt-5.6-sol | ✎ 🗑 |
| vn-test | bedrock-claude-opus-4-8 | ✎ 🗑 |
| vn-coding | valar-glm-5.3-flex | Save Cancel |
| vn-review | valar-qwen3.8-max-flex | ✎ 🗑 |
| vn-verify | valar-glm-5.2-flex | ✎ 🗑 |
↑ the live console — real aliases, real targets
Where to put the effort
Cheapen freely 💵
Phase 3, the grind. High volume, mechanical, and a red suite catches it if the cheap model gets it wrong.
Never cheapen
Judgment. Spec. Ideation. Nothing downstream catches a bad decision — it just gets implemented, faithfully.
The policy, in order
1. Burn the seat first — it's already paid for. 2. When it runs dry, fall through — don't hop. 3. On the metered path, cheap alias for volume, strong for judgment.
Closing · start here
1 · Point your AI at a harnessed repo
“Read the harness in
<one of our repos> and adapt it here.” Take your pick —
van-damme-slack-app or vn-server both carry one.
Don't hand-write it; review the diff.
2 · Get your Valar account
Reach out to me for an
invite. Then verify valar claude status works
today — not at the wall.
3 · Get the panel on your PR
Summon the entire review panel on your PRs — Baz, Cursor Bugbot, Copilot, @VanDamme.
In one line
An interchangeable model is the only cost lever that keeps working after the next release.
Questions — and bring your repo, we'll wire it.
Appendix · the decision ladder
Otherwise you get decision fatigue — and at 6pm you YOLO-click the recommended option a hundred times. That is not review.
The ladder, in order
No approval gates ≠ no gates
Every technical call — naming, decomposition, test structure — is the agent's to make and record. It halts for exactly one thing: a behavioural question nothing written down can settle.
“A good harness should not necessarily aim to fully eliminate human input, but to direct it to where our input is most important.” — Birgitta Böckeler
Appendix · complementary flows
/spec-to-ship
assumes a packet that is already specified. Usually a JIRA gives us that. When it doesn't,
something has to make the packet.
What I use personally
/grill-with-docs — it
interviews me until every branch of the design is resolved, and sharpens the domain language
while it does.
Out comes a packet I can hand to
/spec-to-ship.
Copy the harness. Then add the flows your work actually starts from.
Appendix · seats · the cheapest quota
A SuperGrok subscription authenticates inside OpenCode — device-code OAuth, no API key. Prepaid quota and metered fall-through, in the same tool.
What a seat costs · $ / user / month
| Seat | Price ladder | In OpenCode |
|---|---|---|
| Cursor | 40 · 120 | — |
| Claude Code | 25 · 125 | — |
| Copilot | 19 | yes ⚠ |
| Grok | 10 · 30 · 100 · 300 | yes |
Any plan with Grok API access works — grok.com/plans. SuperGrok is running 3 months for the price of 1.
⚠ the catch on Copilot
A Business seat includes 300 premium requests a month, and since 18 Nov 2025 overage is on by default at $0.04 each.
You cross from prepaid into metered without being asked. Not a wall you can feel.
Cheap seat quota first, Valar second — when it runs out you change a model, not a tool.
Appendix · sources & further reading
Every claim about our repos in this deck is checkable:
cycle-processing branch beta, and .harness/models.json in all
three services. · Logos are each vendor's own:
OpenCode,
AWS architecture icons,
LiteLLM and Valar from their sites.