This is the second post in a two-part series on running AI agents in production engineering teams at Strike. In Building the Strike Software Factory, we covered the initial shift towards upfront alignment, vertical slices, and managing review bottlenecks. Part II explores the economics and architecture of the factory itself - contrasting deterministic build gates with non-deterministic guidance, measuring skills with evals, and maintaining the repository environment as durable institutional capital.

An agent consists of a model, harnessed, operating inside an environment. The harness is the scaffold provided by a vendor - Claude Code, for example - that turns a raw model into an agent capable of taking actions. We inherit the harness and rent the model, but we build the environment. Specifically, we shape that environment into a software factory: a workspace optimised specifically for producing reliable software.

The Economics of the Factory

Every software factory relies on two distinct mechanisms to enforce quality: deterministic gates and non-deterministic guidance.

• Deterministic gates are rule-based, programmatic constraints - such as compilers, linters, static analysers, and automated test suites - where code either passes or hard-fails without probabilistic reasoning.
• Non-deterministic guidance consists of natural language instructions, agent skills, and prompt guardrails that nudge a model towards preferred engineering decisions.

These mechanisms create two entirely different cost profiles:

A deterministic gate represents a fixed cost: you invest once to build the analyser or establish the rule, and it runs indefinitely for free without a model in the loop. A non-deterministic check represents a marginal cost: every execution consumes tokens, scaling directly with the capability of the model and the frequency of runs.

This distinction reframes engineering trade-offs. Converting recurring marginal costs into one-off fixed investments creates immediate operational leverage. Where human or model evaluation remains necessary, routing simpler judgment passes to the cheapest viable model keeps token spend proportional to task complexity.

Human review time is the most expensive line of all. It doesn't scale, it doesn't parallelise cheaply, and human attention remains strictly finite.

Investing in fixed-cost deterministic gates keeps token spend predictable, but more importantly, it ensures PRs arrive clean so human review is spent on architectural intent rather than mechanical validation.

Coercion vs Enforcement

This economic distinction leads directly to an operational reality: non-deterministic guidance is coercion, whereas deterministic tooling is enforcement.

Agents arrive with distinct defaults - habits learned from training data and reinforced by existing repository patterns. For example, agents naturally lean towards generating explanatory comments because documentation is heavily represented in training corpuses. Prompting a skill with "do not write comments" is coercion; you are pushing against an inherent default that tends to resurface. Deterministic enforcement eliminates the habit entirely by removing the option, making compliance the only path to a green build.

Case Study: Comments and Whack-a-Mole

Managing comments illustrates the limits of coercion. When we relied on prompt rules to restrict comments, it turned into an ongoing game of whack-a-mole: the agent still attempted to explain logic, switching to inline comments that mimicked docstrings without technically violating the prompt.

A deterministic gate resolves the cycle permanently. Enforcing a build-blocking rule for missing documentation turns compliance into a hard requirement: public interfaces must either be documented or made private to achieve a green build. Missing docs fail locally via pre-commit hooks and hard-fail in CI, guaranteeing consistent codebase standards, better editor autocompletion, and clean PRs on arrival.

Consistency beats correctness-of-choice. Whether a team mandates documentation everywhere or forbids it entirely matters far less than picking a standard and enforcing it deterministically. Once the team agrees on the convention, encoding it into the build permanently eliminates ongoing debate.

Building the Assembly Line

The shift from prompt coercion to deterministic enforcement extends well beyond code comments into fundamental system architecture. Take intra-service messaging over an event bus as an example: certain message patterns are objectively invalid and fail in production. Catching these in code review and writing guidance skills helped initially, but the failure mode persisted until we built custom static analysis rules to enforce correct patterns at compile time.

Other patterns require no subjective interpretation, such as class inheritance rules. Establishing a standard that classes remain sealed or final by default allows compile-time enforcement through linter rules. Sweeping the repository once aligns the codebase, permanently eliminating the need for engineers or agents to deliberate on class inheritance.

The Limits of Determinism

Determinism has clear boundaries. A compiler can enforce that a docstring exists, but it cannot verify whether the explanation is useful or merely repeats the function signature. Trivial comments add noise without value.

The fix is straightforward: a deterministic gate ensures comments exist, while a lightweight review agent evaluates whether they actually add value. Because this check is scoped strictly to clarity and utility, it runs cost-effectively on cheaper model tiers without requiring frontier models.

Anatomy of the Factory

Maintaining the software factory involves continuous updates across skills, documentation, guardrails, and orchestration. Rather than an ad-hoc collection of tweaks, these improvements cluster into six distinct layers:

• The process spine - operational skills that define how work gets executed (such as an /implement or /submit-for-review workflow), definition-of-ready checks, and review gates.
• Domain guidance - hard-won, service-specific context currently captured in instructions and targeted for future deterministic enforcement.
• Deterministic gates - static analysers, linters, git hooks, and CI checks that eliminate mistake classes without requiring model inference.
• Orchestration - goal prompts, stop conditions, and strategies for coordinating parallel agent execution.
• Corpus hygiene - de-duplicating and refining skills and docs to ensure agents reason over unambiguous, non-contradictory instructions.
• Fold-back - incorporating learnings and spec drift from completed runs back into the foundation.

Proximity to the Keystroke

The organising principle for where a check lives is proximity: the closer the correction sits to the moment the mistake is made, the cheaper it is.

Correcting errors as close to the keystroke as possible minimises disruption and cost:

• As you type - analysers flag issues directly in the editor, preventing the mistake from taking root.
• On save - auto-formatters silently standardise formatting.
• Pre-commit - git hooks validate staged files before changes are committed locally.
• Pre-push - local checks verify test suites before code leaves the machine.
• Agent-hook nudge - hooks pass errors back to the agent for targeted correction during execution.
• CI hard gate - the final backstop ensuring no non-compliant code merges into main branches.

Testing Guidance with Evals

Because human review remains our most constrained resource, automating peer review is essential - but non-deterministic review agents require rigorous measurement to ensure they actually catch regressions.

Review gates act as necessary bottlenecks. In financial software, changes require independent review before merging. This standard applies equally to documentation and agent skills, ensuring compromised credentials or errant updates cannot introduce unvetted modifications into the codebase.

However, review overhead should match operational risk. While documentation and skill updates require validation, they do not carry the same blast radius as financial transaction logic. High-velocity iteration on the factory environment requires reviewing and merging low-risk tooling PRs promptly so subsequent agent runs immediately benefit from improvements.

Automated peer review reduces human overhead. For example, in an /implement workflow, an agent council can run multiple specialised review sub-agents in parallel before opening a PR. Each sub-agent evaluates code against distinct criteria and repository standards, resolving issues prior to human engagement. Human review serves as the final backstop, ensuring human attention is spent on architectural substance rather than basic linting.

Because review agents and skills belong to the non-deterministic layer, we need evals to prove they actually work. Skill updates represent testable hypotheses: that an agent will execute a task with higher reliability. Evals benchmark modifications against established baselines on identical tasks, converting subjective prompt adjustments into measurable improvements. Once validated, the new score becomes the baseline for future iterations.

Evals provide the same regression protection for prompts that automated test suites provide for code: without them, behaviour drifts; with them, quality improves.

The Long Game

Hygiene and Reconciliation

Unchecked growth turns an instruction corpus into an unwieldy maze. As skills, rules, and documents accumulate, maintaining clarity requires active subtraction: pruning redundant prompts, de-duplicating overlapping instructions, and keeping a strict boundary between operational skills and durable documentation. Corpus hygiene is fundamental factory maintenance; without it, yesterday's improvements turn into tomorrow's entropy.

Ambiguous wording acts like a buggy API. Once an orchestrator interprets it and fans out tasks across dozens of agents, ambiguities become executable errors whose costs compound. Factory modifications require the same rigorous feedback loops as production software: observe actual agent behaviours across varied tasks, then tighten and recalibrate.

Reconciliation prevents foundational decay. Following implementation and merge, realigning the original specification with the shipped code ensures documentation reflects reality. Reconciling discrepancies promptly prevents subsequent agent sessions from discovering conflicting patterns and producing misaligned solutions.

Every run builds on the last run's base - a stale base compounds.

Two complementary streams of change run continuously:

• Factory improvements: Updates to skills, static analysers, and orchestration that govern how agents construct software.
• Platform spec refinements: Updates to architecture decision records (ADRs) and ubiquitous language glossaries that define what agents build from.

Keeping both streams synchronised preserves codebase integrity.

The Diminishing Returns Curve

Refining the agent environment follows an expected diminishing-returns curve. Early investments in fundamental skills and compile-time rules yield substantial reliability gains. Over time, improvements shift towards incremental gains and defending high baseline standards.

Maturity varies across services and repositories. Mature repositories with codified domain knowledge and strict gates operate on the flatter end of the curve. Less mature services offer significant room for rapid quality gains through baseline environment investments.

The factory aims to maximise correct, reviewable software delivered within the quality and security constraints of financial software.

A faster agent that creates twice as much review work is not an optimisation.

While token spend and cycle times matter, the ultimate metric of the factory is how effectively the environment minimises review burden.

The Golden Window

Operating as a globally distributed team creates natural quiet stretches outside core collaboration hours - an ideal window for running autonomous, long-running agent swarms on deeper environment maintenance that falls outside real-time delivery cycles.

Large-scale structural updates - such as repository-wide refactors preceding new analyser rules - are well-suited for autonomous multi-agent swarms. Specifying goals and running coordinated sweeps off-hours produces stacked PRs ready for structured team review, shifting mechanical code generation outside core hours.

Optimising the Factory

Engineering focus naturally shifts from writing routine boilerplate to optimising the factory: authoring skills, establishing deterministic gates, configuring review swarms, and maintaining corpus hygiene.

This systems-level focus compounds across every engineer and agent run. While foundational models evolve and vendor harnesses change, repository-level assets - test suites, static analysers, linters, pre-commit hooks, CI gates, and codified architecture decisions - retain value across model generations.

We rent external models and inherit vendor harnesses, but we own and build the environment. Automating mechanical constraints deterministically preserves human attention for core architecture, security, and product design.