Case study · Python · research kernel

SI-Loop

A closed-loop kernel for recursive self-improvement in autonomous software-engineering agents. Every self-modification is isolated, statically validated, tested and either promoted or atomically rolled back, on the strength of a paired statistical gate with a measured false-accept rate.

Role
Author: design, kernel, tests, paper
Status
Kernel implemented and tested. Verifier and LLM meta-agent are interfaces.
Evidence
Study 1 paper ↗ (frozen ablation), plus the Study 2 kernel demo.
7validation layers per patch
12kernel tests
≈4%A/A false-accept at n = 200
1,173lines of kernel code

01 / overview

The judge is part of the system.

A prompt-based agent that edits its own instructions and grades its own work has no fixed point. Two failures follow, and both are structural rather than model-quality problems.

Drift. Each accepted change looks plausible, but acceptance is decided on noisy evidence. Without a controlled error rate, noise-level “improvements” get promoted and the agent random-walks away from its last known-good state.

Reward hacking. The cheapest way to raise a score is to change the measurement: skip or delete tests, weaken the judge, read the oracle, or corrupt the environment the judge runs in. Judge exploitation under self-play [15] and reward-model overoptimisation [6, 4] are the same pressure in learned form.

SI-Loop treats this as a control problem. The plant is the agent codebase, the only subtree the agent may change. The actuator is a structured patch proposal, never a file write. The sensor is a sealed verifier on freshly drawn problems. The controller is a gate whose false-accept rate is measured by simulation, not assumed. The kernel enforcing all of this is immutable to the agent and digest-checked every cycle.

The gate itself came from a measured failure. In Study 1 (a toy few-shot regression), the lower-confidence-bound judge rejected about 73% of rounds, and the no_judge arm beat the full system on the hard split (final loss 0.962 vs 1.048, lower is better; Figures 1 and 2). Study 2 replaces it with a paired non-inferiority test and an explicit A/A target.

02 / architecture & trust boundaries

Three boundaries, no honour system.

Zero-trust isolation

Each cycle begins with git worktree add cand/NNNNNN <parent_sha>. The meta-agent gets read-only access to that worktree and returns a JSON proposal. If the worktree is dirty after propose(), the cycle is rejected: the agent is not allowed to write, only to ask.

The champion lives on main and moves only by fast-forward. If the candidate came from a different lineage, it must beat the current champion again before main moves. The branch is discarded in a finally block unless it was promoted or archived, so there is no path where a failed cycle leaves state behind.

ZoneMay writeEnforcement
Kernelnobody at run timeKernelGuard hashes kernel/*.py (SHA-256) each cycle; any change aborts the run
agent/**meta-agent, via validated proposals onlyPath policy, AST scan, diff audit
Sealedthe experimenterNever mounted into agent processes; the agent sees aggregate signals only

Defence-in-depth AST sanitization

Seven layers, each assuming the one before it failed. The static scan alone would be a weak control, so it is one layer among several and is documented as such.

LayerWhat it enforces
1 · Strict parseJSON schema with key whitelists. At most 8 ops, 20,000 characters, 500-character hypothesis, tag regex. Unknown keys are an error, not ignored.
2 · Path policyWrites only under agent/. Suffixes .py .md .txt .yaml. No traversal, no conftest.py, no dotfiles.
3 · AST scanEvery changed .py file is parsed and walked. See the deny-list below.
4 · Test ratchetThe set of agent self-test ids may only grow. skip, skipif, xfail, importorskip, pytest_plugins and pytestmark are rejected in tests.
5 · Atomic applyreplace_symbol swaps exactly one same-named definition by AST span. Edits are staged in memory and written only if every op succeeds.
6 · Diff auditAfter commit, the files that changed must equal the files named in the proposal. Anything else is a violation.
7 · SandboxSelf-tests run with rlimits, scrubbed env, own session and process-group SIGKILL on timeout. Container mode adds --network none, read-only root, memory, cpu and pids caps.
CategoryBlocked
Importssocket, subprocess, ctypes, multiprocessing, shutil, importlib, pickle, marshal, urllib, http, requests, signal, resource, sys, builtins, sqlite3, gc, kernel, sealed; relative imports
Builtinseval, exec, compile, open, __import__, breakpoint, globals, setattr, delattr; getattr with a computed name
os.*system, popen, remove, unlink, rename, kill, fork, exec*, spawn*, chmod, chown, environ, putenv, chdir, symlink (including aliases and from-os imports)
Dunder routes__subclasses__, __globals__, __builtins__, __code__, __class__, __bases__, __mro__, __dict__

The scan is intentionally strict, so it can reject benign code. It is not a security boundary: real isolation comes from container mode, with no network and no filesystem path from agent to sealed side.

The non-inferiority statistical gate

Parent and candidate run on the same freshly drawn sealed problems, so every problem yields a paired outcome. Let b be the problems the patch fixed, c the problems it broke, and n the number of pairs. Each pair contributes a difference di∈{−1,0,1}, so ∑idi2=b+c. The gate works on the plug-in estimate of the mean difference and its standard error.

δ^=b−cn
Paired effect estimate
se=1nb+c−(b−c)2n
Paired-proportion standard error
L=δ^−z⋅se,z=z0.95=1.645
One-sided lower confidence bound

The improvement criterion is the exact posterior probability that the patch fixes problems more often than it breaks them. Using only the discordant pairs and a flat prior, it reduces to a binomial tail with no normal approximation:

pbetter=P(Beta(1+b,1+c)>12)=P(Bin(b+c+1,12)≤b)
Posterior probability of improvement
pbetter=12b+c+1∑k=0b(b+c+1k)
Exact binomial tail (closed form)

The decision rule combines the three conditions; correctness is a hard prerequisite, not a score term:

accept⟺(L≥−ε)∧(δ^>0)∧(pbetter≥τ)
Accept rule
SymbolCodeMeaning
bbproblems the patch fixed (fail → pass)
ccproblems the patch broke (pass → fail)
nnpaired problems; the gate refuses to decide below min_pairs = 20
δ^deltapaired effect estimate
Llowerone-sided lower bound, z_one_sided = 1.645
εeps_noninferiornon-inferiority margin, 0.02 pass@1
τp_improve_minposterior threshold, A/A-calibrated, default 0.95
VerdictConditionEffect
acceptL ≥ −ε, δ̂ > 0 and p_better ≥ τPromoted by fast-forward merge to main.
accept (efficiency)non-inferior, δ̂ ≥ 0, and ≥ 5% smaller or ≥ 10% fewer tokensCheaper at equal correctness.
stepping_stonenon-inferior and behaviourally novelArchived, not promoted.
rejectanything elseBranch discarded.

Three guards keep the test honest. The gate raises if parent and candidate were scored on different problems, and refuses to decide on fewer than 20 pairs. Cost and size can only break ties among non-inferior candidates. Novelty can never rescue a regression. Drawing fresh problems every cycle limits the adaptive reuse of a fixed evaluation set [5].

A/A calibration

The false-accept rate of null patches (identical true pass rate, independent run-to-run noise) is measured by simulating the gate on T null pairs, then the threshold is the smallest grid value that meets a pre-registered target α:

FAR^(τ)=1T⋅|{t≤T: gate accepts null pair t}|
Empirical false-accept rate
τ∗=min{τ∈G:FAR^(τ)≤α}
Calibrated threshold

With n=200, p=0.5, noise=0.1 and α=0.05, the smallest threshold that meets the target is the default τ=0.95.

τ (p_improve_min)False-accept rate
0.80≈18%
0.90≈9%
0.95 (default)≈3.6–4.1%
0.975≈2%
0.99≤ 0.5%

End-to-end demo

ScenarioEvidenceOutcome
Good patch (structural repair)b = 30, c = 1, n = 200accepted, promoted
Kernel tampering attemptpath kernel/judge.py is outside agent/rejected before anything runs
Degrading patch (retries capped at 0)b = 0, c = 65, n = 200, δ̂ = −0.325, L = −0.379rejected: not non-inferior

The demo runs against a simulated verifier with scripted proposals. It checks that the kernel promotes the good patch exactly once, blocks the hostile one statically, rejects the regression on the gate, leaves no cand/* branches behind and does not touch the kernel.

03 / engineering highlights

Small kernel, strict seams.

Modular design

patching.py is pure: parse, validate, apply. orchestrator.py composes Config, Budget, Sandbox, GitRepo, KernelGuard, Ledger, Gate, Archive and attribution. The two experiment-specific parts, the sealed Verifier and the MetaAgent, are abstract interfaces, so the kernel is testable without a model or a benchmark.

Shapley credit over code hunks

Each edit op is one hunk, and credit is the Shapley value [11] over hunks, computed on a separate sealed credit split (see below). The agent only ever sees hunk-level values.

Quality-diversity archive

Non-inferior but novel candidates are kept as stepping stones, described by per-family pass profile, agent size and tokens. The next parent is the champion with probability p_exploit (0.5), otherwise sampled by performance × novelty ÷ visits, so the search does not collapse to one lineage.

Ledger-based crash recovery

An fsynced, append-only JSONL write-ahead log records begin and end per cycle. A begin with no end marks a crash; recover() discards that branch and closes the cycle as rejected on the next start. After 8 consecutive failed cycles a circuit breaker cools the loop off for 600 s.

Concurrency

With workers > 1, candidates are evaluated in separate worktrees. Promotion is serialised by a lock plus the fast-forward or champion re-gate check, so parallel candidates cannot both win.

Memory treated as untrusted

The “Mom Center” workspace stores patch, failure and credit entries in SQLite, with outcome-based Beta trust, TTL and capacity-limited broadcast using salience plus MMR diversity. Every payload is sanitised and marked untrusted before it reaches the agent.

Hunk-level credit assignment

Let N be the hunks of a patch and v(S) the pass@1 of the parent with only the hunks in S applied, on the credit split. A coalition whose ops cannot be applied scores 0.

φi=∑S⊆N∖{i}|S|!(|N|−|S|−1)!|N|![v(S∪{i})−v(S)]
Shapley value of hunk i
∑i∈Nφi=v(N)−v(∅)
Efficiency: credit sums to the total gain

The sum is exact for up to 6 hunks (at most 2⁶ coalitions, each scored in its own worktree). Beyond that the kernel samples 24 seeded permutations. Failure to compute credit never undoes a promotion.

04 / impact & stack

What it shows, and what it doesn’t.

The result is a kernel where any claim of “the agent improved” arrives with a pre-registered false-accept rate attached, and where the failure modes above are blocked structurally instead of by prompt. The design also carries a recorded negative result: Study 1’s judge was too conservative, and that finding drove the Study 2 gate.

The repo documents its own limits, listed below. Closing the first one, a real sealed verifier and LLM meta-agent, is the next milestone.

LayerDetail
LanguagePython ≥ 3.10, standard library only for the kernel
Version controlGit CLI (≥ 2.30) driven through subprocess: worktrees, fast-forward merges, tags, refs
Static analysisPython ast: symbol-span replacement, deny-list scan, test-id collection
StatisticsPaired proportions, exact binomial tail, Monte Carlo A/A calibration, exact and sampled Shapley
Persistencefsynced append-only JSONL ledger, JSON archive, SQLite (Mom Center)
RuntimeLocal rlimit sandbox (POSIX) or docker/podman container mode
Testingpytest kernel suite and a 3-scenario end-to-end demo

Known limitations

  • The sealed Verifier and the LLM MetaAgent are abstract interfaces. The repo ships no problem families and no model client, and example_patch.py uses a simulated verifier. The demo exercises the kernel; it is not evidence of self-improvement.
  • The AST scan is a first filter, not a security boundary. It has known blind spots and rejects some benign code. Untrusted runs need container mode.
  • Adaptive overfitting to the sealed distribution is still possible over many cycles [5]. The mitigation is a rarely-read audit split compared against the selection split.
  • The A/A noise model is an assumption. The real run-to-run noise has to be estimated from repeated evaluations of an unchanged agent before a pre-registered run.
  • A learned or LLM-based judge, the main reward-hacking risk in the wider literature [15, 6], is out of scope.

05 / empirical results & ablation study

The data behind the pivot.

Study 1 is a pre-registered, frozen ablation on a toy few-shot regression task: four arms (full, no_ws, random_ws, no_judge), ten paired seeds, twelve rounds, query MSE as the metric (lower is better). The primary metric is the final loss on the hard split.

The full system gated every round through the judge. The ablation removed it in one arm. Both figures below come straight from the repo’s results/ directory.

What the judge cost. The full arm accepted 26.7% of proposed rounds, so roughly 73% were rejected. The no_judge arm accepted all of them and ended lowest on both evaluation sets.

What this does not show. The toy task has no adversarial pressure, so this measures the price of an over-conservative gate, not whether a gate is needed.

Two line charts of query MSE over twelve rounds for the arms full, no_ws, random_ws and no_judge, on a uniform-difficulty set and on the hard set. no_judge ends lowest on both; the hard-set curve is noisier with spikes near rounds 3 and 6; shaded bands overlap heavily.
Figure 1. Study 1 multi-seed ablation curves: per-round query MSE (mean ± s.e. over 10 seeds) on the uniform-difficulty set (left) and the hard set (right). no_judge descends fastest and ends lowest on both, but its hard-set trajectory is the noisiest, and the bands overlap for most of the run.
Bar chart of final hard-split query MSE per arm with standard-error bars and one dot per seed: full 1.048, no_ws 1.038, random_ws 1.022, no_judge 0.962.
Figure 2. Final hard-split performance per arm: bars are means with standard errors, dots are individual seeds (query MSE, lower is better). Per-seed spread is large relative to the gap between arms.
Armfinal_hardfinal_allaccept rate
full1.048 ± 0.1900.872 ± 0.1480.267
no_ws1.038 ± 0.1890.872 ± 0.1660.275
random_ws1.022 ± 0.1620.848 ± 0.1540.275
no_judge0.962 ± 0.1460.755 ± 0.0921.000

Mean ± SD over seeds 1–10. Accept rate is the share of proposed rounds the judge let through.

Pre-registered decision rule

A component counts as supported only if the Bonferroni-adjusted interval for the paired differenceΔ=arm−full excludes zero in its favour and the mean clears the smallest effect of interest. Negative values mean the arm beat full.

Arm − fullMean Δ95% CIBonferroni 98.33% CIMDE (80% power)Verdict
no_ws−0.010[−0.050, +0.026][−0.060, +0.034]0.068no detectable difference
random_ws−0.027[−0.062, +0.007][−0.071, +0.015]0.061no detectable difference
no_judge−0.086[−0.151, −0.025][−0.168, −0.014]0.111full clearly worse

Only the no_judge interval excludes zero, and it excludes it in the arm’s favour: the judge component hurt on this task. The other two verdicts mean “no detectable difference”, not “no effect”: their observed gaps (0.010 and 0.027) sit below the minimum detectable effect for ten seeds.

From this result to the Study 2 gate

The negative result is the specification for the redesign. Each row pairs a measured weakness with the part of the Study 2 gate that addresses it.

Study 1 findingStudy 2 response
The LCB judge accepted 26.7% of rounds, so about 73% were rejected, and the unjudged arm finished lower.Accept on non-inferiority (L ≥ −ε), so a patch that does not regress is not discarded for failing to prove a gain.
Improvement and safety were one test, with a hand-set threshold and no known error rate.Two separate conditions: non-inferiority for safety, posterior p_better ≥ τ for gain, with τ A/A-calibrated to a 5% target.
Seed-to-seed SD of 0.15–0.19 on final_hard; the smallest detectable difference was 0.06–0.11.Paired evaluation on identical problems (the gate raises otherwise) and a hard floor of 20 pairs, so noise is cancelled per problem.
no_judge accepted every round, so it measures the cost of over-rejection, not the value of having no gate.The gate stays; what changes is that its false-accept rate is measured before it is trusted.

06 / references

Bibliography, Study 1.

References cited in the Study 1 paper. Numbers in the text above point here. External related-work links in the overview are listed inline.

  1. Bernard J. Baars. A Cognitive Theory of Consciousness. Cambridge University Press, 1988.
  2. Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, and Robert Tjarko Lange. Text-to-LoRA: Instant transformer adaption. In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of PMLR, 2025. arXiv:2506.06105 ↗
  3. Mingguang Chen, Licheng Wang, and Bo Qu. Recursive self-improvement in AI: From bounded self-refinement to autonomous research loops. arXiv preprint, 2026. arXiv:2607.07663 ↗
  4. Thomas Coste, Usman Anwar, Robert Kirk, and David Krueger. Reward model ensembles help mitigate overoptimization. In International Conference on Learning Representations (ICLR), 2024. arXiv:2310.02743 ↗
  5. Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Aaron Roth. The reusable holdout: Preserving validity in adaptive data analysis. Science, 349(6248):636–638, 2015.
  6. Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023. arXiv:2210.10760 ↗
  7. Bochen Han and Songmao Zhang. Exploring advanced LLM multi-agent systems based on blackboard architecture. arXiv preprint, 2025. arXiv:2507.01701 ↗
  8. Prannay Hebbar, Yogendra Manawat, Samuel Verboomen, Alesia Ivanova, Selvam Palanimalai, Kunal Bhatia, and Vignesh Baskaran. SIA: Self improving AI with harness & weight updates. arXiv preprint, 2026. arXiv:2605.27276 ↗
  9. Zhiyuan Liang, Dongwen Tang, Yuhao Zhou, Xuanlei Zhao, Mingjia Shi, Wangbo Zhao, Zekai Li, Peihao Wang, Konstantin Schürholt, Damian Borth, Michael M. Bronstein, Yang You, Zhangyang Wang, and Kai Wang. Drag-and-drop LLMs: Zero-shot prompt-to-weights. arXiv preprint, 2025. arXiv:2506.16406 ↗
  10. Wenlong Shang. “Theater of mind” for LLMs: A cognitive architecture based on global workspace theory. arXiv preprint, 2026. arXiv:2604.08206 ↗
  11. Lloyd S. Shapley. A value for n-person games. In Harold W. Kuhn and Albert W. Tucker, editors, Contributions to the Theory of Games, Volume II, pages 307–317. Princeton University Press, 1953.
  12. Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel machine: Open-ended evolution of self-improving agents. arXiv preprint, 2025. arXiv:2505.22954 ↗
  13. Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. Hyperagents. arXiv preprint, 2026. arXiv:2603.19461 ↗
  14. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint, 2025. arXiv:2505.03335 ↗
  15. Chenyu Zhou. More convincing, not more correct: Self-play reward hacking of reference-free LLM judges. arXiv preprint, 2026. arXiv:2607.05904 ↗
  16. Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, and Pulkit Agrawal. Self-adapting language models. In Advances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2506.10943 ↗