| Measure | CPython | Monty |
|---|---|---|
| First callfresh process, includes runtime load | 18.85 ms | 0.575 ms |
| Next callsame process, new interpreter state | 4.412 ms | 0.049 ms |
| Warm print(1)median per call in a long-lived session | 4.424 ms | 0.018 ms |
Benches
Latest benchmark snapshot
Static aggregate generated from repository result artifacts. Use the linked files for raw measurements and full eval traces.
Python startup history
CPython per run
| Run | First call | Warm print(1) |
|---|---|---|
| 2026-10-07import http.client: 461.12 ms | 18.85 ms | 4.424 ms |
| 2026-10-06import timing not recorded | 21.896 ms | 6.312 ms |
| Gap | Calls now | Recorded |
|---|---|---|
unterminated command substitutionparse error | 24 | 30 |
in conditional expression: unexpected token `)'parse error | 13 | 1 |
python3missing command | 5 | 6 |
diff runknown option | 5 | 4 |
in conditional expression: unexpected token `token'parse error | 4 | 0 |
(no detail)parse error | 3 | 12 |
nodemissing command | 3 | 4 |
perlmissing command | 3 | 3 |
pythonmissing command | 3 | 2 |
near unexpected token `|'parse error | 2 | 0 |
unexpected EOF while looking for matching `''parse error | 2 | 0 |
Error -x.unknown option | 1 | 1 |
Errorunsupported | 1 | 1 |
Error -zunknown option | 1 | 0 |
awk Wunknown option | 1 | 0 |
diff --normalunknown option | 1 | 0 |
diff --unchanged-line-format=unknown option | 1 | 0 |
tar --transformunknown option | 1 | 0 |
tar --wildcardsunknown option | 1 | 0 |
tar hunknown option | 1 | 0 |
Gap history
Calls hitting a gap
| Replay | Now | Recorded |
|---|---|---|
| 2026-10-096,246 calls, bashkit 0.18.2 | 73 | 336 |
| 2026-10-075,206 calls, bashkit 0.18.2 | 125 | 319 |
| Spec file (most misses) | bashkit | bash |
|---|---|---|
background74.1% of 27 cases | 20 | 27 |
bugs82.8% of 29 cases | 24 | 29 |
builtin-trap87.9% of 33 cases | 29 | 33 |
brace-expansion94.5% of 55 cases | 52 | 55 |
builtin-meta83.3% of 18 cases | 15 | 18 |
builtin-trap-bash87% of 23 cases | 20 | 23 |
sh-options92.3% of 39 cases | 36 | 39 |
assign-extended94.3% of 39 cases | 33 | 35 |
builtin-read96.9% of 64 cases | 62 | 64 |
builtin-trap-err90.9% of 23 cases | 20 | 22 |
builtin-umask91.7% of 24 cases | 22 | 24 |
fatal-errors60% of 5 cases | 3 | 5 |
Oils spec history
Cases passed vs real bash
| Run | Rate | bashkit / bash |
|---|---|---|
| 2026-10-09Oils 57d3f0d088c3, bashkit 298bf8f9cfd1 | 97.3% | 2,618 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit 74d816bc7893 | 95.8% | 2,578 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit 38a65ba78b0b | 92.8% | 2,500 / 2,693 |
| 2026-10-08Oils 57d3f0d088c3, bashkit fa242de13904 | 89% | 2,395 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit 26a3acc75177 | 86% | 2,316 / 2,693 |
| 2026-10-08Oils 57d3f0d088c3, bashkit 017d9bfb4f3d | 82.1% | 2,211 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit 5f04b4810ec2 | 78.8% | 2,120 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit bae540476f21 | 77.3% | 2,080 / 2,692 |
| 2026-10-08Oils 57d3f0d088c3, bashkit f86bd7c81ae3 | 74.2% | 1,997 / 2,693 |
| Category | Cases | Last run |
|---|---|---|
| startupSmall commands where interpreter startup dominates runtime. | 4 | 0.016 msbash median: 4.769 ms |
| arithmeticInteger math, substitutions, and expression-heavy shell snippets. | 6 | 0.018 msbash median: 4.313 ms |
| stringsString expansion, pattern handling, and text manipulation. | 8 | 0.018 msbash median: 4.378 ms |
| variablesVariable assignment, lookup, expansion, and environment handling. | 8 | 0.019 msbash median: 4.096 ms |
| arraysIndexed array reads, writes, expansion, and iteration. | 6 | 0.019 msbash median: 4.064 ms |
| pipesPipeline construction, streaming, and command chaining. | 6 | 0.021 msbash median: 5.898 ms |
| subshellCommand substitution and nested shell execution paths. | 6 | 0.024 msbash median: 5.484 ms |
| controlConditionals, loops, case statements, and branching scripts. | 9 | 0.025 msbash median: 3.812 ms |
| Category | Passed | Pass rate |
|---|---|---|
| build_system | 0/1 | 0%tasks passed |
| repo_workflow | 5/8 | 62.5%tasks passed |
| json_processing | 9/9 | 100%tasks passed |
| scripting | 7/7 | 100%tasks passed |
| text_processing | 6/6 | 100%tasks passed |
| data_transformation | 6/6 | 100%tasks passed |
| complex_tasks | 6/6 | 100%tasks passed |
| pipelines | 5/5 | 100%tasks passed |
| Run | Score | Tools |
|---|---|---|
| claude-opus-5-52026-10-09 | 98.9% | 92.1% |
| claude-sonnet-5-52026-10-09 | 96.6% | 91.7% |
| gemini-3.8-flash2026-10-09 | 84.1% | 94.1% |
| kimi-k32026-10-09 | 97.7% | 92.4% |
| muse-spark-1.3-contributor2026-10-09 | 95.5% | 96.8% |
| claude-opus-5-52026-10-09 | 98.3% | 94.6% |
Indexes