2026-10-24 21:10:23 - No Comments
Local model benchmark — harness: opencode 1.18.16
| Model | Pass rate | First-try validity | Avg repairs | Avg generation | Avg execution | Tokens/sec |
|---|---|---|---|---|---|---|
laguna-xs.2:latest |
91/99 (91.9%) | 100.0% | 0.00 | 93.31 s | 224 ms | 34.1 |
muse-glimmer:30b |
87/99 (87.9%) | 100.0% | 0.00 | 2m 11s | 172 ms | 11.6 |
nemotron-3.5-lightning:30b |
78/99 (78.8%) | 98.9% | 0.01 | 2m 45s | 124 ms | 46.7 |
ornith:35b |
72/99 (72.7%) | 100.0% | 0.00 | 49.96 s | 3.40 s | 34.6 |
qwen3.6:27b |
85/99 (85.9%) | 100.0% | 0.00 | 2m 18s | 129 ms | 3.9 |
qwen3.6:35b |
78/99 (78.8%) | 100.0% | 0.00 | 53.89 s | 3.43 s | 23.9 |
qwen3.8-flash-next-coder-iq1_m |
97/99 (98.0%) | 99.0% | 0.01 | 81.92 s | 158 ms | 10.5 |
qwen3.8-flash-next-iq3_s |
99/99 (100.0%) | 100.0% | 0.00 | 88.81 s | 173 ms | 12.9 |
qwen3.8:27b |
91/99 (91.9%) | 100.0% | 0.00 | 72.79 s | 129 ms | 15.4 |
Run dates 2026-08-13 to 2026-10-24 · repeats 3 · exec timeout 300 s · repair budget 3 · prompt 1.0.0
Be the first to leave a comment!