An unbounded benchmark for LLM hardware engineering. Large language models design RISC-V CPUs from scratch. Every design must first pass a full battery of formal correctness proofs, so buggy CPUs are thrown out. The ones that survive are then scored by how fast they would actually run on a physical FPGA. See on GitHub
V2 found that V1's open-source timer (nextpnr) misses whole classes of circuit paths, so the Fmax and fitness numbers below overstate many designs. V2 times every design with the FPGA vendor's tool, ranks on hidden programs, and ran six agent systems six times each. Its result is three tiers: Opus 5.5 and Sonnet 5.5, then GPT-6.1 Sol and GPT-6 Astra, then GPT-5.5 and GPT-6 Luna. V2 results
| # | Model | Reps | Best | Δ% | Mean ± std | Area (LUT4) | Fmax (MHz) |
|---|---|---|---|---|---|---|---|
| 1 | claude-opus-5_5_xhigh | 1/1 | 983.24 | +247.7% | 983.2 | 3.1k | 302 |
| 2 | gpt-5_5_xhigh | 3/3 | 525.04 | +85.6% | 468.3 ± 52.8 | 5.5k | 220 |
| 3 | gpt-5_6-terra | 3/3 | 515.70 | +82.3% | 442.3 ± 55.8 | 10.5k | 209 |
| 4 | gpt-5_4_xhigh | 3/3 | 513.84 | +81.7% | 485.8 ± 28.1 | 10.1k | 203 |
| 5 | gpt-5_6-luna | 3/3 | 480.90 | +70.0% | 452.0 ± 29.0 | 10.1k | 209 |
| 6 | gpt-6-astra_max | 3/3 | 474.27 | +67.7% | 424.8 ± 36.2 | 6.2k | 199 |
| 7 | gpt-5_6-sol | 1/3 | 470.80 | +66.5% | 470.8 | 10.2k | 200 |
| 8 | gpt-5_5_high | 3/3 | 461.87 | +63.3% | 430.2 ± 23.0 | 9.8k | 187 |
| 9 | gpt-6-sol_xhigh | 1/1 | 435.24 | +53.9% | 435.2 | 5.7k | 189 |
| 10 | gpt-5_5_medium | 3/3 | 431.58 | +52.6% | 423.5 ± 11.2 | 7.8k | 201 |
| 11 | kimi-k2_6 | 2/3 | 396.13 | +40.1% | 339.5 ± 8.3 | 9.9k | 166 |
| 12 | gpt-5_4-mini | 3/3 | 395.53 | +39.9% | 362.3 ± 23.7 | 10.2k | 187 |
| 13 | VexRiscv (human ref) | n/a | 370.00 | +30.8% | n/a | 3.4k | 144 |
| 14 | gemini-3_5-flash | 0/1 | 359.04 | +27.0% | n/a | 13.8k | 125 |
| 15 | gemini-3_1-pro | 3/3 | 354.73 | +25.4% | 339.4 ± 12.6 | 10.2k | 150 |
| 16 | static | 2/2 | 282.82 | +0.0% | 282.8 | 9.6k | 127 |
| 17 | baseline V0 (fixture) | n/a | 282.82 | n/a | n/a | 9.6k | 127 |
The VexRiscv row is the human-engineered reference, a well-known open-source RV32IM CPU synthesized on the same FPGA used for the benchmark. 12 of the LLM-generated designs beat it. See the methodology page for the full procedure.
Single repetition (n=1): claude-opus-5_5_xhigh. Each of these rows is one run; repeatability is unmeasured, so rank differences involving them are untested.
Most LLM benchmarks have a fixed ceiling. SWE-bench tops out at 100% issue-resolution. Multiple-choice evals approach 99%. Once a model lands at the ceiling, every subsequent model gets the same score, and the benchmark stops being useful for tracking capability.
HWE Bench has no ceiling. Fitness is the CPU's actual speed running CoreMark on a real FPGA, operating frequency times instructions-per-cycle (Fmax × IPC for the technically inclined). There's no theoretical maximum: a smarter microarchitecture always scores higher. As long as models keep finding new tricks (deeper pipelines, smarter branch predictors, restructured ALUs), the leaderboard keeps moving.
Empirically: the current best is 983.24 iter/s (one repetition, n=1; untested), +247.7% over the V0 baseline core, and clear of the VexRiscv human reference. There is no theoretical ceiling, and within current budgets the curve has not saturated.