BrickBench / Overall
Design quality vs. build cost
One dot per system. Higher scores mean better designs. Lower costs sit farther left.
Design rating (Elo)
Average token cost per build (USD, log scale)
Best rating within budget
Compare how AI systems design brick models, follow assembly steps and create functional objects.
How rankings workBrickBench · 12 systems · 300 design prompts
BrickBench / Overall
One dot per system. Higher scores mean better designs. Lower costs sit farther left.
Design rating (Elo)
Average token cost per build (USD, log scale)
Best rating within budget
| System | Elo ± 95% CI | $/build |
|---|---|---|
| GPT-6 Astra | 1299 ± 27 | 4.06 |
| Claude Opus 5.5 | 1287 ± 27 | 6.33 |
| GPT-6.1 Sol | 1286 ± 26 | 0.87 |
| Claude Opus 5 | 1108 ± 20 | 16.47 |
| BrickWright (Astra) | 1107 ± 18 | 1.43 |
| Qwen 3.8 Flash | 1057 ± 20 | 1.75 |
| Gemini 3.8 Flash | 1024 ± 20 | 3.70 |
| DeepSeek V4.1 Flash | 1015 ± 20 | 0.71 |
| GPT-5.6 Sol | 988 ± 19 | 1.02 |
| GPT-5.6 Luna | 879 ± 20 | 0.75 |
| Muse Spark 1.3 | 878 ± 23 | 6.68 |
| GLM 5.3 Flash | 753 ± 29 | 0.54 |
BC-Bench · Five models following the same brick assembly instructions.
| Rank | Model | Selection % | Position error LDU | Rotation error ° | Step success % |
|---|---|---|---|---|---|
| 1 | GPT-5.4 | 43.88 | 310.78 | 74.85 | 0.18 |
| 2 | Qwen-3.5-VL-27B | 37.44 | 314.32 | 82.94 | 0.18 |
| 3 | Qwen-3-VL-8B | 22.76 | 210.14 | 62.47 | 0.36 |
| 4 | InternVL-3.5-8B | 13.24 | 221.69 | 88.43 | 0.00 |
| 5 | Gemma-3-12B | 4.35 | 269.09 | 63.00 | 0.09 |
Liu et al., BC-Bench (2026), Table 2. Scores are the authors’ overall averages across objects.
LMBuild evaluates physical products across soundness, affordance, design and realization.
LMBuild / Task results
Tier B · Name and image input · Final checkpoint
Five highest reported R.3 operability scores. The evaluator marked all five as degraded.
Each discipline uses the measures published by its research teams. Compare systems within the same benchmark and evaluation setting.
BrickBench assigns Design Elo from pairwise judgments of build quality. A higher rating means the judges more often preferred the system’s designs. The table gives each estimate and its 95% interval. Nearby point estimates can have overlapping intervals.
Each step marks the best published Design Elo available at that mean cost per build. The line rises when a more expensive system sets a higher score. Horizontal spacing represents proportional changes in mean token cost per assembly.
BC-Bench reports average performance on assembly steps. Selection and success improve as scores rise. Position and rotation errors improve as they fall.
The LMBuild view shows operability on one bar-stool task under a fixed input condition. The studies below document the evaluation conditions.