Sign in

Privacy
Model Angle
BenchmarksModel Comparisons
All sections
All articlesBenchmarksModel ComparisonsTraining Trade-offsInferenceMultimodalRobustnessReproducibility
SearchArchive

Comparison index

AnalysisEvaluation topicPublished
RFBench tests whether multimodal models can connect a circuit drawing to its physicsMultimodalNov 20, 2026
A patch can be scored without running it. What does that score actually measure?BenchmarksNov 20, 2026
A fixed budget can tell evaluators how much their model score may moveReproducibilityNov 20, 2026
Visual agents can choose a good first crop and waste the next oneMultimodalNov 20, 2026
A new science benchmark asks models to improve a method, not recite an answerBenchmarksNov 20, 2026

Latest articles

All articlesBenchmarksModel ComparisonsTraining Trade-offsInferenceMultimodalRobustnessReproducibility
All articlesBenchmarksModel ComparisonsTraining Trade-offsInferenceMultimodalRobustnessReproducibility
ArchiveSearch
RSS feed
Model Angle

Model comparisons, evaluation, and their limits.

AboutEditorial policyCorrectionsPrivacyTerms of useCookie policy
MultimodalNov 20, 2026

RFBench tests whether multimodal models can connect a circuit drawing to its physics

A new benchmark aligns editable layouts, images, geometry grids and simulated RF responses across 10,000 passive designs. Fine-tuning makes generated layouts far more valid, while closed-loop electrical specification success remains rare.

BenchmarksNov 20, 2026

A patch can be scored without running it. What does that score actually measure?

A repository-grounded rubric gives a coding-agent patch a weighted score for task requirements visible in the issue, repository and diff. On SWE-bench Verified, the scores mostly ranked resolved patches above unresolved ones, but the authors say they do not establish that a patch builds, runs or satisfies every requirement.

ReproducibilityNov 20, 2026

A fixed budget can tell evaluators how much their model score may move

A new framework allocates evaluation outputs across benchmark tasks and repeated runs, then uses disagreement to calculate an honest confidence interval. In a LiveCodeBench replay it produced narrower intervals in 15 of 16 configurations at the same 1,100-output budget.

MultimodalNov 20, 2026

Visual agents can choose a good first crop and waste the next one

A counterfactual audit pairs nearly identical document-reading decisions under different observation budgets. In controlled tests, five open-weight vision models never made the optimal switch across forty pairs each. Their later reads explained most of the measured loss, while results on real forms depended on how answers were scored.

BenchmarksNov 20, 2026

A new science benchmark asks models to improve a method, not recite an answer

STEMBench scores twelve computational research problems against the measures scientists already use, then places each result between a generic baseline and a reproduced expert method. Its early results map task-specific strengths, with expert validation of the scoring still to come.

Lead storyBenchmarksNov 20, 2026

AWS puts model scores beside deployment costs. The configuration still decides the comparison.

The new JumpStart leaderboard can help choose a first shortlist. Its strongest contribution is making the assumptions behind quality, speed and spending easier to examine together.

Concept illustration of two coding workstations sharing a controlled evaluation bench.