command-r vs llama-3.1-8b-instant

Capability for the money

llama-3.1-8b-instant is 4.6× cheaper than command-r and scores higher on coding (LiveCodeBench) — there is no case for paying more here.

1. Coding

   writing and fixing code

   LiveCodeBench
   command-r4.8%

   llama-3.1-8b-instant11.6%

   llama-3.1-8b-instant +6.8

2. Hard reasoning

   graduate-level science and multi-step analysis

   GPQA Diamond
   command-r28.4%

   llama-3.1-8b-instant25.9%

   command-r +2.5

Averaged over the benchmarks both models sat, llama-3.1-8b-instant returns 326.1 points of pass rate per dollar of blended price against 63.2 — the ratio that decides whether the dearer model earns its rate.

Each figure is a pass rate: the share of that test set the model answered correctly, measured by a third party rather than self-reported by the lab. Rows are different tests of different difficulty, so compare a model against the other model, not one row against the next. No published following instructions or tool use score for one of these models, so those lenses are omitted. And a benchmark is not your workload — it narrows the shortlist, it does not pick for you.