01 / FIELD NOTES
Loading a model is the start of the test
This article is a benchmark protocol, not a report of measured hardware results. A local model needs to do more than return text: it must complete the workload at useful quality and speed, within a memory budget, while serving the people who rely on it.
A credible hardware case study starts with a real bill of materials and a reproducible workload. Record hardware configuration, memory, operating system, model identifier, quantization, serving version, context settings, and every relevant runtime option. A price without that context cannot tell another team what it is buying.
02 / FIELD NOTES
Use a representative workload and an acceptance rubric
Sample tasks from the work you intend to move, with confidential material removed or handled under an approved process. Preserve the range of prompt lengths and difficulty. Agree on acceptance criteria before running the models.
Run both cold and warmed requests, report them separately, and keep errors. Repeat enough tasks to show variation rather than selecting the most flattering run. Reviewer acceptance belongs in the results file; token throughput cannot determine whether an answer was useful.
- Record time to first token and full completion time with a streaming client.
- Measure accepted results, retries, failures, and active human repair.
- Test the intended context lengths and peak concurrency separately.
- Record wall power at idle and under load, with the measurement method.
03 / FIELD NOTES
Start with the downloadable run recorder
The accompanying Node.js script sends a JSONL task file to a local OpenAI-compatible chat endpoint and writes a JSONL ledger containing configuration, returned usage, completion time, output, and errors. It intentionally leaves reviewer acceptance blank.
This first recorder uses non-streaming sequential requests. It measures total completion time; it does not measure first-token latency, peak concurrency, power, or end-to-end service capacity. Use it as a smoke test and evidence format, then extend the workload for those separate measurements. Never label its output a production-capacity benchmark.
Run with Node 22 or newer: node record-local-benchmark.mjs tasks.jsonl results.jsonl. Set LOCAL_LLM_MODEL and, if needed, LOCAL_LLM_URL. Each task line needs an id and a prompt. The default URL is http://127.0.0.1:11434/v1/chat/completions. The script accepts loopback destinations only and will not overwrite an existing results file.
04 / FIELD NOTES
Price the configuration that passed
Use the actual hardware quote or sunk-cost decision, measured power, hosting, support, operating labor, and remaining API bill. Enter the fraction of API dollars your pilot eliminated, rather than the fraction of requests it processed.
A publishable result should include the task-selection method, acceptance rubric, complete ledger, software versions, power method, throughput under the intended load, and limitations. A configuration that passes a narrow workflow is a useful finding even if it cannot replace a general-purpose cloud model.
The measured workstation case study will require those inputs. Until then, a transparent protocol is more useful than an invented speed or savings claim.
Sources & next steps
- Download the local benchmark recorder
- vLLM benchmarking documentation
- Compare your measured setup in the cost calculator
Published November 2, 2026. Worked examples are hypothetical; benchmark methods describe measurements to perform.
MAKE THE ECONOMICS VISIBLE