AI model
Qwen2.5-7B is a 7-billion-parameter open-source large language model from Alibaba's Qwen2.5 series, used for text generation and local inference benchmarking.
The daemon's prompt_eval_count includes cache reads and prompt_eval_duration does not, so one qwen2.5:7b prompt reported 13,826 tokens a second and 43 tokens a second half a minute apart. One subtraction fixes it.
Reality
Five bots posted hundreds of closed positions while the broker rejected every sell, and the only record that knew the difference was the account balance, which nothing in the pipeline was wired to read.
Reality
Across 20 paired writing-correction cases on Ollama for Windows, the two models succeeded on the same 18 and failed on the same 2, while the 4B averaged 23.99 seconds of cold start against 54.37. That reorders local shortlisting.
Reality
Constrained decoding, a repair parser, validation and feedback retries do four different jobs. A dev.to walkthrough shows where each one stops working, and where a 30-second retry budget runs out.
Reality
A 7B model split across Iowa and Oregon on free T4s went from 4.92 to 28.10 tokens per second. Most of the gain came from a drafter that stopped launching kernels one at a time.
Reality