AI model
Text embedding model used to compute cosine similarities between the twenty Spanish question pairs.
Ollama's OpenAI-compatible endpoint reported 3,875 prompt tokens per second where the real rate was 125, a dev.to author found. Any benchmark that repeats a prompt and reads that field overstates prefill speed by the share served from cache.
Reality
A dev.to walkthrough replaces a managed vector store and a hosted embedding API with sqlite-vec and a local model on port 11434. Its schema declares float[768], so changing model dimension means re-embedding everything.
Reality
An RTX 3090 holding transcription and embedding models in VRAM averaged 25 W over the month at a cost of 2 euros, while the same post puts hardware amortisation at about 25 euros a month, twelve times the power bill.
Reality
A dev.to writeup instrumented 1,180 local requests over 24 hours and counted 214 cold model loads, including a summarizer on a 10-minute cron that came up cold every time.
Reality
Six days after a reviewer asked him to prove a 0.92 similarity threshold was safe, the author switched his cache off. Then he found two of his own test labels were wrong.
Reality