Foundry Local ships inference as a native library the app loads in-process

When the app calls the Core API, that library detects the hardware, picks an execution provider, downloads the model if it is not already cached, manages the session and runs the inference ^[3]. It ships as a .dll on Windows, a .so on Linux and a .dylib on macOS ^[3]. The post's architecture section is titled "Four Layers, Zero Daemons" ^[8].

The dev.to post argues Foundry Local is "closer to embedding SQLite than to calling a hosted database service" ^[16]. That comparison is the accurate one. An embedded database moves operational work into the client: file location and version skew between installs. Foundry Local moves the equivalents. The lifecycle it documents runs download, load, infer, unload ^[9], and catalog models are versioned and cached locally after first download ^[5]. Cache state on a customer's disk is part of the support surface.

The band the post nominates is quantized 0.5B to 8B parameters, naming Qwen2.5, Phi, DeepSeek-distilled variants and Mistral small ^[13]. The top of that band has sixteen times the parameters of the bottom ^[17]. An installer that ships one model choice has to make it against the weakest machine it will land on. The hardware the post counts as ready spans NPUs now standard on Copilot+ PCs, Apple Silicon GPUs through Metal, and commodity laptops with usable GPU paths via WebGPU ^[14].

The case against the cloud default for small tasks, as the post puts it, is per-call: round-trip latency measured in hundreds of milliseconds and sometimes seconds, plus a token charge for even a one-line classification ^[15]. Against that it sets a quality claim: those quantized small models "handle summarization, classification, extraction, and simple tool calling at quality levels that were frontier-only two years ago" ^[13]. For that to transfer, your accuracy bar has to sit below where a quantized model lands on your data, not on the catalog's examples.

Microsoft's FAQ says "Foundry Local is not a web server and CLI tool", and the post notes it appears there almost as a disclaimer ^[6][7]. The same post then documents an OpenAI-compatible server mode, alongside a streaming chat client in about 30 lines ^[11][10]. In my view the server mode exists so existing OpenAI client code keeps working while you develop, and the in-process library is the thing you ship. Which of the two you build against decides whether a later model swap is a config change or a new build. The post's worked scenario for the local path is a field inspection app that must work offline ^[12].