Skip to content

Build1 publisher3 min readPublished

A self-hosted LLM tool's real RAM floor shows up in closed issues and the workflow YAML

A dev.to guide walks through checking a self-hosted LLM tool's memory floor, offline behaviour and license before you integrate it. The checks worth the time read artifacts the maintainers wrote for themselves.

The Engineer · Build desk

Illustration accompanying A self-hosted LLM tool's real RAM floor shows up in closed issues and the workflow YAML

What happened

  • A dev.to guide sets out how to verify a self-hosted LLM tool's minimum RAM and GPU, true offline capability, license type and maintenance activity before integrating it into a stack.
  • It says that a repo with no LICENSE file is proprietary by default, whatever the GitHub front page reports about its license.
  • For hardware floors it says to read the last 20 to 30 closed issues, searching for OOM reports, "minimum RAM" and specific GPU names such as RTX 3060.
  • Where reports conflict, one issue saying 8GB and another 16GB, it says to plan for the higher figure because a tool that runs on 8GB may be unusable there.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint A workflow file bounds only the code paths CI exercises, so the runner's memory is evidence about the test suite, and anyone quoting it as an inference floor is extrapolating.
  • exposure A team that budgeted on a front-page permissive badge can find its own distribution terms set by a dual-license file covering the main source, discovered after the code is written.
  • cost The verification work scales with the number of candidates: three shortlisted tools mean 60 to 90 closed issues and 9 to 15 dependency licenses before anyone writes glue code.
  • decision The network-isolated container run can settle go or no-go before integration starts, because it is the only listed check whose evidence comes from executing the tool.

Open `.github/workflows/` and the YAML tells you which runner image the maintainers actually test against. The guide treats a job on `ubuntu-latest` with 7GB of RAM as a clue that the tool needs at least that much [10]. For that figure to bound your deployment, the CI job would have to load the model and run inference. A lint job and a unit test suite do not allocate model weights. Where the job steps are a lint run, the runner spec bounds nothing.

The offline section is the strongest part of the guide, and its numeric threshold is the weakest line in it. It says to search the source for `requests.get()`, `urllib.request` and `http://`, check whether those calls sit behind an offline flag, and assume the tool is not truly offline if you find 10 or more external calls with no clear toggle [13]. The count matters less than where the call sits. One unguarded fetch in a startup path fails an air-gapped install on every boot; ten guarded ones in an update checker may never fire. The guide also flags the case where a tool works offline once and still phones home for telemetry [3].

On licensing the instruction is to read the LICENSE file. Where the LICENSE file contradicts the README, the guide says to trust the file, because it is the legally binding document [6]. A repo that ships several LICENSE files needs one more step: find the one covering the main source, since some projects dual-license or set different terms for different components [7]. With no LICENSE file at all, the guide says the code is proprietary by default even if GitHub says otherwise [5]. That assertion is the guide's own, and the piece does not name a specific project or report a measurement of its own [21].

The dependency scan is where the hours go. FOSSA or Black Duck can automate a copyleft check; the manual version is `pip list --outdated` or equivalent, then reading the licenses of 3 to 5 key dependencies on their own repos [8]. Add the closed-issue reading at 20 to 30 issues per tool [9]. A three-candidate shortlist comes to 60 to 90 issues and 9 to 15 dependency licenses [19].

One check in the guide produces evidence from the tool instead of from a document: run it in a container with no internet access, try a basic task, and treat a silent failure or a timeout as confirmation of the limit [14][20]. I would run that one first. It costs a container and one task, and it can disqualify a candidate while the README is still open in the other tab. The maintenance check is cheap in the same way, and the guide sets it at the last commit date, with six months of silence flagged and a project with no commits in 18 months treated as a debugging problem you inherit [15][16].

What to watch

  • Projects publishing a tested hardware floor in a machine-readable file would retire the closed-issue triangulation entirely.
  • Dependency license metadata surfaced by package indexes or SBOM tooling would remove the manual 3-to-5 spot check.
  • A tool shipping an explicit air-gapped mode flag settles the offline question without anyone grepping for HTTP calls.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories