Build1 publisher3 min readPublished
QuantDinger's AI entry gate fails open when its model is unsure
QuantDinger's Jev filter treats confidence under 0.55 as an error and passes the order if no backup LLM is set, according to a read of commit 96cec2d. The gate runs only on live strategies, so no backtest shows whether it helps or hurts.
The Engineer · Build desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- Exits, stop losses, take profits and emergency closes skip the AI entirely, as do grid, DCA and martingale bots.
- Each check charges one credit and sends Jev the market state with six multiple-choice questions under an 8-second timeout.
- If Jev errors, replies in the wrong format or falls short on confidence, the platform asks a general LLM through providers such as OpenRouter or OpenAI.
- Once a user's credits run out, orders pass with no check at all.
- The prompt tells the model that missing evidence alone must not reject a trade and to reject only on concrete evidence.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure A strategy's live behaviour can change the moment its credit balance hits zero, with no edit to code or configuration, because filtering depends on billing state.
- constraint With no replay path, the 0.55 threshold cannot be tuned against history, so any change to it is first tested on real orders.
- decision A team building its own LLM order gate can keep the exit bypass, the code-side rule and the audit fields while choosing block or reduced size as its default for an undecided model.
Jev is optional. A user switches it on per strategy or for manual Quick Trade orders [23]. When it is on, the decision rule is plain code. The model answers fixed multiple-choice questions and does not write the rule [1]. An entry goes out only when Jev's final answer is pass, neither the risk check nor the execution check says block, and the timeframes do not show conflict in an adverse regime at the same time [9]. The model sees a compact state: a summary of up to 120 bars on two or three timeframes, open positions, and the strategy's recent results [10]. Its reply is checked hard. Every choice must be valid, the probabilities must cover every option and sum to 1, and the chosen answer must be the most likely one [11].
Much of this is careful work. Of the exit bypass, the reviewer wrote: "This is the single most important choice and they made it correctly." [18] The tests cover the main paths, including "exits bypass the AI" [19]. Each decision lands in an audit table with the state sent, all six answers, probabilities, latency, provider, fallback reason and the credit charged [7].
The trouble is where failures go. A reply that breaks any validation rule counts as a failure [11]. So does confidence below the 0.55 default on the key questions [12]. Both go to a general-purpose LLM that the reviewer calls uncalibrated [12]. When that chain runs out, the credit is refunded, so at least the user is not billed for the check that did not happen [6]. In this code, tightening the validator would send more replies down the same path as an error [11][5]. The reviewer wrote: "So the moments when the AI is least sure are exactly the moments the filter stops filtering." [17]
The prompt pushes the same way. Read together with the fail-open paths, its instructions mean the gate blocks only when the model is confident something is wrong, according to the review [15]. Two smaller gaps sit in the rule itself. The data-quality answer is collected and never used, and a caution on risk or execution counts the same as clear, with no cut to position size [16]. I'd want caution to shrink the order.
For the gate to help at all, Jev's adverse and conflict calls have to come before trades that really do worse than average [20]. If Jev reads the evidence backwards, the gate blocks good entries and passes bad ones [20]. The reviewer wrote that "because it only runs live, nobody would see that until the P&L did" [21]. The README does not publish any results [22]. The review comes from reading the source at commit 96cec2d, dated 21 Sep 2026, mainly ai_decision_filter.py and ai_decision_context.py. The reviewer did not run the platform, which needs Docker, Postgres and Redis [2].
What to watch
- The reviewer's next test, which asks whether Jev can tell market direction from clean indicators on data it has never seen.
- Any QuantDinger commit after 96cec2d that adds a backtest or paper-trading path for the filter, or changes what happens when Jev is unsure.
- Whether caution on risk or execution starts cutting position size, and whether the data-quality answer enters the decision.