Build 1 publisher2 min readPublished September 18, 2026 at 8:29 AM PDT

A 20x backbone-to-head learning rate gap only pays when the checkpoint solves a different task

A dev.to write-up gives DEIMv2's DINOv3 backbone 5e-6 and its decoder head 1e-4 while fine-tuning a face detector, then explains why a second detector from the same week kept one global learning rate.

The Engineer · Build desk

Illustration accompanying A 20x backbone-to-head learning rate gap only pays when the checkpoint solves a different task

What happened

A dev.to write-up describes fine-tuning DEIMv2, a DETR-style detector with a DINOv3 Vision Transformer backbone, from a COCO-pretrained checkpoint into a single-class face detector.
The author set the backbone learning rate to 5e-6 and the decoder head to 1e-4, a 20x gap he names as discriminative fine-tuning, also called layer-wise learning rate decay.
The common recipe uses one learning rate for the whole network on the assumption that every layer needs the same update rate, which the post says holds only when the checkpoint already solves the task.
In the same week the author fine-tuned two different detectors for the same face-detection task and applied layer-wise decay to only one of them.
The technique dates to ULMFiT in 2018 and now appears in BERT-family fine-tuning and in MAE and BEiT-style vision recipes that decay a factor per layer from the output back toward the input.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

decision The optimizer config now turns on a question about the checkpoint's prior task, so two runs on the same dataset, in the same framework, on the same architecture can justify opposite settings.
cost Both wrong answers are paid in GPU hours: a throttled head spends epochs converging that produce little, and an over-driven backbone spends them degrading the pretraining you paid nothing for.
exposure Teams that keep one global rate because their checkpoint already solves the task take on a different risk. The post points at warmup schedules that perturb an optimum which did not need perturbing.
constraint The contrast rests on two runs by one practitioner, and no numbers were published, so a team can adopt the decision rule but has to spend its own compute to size the effect.

The gap is a ratio. Divide the head's 1e-4 by the backbone's 5e-6 and you get 20, so for an equal gradient signal the head's weights move twenty times as far per step as the backbone's ^[4]. That backbone came in from a COCO-pretrained DINOv3 checkpoint and already encodes what an edge or a texture looks like ^[1]^[8]. The classification head had never seen "face" as its own class, and has to learn the concept close to from scratch ^[8].

Share one rate across both and you pick which side to damage. Set it high enough for the head to converge quickly and the backbone is dragged along at the same pace, its general-purpose features degrading before the head stabilizes, the classic symptom of catastrophic forgetting ^[9]. Set it low enough to keep the backbone safe and the head converges painfully slowly, burning epochs and compute ^[10].

Which of those you are exposed to depends on the checkpoint, not the model class. The post's self-test is one question: has this checkpoint already solved this exact task before, just on less data, or is it solving a different task whose learned features you are borrowing ^[17]. Repurposing an 80-class COCO detector to find one category it was never trained to recognize as a distinct class falls on the second side ^[7]. The answer decides it, the author wrote, "not the architecture, not the framework" ^[17].

The same author wrote that discriminative fine-tuning should not be switched on by default ^[15]. The two runs are the only evidence for that claim. The post does not publish the second detector's optimizer settings or accuracy figures for either run ^[18]. A single global rate is also the default in almost every framework's train() call ^[5].

So the direction transfers, and only the direction. Per-layer decay is standard practice in the recipes the post cites, from output layers back toward the input ^[13], and the rule of thumb is only an ordering: closer to the input, lower ^[12]. For 20x specifically to be right on your run, your head would have to sit about as far from its pretrained objective as a one-class face head sits from 80-class COCO detection ^[7]^[8].

What to watch

A follow-up that names the second detector and its optimizer settings would make the same-week contrast checkable.
Published metrics for one DETR-style fine-tune run with and without a 20x split would show the size of the effect on a detector rather than a classifier.
Whether detection training recipes start shipping the per-layer decay factor that MAE and BEiT-style vision recipes already use.

Claim ledger

Ranked by verification strength, evidence, and original report placement.

[1]
The author was fine-tuning DEIMv2, a DETR-style detector with a DINOv3 Vision Transformer backbone, starting from a COCO-pretrained checkpoint, to build a single-class face detector.
ReportedSupportedView cited source
[2]
While wiring up the optimizer, the author gave the backbone a learning rate of 5e-6 and the decoder head 1e-4, described as a 20x gap.
ReportedSupportedView cited source
[3]
The technique is named discriminative fine-tuning, also called layer-wise learning rate decay (LLRD).
ReportedSupportedView cited source
[5]
The most common fine-tuning recipe loads a pretrained checkpoint, drops the learning rate relative to training from scratch, and continues training with one single learning rate for the entire network; it is the default in almost every framework's train() call.
ReportedSupportedView cited source
[11]
Discriminative fine-tuning gives every part of the network its own learning rate, scaled inversely to how general that part's learned representations already are.
ReportedSupportedView cited source
[12]
The rule of thumb is that layers closer to the input, being more general, get a lower learning rate, and layers closer to the output, being more task-specific, get a higher one.
ReportedSupportedView cited source
[13]
The technique traces back to ULMFiT (Howard and Ruder, 2018), where it was introduced for fine-tuning language models in NLP under the name discriminative fine-tuning, and layer-wise learning rate decay now shows up in BERT-family fine-tuning recipes and in vision transformer work such as MAE and BEiT-style recipes, which commonly apply a per-layer decay factor from the output layers back toward the input.
ReportedSupportedView cited source
[14]
In the same week the author was fine-tuning two different detectors for the exact same downstream task, face detection, and applied LLRD to only one of them.
ReportedSupportedView cited source
[18]
The published text describes the same-week contrast between the two detectors but does not give the second detector's learning-rate configuration or report accuracy results for either run.
ReportedSupportedView cited source
[4]
The head's learning rate is 20 times the backbone's, so for an equal gradient signal the head's weights move twenty times as far per step.
DerivedSupported1 source— sign in to open
[6]
That recipe carries a hidden assumption that every layer needs to be updated at the same rate, which the post says holds in exactly one situation: when the pretrained checkpoint already solves your task and you are simply continuing training on more or different data.
ReportedInsufficientView cited source
[7]
The assumption breaks down for transfer across tasks or domains, for example taking a general-purpose 80-class COCO detector and repurposing it to detect exactly one thing it was never trained to recognize as a distinct category: faces.
ReportedInsufficientView cited source
[8]
The backbone has already learned what an edge, a texture and a spatial gradient look like, knowledge broadly reusable across almost any image, while the classification head has never seen 'face' as its own class and needs to learn that concept close to from scratch.
ReportedInsufficientView cited source
[9]
With a shared learning rate high enough for the head to converge quickly, the backbone gets dragged along at the same pace and its general-purpose features start degrading before the head stabilizes, the classic symptom of catastrophic forgetting, in which earlier general representations are overwritten by gradient updates optimized for a narrower newer objective.
ReportedInsufficientView cited source
[10]
With a shared learning rate low enough to keep the backbone safe, the head converges painfully slowly, burning epochs and compute that would otherwise be productive.
ReportedInsufficientView cited source
[15]
The author wrote that discriminative fine-tuning "is not a 'just add it, can't hurt' default" and is a targeted answer to cross-task or cross-domain transfer, where different parts of the network have different amounts of relearning to do.
ReportedInsufficientView cited source
[16]
When the starting checkpoint already solves the task and training is simply being extended, the post says a single global learning rate is often the right and simpler choice, and the real risks tend to be elsewhere, such as warmup schedules that perturb an optimum that did not need perturbing.
ReportedInsufficientView cited source
[17]
The post's self-test asks whether the pretrained checkpoint has already solved this exact task before, just on less data, or is solving a different task whose learned features are being borrowed; the answer determines whether discriminative learning rates are needed, "not the architecture, not the framework".
ReportedInsufficientView cited source

Sources

Publishers with included, body-backed reporting in this cluster.

dev.to
1 article · September 18, 2026
Discriminative Fine-Tuning: Why Your Backbone and Your Head Shouldn't Learn at the Same Speed

Ask Clarity

Have a question this piece left open? The answer comes from its reporting and claim ledger — nothing else.

Try

Clarity analysis

How the coverage leans and what the record actually supports. Read the methodology

Perspective Coverage

How much of the coverage speaks to builders, operators and investors. It measures emphasis, not whether the story is right.

The condition is the usable part here: the author's test is whether your checkpoint has already solved this exact task on less data, or is solving a different task whose features you are borrowing, with only the second case arguing for splitting rates. For the DEIMv2 run that meant 5e-6 on the DINOv3 backbone, 3e-5 on the encoder and 1e-4 on the decoder head, assigned as AdamW parameter groups at optimizer construction, with no change inside the model. DEIMv2, D-FINE and RT-DETR let you declare the same split as regex patterns in config. In Ultralytics YOLO you cannot: build_optimizer splits parameters by type only, and every group keeps the same lr, so you build the optimizer yourself and hand it to a custom trainer. The post never reports whether the split actually helped.

Reality

Evidence, adoption, hype gap, incentives and confidence, each read independently from the current record on the scale shown beneath it. How these are measured.

Evidence40

0100

Checkable plumbing, unmeasured runs

Two kinds of evidence sit side by side here. The configuration and the framework behaviour can be checked: 5e-6 and 1e-4 are stated outright, and the quoted Ultralytics code does hand every parameter group the same lr. What a reader would actually act on rests elsewhere: the failure modes, and any benefit from the 20x split, come with no loss curves, no accuracy figures and no dataset size. The line back to ULMFiT and the MAE and BEiT recipes is real published work, cited rather than reproduced.

Adoption45

0100

Ordinary in DETR-family configs, absent from Ultralytics

Per-layer rates are routine in the frameworks this post works in, and the post shows where: DEIMv2, D-FINE and RT-DETR take regex patterns with a rate per group. Ultralytics YOLO does not, so the same split costs a custom optimizer and trainer. Outside those framework facts, the only usage on the record is the author's own pair of runs.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Entities

build
Sixteen times more training code buys the same gain as reading the rest of the file
1 publisher
build
A vLLM request crosses two IPC queues before its result reaches the caller
1 publisher
build
Fixing one authority case cost a 4B local model six others on the same 24-case suite
1 publisher
build
Moving only the runtime slot on one G5g instance exposed five wrong performance claims
1 publisher

What happened

Why it matters

What to watch

Claim ledger

Sources

Let Clarity write the post for you.

Topics and entities

Topics

Entities

Related stories

Claim ledger

Let Clarity write the post for you.

Topics and entities

Topics

Entities

Perspective Coverage

Reality

Related stories

Sources