Topic

Discriminative fine-tuning

A fine-tuning method, also called layer-wise learning rate decay, that assigns different learning rates to different parts of a network, typically lower near the input and higher near the output.

Current clusters

build1 publisher

A 20x backbone-to-head learning rate gap only pays when the checkpoint solves a different task

A dev.to write-up gives DEIMv2's DINOv3 backbone 5e-6 and its decoder head 1e-4 while fine-tuning a face detector, then explains why a second detector from the same week kept one global learning rate.

Publishers:dev.to

Reality

Evidence40
Adoption45
Hype gap+12
Incentives20
Confidence55