build1 distinct publisher
Your fine-tune did not get better at HellaSwag, it got shorter
The acc and acc_norm split in lm-eval-harness can move in opposite directions on one checkpoint. Pick the metric before you train, and say which one you picked.
Publishers:dev.to
Reality
- Evidence52
- Adoption
- Insufficient
- Hype gap+14
- Incentives18
- Confidence46