build1 publisher
The softmax bottleneck leaks a model's hidden size to anyone who can read its logits
A paper on arXiv put under $1,000 of queries through gpt-3.5-turbo and estimated its embedding size at about 4,096, then reused the same output basis as a fingerprint sensitive to weight changes.
Publishers:arxiv.org
Reality
- Evidence66
- Adoption
- Insufficient
- Hype gap+15
- Incentives40
- Confidence58