原推：People confidently state that neural nets will not be able to do X.
And yet at certain dataset-size/compute/model-depth thresholds models prove suddenly and unexpectedly capable (of things where they hadn’t shown any previous trend improvement.) https://t.co/cekPpuLTa3
New survey paper! We discuss “emergent abilities” of large language models.
Emergent abilities are only present in sufficiently large models, and thus they would not have been predicted simply by extrapolating the scaling curve from smaller models.
?⬇️ https://t.co/CNiExpxjD1