impossible → routine
Competition mathematics
Solving competition mathematics problems and showing the working.
Impossible
The MATH paper concludes that scaling to 40% accuracy would need around 10^35 parameters, which it calls impractical.
GPT-3, 175B parameters: 5.2%
source3 years, 10 months
Routine
A model published with open weights reports 97.3% on MATH-500 and is distilled into versions from 1.5B parameters up.
97.3% pass@1 on MATH-500
source