Researchers trained an OpenAI rival in half an hour for less than $50
Feb 06, 2025
Researchers created a low-cost AI model in just 26 minutes for less than $50. They used a method called distillation to refine the model, s1. This method allows smaller models to learn from larger ones. The researchers used answers from Google’s AI reasoning model, Gemini 2.0 Flash Thinking Experimental.
The s1 model is based on Qwen2.5 and was trained on a small dataset of 1,000 questions. They found that a larger dataset of 59,000 questions didn’t offer much improvement. The model was trained on just 16 Nvidia H100 GPUs.
The s1 model also uses test-time scaling to double-check its reasoning before giving an answer. This helps the model fix mistakes in its logic. The researchers claim that s1 performs better than OpenAI’s o1 model on competition math questions.
Smaller and cheaper AI models like s1 could change the way big companies train their AI. They may not need to spend billions of dollars on training AI anymore. This could disrupt major companies like OpenAI, Microsoft, Meta, and Google who need massive data centers with thousands of Nvidia GPUs.
Overall, the rise of smaller and cheaper AI models could shake up the industry in a big way. It shows that you don’t always need lots of money and resources to create powerful AI technology. It’s amazing how researchers can create something so advanced in such a short time and with limited resources.