Researchers at UC Berkeleys Sky Computing Lab have open-sourced Sky-T1, a reasoning AI model that costs less than $450 to train. The model, Sky-T1, is the first truly open-source reasoning model that can be replicated from scratch. It was trained for less than $450 and competes with OpenAIs o1 on key benchmarks. Synthetic training data has helped drive down costs of training models, with Palmyra X 004 only costing $700,000. Reasoning models, such as Sky-T1, are more reliable in domains such as physics, science, and mathematics. The NovaSky team used Alibabas QwQ-32B-Preview to generate initial training data for Sky-T1 and then leveraged OpenAIs GPT-4o-mini for data refactoring. Sky-T1 outperforms o1 on MATH500 math challenges but falls short on GPQA-Diamond. The model was trained using 8 Nvidia H100 GPUs over 19 hours. Overall, Sky-T1 represents a cost-effective approach to developing high-level reasoning AI models.
