Open-sourced AI models may be more costly in the long run, study finds

Open-source AI models use more computing power, increasing costs over time compared to closed models.

: A recent study by Nous Research highlights the higher long-term costs of open-source AI models due to their increased computing resource demands. They assessed AI models, including Google's and OpenAI's closed systems and DeepSeek's and Magistral's open-source ones, by measuring tokens used in tasks. Open models used between 1.5 and 4 times more tokens, indicating higher resource consumption and latency. Factors such as token efficiency are crucial, with OpenAI's models being particularly efficient in handling math problems.

Open-source artificial intelligence (AI) models, once considered the lower-cost option, may turn out to be more expensive in the long run due to their higher consumption of computing resources. This finding comes from a study conducted by Nous Research, which tested various AI models, including closed-source systems from industry leaders like Google and OpenAI along with open-source models from DeepSeek and Magistral. The key metric used in the study was 'tokens', discrete units of text or data that models process incrementally. The study found that open-source AI models tended to use significantly more tokens than their closed-source counterparts—anywhere from 1.5 to up to 4 times more in some applications like simple knowledge questions.

For tasks requiring basic knowledge, open-source models were notably inefficient, sometimes consuming up to ten times more tokens than closed-source models. In mathematical and logical problem-solving, the gap narrowed but remained; open-source models still used nearly twice as many tokens. The researchers leveraged these token counts as a proxy measurement for computing resources, since many closed-source models do not disclose their underlying processes. This heavy token usage by open-source models translates into longer generation times and increased latency, factors that render them less cost-effective over time despite potentially lower per-token costs.

Token efficiency is crucial for AI economics because models are charged based on their total token output. Every additional token represents not just added computing effort but also longer process completion times. Companies utilizing AI for various applications must weigh the upfront savings of an open-source model against these hidden costs. The Nous Research report emphasizes this balance, suggesting that while open-source models allow cheap hosting, their inefficiency in token usage could negate these savings.

Among the models tested, OpenAI's entries, particularly their innovative open-weight GPT-OSS series, stood out for their token efficiency, especially on mathematical problems. This efficiency is achieved through optimized processes like concise chains-of-thought, and sets a benchmark for improving efficiencies in other open-source models. While open-source models like DeepSeek and Qwen were less efficient with their token usage, OpenAI's methodologies potentially offer strategies for improving open-source models' efficiency.

In conclusion, the choice between open-source and closed-source AI models involves more than just the apparent immediate costs. It demands a detailed consideration of long-term resource efficiencies. The Nous Research study provides invaluable insights into this aspect, encouraging companies to look beyond surface-level economic benefits when implementing AI solutions. Ultimately, while the open-source models grant access and potential flexibility, their enhanced token use could prove more costly over time.

Sources: Gizmodo, Nous Research

AIMachine LearningResearch