OpenAI makes its Decisions API widely available as competition grows among fast, low-cost AI decision models

What happened

OpenAI has made its Decisions API available to all customers for the first time, TechCrunch reported. The company first announced the product at its developer day on September 29, and it opened to everyone on Thursday.

The launch puts OpenAI in a market for fast, inexpensive AI. According to TechCrunch, that market recently made TypeSafe, a startup founded by a former researcher at the frontier lab, a $7.6 billion company.

What a decision model does

TechCrunch reported that TypeSafe's Jev model created a category of deep learning models that work as super classifiers. Given a set of options, they make a choice quickly and give a confidence rating for that decision.

Software developers found that these decision models, also called System One models, could replace large language models in a variety of workflows, according to TechCrunch. They run faster and at a much lower cost.

How OpenAI built its version

Nikunj Handa, the product lead for OpenAI's APIs, told TechCrunch his team moved fast to offer its own version after seeing Jev. Instead of building a model calibrated specifically for decisions, the team took ChatGPT Luna and reconfigured it to speed up inference and lower cost.

"The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds," Handa told TechCrunch.

What changed in the public release

OpenAI added new voice and visual modalities to the public version, TechCrunch reported. The company also worked to lower latency in regions around the world by targeting its inference resources geographically.

Early customer results

In initial trials, customers including Box, Datadog, Ramp and Speak used the API to replace OpenAI's large language models and get higher throughput, according to TechCrunch. Andrew Hsu, chief technology officer at Speak, which builds AI language tutors, said the tool cut the time between actions from 1.4 seconds to .13 seconds.

OpenAI also used the product to make its Dots voice assistant more responsive, TechCrunch reported.

How the models compare

TechCrunch noted that it is not clear what makes a good decision model, since the balance of intelligence, cost, calibration and speed seems to differ by workload. TypeSafe CEO Diogo Almeida distrusts benchmarks and urges people to test models in their own work, though TypeSafe released its own rankings last week alongside its latest Jev model.

Public benchmark rankings

Public benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index tend to favor D1 Ultra, which Liquid released last week, and Base Lab's Elephant-1, according to TechCrunch. The latest Jev model ranks seventh and third on those indices, while OpenAI's Decision Model ranks eighth and fourteenth.

TypeSafe's models are currently the most widely queried on OpenRouter, TechCrunch reported. The newest Jev model, which adds support for images and smart-if statements, also tops JevMan, a benchmark built by Opper AI in which decision models play PacMan.

Where OpenAI may have an edge

TechCrunch wrote that OpenAI's model, which it called arguably more capable, may have an advantage in handling multiple types of input. On image classification, Handa told TechCrunch, "Luna is extremely powerful and has been showing better calibrations."

What to watch next

• Whether OpenAI adds specific training to improve its decision models, which Handa told TechCrunch the company may do in the future.
• OpenAI customers want the Decisions API to be more flexible about using images as outputs, according to Handa.
• Handa raised the question of whether response times could reach about 60 milliseconds, which he said could allow an API call for every frame of a video.