Signpost: Decision engines are the next big hype

On 15 September, hardly anyone had heard of Jev. By the start of October, OpenAI, AWS and Cloudflare had all produced their own versions of the idea.

Jev has one job: it chooses. Give it a set of possible answers to a question, and it offers probabilities for each choice, minus the prose that has made generative AI a standard writing tool.

Weirdly, that modest capability has caught fire because, it turns out, AI has to make constant tiny decisions, and throwing a large language model like Claude or ChatGPT at the problem is massive overkill. Not to mention a massive waste of costly resources.

For a business, those decisions could include whether a transaction seems fraudulent, whether an insurance claim can be approved automatically, which customer offer should be presented, or whether a loan application needs further checks. In each case, the task is to make a defined operational choice quickly and at very high volume.

Jev was created by TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, who helped develop reinforcement learning from human feedback, or RLHF.

“We have been super good at human language for four years, but it’s not useful for automation because computers speak a different language,” Almeida said after Jev was launched.

An automated support system may only need to know which department should receive a customer complaint. Jev can receive the complaint together with the permitted choices and provide probabilities for each.

Traditional classifiers are trained to sort inputs into existing categories. Jev lets one define the choices at the time of the request without training a new classifier for each variation.

This translates into dramatic shortcuts in time and resources. Large language models build an answer sequentially, one token after another, with each token representing a small piece of text generated after the previous one. That adds time and cost even when the business only needs a simple answer like “approve” or “decline”. A decision model can evaluate the choices directly and give a result without first generating a string of tokens.

TypeSafe says Jev always returns one of the permitted answer formats, so software expecting “approve” or “decline” will not instead receive a paragraph of hallucinations.

Much of the excitement has been generated by something as basic as price. TypeSafe charges $42 per billion input tokens, or $0.042 per million, and charges nothing for output tokens. By comparison, OpenAI’s GPT-5.6 Sol costs $4 per million input tokens and $20 per million output tokens. On input alone, Jev is about 95 times cheaper. For a business making millions of automated decisions, that rewrites the rules of AI economics.

OpenAI is nothing if not agile. On 29 September it announced Decisions API in limited preview. Like Jev, it classifies inputs or chooses actions from answers defined by the developer.

Said OpenAI CEO Sam Altman: “By focusing the model on that choice, we can make it extremely fast while keeping capabilities like image understanding, broad language support, and safety protections.”

AWS followed just two days later, on 1 October, with Strands Decider 2B, an open-source model designed for agent workflows. AWS distinguished engineer Marc Brooker commented: “What originally piqued my interest in this class of models was that they make a perfect decider for a workflow step – ‘what is the next thing for me to do here, based on where I am?’”

Cloudflare released Clef and Clef-flash on the same day, offering its own take on the concept. Cloudflare says Clef-flash made decisions in about 39 milliseconds in tests, compared with about 524 milliseconds for Jev. Those are Cloudflare’s own figures, but they show how quickly the competition has moved from copying Jev to trying to beat it on speed.

As usual, there is a catch. Check Point researchers found that Jev can still be tricked into making the wrong decision if the information it receives has been manipulated. In one test, they altered the material Jev was asked to assess and persuaded it to rate a risky investment as safer than it was. Jev gave the answer in the correct format, but the decision itself was wrong.

So companies will still need controls over the information fed into decision models and over the actions that software is allowed to take. In other words, decision engines remove a technical weakness of generative AI, but they do not remove the need to govern the AI itself.

* Arthur Goldstuck is CEO of World Wide Worx, editor-in-chief of Gadget.co.za, and author of “The Hitchhiker’s Guide to AI – The African Edge”.

The post Signpost: Decision engines <br>are the next big hype appeared first on Gadget.