Jev says it is 95% sure. Check that on your own files.

A credit union’s fraud queue does not need an essay. For each alert it needs to know one thing: clear it, escalate it, or put it in front of a person. That is a decision with three possible answers, and most of the work in a regulated office has the same shape.

In September a startup called TypeSafe AI released a model built for that shape. It is called Jev, and it does not write text. You hand it a case and a fixed set of questions, and it hands back an answer to each one with a probability attached. TechSpot’s write-up shows the difference with a developer’s demo. Asked whether an invoice is fraudulent, a chat model spends more than eight seconds writing a sentence. Jev answers in about a tenth of a second with three numbers: 7% fraud, 88% clean, 5% needs review.

The possible answers are fixed before the question is asked.

Jev takes three kinds of question. Pick one option from a list. Place the case on a scale. Say how likely a yes-or-no statement is. The caller writes the list of possible answers in advance, so the model cannot reply with anything outside it. That matters more than it sounds. Most of the trouble with putting a chat model inside software is checking that its answer is one of the allowed ones, and deciding what to do when it is something else.

TypeSafe also says the model is fast and cheap: 70 to 500 milliseconds an answer, $0.042 per million input tokens, and no charge for output. Those are the company’s own figures, from its launch post.

The probability is the whole design.

The interesting part is the number attached to each answer. An engineer quoted by TechSpot called Jev “an AI-native if statement.” A rule that flags every transaction over $10,000 for review becomes a rule that sends a transaction to a person when the model is more than 95% confident it looks suspicious.

TypeSafe’s launch post puts the reason plainly. A model that can do a task 95% of the time but “doesn’t say when it’s in the 5%, it can’t automate that task.” That is right, and it is the same line every bank, clinic and firm draws when it decides what a machine may do alone. Above the line, the case runs by itself. Below it, the case goes to a person with a name.

So the line is only as good as the number. If the model says 95% and is right 95% of the time at that level, the design works. If it says 95% and is right 80% of the time, a fifth of the cases you let run alone are mistakes, and nobody is looking at them.

Calibration is measured on somebody’s data.

TypeSafe calls its probabilities calibrated: higher confidence means higher accuracy. It trained the model for that with a method it calls Reinforcement Learning for Calibrated Decisions. I have no reason to doubt the method. I would still want to know what the calibration was measured against.

TypeSafe is open about this. Its published evaluations run four workflows written by its own team, and the reference answer for each question is the average of what two large models, GPT-6 Astra and Fable 5.1, said. The company also says its headline speed and cost gains are likely at the high end of what customers will see. That is a fair way to compare models with each other. It says nothing yet about whether 95% means 95% on a loan file from a credit union in Oklahoma.

The first independent test points the same way. Five days after the launch, Praneeth Paikray ran Jev on 300 sentences from medical case reports, asking whether each described a drug side effect, and compared its answers with the labels human annotators had given them. Jev found almost every side effect. It also reported complete confidence, exactly 1.0, on 150 of the sentences, and ten of those answers were wrong. Among the 233 answers it gave at 90% confidence or more, 30 were wrong. Rewriting the instructions it was given improved its probabilities a great deal. He also measured median response times of about fifteen seconds, against TypeSafe’s stated 70 to 500 milliseconds, while noting he could not separate the model from the network in between.

One test on one kind of text settles nothing about Jev in general. It does show the check working: the confidence scores looked trustworthy until someone compared them with answers already known.

I spent three years at LexisNexis building AI systems, and this is the lesson that stuck. A model’s accuracy belongs to two things at once: the model, and the data it was measured on. Calibration is a kind of accuracy, so the same rule applies. Move to different data and the probabilities can drift. They drift quietly, because the model keeps reporting a confident number.

We have watched it happen since. A law firm we built for had already tried the leading legal AI products for triaging its documents, and none of them could do the job on its documents. We built an agent for the firm’s own documents and categories.

You can check it with files you already have.

The test needs past cases where you already know the right answer. Fraud alerts your team cleared or confirmed. Referrals that came back complete or incomplete. Contracts where a lawyer already marked the clauses. A few hundred is enough to start.

Run the model on them without the answers. Then sort its answers by the confidence it reported, and count. Of the cases it called 90 to 95% certain, how many did it get right? If it is about nine in ten, the probabilities hold on your files. If it is seven in ten, the number is decoration, and the line you drew at 95% sits somewhere else.

Two more counts are worth the time. Look hardest at the band just above your line, because those are the cases that will run with no one looking. And look for clusters: mistakes that gather on one product, one clinic, one kind of document. An average can look fine while one kind of case is badly off.

Write down what counts as passing before you run anything. A bar picked after the results tends to land wherever the results are.

This is how we report our own work. We trained a model on a regional health system’s patient records to find and rank the patients most likely to be at risk. The number we give for it is how often it agreed with that health system’s clinicians: about 93%. Their calls were the answer key, on their records.

The shape fits a lot of regulated work.

In a bank or credit union: triage a fraud alert, check each item on a loan file checklist, route a member complaint or a dispute to the team that handles it. In a clinic or hospital: check whether each required field in a referral is present, check a record against each criterion for a prior authorization, place a request on an urgency scale. In a law firm: sort documents in a first pass, check for each clause the firm cares about, flag documents that may be privileged.

We have built for this shape before. For a bank, we fine-tuned a language model on the bank’s own data to classify fraud and abuse. For a hospital system, we built an agent that researches and prepares prior authorizations, with staff deciding wherever a decision is needed.

Each of those is a decision somebody already makes many times a day, with a right answer known afterward. That is what makes them testable. It is also why a person should keep the cases the model is unsure about.

Settle the terms before a real file goes near it.

Jev is in early access. It reads text and structured data only. TypeSafe has not published a technical paper or how the model is built, and its launch post says the service currently runs from the US West Coast.

Its customer agreement says TypeSafe will not use your data to train its models without your prior consent. The same agreement lets it use what it calls telemetry, including logs, summary statistics and classifications drawn from your use, without restriction, and says it is under no obligation to keep your data. Its preview terms say the service may not be suitable for production. We found no Business Associate Agreement among its published documents. A bank, clinic or firm should put those terms in front of its compliance team before any member, patient or client file goes in, as it would for any outside AI service, and should expect the model and its prices to change while early access lasts.