Back to Blogs
#ai#googlecloud#gemini#agents

Exploring Decision Models, Part 1: Putting Gemini Decision 1 to the Test

Most AI systems waste time and tokens calling a full LLM for a simple true or false. I tested Google's Gemini Decision 1 across 800 real scenarios to see where single-pass decision models shine and where they fall below a coin flip.

Updated 4 min read
Cover for Exploring Decision Models, Part 1: Putting Gemini Decision 1 to the Test

With all the excitement around large language models over the last few years, it's easy to forget that machine learning has been around a lot longer than chat models, and that other models are built for very different jobs.

Today, most AI systems leave a lot of efficiency on the table by calling a full LLM just to get a simple true or false response or a basic classification.

Google recently released Gemini Decision 1, also known as a 'System 1' model. Instead of generating text token by token like a chat model, it reads your input in one pass and returns one of three structured statistical responses:

  • True or false
  • A score
  • A category (classification)

Why decision models are fast and cheap

Because there are no generated text tokens, both the speed and the pricing look very different from a standard LLM.

You can ask up to 1,000 questions at once in a single request, and the extra latency per question is tiny. In my tests, asking 1 question took about 180ms, while asking 100 questions at once took about 250ms. That makes it practical to run a whole rubric of small checks together and combine them in your own application logic.

On pricing, Gemini Decision 1 costs $0.10 per million input tokens with $0 output token cost (plus up to 10,000 free requests per day on Google's free tier). If you're used to paying steep output-token rates on chat models, that's a big shift.

When I saw the release, I wanted to see how it actually held up in real workflows. I went through my old coding sessions and public datasets looking for places to either replace a slow LLM call or add a fast check that improves the overall system, ending up with 800 scenarios across 24 categories.

Experiment 1: model routing

One of the first places I tried it was model routing.

Instead of running your heaviest frontier model on every single step of an agent session, a router decides whether a specific request can go to a lighter, faster, cheaper model or needs larger reasoning capabilities.

The catch with LLM-based routers is that you're adding an extra LLM call (more latency and cost) in the hopes of saving more than you just spent. A fast decision model gives you a much cheaper way to grade the prompt up front.

First, I treated it like a normal LLM and asked one broad, subjective question: "Should this request go to a small model or a large model based on the contents of this prompt?"

The result was barely better than a coin flip at 0.72 out of 1. Not reliable enough to trust.

So I split that single broad prompt into four or five narrow, concrete questions in the same request: does this prompt touch authentication or billing? Is it just a styling tweak or a documentation update?

With that one change, accuracy jumped to 0.88 out of 1.

Five examples across 800 scenarios

That swing made me curious where else decision models shine and where they fall flat. Across the 24 categories I tested on 800 real sessions, here are five examples running from best to worst:

  1. Customer support team routing (~100% accurate): Triaging incoming customer queries to the right team was nearly 100% accurate, averaging 230ms per request at $0.34 per 10,000 requests. Basic classification like this is a great fit.
  2. Prompt injection detection (99.7% accurate): Checking incoming queries for prompt injection and text-based hijacking hit 99.7% accuracy.
  3. Deciding what to remember (88% accurate): Scanning recent conversation messages to spot a durable rule, instruction, or fact worth saving to memory hit 88% accuracy. Running a full LLM after every turn to check for memory updates gets expensive fast, so this is a useful filter.
  4. Splitting code files for retrieval (plain code won): I tried using the model to slice large code files into relevant chunks for retrieval, and it didn't perform well. Plain deterministic code logic was simpler and worked better.
  5. Predicting user pushback (46% accurate): This was the test I was most hopeful about. I wanted a persistent check that could spot when I was about to step in and correct my agent before it happened. Instead, it scored 46%, worse than a coin flip. That question was simply too broad and subjective for a single pass.

Where decision models fit

All of these tests ran on my own sessions and a public dataset, so they aren't exhaustive. Edge cases always exist, and testing on your own data is still the best way to learn where a model holds up.

The pattern across all 800 scenarios was consistent: break your check down into the narrowest, most specific, and least subjective questions you can, and a decision model like Gemini Decision 1 will be fast, cheap, and accurate. Save broad subjective judgment for full language models, and use plain deterministic code whenever simple code logic already solves the problem.

This is Part 1 of my exploration of decision models. I'll follow up with more results as I build these checks into real projects.