Germany · privatemode.ai

Turning GLM-5.3-Flash into a Jev-like decision model

Sat, Sep 26 · 3:49 PM UTC 2 min read

Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.

◌ Single Source

The prompt numbers the options and it ends with the assistant’s answer already begun as choice_index: , so the next token is the index. A mask on the vocabulary allows only the option indexes, the API returns their log probabilities, and normalizing them over the options gives a probability for each.

Typed decisions with a probability for every option, in a single forward pass: matching Jev's accuracy and speed with an LLM.

Key facts

On TREC, going from 6 to 42 options, Jev drops from 92.1% to 85.6%, GLM-5.3-Flash from 91.2% to 79.6%, and Laya from 88.4% to 51.2%
On RVL-CDIP, a set of 1,600 scanned business documents in 16 classes, GLM-5.3-Flash reaches an accuracy of 70.2% and is the only one of the three that can answer
On MASSIVE, going from 18 scenarios to 59 intents raises the scores of Jev and GLM-5.3-Flash in both languages, while Laya's score drops
The GLM-5.3-Flash prompt adds about 55 tokens of fixed overhead and about 20 per option

Summary

Marko Rosenmüller, PhD. TL;DR: In this post, they show how an off-the-shelf LLM can make typed decisions in a single forward pass. The team evaluate the approach using GLM-5.3-Flash running on Privatemode. As a bonus, the setup with GLM-5.3-Flash enables typed decisions on images, which is not possible with Jev.

Read full article at privatemode.ai →

#Germany