ZAZUMODELSBLOG

Model library · Gemma 4 12B · 2026-11-07

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
20 simulations
TaskTrialResultEndedMessagesCost
100failuser_stop30$0.009Transcript →
110failuser_stop26$0.007Transcript →
120failmax_steps202$0.054Transcript →
140failuser_stop50$0.022Transcript →
160failmax_steps202$0.049Transcript →
180failuser_stop86$0.051Transcript →
20failuser_stop38$0.008Transcript →
230failuser_stop60$0.029Transcript →
240failuser_stop8$0.001Transcript →
290failuser_stop38$0.012Transcript →
30failuser_stop12$0.001Transcript →
330failuser_stop36$0.010Transcript →
340failuser_stop32$0.008Transcript →
350failuser_stop32$0.007Transcript →
370failuser_stop36$0.012Transcript →
380failuser_stop54$0.020Transcript →
420failuser_stop62$0.030Transcript →
440failuser_stop62$0.025Transcript →
80failmax_steps202$0.047Transcript →
90failuser_stop54$0.021Transcript →