ZAZUMODELSBLOG

Model library · Grok 4.20 · 2026-10-27

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
100 simulations
TaskTrialResultEndedMessagesCost
00failuser_stop46$0.245Transcript →
01failuser_stop12$0.032Transcript →
10failuser_stop27$0.085Transcript →
11failuser_stop25$0.077Transcript →
100failuser_stop104$0.876Transcript →
101passuser_stop62$0.306Transcript →
110passuser_stop23$0.074Transcript →
111passuser_stop16$0.048Transcript →
120passuser_stop36$0.128Transcript →
121failuser_stop32$0.115Transcript →
130passuser_stop12$0.033Transcript →
131passuser_stop20$0.071Transcript →
140failuser_stop39$0.193Transcript →
141failuser_stop33$0.148Transcript →
150passuser_stop32$0.128Transcript →
151passuser_stop26$0.096Transcript →
160passuser_stop29$0.118Transcript →
161passuser_stop27$0.101Transcript →
170passuser_stop31$0.113Transcript →
171passuser_stop35$0.139Transcript →
180passuser_stop54$0.286Transcript →
181failuser_stop44$0.205Transcript →
190passuser_stop27$0.088Transcript →
191passuser_stop27$0.091Transcript →
20failuser_stop31$0.109Transcript →
21failuser_stop37$0.178Transcript →
200passuser_stop29$0.113Transcript →
201passuser_stop22$0.082Transcript →
210failuser_stop39$0.174Transcript →
211failuser_stop46$0.212Transcript →
220passuser_stop29$0.105Transcript →
221passuser_stop31$0.108Transcript →
230failuser_stop62$0.401Transcript →
231failuser_stop33$0.142Transcript →
240failuser_stop36$0.147Transcript →
241failuser_stop35$0.147Transcript →
250passuser_stop24$0.080Transcript →
251passuser_stop26$0.092Transcript →
260failuser_stop21$0.064Transcript →
261failuser_stop24$0.072Transcript →
270passuser_stop20$0.062Transcript →
271passuser_stop22$0.073Transcript →
280failuser_stop22$0.071Transcript →
281failuser_stop20$0.064Transcript →
290failuser_stop27$0.102Transcript →
291failuser_stop26$0.102Transcript →
30failuser_stop18$0.054Transcript →
31passuser_stop16$0.044Transcript →
300failuser_stop24$0.085Transcript →
301failuser_stop28$0.109Transcript →
Page 1 of 2Next →