ZAZUMODELSBLOG

Model library · Gemma 4 31B · 2026-11-09

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
30 simulations
TaskTrialResultEndedMessagesCost
00passuser_stop16$0.007Transcript →
100passuser_stop54$0.038Transcript →
120passuser_stop31$0.017Transcript →
130passuser_stop14$0.006Transcript →
150passuser_stop27$0.019Transcript →
170passuser_stop29$0.015Transcript →
190passuser_stop23$0.011Transcript →
20passuser_stop154$0.068Transcript →
200passuser_stop32$0.020Transcript →
220passuser_stop21$0.009Transcript →
240passuser_stop29$0.015Transcript →
250passuser_stop40$0.025Transcript →
260passuser_stop17$0.007Transcript →
270passuser_stop28$0.013Transcript →
280passuser_stop18$0.008Transcript →
300passuser_stop21$0.010Transcript →
310passuser_stop20$0.009Transcript →
320passuser_stop38$0.026Transcript →
370passuser_stop27$0.013Transcript →
380passuser_stop20$0.010Transcript →
390passuser_stop26$0.010Transcript →
40passuser_stop23$0.009Transcript →
400passuser_stop16$0.007Transcript →
450passuser_stop20$0.009Transcript →
460passuser_stop8$0.003Transcript →
470passuser_stop21$0.010Transcript →
480passuser_stop14$0.006Transcript →
490passuser_stop14$0.006Transcript →
50passuser_stop29$0.017Transcript →
80passuser_stop30$0.017Transcript →