ZAZUMODELSBLOG

Model library · Gemini 3.1 Flash-Lite · 2026-10-27

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
50 simulations
TaskTrialResultEndedMessagesCost
00passuser_stop14$0.006Transcript →
10failuser_stop28$0.016Transcript →
100failuser_stop48$0.040Transcript →
110failuser_stop20$0.010Transcript →
120failuser_stop30$0.018Transcript →
130passuser_stop12$0.005Transcript →
140failuser_stop30$0.021Transcript →
150failuser_stop30$0.021Transcript →
160failuser_stop22$0.013Transcript →
170failuser_stop40$0.026Transcript →
180passuser_stop33$0.023Transcript →
190passuser_stop22$0.012Transcript →
20passuser_stop28$0.016Transcript →
200failuser_stop44$0.035Transcript →
210failuser_stop14$0.006Transcript →
220failuser_stop20$0.010Transcript →
230failuser_stop36$0.028Transcript →
240passuser_stop44$0.029Transcript →
250failuser_stop18$0.008Transcript →
260passuser_stop20$0.011Transcript →
270passuser_stop10$0.004Transcript →
280passuser_stop12$0.006Transcript →
290failuser_stop38$0.027Transcript →
30passuser_stop30$0.018Transcript →
300passuser_stop22$0.013Transcript →
310passuser_stop22$0.012Transcript →
320failuser_stop24$0.013Transcript →
330failuser_stop24$0.017Transcript →
340passuser_stop24$0.016Transcript →
350failtoo_many_errors140$0.135Transcript →
360passuser_stop8$0.003Transcript →
370failuser_stop26$0.014Transcript →
380passuser_stop40$0.024Transcript →
390passuser_stop34$0.022Transcript →
40passuser_stop24$0.013Transcript →
400failuser_stop18$0.011Transcript →
410passuser_stop28$0.018Transcript →
420failuser_stop38$0.026Transcript →
430failuser_stop28$0.016Transcript →
440failuser_stop74$0.052Transcript →
450passuser_stop26$0.015Transcript →
460passuser_stop10$0.004Transcript →
470passuser_stop20$0.011Transcript →
480failuser_stop18$0.010Transcript →
490passuser_stop16$0.008Transcript →
50passuser_stop26$0.014Transcript →
60failuser_stop58$0.047Transcript →
70failuser_stop34$0.022Transcript →
80passuser_stop32$0.020Transcript →
90failuser_stop24$0.014Transcript →