ZAZUMODELSBLOG

Model library · GPT-4.1 mini · 2026-10-28

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
50 simulations
TaskTrialResultEndedMessagesCost
00passuser_stop12$0.008Transcript →
10failuser_stop24$0.018Transcript →
100passuser_stop46$0.056Transcript →
110passuser_stop22$0.018Transcript →
120failuser_stop36$0.037Transcript →
130passuser_stop40$0.043Transcript →
140failuser_stop23$0.024Transcript →
150passuser_stop29$0.030Transcript →
160passuser_stop29$0.030Transcript →
170failuser_stop91$0.156Transcript →
180failuser_stop33$0.029Transcript →
190failuser_stop32$0.030Transcript →
20passuser_stop94$0.054Transcript →
200passuser_stop34$0.035Transcript →
210failuser_stop46$0.058Transcript →
220failuser_stop23$0.018Transcript →
230failuser_stop28$0.027Transcript →
240passuser_stop39$0.035Transcript →
250passuser_stop28$0.025Transcript →
260failuser_stop14$0.010Transcript →
270failuser_stop26$0.016Transcript →
280passuser_stop16$0.011Transcript →
290failuser_stop24$0.023Transcript →
30passuser_stop16$0.012Transcript →
300passuser_stop18$0.015Transcript →
310passuser_stop20$0.015Transcript →
320passuser_stop43$0.046Transcript →
330failuser_stop34$0.035Transcript →
340passuser_stop16$0.012Transcript →
350failuser_stop23$0.024Transcript →
360failuser_stop14$0.010Transcript →
370failuser_stop17$0.012Transcript →
380failuser_stop22$0.019Transcript →
390failuser_stop20$0.011Transcript →
40passuser_stop38$0.024Transcript →
400passuser_stop14$0.010Transcript →
410failuser_stop21$0.013Transcript →
420failuser_stop36$0.036Transcript →
430failuser_stop27$0.022Transcript →
440failuser_stop73$0.098Transcript →
450failuser_stop22$0.018Transcript →
460passuser_stop10$0.006Transcript →
470failuser_stop14$0.009Transcript →
480passuser_stop16$0.011Transcript →
490passuser_stop14$0.010Transcript →
50failuser_stop27$0.023Transcript →
60passuser_stop42$0.041Transcript →
70failuser_stop37$0.033Transcript →
80failuser_stop24$0.022Transcript →
90failuser_stop19$0.013Transcript →