ZAZUMODELSBLOG

Model library · Mistral NeMo 12B · 2026-11-07

Airline simulations

Tests instruction following over a long booking policy.

Allpassfailerror
44 simulations
TaskTrialResultEndedMessagesCost
00failuser_stop10$0.001Transcript →
10failuser_stop22$0.002Transcript →
100failuser_stop96$0.019Transcript →
110failuser_stop40$0.003Transcript →
120failuser_stop44$0.004Transcript →
140failuser_stop62$0.008Transcript →
150failtoo_many_errors67$0.007Transcript →
160failtoo_many_errors70$0.009Transcript →
170failuser_stop69$0.008Transcript →
180failuser_stop68$0.006Transcript →
190failuser_stop34$0.003Transcript →
200failuser_stop42$0.005Transcript →
210failuser_stop50$0.004Transcript →
220failuser_stop21$0.002Transcript →
230failuser_stop62$0.006Transcript →
240failuser_stop72$0.014Transcript →
250failuser_stop74$0.010Transcript →
260failuser_stop21$0.001Transcript →
290failuser_stop32$0.004Transcript →
30failuser_stop20$0.002Transcript →
300failuser_stop51$0.005Transcript →
310failtoo_many_errors96$0.009Transcript →
320failuser_stop38$0.004Transcript →
330failuser_stop67$0.007Transcript →
340failuser_stop30$0.003Transcript →
350failtoo_many_errors106$0.012Transcript →
360failuser_stop32$0.003Transcript →
370failuser_stop18$0.001Transcript →
390failuser_stop34$0.002Transcript →
40failuser_stop34$0.003Transcript →
400failuser_stop18$0.001Transcript →
410failuser_stop28$0.002Transcript →
420failuser_stop33$0.002Transcript →
440failuser_stop36$0.003Transcript →
450failuser_stop84$0.008Transcript →
460failuser_stop25$0.001Transcript →
470failuser_stop20$0.001Transcript →
480failuser_stop19$0.001Transcript →
490failuser_stop22$0.001Transcript →
50failuser_stop27$0.002Transcript →
60failuser_stop40$0.003Transcript →
70failuser_stop22$0.001Transcript →
80failtoo_many_errors76$0.009Transcript →
90failuser_stop18$0.001Transcript →