ZAZUMODELSBLOG

Model library · Google · 12B

Gemma 4 12B

From the transcripts Solves about half the tasks, but reads its reasoning out to customers in 1,264 of 3,056 replies even with thinking barred, and its empty replies crashed 30 of 278 chats.

MiniMax-strict customer

Customer simulator: minimax/minimax-m3.1-flash (strict customer patch).

2026-11-07

self-hostedbf16reasoning off
Domainpass^1Higher kSimsInfra errorsAgent cost / taskTranscripts
AirlineInstruction following38.0%—5011$0.0086View →
RetailTool calling55.3%—11419$0.0082View →
TelecomReasoning50.9%—1140$0.0088View →

GLM-5.3 Flash customer

Customer simulator: glm-5.3-flash (gpt-oss-120b on *_fixoss reruns).

2026-10-07

self-hostedreasoning off
Domainpass^1Higher kSimsInfra errorsAgent cost / taskTranscripts
RetailTool calling74.4%—9024—View →