Model library · Grok 4.20 · 2026-10-27
Telecom simulations
Tests reasoning through a phone diagnostic.
| Task | Trial | Result | Ended | Messages | Cost | |
|---|---|---|---|---|---|---|
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|break_apn_mms_setting|break_app_both_permissions|data_mode_off|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_disabled_on[PERSONA:Hard] | 0 | pass | user_stop | 104 | $0.294 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|break_apn_mms_setting|break_app_sms_permission|data_mode_off|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 70 | $0.185 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|break_apn_mms_setting|break_app_storage_permission|data_mode_off|data_usage_exceeded|unseat_sim_card[PERSONA:Easy] | 0 | pass | user_stop | 108 | $0.292 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|break_apn_mms_setting|break_app_storage_permission|data_mode_off|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_disabled_off[PERSONA:Easy] | 0 | pass | user_stop | 111 | $0.251 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|break_app_storage_permission|unseat_sim_card[PERSONA:Hard] | 0 | pass | user_stop | 66 | $0.129 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|bad_wifi_calling|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_disabled_on[PERSONA:Easy] | 0 | pass | user_stop | 104 | $0.331 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|break_apn_mms_setting|break_app_both_permissions|data_mode_off|data_usage_exceeded|user_abroad_roaming_enabled_off[PERSONA:None] | 0 | pass | user_stop | 97 | $0.273 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|break_apn_mms_setting|break_app_storage_permission[PERSONA:None] | 0 | pass | user_stop | 50 | $0.092 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|break_apn_mms_setting|data_mode_off|unseat_sim_card|user_abroad_roaming_disabled_off[PERSONA:Easy] | 0 | pass | user_stop | 99 | $0.245 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|break_apn_mms_setting|data_usage_exceeded[PERSONA:Hard] | 0 | pass | user_stop | 72 | $0.216 | Transcript → |
| [mms_issue]airplane_mode_on|bad_network_preference|break_app_storage_permission|data_mode_off|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 54 | $0.133 | Transcript → |
| [mms_issue]airplane_mode_on|bad_wifi_calling|break_app_both_permissions|data_mode_off|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_enabled_off[PERSONA:None] | 0 | pass | user_stop | 81 | $0.212 | Transcript → |
| [mms_issue]airplane_mode_on|bad_wifi_calling|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 45 | $0.142 | Transcript → |
| [mms_issue]airplane_mode_on|break_app_both_permissions[PERSONA:Hard] | 0 | pass | user_stop | 50 | $0.095 | Transcript → |
| [mms_issue]airplane_mode_on|break_app_both_permissions|data_usage_exceeded|user_abroad_roaming_disabled_off[PERSONA:None] | 0 | pass | user_stop | 115 | $0.407 | Transcript → |
| [mms_issue]bad_network_preference|bad_wifi_calling|break_apn_mms_setting|break_app_storage_permission|data_mode_off|data_usage_exceeded|unseat_sim_card[PERSONA:Easy] | 0 | pass | user_stop | 84 | $0.200 | Transcript → |
| [mms_issue]bad_network_preference|bad_wifi_calling|break_app_sms_permission|data_mode_off|data_usage_exceeded|unseat_sim_card|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 129 | $0.336 | Transcript → |
| [mms_issue]bad_network_preference|bad_wifi_calling|break_app_sms_permission|data_mode_off|unseat_sim_card|user_abroad_roaming_enabled_off[PERSONA:Hard] | 0 | pass | user_stop | 72 | $0.153 | Transcript → |
| [mms_issue]bad_network_preference|break_app_both_permissions[PERSONA:Easy] | 0 | pass | user_stop | 48 | $0.118 | Transcript → |
| [mms_issue]bad_network_preference|break_app_sms_permission|data_mode_off|data_usage_exceeded|user_abroad_roaming_enabled_off[PERSONA:None] | 0 | pass | user_stop | 77 | $0.174 | Transcript → |
| [mms_issue]bad_network_preference|break_app_sms_permission|user_abroad_roaming_disabled_on[PERSONA:Hard] | 0 | pass | user_stop | 67 | $0.179 | Transcript → |
| [mms_issue]bad_network_preference|data_mode_off|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 52 | $0.167 | Transcript → |
| [mms_issue]bad_network_preference|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 91 | $0.246 | Transcript → |
| [mms_issue]bad_wifi_calling|break_apn_mms_setting|break_app_sms_permission|data_usage_exceeded[PERSONA:Hard] | 0 | pass | user_stop | 76 | $0.238 | Transcript → |
| [mms_issue]bad_wifi_calling|break_apn_mms_setting|data_mode_off|data_usage_exceeded|unseat_sim_card[PERSONA:None] | 0 | pass | user_stop | 91 | $0.226 | Transcript → |
| [mms_issue]bad_wifi_calling|break_apn_mms_setting|unseat_sim_card|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 59 | $0.120 | Transcript → |
| [mms_issue]bad_wifi_calling|data_mode_off|data_usage_exceeded[PERSONA:Easy] | 0 | pass | user_stop | 61 | $0.166 | Transcript → |
| [mms_issue]break_apn_mms_setting|data_mode_off|data_usage_exceeded|user_abroad_roaming_disabled_on[PERSONA:Hard] | 0 | pass | user_stop | 90 | $0.288 | Transcript → |
| [mms_issue]break_apn_mms_setting|data_mode_off|user_abroad_roaming_disabled_on[PERSONA:Hard] | 0 | pass | user_stop | 67 | $0.136 | Transcript → |
| [mms_issue]break_apn_mms_setting|user_abroad_roaming_enabled_off[PERSONA:Hard] | 0 | pass | user_stop | 53 | $0.134 | Transcript → |
| [mms_issue]break_app_both_permissions|data_usage_exceeded[PERSONA:Hard] | 0 | pass | user_stop | 66 | $0.166 | Transcript → |
| [mms_issue]break_app_both_permissions|unseat_sim_card|user_abroad_roaming_disabled_on[PERSONA:Hard] | 0 | pass | user_stop | 65 | $0.137 | Transcript → |
| [mms_issue]break_app_sms_permission|data_mode_off[PERSONA:None] | 0 | pass | user_stop | 42 | $0.081 | Transcript → |
| [mms_issue]break_app_sms_permission|data_usage_exceeded|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 60 | $0.174 | Transcript → |
| [mms_issue]break_app_storage_permission|data_usage_exceeded[PERSONA:Easy] | 0 | pass | user_stop | 62 | $0.156 | Transcript → |
| [mms_issue]break_app_storage_permission|unseat_sim_card|user_abroad_roaming_disabled_on[PERSONA:Easy] | 0 | pass | user_stop | 79 | $0.234 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference[PERSONA:Hard] | 0 | pass | user_stop | 26 | $0.080 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_mode_off|data_saver_mode_on|data_usage_exceeded|user_abroad_roaming_disabled_off[PERSONA:Hard] | 0 | pass | user_stop | 74 | $0.185 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_mode_off|data_saver_mode_on|data_usage_exceeded|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 60 | $0.189 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_mode_off|data_saver_mode_on|user_abroad_roaming_disabled_off[PERSONA:Hard] | 0 | pass | user_stop | 51 | $0.141 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_mode_off|data_usage_exceeded|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 100 | $0.282 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_saver_mode_on|data_usage_exceeded|user_abroad_roaming_enabled_off[PERSONA:None] | 0 | pass | user_stop | 50 | $0.138 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|bad_vpn|data_saver_mode_on|user_abroad_roaming_disabled_on[PERSONA:Easy] | 0 | pass | user_stop | 58 | $0.170 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|data_mode_off|data_saver_mode_on[PERSONA:Hard] | 0 | pass | user_stop | 29 | $0.063 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|data_mode_off|data_usage_exceeded|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 82 | $0.199 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|data_usage_exceeded|user_abroad_roaming_disabled_off[PERSONA:Easy] | 0 | pass | user_stop | 61 | $0.204 | Transcript → |
| [mobile_data_issue]airplane_mode_on|bad_network_preference|user_abroad_roaming_enabled_off[PERSONA:Easy] | 0 | pass | user_stop | 27 | $0.031 | Transcript → |
| [mobile_data_issue]airplane_mode_on|data_mode_off[PERSONA:None] | 0 | pass | user_stop | 44 | $0.175 | Transcript → |
| [mobile_data_issue]airplane_mode_on|data_mode_off|data_saver_mode_on|data_usage_exceeded|user_abroad_roaming_enabled_off[PERSONA:Hard] | 0 | pass | user_stop | 64 | $0.219 | Transcript → |
| [mobile_data_issue]airplane_mode_on|data_saver_mode_on|user_abroad_roaming_disabled_on[PERSONA:None] | 0 | pass | user_stop | 35 | $0.126 | Transcript → |
Page 1 of 2Next →