How models perform on this prompt
Arcee.ai Spotlight
Model answer: 36
ChatGPT-4o (High Reasoning)
Model answer: 33
ChatGPT-4o (Medium Reasoning)
Model answer: 34
Claude 3.5 Haiku
Model answer: 25
Claude 3.7 Sonnet
Model answer: Based on what I can see in the image, there appear to be approximately 20-25 bottles of Sprite in the refrigerator or cooler. The green bottles are arranged on the white rack shelves, with some standing upright and others lying on their sides. It's difficult to give an exact count because some bottles may be partially obscured by others.
Claude 4 Opus
Model answer: I can see multiple green Sprite bottles in this refrigerator image. Let me count them carefully. Looking at the bottles visible in the image, I can count approximately 20 bottles. 20
Claude 4 Sonnet
Model answer: 24
Claude 4.1 Opus
Model answer: I can see multiple green Sprite bottles in this refrigerator image. Counting all the visible bottles, including those partially visible, I count approximately 20 bottles. 20
Cohere Aya Vision 32B
Model answer: 48
Cohere Aya Vision 8B
Model answer: There are 21 bottles in the image.
GPT-4.1
Model answer: 20
GPT-4.1 Mini
Model answer: 29
GPT-4.1 Nano
Model answer: 20
GPT-5
Model answer: 24
GPT-5 (high reasoning)
Model answer: 25
GPT-5 Mini
Model answer: 18
GPT-5 Nano
Model answer: 24
GPT-5 with High Reasoning
Model answer: 26
Gemini 1.5 Flash
Model answer: 20
Gemini 1.5 Pro
Model answer: 22
Gemini 2.0 Flash
Model answer: 30
Gemini 2.0 Flash Lite
Model answer: 35
Gemini 2.5 Flash
Model answer: 25
Gemini 2.5 Flash Lite
Model answer: 33
Gemini 2.5 Pro
Model answer: 31
Gemma 3 27b
Model answer: 31
Gemma 3 4B
Model answer: 33
Llama 4 Maverick 17B
Model answer: 51
Llama 4 Scout 17B
Model answer: 36
Mistral Medium 3
Model answer: 34.
Mistral Small 3.1 24B
Model answer: I don't have the ability to count objects in images.
OpenAI O1
Model answer: 20
OpenAI O3 (High Reasoning)
Model answer: 26
OpenAI O3 (Medium Reasoning)
Model answer: 23
OpenAI O4 Mini (High Reasoning)
Model answer: 15
OpenAI O4 Mini (Medium Reasoning)
Model answer: 20
OpenAI o3-pro
Model answer: 24
Phi 4 Multimodal
Model answer: 54
Qwen 2.5 VL 7B
Model answer: 14