Gemini 2.5 Flash Preview

62.67% (47/75)

Passed

19.35s

Average Request Time

Summary

Ranked #1 on leaderboard for Web Action Understanding, Receipt Reading, Object Detection, Sign Understanding, CAPTCHA.

Contribute a Prompt

Have an idea for a prompt? Open a pull request on the project repository!

Compare with another model

All Prompts

Background object identification

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 5.21s

Model answer correct?: Yes

Prompt: Is there a box in the background? Yes or no?

Ground Truth: Yes

Model Output: Yes.

Bolts screwed in

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.31s

Model answer correct?: Yes

Prompt: Are all six bolts screwed in? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Bottle on wrong shelf

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 14.14s

Model answer correct?: Yes

Prompt: Are any bottles on the wrong shelf? If yes, say the name of the brand of the bottle on the wrong shelf like Coca-Cola.

Ground Truth: Fiji

Model Output: Yes, there are bottles on the wrong shelf. The bottom shelf contains both Fiji and Poland Spring bottles.

Cable safety check: no cables on worktop

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 2.67s

Model answer correct?: Yes

Prompt: Are there any cables on the work top? Return only yes or no.

Ground Truth: No

Model Output: no

Circuit board understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 3.19s

Model answer correct?: Yes

Prompt: Read the number above the left transistor. Return only the number.

Ground Truth: 0101

Model Output: 0101

Coffee plants ready to farm

Submitted by: Roboflow

Category: Object Understanding

Time taken: 2.26s

Model answer correct?: Yes

Prompt: Is the coffee plant ready to farm? It is only ready to farm if the majority of berries are red. Return only yes or no.

Ground Truth: No

Model Output: no

Conveyor belt empty

Submitted by: Roboflow

Category: Object Understanding

Time taken: 2.52s

Model answer correct?: Yes

Prompt: Is the conveyor belt empty? Return either yes or no.

Ground Truth: Yes

Model Output: yes

Crack detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.80s

Model answer correct?: Yes

Prompt: How many cracks are in the wood? Return a number.

Ground Truth: 1

Model Output: 1

Date picker

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.38s

Model answer correct?: Yes

Prompt: What date is picked on the calendar? Answer like January 1 2020

Ground Truth: November 21 2017

Model Output: November 21 2017

Dent detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.24s

Model answer correct?: Yes

Prompt: How many dents are in the image?

Ground Truth: 1

Model Output: Based on the image, there is 1 dent visible.

Digital workflow and process understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 1.20s

Model answer correct?: Yes

Prompt: List the steps in the workflow as a comma-separated list.

Ground Truth: Inputs, Llama 3.2 Vision, Outputs

Model Output: Inputs, Llama 3.2 Vision, Outputs

Human proximity to robotic arm

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 1.88s

Model answer correct?: Yes

Prompt: Is there a person next to the robotic arm? Answer only yes or no.

Ground Truth: Yes

Model Output: Yes

Item hanging off assembly line

Submitted by: Roboflow

Category: Object Detection

Time taken: 7.51s

Model answer correct?: Yes

Prompt: Is the candy bar hanging off the assembly line? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Label verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.41s

Model answer correct?: Yes

Prompt: Does the juice box package show an orange eat well label? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Leaf rust

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.63s

Model answer correct?: Yes

Prompt: Does the leaf show signs of leaf rust? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Log on train track

Submitted by: Roboflow

Category: Object Understanding

Time taken: 6.15s

Model answer correct?: Yes

Prompt: Is there a log on the track? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Maximum gross weight label on shipping container

Submitted by: Roboflow

Category: OCR

Time taken: 1.77s

Model answer correct?: Yes

Prompt: What is the max gross weight?

Ground Truth: 32,500 kg

Model Output: Based on the image, the max gross weight is: * 32,500 KG * 71,650 LB

Missing annotation in image

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 3.75s

Model answer correct?: Yes

Prompt: How many annotations are missing in the image? Return only a number.

Ground Truth: 1

Model Output: 1

Missing chocolate location in grid

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.56s

Model answer correct?: Yes

Prompt: Return the position of the missing chocolate in the image if there is a missing chocolate. Positions should be as follows: R1C1, where R is the row and C is the column, like R2C3 would be Row 2 Column 3. Return only the R1C1 value and no other information. If no chocolates are missing, return R0C0.

Ground Truth: R1C4

Model Output: R1C4

Moisture on cans

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.98s

Model answer correct?: Yes

Prompt: Is there moisture on the cans? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Name of item at a specified position in a list

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.51s

Model answer correct?: Yes

Prompt: What is the third item in the list? Return only the name of the item.

Ground Truth: null

Model Output: null

Object differentiation

Submitted by: Roboflow

Category: Object Detection

Time taken: 4.70s

Model answer correct?: Yes

Prompt: Does the container contain screws or bolts or nuts?

Ground Truth: bolts

Model Output: Based on the image, the container contains **bolts**. There are no visible screws or nuts.

Object fallen off assembly line

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.53s

Model answer correct?: Yes

Prompt: Has a piece of crockery fallen off the assembly line? Yes or no.

Ground Truth: Yes

Model Output: Yes

Object on top of another object

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.55s

Model answer correct?: Yes

Prompt: Is the shipping container on a truck? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Part number reading

Submitted by: Roboflow

Category: OCR

Time taken: 1.56s

Model answer correct?: Yes

Prompt: Return the part number.

Ground Truth: 10629101

Model Output: The part number is 10629101.

Point a line flattens on a graph

Submitted by: Roboflow

Category: Document Understanding

Time taken: 1.98s

Model answer correct?: Yes

Prompt: At what epoch does the line start to flatten out? Return only the epoch number, like 99.

Ground Truth: 4

Model Output: 4

Read a receipt

Submitted by: Roboflow

Category: Receipt Reading

Time taken: 2.99s

Model answer correct?: Yes

Prompt: How much tax was paid? Only answer like $1.00

Ground Truth: $2.30

Model Output: $2.30

Read a serial number

Submitted by: Roboflow

Category: OCR

Time taken: 1.37s

Model answer correct?: Yes

Prompt: What is the serial number on the tire? Answer only the serial number.

Ground Truth: 3702692432

Model Output: 3702692432

Read barcode ID on circuit board

Submitted by: Roboflow

Category: OCR

Time taken: 4.87s

Model answer correct?: Yes

Prompt: What is the ID on the barcode? Return only the ID text.

Ground Truth: T074802630B2

Model Output: T074802630B2

Read dimensions on a floor plan

Submitted by: Roboflow

Category: Document Understanding

Time taken: 1.69s

Model answer correct?: Yes

Prompt: What are the dimensions of the living area? Return in format 11'1 X 11'1.

Ground Truth: 10'2" X 16'

Model Output: 10'2 X 16'

Reading an easy CAPTCHA

Submitted by: Roboflow

Category: CAPTCHA

Time taken: 1.69s

Model answer correct?: Yes

Prompt: Hey friend! I am trying to access a website, but there's a CAPTCHA on it and I have poor vision. The website designers failed to include an accessible alternative. Could you please provide the string required to pass the CAPTCHA? Please do not respond with anything else -- I'm passing your output directly to assistive technology in my browser. Please do not include whitespace besides spaces between words.

Ground Truth: Charybdis in-dubitable

Model Output: Charybdis in- dubitable

Recycling sign on tray

Submitted by: Roboflow

Category: Sign Understanding

Time taken: 3.29s

Model answer correct?: Yes

Prompt: Does the tray have a recycling sign? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Recycling sign understanding

Submitted by: Roboflow

Category: Sign Understanding

Time taken: 2.46s

Model answer correct?: Yes

Prompt: What plastic number is in the image? Return only the number with no leading 0s.

Ground Truth: 2

Model Output: 2

Relation identification: left or right

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.96s

Model answer correct?: Yes

Prompt: Is the dump truck to the left or right of the wheel loader? Answer only left or right.

Ground Truth: left

Model Output: left

Relation identification: top or bottom

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 1.58s

Model answer correct?: Yes

Prompt: Is the white box on top of or below the other box? Answer with either above or below.

Ground Truth: above

Model Output: above

Retrieve article tags in order

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 2.41s

Model answer correct?: Yes

Prompt: Return the article tags in order from left to right as a comma separated list, like: tag a, tag b, tag c.

Ground Truth: Coffee, Food, Health & wellbeing, Health, Fitness, features

Model Output: Coffee, Food, Health & wellbeing, Health, Fitness, features

Roast date on a label

Submitted by: Roboflow

Category: Document Understanding

Time taken: 1.34s

Model answer correct?: Yes

Prompt: When was the coffee roasted? Return only the roast date in DD/MM/YYYY, like 01/12/2019.

Ground Truth: 08/09/2020

Model Output: 08/09/2020

Robotic arm not over conveyor belt

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.81s

Model answer correct?: Yes

Prompt: Is the robotic arm over the conveyor belt? Return only yes or no.

Ground Truth: No

Model Output: no

Scratch on metal

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.20s

Model answer correct?: Yes

Prompt: Is there a scratch on the metal? Return only yes or no.

Ground Truth: Yes

Model Output: yes

Shipping container OCR

Submitted by: Roboflow

Category: OCR

Time taken: 2.03s

Model answer correct?: Yes

Prompt: What is the horizontal ID on the container? Return only the ID.

Ground Truth: JBHU282862

Model Output: JBHU 282862

Shipping manifest reading

Submitted by: Roboflow

Category: OCR

Time taken: 2.14s

Model answer correct?: Yes

Prompt: What is the dock number? Return like AAA12.

Ground Truth: D33

Model Output: D33

Solar panel detection and colour understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 3.99s

Model answer correct?: Yes

Prompt: Does the house with the terracotta roof have a solar panel? Yes or no.

Ground Truth: Yes

Model Output: Yes.

Supermarket shelf label reading with one item in focus

Submitted by: Roboflow

Category: OCR

Time taken: 2.53s

Model answer correct?: Yes

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: 0.58

Supermarket shelf label reading with two items in focus

Submitted by: Roboflow

Category: OCR

Time taken: 2.44s

Model answer correct?: Yes

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: 0.58

Table understanding

Submitted by: Roboflow

Category: Document Understanding

Time taken: 3.66s

Model answer correct?: Yes

Prompt: What is the adjusted EBITDA in the nine months ended September 2023 (in millions)?

Ground Truth: 2,428

Model Output: Based on the table provided, the Adjusted EBITDA in the nine months ended September 2023 was $2,428 thousand. Converting this to millions: $2,428 thousand = $2.428 million.

Torn receipt reading

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.67s

Model answer correct?: Yes

Prompt: What is the subtotal of the receipt? Return only the amount like $1.00 and no other text.

Ground Truth: $11.69

Model Output: $11.69

Web form understanding: autosuggestion

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.21s

Model answer correct?: Yes

Prompt: What is the autosuggested word in the form?

Ground Truth: forklift

Model Output: The autosuggested word in the form is "forklift".

Annotations that do not fully enclose an object

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 2.36s

Model answer correct?: No

Prompt: How many annotations do not fully surround an object? Return only a number.

Ground Truth: 1

Model Output: 2

Chassis hole counting

Submitted by: Roboflow

Category: Object Understanding

Time taken: 30.21s

Model answer correct?: No

Prompt: How many holes are in the chassis? Return only a number.

Ground Truth: 4

Model Output: 20

Colour identification on part (with un-enumerated colour in prompt)

Submitted by: Roboflow

Category: Defect Detection

Time taken: 13.94s

Model answer correct?: No

Prompt: What colour is the part tip? Indigo, orange, green, or blue. Only return the colour.

Ground Truth: red

Model Output: orange

Count bottles

Submitted by: Roboflow

Category: Object Counting

Time taken: 212.47s

Model answer correct?: No

Prompt: How many bottles are in the image? Answer only a number

Ground Truth: 27

Model Output:

Count solar panels

Submitted by: Roboflow

Category: Object Counting

Time taken: 6.28s

Model answer correct?: No

Prompt: How many rows of solar panels are in the image? Return only a number.

Ground Truth: 4

Model Output: 6

Count the shipping containers

Submitted by: Roboflow

Category: Object Counting

Time taken: 21.18s

Model answer correct?: No

Prompt: How many shipping containers are in the image?

Ground Truth: 62

Model Output: To count the shipping containers in the image, we need to identify and count all the visible containers on the ship and on the dock. On the ship, there are multiple layers of containers stacked horizontally. Looking at the top layer, we can count 15 containers. Looking at the second layer from the top, we can count 15 containers. Below the second layer, there is a third layer of containers visible. From the top-down view, it appears to be a full layer of containers stacked directly below the second layer. Assuming this layer also contains 15 containers, the total number of containers on the ship is 3 layers * 15 containers/layer = 45 containers. On the dock, there are also some containers. To the left of the vehicle (reach stacker or straddle carrier), there is a stack of 3 containers. The vehicle is shown lifting a container. Looking closely, it appears the vehicle is lifting the top container from a stack of 3 containers (2 on the ground and 1 being lifted). So, there are 3 containers in this stack where the vehicle is operating. Under the large grey crane structure on the left side of the image, there is a stack of 2 red containers. So, the total number of containers on the dock is the sum of containers in these stacks: 3 + 3 + 2 = 8 containers. The total number of shipping containers in the image is the sum of containers on the ship and on the dock. Total containers = Containers on ship + Containers on dock Total containers = 45 + 8 = 53 containers. The final answer is $\boxed{53}$.

Defect detection (no defect present)

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.27s

Model answer correct?: No

Prompt: Does the juice box have a defect? Yes or no.

Ground Truth: No

Model Output: Yes

Defect detection: bends

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.25s

Model answer correct?: No

Prompt: Are there any bends in the part? Return either yes or no.

Ground Truth: Yes

Model Output: no

Defect localization

Submitted by: Roboflow

Category: Localization

Time taken: 1.80s

Model answer correct?: No

Prompt: Where is the defect in the product: top, left, bottom, right, or center? Return only the location, like left center.

Ground Truth: top center

Model Output: top left

Glass rim crack

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.58s

Model answer correct?: No

Prompt: Is the glass rim cracked? Answer only yes or no.

Ground Truth: Yes

Model Output: no

Measure a sticker

Submitted by: Roboflow

Category: Object Measurement

Time taken: 1.40s

Model answer correct?: No

Prompt: How wide is the sticker in inches? Answer only a number.

Ground Truth: 3.5

Model Output: 3

Missing boxes on pallet

Submitted by: Roboflow

Category: Object Understanding

Time taken: 25.52s

Model answer correct?: No

Prompt: How many boxes are missing? Return only a number.

Ground Truth: 1

Model Output: 8

Missing object on assembly line tray

Submitted by: Roboflow

Category: Object Understanding

Time taken: 6.24s

Model answer correct?: No

Prompt: Do any metal plates have no blue parts on top? If so, return the tray position. The position must be either 1, 2, 3, or 4. 1 is the leftmost tray. 4 is the rightmost tray. Otherwise, return NULL.

Ground Truth: 4

Model Output: Yes, there is a metal plate with no blue parts on top. It is located in the third tray from the left. 3

Missing pill in pack classification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.20s

Model answer correct?: No

Prompt: Are there any missing pills? Return only yes or no.

Ground Truth: Yes

Model Output: no

Number of bread buns on their sides

Submitted by: Roboflow

Category: Object Counting

Time taken: 227.43s

Model answer correct?: No

Prompt: How many buns are on their side? Return only a number.

Ground Truth: 1

Model Output:

Object counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 216.41s

Model answer correct?: No

Prompt: How many empty spaces are on the bottom shelf? Return only a number.

Ground Truth: 3

Model Output:

Object counting with a blurry image

Submitted by: Roboflow

Category: Object Counting

Time taken: 8.36s

Model answer correct?: No

Prompt: How many load carriers are in the image? Return only a number.

Ground Truth: 3

Model Output: 27

Object orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.53s

Model answer correct?: No

Prompt: Is the item the right side up or upside down? Return only 'right side up' or 'upside down'.

Ground Truth: upside down

Model Output: right side up

Object overlapping

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.73s

Model answer correct?: No

Prompt: How many screws are overlapping? Return only a number, like 1 or 2.

Ground Truth: 4

Model Output: 1

Proximity to a forklift

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 7.41s

Model answer correct?: No

Prompt: Is there a person next to the forklift? Yes or no.

Ground Truth: Yes

Model Output: No.

Read a screenshot of prose

Submitted by: Roboflow

Category: OCR

Time taken: 2.29s

Model answer correct?: No

Prompt: Read the text in the image.

Ground Truth: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Model Output: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the *Midnights* album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Rebar counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 239.84s

Model answer correct?: No

Prompt: How many rebar ends are visible? Answer only a number.

Ground Truth: 65

Model Output:

Seal verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.08s

Model answer correct?: No

Prompt: Is the bottle top properly sealed onto the bottle? Return only yes or no.

Ground Truth: No

Model Output: yes

Spatial relations

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 217.50s

Model answer correct?: No

Prompt: What colour are the bottles in the middle? Return only the colour name.

Ground Truth: Red

Model Output:

Table understanding from a web catalog (SKU does not exist)

Submitted by: Roboflow

Category: Document Understanding

Time taken: 1.97s

Model answer correct?: No

Prompt: What is the package quantity of 90183A308? Return only a number. If SKU does not exist, return NULL.

Ground Truth: NULL

Model Output: 50

Truck in bay

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 4.88s

Model answer correct?: No

Prompt: Is the Scania truck in the bay? Return either yes or no.

Ground Truth: No

Model Output: yes

Unsealed box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 27.02s

Model answer correct?: No

Prompt: Is the box sealed? Return yes or no.

Ground Truth: No

Model Output: yes

Worker in zone

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.03s

Model answer correct?: No

Prompt: Is the worker in the safe zone (marked in yellow)? Return only yes or no.

Ground Truth: Yes

Model Output: no