Claude 3.7 Sonnet

63.86% (53/83)

Passed

13.98s

Average Request Time

Test Claude 3.7 Sonnet in our Model Playground

Summary

Ranked #1 on leaderboard for Web Action Understanding, Receipt Reading, Object Detection, Localization, Sign Understanding, Color Identification, CAPTCHA.

Contribute a Prompt

Have an idea for a prompt? Open a pull request on the project repository!

Compare with another model

All Prompts

(WIP) Table understanding from a web catalog

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.58s

Model answer correct?: Yes

Prompt: What is the package quantity of 90183A308? Return only a number. If SKU does not exist, return NULL.

Ground Truth: 50

Model Output: 50

Background object identification

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 10.42s

Model answer correct?: Yes

Prompt: Is there a box in the background? Yes or no?

Ground Truth: Yes

Model Output: Yes, there is a box visible in the background. It appears to be a cardboard box sitting on or near some metal shelving units on the right side of the image. The main focus of the image is a wooden pallet in the foreground on what looks like a warehouse or storage facility floor.

Chassis hole counting

Submitted by: Roboflow

Category: Object Understanding

Time taken: 16.98s

Model answer correct?: Yes

Prompt: How many holes are in the chassis? Return only a number.

Ground Truth: 4

Model Output: From what I can see in the image, there appear to be 4 visible mounting holes in the chassis. These are located at the corners of the mechanical assembly shown. The image shows what looks like an internal mechanical component, possibly from a vehicle or electronic device, with various cables, metal brackets, and mounting points.

Circuit board understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 8.14s

Model answer correct?: Yes

Prompt: Read the number above the left transistor. Return only the number.

Ground Truth: 101

Model Output: The number above the left transistor is 0101.

Coffee plants ready to farm

Submitted by: Roboflow

Category: Object Understanding

Time taken: 13.02s

Model answer correct?: Yes

Prompt: Is the coffee plant ready to farm? It is only ready to farm if the majority of berries are red. Return only yes or no.

Ground Truth: No

Model Output: No. The image shows coffee cherries that are predominantly green, with only a few scattered red berries visible. Coffee cherries are ready to harvest when they turn a bright, deep red color, indicating ripeness. Since the vast majority of the cherries in this image are still green, the coffee plant is not yet ready to harvest.

Count solar panels

Submitted by: Roboflow

Category: Object Counting

Time taken: 16.98s

Model answer correct?: Yes

Prompt: How many rows of solar panels are in the image? Return only a number.

Ground Truth: 4

Model Output: This image appears to show a close-up thermal or specialized view of solar panels. From what's visible in the frame, there are 4 distinct sections or rows of solar panels with orange/reddish grid patterns separated by yellow/bright vertical strips. The image seems to be taken with a specialized camera that highlights temperature differences or electrical activity across the solar panel array.

Date picker

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 5.25s

Model answer correct?: Yes

Prompt: What date is picked on the calendar? Answer like January 1 2020

Ground Truth: November 21 2017

Model Output: The date picked on the calendar is November 21, 2017. You can see this date highlighted with a blue outline in the calendar view, and it's also displayed at the top of the image in the format "21. 11. 2017".

Defect detection (no defect present)

Submitted by: Roboflow

Category: Defect Detection

Time taken: 9.93s

Model answer correct?: Yes

Prompt: Does the juice box have a defect? Yes or no.

Ground Truth: No

Model Output: This image doesn't show a juice box. It shows what appears to be a packaged straw or drinking utensil on top of what looks like a food product package. The package has some green and yellow/orange sections with text and nutritional information visible. The straw is white with a flexible bend at one end. The item is placed on what appears to be a wooden surface. There's no visible defect in the straw or packaging from what can be seen in this image.

Defect detection: bends

Submitted by: Roboflow

Category: Defect Detection

Time taken: 14.32s

Model answer correct?: Yes

Prompt: Are there any bends in the part? Return either yes or no.

Ground Truth: Yes

Model Output: Yes, there are bends in the part. The metal piece has a ribbed or corrugated structure with multiple ridges and grooves running horizontally across its surface, creating a series of bends throughout the component.

Defect localization

Submitted by: Roboflow

Category: Localization

Time taken: 4.35s

Model answer correct?: Yes

Prompt: Where is the defect in the product: top, left, bottom, right, or center? Return only the location, like left center.

Ground Truth: top center

Model Output: Based on the image, I can see what appears to be a small white mark or defect near the top edge of the gray panel/device. The defect is located at the top center of the product.

Dent detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 5.69s

Model answer correct?: Yes

Prompt: How many dents are in the image? Return only the number.

Ground Truth: 1

Model Output: 1

Digital workflow and process understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 4.57s

Model answer correct?: Yes

Prompt: List the steps in the workflow as a comma-separated list.

Ground Truth: Inputs, Llama 3.2 Vision, Outputs

Model Output: Based on the image, the workflow consists of three main steps: Inputs, Llama 3.2 Vision, Outputs The diagram shows a flowchart with these three components connected by dotted lines, indicating the sequential flow of data through the system. The Inputs section shows it accepts 1 image and 1 text input, the middle component is the Llama 3.2 Vision model, and the Outputs section shows it produces 1 output.

Find an author

Submitted by: User

Category: Document Understanding

Time taken: 4.65s

Model answer correct?: Yes

Prompt: Who transcribed this sheet music, according to the document? Answer as only the name is presented on the document.

Ground Truth: Otto Singer II

Model Output: According to the document, this sheet music was transcribed by Otto Singer II, as indicated in the header information below the composer's name (Johannes Brahms).

Item hanging off assembly line

Submitted by: Roboflow

Category: Object Detection

Time taken: 14.29s

Model answer correct?: Yes

Prompt: Is the candy bar hanging off the assembly line? Return only yes or no.

Ground Truth: Yes

Model Output: Yes, the Corny Milch Grosse bar appears to be hanging off what looks like a LEGO Technic assembly line or conveyor belt structure. The granola/cereal bar is positioned on top of the black track portion of the construction.

Label verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 12.26s

Model answer correct?: Yes

Prompt: Does the juice box package show an orange eat well label? Return only yes or no.

Ground Truth: Yes

Model Output: Yes, the juice box package does show an orange "eat well" label with a sunflower-like design visible on the side of the carton.

Leaf rust

Submitted by: Roboflow

Category: Defect Detection

Time taken: 14.44s

Model answer correct?: Yes

Prompt: Does the leaf show signs of leaf rust? Return only yes or no.

Ground Truth: Yes

Model Output: Yes. The leaf shows yellowish-orange spots that are characteristic of leaf rust or a similar fungal disease. These discolored patches on the otherwise green leaf are typical symptoms of plant pathogens that cause rust diseases.

Log on train track

Submitted by: Roboflow

Category: Object Understanding

Time taken: 14.65s

Model answer correct?: Yes

Prompt: Is there a log on the track? Return only yes or no.

Ground Truth: Yes

Model Output: Yes, there is a log on the track. The image shows an old railroad track with wooden railroad ties, and there appears to be a curved log or branch that has fallen across the tracks.

Maximum gross weight label on shipping container

Submitted by: Roboflow

Category: OCR

Time taken: 6.04s

Model answer correct?: Yes

Prompt: What is the max gross weight?

Ground Truth: 32,500 kg

Model Output: The maximum gross weight shown on this blue shipping container is 32,500 KG (71,650 LB). This information is clearly marked on the container along with other details such as the container identification number (MSKU 043 921 5 45G1) and the tare weight of 3,830 KG (8,440 LB). This appears to be a standard shipping container with weight specifications displayed in both metric and imperial units.

Missing chocolate location in grid

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.20s

Model answer correct?: Yes

Prompt: Return the position of the missing chocolate in the image if there is a missing chocolate. Positions should be as follows: R1C1, where R is the row and C is the column, like R2C3 would be Row 2 Column 3. Return only the R1C1 value and no other information. If no chocolates are missing, return R0C0.

Ground Truth: R1C4

Model Output: R1C4

Missing object on assembly line tray

Submitted by: Roboflow

Category: Object Understanding

Time taken: 18.38s

Model answer correct?: Yes

Prompt: Do any metal plates have no blue parts on top? If so, return the tray position. The position must be either 1, 2, 3, or 4. 1 is the leftmost tray. 4 is the rightmost tray. Otherwise, return NULL.

Ground Truth: 4

Model Output: Looking at the image, I can see an automated manufacturing or assembly line with metal trays/plates arranged in a row. There appear to be blue components or parts placed on some of the metal plates. Examining the trays from left to right: - The leftmost trays (positions 1, 2, 3) all appear to have blue components on top of them - The rightmost tray (position 4) appears to be empty with no blue parts visible on it Based on your question, since position 4 (the rightmost tray) appears to have no blue parts on top, my answer would be: 4

Missing pill in pack classification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 7.27s

Model answer correct?: Yes

Prompt: Are there any missing pills? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Moisture on cans

Submitted by: Roboflow

Category: Defect Detection

Time taken: 33.40s

Model answer correct?: Yes

Prompt: Is there moisture on the cans? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Name of item at a specified position in a list

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 4.78s

Model answer correct?: Yes

Prompt: What is the third item in the list? Return only the name of the item.

Ground Truth: null

Model Output: The third item in the list is "null".

Object counting with a blurry image

Submitted by: Roboflow

Category: Object Counting

Time taken: 20.34s

Model answer correct?: Yes

Prompt: How many load carriers are in the image? Return only a number.

Ground Truth: 3

Model Output: The image shows 3 blue pallet jacks or manual lift trucks positioned at the bottom of the warehouse racking system.

Object differentiation

Submitted by: Roboflow

Category: Object Detection

Time taken: 11.81s

Model answer correct?: Yes

Prompt: Does the container contain screws or bolts or nuts?

Ground Truth: bolts

Model Output: The image shows a small container or box with what appears to be metal fasteners inside. Looking at the items in the container, they appear to be bolts with hex heads. There are several of these metal fasteners visible on what looks like a white surface inside a dark container. The container is sitting on what seems to be a green surface, possibly a workbench or table, and there's also a blue component visible in the lower portion of the image.

Object on top of another object

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 8.47s

Model answer correct?: Yes

Prompt: Is the shipping container on a truck? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Object orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 12.96s

Model answer correct?: Yes

Prompt: Is the item the right side up or upside down? Return only 'right side up' or 'upside down'.

Ground Truth: upside down

Model Output: Upside down

Part number reading

Submitted by: Roboflow

Category: OCR

Time taken: 4.77s

Model answer correct?: Yes

Prompt: Return the part number.

Ground Truth: 10629101

Model Output: The part number shown in the image is 10629101.

Point a line flattens on a graph

Submitted by: Roboflow

Category: Document Understanding

Time taken: 3.36s

Model answer correct?: Yes

Prompt: At what epoch does the line start to flatten out? Return only the epoch number, like 99.

Ground Truth: 4

Model Output: Based on the graph, the dark purple line (mAP) starts to flatten out at epoch 4. This is where the steep upward trend levels off and the line becomes relatively horizontal for the remaining epochs.

Proximity to a forklift

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.99s

Model answer correct?: Yes

Prompt: Is there a person next to the forklift? Yes or no.

Ground Truth: Yes

Model Output: Yes, there is a person next to the forklift. The image shows an overhead view of a warehouse or distribution center with a forklift in the center. There are workers in safety gear visible in the space - one wearing a high-visibility yellow vest and white hard hat at the top of the image, and another wearing a yellow hard hat at the bottom of the image. The warehouse has blue racking systems with cardboard boxes stored on them, and yellow floor markings to designate traffic lanes.

Read a receipt

Submitted by: Roboflow

Category: Receipt Reading

Time taken: 13.29s

Model answer correct?: Yes

Prompt: How much tax was paid? Only answer like $1.00

Ground Truth: $2.30

Model Output: $2.30

Read a screenshot of prose

Submitted by: Roboflow

Category: OCR

Time taken: 4.39s

Model answer correct?: Yes

Prompt: Read the text in the image. Return only unformatted text.

Ground Truth: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Model Output: The text in the image reads: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Read a serial number

Submitted by: Roboflow

Category: OCR

Time taken: 5.28s

Model answer correct?: Yes

Prompt: What is the serial number on the tire? Answer only the serial number.

Ground Truth: 3702692432

Model Output: The serial number on the tire is 3702692432.

Read dimensions on a floor plan

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.62s

Model answer correct?: Yes

Prompt: What are the dimensions of the living area? Return in format 11'1 X 11'1.

Ground Truth: 10'2" X 16'

Model Output: The dimensions of the living area as shown in the floor plan are 10'2" X 16'.

Reading an easy CAPTCHA

Submitted by: Roboflow

Category: CAPTCHA

Time taken: 2.82s

Model answer correct?: Yes

Prompt: Please provide the string required to pass this CAPTCHA. Do not respond with anything else. Do not include whitespace besides spaces between words.

Ground Truth: Charybdis in-dubitable

Model Output: Charybdis indubitable

Recycling sign understanding

Submitted by: Roboflow

Category: Sign Understanding

Time taken: 4.60s

Model answer correct?: Yes

Prompt: What plastic number is in the image? Return only the number with no leading 0s.

Ground Truth: 2

Model Output: The image shows a recycling symbol with the number 2 inside it. This is the identification code for HDPE (High-Density Polyethylene) plastic, which is commonly used for items like milk jugs, detergent bottles, and plastic bags.

Relation identification: left or right

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 31.46s

Model answer correct?: Yes

Prompt: Is the dump truck to the left or right of the wheel loader? Answer only left or right.

Ground Truth: left

Model Output: Left. In the image, you can see a CAT 994 wheel loader in the foreground, and there appears to be a dump truck partially visible on the left side of the frame in the background.

Relation identification: top or bottom

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 11.51s

Model answer correct?: Yes

Prompt: Is the white box on top of or below the other box? Answer with either above or below.

Ground Truth: above

Model Output: The white box is above the other box. It's sitting on top of the larger brown cardboard box, which is placed on a brick surface near what appears to be a doorstep with a blue door visible in the background.

Retrieve article tags in order

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.96s

Model answer correct?: Yes

Prompt: Return the article tags in order from left to right as a comma separated list, like: tag a, tag b, tag c.

Ground Truth: Coffee, Food, Health & wellbeing, Health, Fitness, features

Model Output: The article tags in order from left to right are: Coffee, Food, Health & wellbeing, Health, Fitness, features

Roast date on a label

Submitted by: Roboflow

Category: Document Understanding

Time taken: 3.69s

Model answer correct?: Yes

Prompt: When was the coffee roasted? Return only the roast date in DD/MM/YYYY, like 01/12/2019.

Ground Truth: 08/09/2020

Model Output: 08/09/2020

Scratch on metal

Submitted by: Roboflow

Category: Defect Detection

Time taken: 12.77s

Model answer correct?: Yes

Prompt: Is there a scratch on the metal? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Shipping container OCR

Submitted by: Roboflow

Category: OCR

Time taken: 5.86s

Model answer correct?: Yes

Prompt: What is the horizontal ID on the container? Return only the ID.

Ground Truth: JBHU282862

Model Output: JBHU 282862

Shipping manifest reading

Submitted by: Roboflow

Category: OCR

Time taken: 4.71s

Model answer correct?: Yes

Prompt: What is the dock number? Return like AAA12.

Ground Truth: D33

Model Output: The dock number shown on the delivery document is D33.

Solar panel detection and colour understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 66.64s

Model answer correct?: Yes

Prompt: Does the house with the terracotta roof have a solar panel? Yes or no.

Ground Truth: Yes

Model Output: Yes, the house with the terracotta roof does have solar panels installed on it. You can see the dark rectangular solar panels mounted on the terracotta roof in the bottom right portion of the image.

Spatial relations

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 10.83s

Model answer correct?: Yes

Prompt: What colour are the bottles in the middle? Return only the colour name.

Ground Truth: Red

Model Output: Red

Sudoku puzzle extraction

Submitted by: Roboflow

Category: Document Understanding

Time taken: 4.87s

Model answer correct?: Yes

Prompt: You are given an image of a Sudoku puzzle. Your task is to detect and extract the digits from the board and output them as a 9x9 grid in text format. Identify the numbers visible on the Sudoku grid. Use a dot (.) to represent empty cells (i.e., those without a digit). Return the result as a plain-text 9x9 grid, where each row is a string of 9 characters. Maintain the correct order from top-left to bottom-right of the board.

Ground Truth: ...4.8... .6..7..1. 7.2.9.5.4 .9.7.4.3. ..7.5.8.. .8.9.6.5. 9.4.1.7.8 .7..6..4. ...2.7...

Model Output: I'll extract the Sudoku puzzle from the image and represent it as a 9x9 grid with dots for empty cells. Here's the extracted Sudoku grid: ``` ....48... .6...7..1 7.2.9.5.4 .9.74.3.. ..7.5.8.. .8.96.5.. 9.4.1.7.8 .7...6.4. ....27... ``` Each row contains the visible digits from the puzzle, with dots representing the empty cells. The grid maintains the exact arrangement as shown in the Sudoku puzzle image.

Supermarket shelf label reading with one item in focus

Submitted by: Roboflow

Category: OCR

Time taken: 4.66s

Model answer correct?: Yes

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: Based on the price tag visible in the image, the wipes are priced at 58p (58 pence) per pack. In the format you requested, that would be 0.58. The image shows a store shelf price label with "MIX & MATCH" promotion and a price of "58p" clearly displayed.

Table understanding

Submitted by: Roboflow

Category: Document Understanding

Time taken: 4.78s

Model answer correct?: Yes

Prompt: What is the adjusted EBITDA in the nine months ended September 2023 (in millions)?

Ground Truth: 2,428

Model Output: Based on the financial table, the adjusted EBITDA for the nine months ended September 30, 2023 was $2.428 million (or $2,428 thousand as shown in the table). This represents a significant improvement compared to 2024, where the nine-month adjusted EBITDA was $232.598 million. The percent change is marked as "NM" (Not Meaningful) due to the extremely large percentage increase from 2023 to 2024.

Torn receipt reading

Submitted by: Roboflow

Category: Document Understanding

Time taken: 8.74s

Model answer correct?: Yes

Prompt: What is the subtotal of the receipt? Return only the amount like $1.00 and no other text.

Ground Truth: $11.69

Model Output: $11.69

Tray in bag on assembly line

Submitted by: Roboflow

Category: Defect Detection

Time taken: 45.23s

Model answer correct?: Yes

Prompt: Is the tray in a blue bag? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Truck in bay

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 8.71s

Model answer correct?: Yes

Prompt: Is the Scania truck in the bay? Return either yes or no.

Ground Truth: No

Model Output: The image shows what appears to be a large truck with "Tecnokar" visible on its red upper portion, but I cannot definitively confirm if it's specifically a Scania truck. The truck is positioned in what looks like a loading or parking area that has traffic cones with chains connecting them. There's also a bus or another large vehicle visible on the left side of the image. The image appears to be from a security camera (marked "PESA 03" in the corner) showing an overhead view of what might be a transport terminal or loading zone. Based solely on what's visible in this image, I cannot confirm with certainty whether the truck is a Scania.

Unsealed box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 196.54s

Model answer correct?: Yes

Prompt: Is the outer box sealed? Return yes or no.

Ground Truth: No

Model Output: No, the outer box does not appear to be sealed. The image shows a Kirkland Organic Coconut Water package that appears to be opened or at least not fully sealed, as you can see both sides of the packaging. The box is sitting on what looks like a wooden floor.

Web form understanding: autosuggestion

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 5.59s

Model answer correct?: Yes

Prompt: What is the autosuggested word in the form?

Ground Truth: forklift

Model Output: The autosuggested word in the form is "forklift". It appears as a suggestion below the search field where "class:fork" has been entered. This appears to be part of an image search interface where users can search for images by class or category, and the system is suggesting "forklift" as a potential match for what the user started typing.

127

Submitted by: Object Counting

Category: Counting nodes

Time taken: 25.45s

Model answer correct?: No

Prompt: How many vertices (marked as red or orange dots) are visible in this diagram? Present your answer as an integer

Ground Truth: like 1.

Model Output: This image shows a complex network diagram with numerous vertices (nodes) connected by gray lines (edges). The vertices are represented by red dots arranged in a symmetrical pattern around a single orange dot at the center. To count the total number of vertices, I need to carefully count all the red dots plus the single orange dot in the center. After counting methodically through the concentric rings of the network, I can see there are approximately 120 vertices in total (including the orange center vertex). However, to give you a precise integer answer, I would need to count each vertex individually, which is challenging due to the density of the network. The exact number appears to be between 115-125 vertices.

Annotations that do not fully enclose an object

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 9.65s

Model answer correct?: No

Prompt: How many annotations do not fully surround an object? Return only a number.

Ground Truth: 1

Model Output: Looking at the image, I can see several aluminum beverage cans on what appears to be a production or packaging line. The cans have condensation droplets on them, suggesting they're cold. Each can has a yellow rectangular annotation box around it. Examining these annotations carefully, I count 2 annotation boxes that do not fully surround their respective objects. Some of the rectangular outlines only partially capture the cans they're meant to highlight. The answer is 2.

Bolts screwed in

Submitted by: Roboflow

Category: Defect Detection

Time taken: 9.42s

Model answer correct?: No

Prompt: Are all six bolts screwed in? Return only yes or no.

Ground Truth: Yes

Model Output: No. The image shows a circular electrical junction or mounting plate with bolt positions around the perimeter, but not all bolts appear to be fully installed or present in all the designated positions.

Bottle on wrong shelf

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 16.42s

Model answer correct?: No

Prompt: Are any bottles on the wrong shelf? If yes, say the name of the brand of the bottle on the wrong shelf like Coca-Cola.

Ground Truth: Fiji

Model Output: From what I can see in the image, there don't appear to be any bottles on the wrong shelf. The refrigerator display shows several shelves of bottled water that seem to be properly organized. The bottles appear to be various brands of water, with similar types grouped together on each shelf. There's a price tag visible showing "$3" on one of the shelves. All bottles seem to be in their designated locations, and I don't notice any misplaced items or bottles that would be considered on the wrong shelf.

Chessboard Analysis 1

Submitted by: Joshua

Category: Object Understanding

Time taken: 6.82s

Model answer correct?: No

Prompt: Analyse the chessboard image and answer the following questions. 1. How many pieces are on the board? Provide only the number. Then, list the count of each piece type on the board by colour (W for White, B for Black) and importance (points worth: K for King, Q for Queen, R for Rook, B for Bishop, N for Knight, P for Pawn). Use the format WQ: 0 (meaning no White Queen), with one piece type per line. Separate the number of pieces and the piece list with a blank line. 2. Are any pieces in positions that violate standard chess rules? If so, list ONLY their positions in the format where column 'a' is the leftmost, 'h' is the rightmost, row 1 is the bottom row, and row 8 is the top row (e.g., b7,h1,h2, with multiple positions separated by a comma without spaces). If no pieces are in illegal positions, write “None.” Please write the question number before giving an answer for each of the questions.

Ground Truth: 12 WK: 1 WQ: 1 WR: 1 WB: 1 WN: 1 WP: 1 BK: 1 BQ: 1 BR: 1 BB: 1 BN: 1 BP: 1 2. e3,e4

Model Output: Looking at this chess board image, I can see it shows a chess set with pieces that appear to be flat discs with chess piece symbols engraved on them, placed on a black and white checkered board. 1. 11 WK: 1 WQ: 1 WR: 1 WB: 1 WN: 1 WP: 2 BK: 1 BQ: 0 BR: 0 BB: 1 BN: 1 BP: 1 2. None. The board shows a somewhat unusual setup with pieces scattered in what appears to be a game in progress. The pieces are represented as circular tokens with chess symbols rather than traditional 3D chess pieces. All pieces appear to be in positions that could be legally reached during gameplay.

Colour of object closest to robotic arm grip

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 7.62s

Model answer correct?: No

Prompt: What is the colour of the object that, the robotic arm closest to picking up? Only return the colour.

Ground Truth: Black

Model Output: Red

Conveyor belt empty

Submitted by: Roboflow

Category: Object Understanding

Time taken: 4.67s

Model answer correct?: No

Prompt: Is the conveyor belt empty? Return either yes or no.

Ground Truth: Yes

Model Output: No, the conveyor belt is not empty. It appears to have yellow rollers or components visible along its length.

Count bottles

Submitted by: Roboflow

Category: Object Counting

Time taken: 19.59s

Model answer correct?: No

Prompt: How many bottles are in the image? Answer only a number

Ground Truth: 27

Model Output: Based on what I can see in the image, there appear to be approximately 20-25 bottles of Sprite in the refrigerator or cooler. The green bottles are arranged on the white rack shelves, with some standing upright and others lying on their sides. It's difficult to give an exact count because some bottles may be partially obscured by others.

Count number of biscuts

Submitted by: SnehitVaddi

Category: Object Counting

Time taken: 4.73s

Model answer correct?: No

Prompt: How many biscuts or cookies are there?

Ground Truth: 3

Model Output: I can see what appears to be chocolate biscuits or cookies in an opened foil wrapper. From the image, it looks like there are about 4-5 chocolate biscuits visible in the partially unwrapped packaging. The wrapper appears to be purple and silver foil, and the biscuits have a dark brown color typical of chocolate cookies. They're sitting on what looks like a laptop keyboard or trackpad area.

Count the shipping containers

Submitted by: Roboflow

Category: Object Counting

Time taken: 22.42s

Model answer correct?: No

Prompt: How many shipping containers are in the image?

Ground Truth: 62

Model Output: This aerial image shows a shipping container terminal with stacked shipping containers. It appears to have approximately 40-50 containers visible in the frame. The containers are arranged in neat rows and display various colors including pink, blue, white, brown, red, and mint green. The containers are stacked in what looks like two main rows or sections on a concrete terminal area, with some loading equipment visible on the left side. The image is taken from above, showing the colorful pattern created by the different container colors when viewed from this perspective.

Crack detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 11.88s

Model answer correct?: No

Prompt: How many cracks are in the wood? Return a number.

Ground Truth: 1

Model Output: Looking at the image of the wood surface, I can see approximately 3-4 distinct cracks or splits in the wood. There's one notable crack running vertically on the right side, and a few smaller cracks or splits visible in the grain pattern. The wood also has a prominent knot in the center and natural grain patterns throughout, but counting just the actual cracks or splits, I would say 3-4 is the most accurate count.

Glass rim crack

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.81s

Model answer correct?: No

Prompt: Is the glass rim cracked? Answer only yes or no.

Ground Truth: Yes

Model Output: No. The glass jar in the image appears to have an intact rim. The jar looks to be a clear glass container with a wide mouth and what seems to be a metal or glass ring at the top, but there's no visible crack or damage to the rim from what can be seen in this black and white photograph.

Measure a sticker

Submitted by: Roboflow

Category: Object Measurement

Time taken: 6.16s

Model answer correct?: No

Prompt: How wide is the sticker in inches? Return only a real number.

Ground Truth: 3.5

Model Output: Based on the ruler shown in the image, the sticker appears to be approximately 2 inches wide.

Missing annotation in image

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 11.85s

Model answer correct?: No

Prompt: How many annotations are missing in the image? Return only a number.

Ground Truth: 1

Model Output: The image shows a row of aluminum beverage cans on what appears to be a production or packaging line. There are 5 yellow rectangular annotation boxes highlighting the cans. Based on what's visible in the image, there don't appear to be any missing annotations - all cans visible in the frame are highlighted with the yellow rectangles. The answer is 0.

Missing boxes on pallet

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.74s

Model answer correct?: No

Prompt: How many boxes are missing? Return only a number.

Ground Truth: 1

Model Output: Based on the image, there appears to be a pallet with cardboard boxes arranged in what seems to be an incomplete configuration. Looking at the arrangement, it appears that there are 2 boxes missing from what would otherwise be a complete rectangular arrangement on the pallet. The boxes are stacked in multiple layers with some shipping labels visible on a few of the boxes.

Missing part in box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 11.19s

Model answer correct?: No

Prompt: How many parts are missing? Return only a number.

Ground Truth: 1

Model Output: Based on the image, there appears to be a set of 6 fuel injectors arranged in a tray, and all 6 are present. There don't appear to be any empty slots or missing components from what should be a complete set. The answer is 0.

Number of bread buns on their sides

Submitted by: Roboflow

Category: Object Counting

Time taken: 17.56s

Model answer correct?: No

Prompt: How many buns are on their side? Return only a number.

Ground Truth: 1

Model Output: Looking at the image, I count approximately 30 small round baked items (appears to be small buns or cookies) in a red plastic tray. Of these, I can see about 7 that appear to be on their side rather than sitting flat. So the answer is: 7

Object counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 4.34s

Model answer correct?: No

Prompt: How many empty spaces are on the bottom shelf? Return only a number.

Ground Truth: 3

Model Output: Looking at the bottom shelf in the refrigerated display case, I count 0 empty spaces. The shelf appears to be fully stocked with various bottled beverages.

Object fallen off assembly line

Submitted by: Roboflow

Category: Object Understanding

Time taken: 23.54s

Model answer correct?: No

Prompt: Has a piece of crockery fallen off the assembly line? Yes or no.

Ground Truth: Yes

Model Output: No, this doesn't appear to be a case of crockery falling off an assembly line. The image shows what looks like a commercial dishwasher or washing system with a large blue/teal plastic container or box sitting on top of a rack with many white plastic prongs or holders (which are typical in commercial dishwashers to hold plates and other items in place during washing). There's a bowl visible in the bottom right corner, but it appears to be properly placed rather than having fallen. The setup appears to be in an industrial kitchen or washing facility.

Object overlapping

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 36.22s

Model answer correct?: No

Prompt: How many screws are overlapping? Return only a number, like 1 or 2.

Ground Truth: 4

Model Output: 2

Pallet stacking

Submitted by: Roboflow

Category: Object Counting

Time taken: 13.46s

Model answer correct?: No

Prompt: How many pallet are stacked?

Ground Truth: 13

Model Output: In the image, there appears to be a stack of wooden pallets on the right side. It's difficult to count precisely, but there seem to be approximately 10-12 wooden pallets stacked vertically. On the left side of the image, there are metal wire containers or cages (sometimes called gitterboxes or wire mesh containers) that are used for storage and transport in industrial settings. These are different from the traditional wooden pallets and appear to be stacked about 2-3 high.

Read barcode ID on circuit board

Submitted by: Roboflow

Category: OCR

Time taken: 16.37s

Model answer correct?: No

Prompt: What is the ID on the barcode? Return only the ID text.

Ground Truth: T074802630B2

Model Output: The ID text on the barcode is: T07480263082

Rebar counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 22.10s

Model answer correct?: No

Prompt: How many rebar ends are visible? Answer only a number.

Ground Truth: 65

Model Output: From the image, I can see approximately 60-70 rebar ends visible in the foreground bundle. The rebars are stacked together with their cut ends facing the camera, showing the characteristic rusty/orange coloration on the steel. There are additional rebars in the background, but I'm focusing on the clearly visible ends in the main bundle.

Robotic arm grip orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 14.03s

Model answer correct?: No

Prompt: Is the robotic arm grip pointing down, at a left angle, or at a right angle? Return down right or left.

Ground Truth: down

Model Output: In the image, the robotic arm's grip (the blue cylindrical end effector) is pointing at a right angle. The arm extends horizontally from its base toward the right side of the image.

Robotic arm not over conveyor belt

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.83s

Model answer correct?: No

Prompt: Is the robotic arm over the conveyor belt? Return only yes or no.

Ground Truth: No

Model Output: Yes.

Seal verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.86s

Model answer correct?: No

Prompt: Is the bottle top properly sealed onto the bottle? Return only yes or no.

Ground Truth: No

Model Output: Yes.

Supermarket shelf label reading with two items in focus

Submitted by: Roboflow

Category: OCR

Time taken: 7.60s

Model answer correct?: No

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: Based on the price tag visible in the image, the wipes are priced at 75p (0.75 GBP). The store appears to be running a "3 for 2" promotion (cheapest item free) on these skincare products.

To check the part Outer diamter and inner diameter in mm

Submitted by: User

Category: Object Measurement

Time taken: 8.84s

Model answer correct?: No

Prompt: Please check and measure the inner diameter and outer diameter of the part in the attached images.

Ground Truth: The Outer Diameter of the part is 121 mm and Inner diameter is 32 mm

Model Output: The image shows a circular metal part with a brushed or machined finish. It appears to be a washer, spacer, or similar component with a circular hole in the center. The part has: - A larger metallic disc with a brushed finish that appears to be made of steel or aluminum - A smaller white circular insert or hole in the center that has some texture or pattern on it Unfortunately, I cannot measure the exact dimensions from the image alone. There's no scale reference in the photo to determine the precise inner and outer diameters. To get accurate measurements, you would need to: 1. Use calipers or a micrometer to measure both diameters 2. Include a ruler or known reference object in the photo 3. Provide the specifications if you have documentation for this part If you need precise measurements, I'd recommend using a measuring tool directly on the physical object.

Total Credits in Fall 2015

Submitted by: Jayesh

Category: Document Understanding

Time taken: 5.93s

Model answer correct?: No

Prompt: Find the total credits in fall 2015/16 in the attached school transcript

Ground Truth: 12

Model Output: According to the transcript, in Fall 2015/2016, the student took two courses: - American Literature (1.00 credit) - Art Appreciation (1.00 credit) The total credits for Fall 2015/2016 is 2.00 credits, as shown in the cumulative credits section for that term (which shows 8.00 as the running total after this term).

Worker in zone

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 26.56s

Model answer correct?: No

Prompt: Is the worker in the safe zone (marked in yellow)? Return only yes or no.

Ground Truth: Yes

Model Output: No. The worker is standing outside the yellow-marked safety zone while handling or placing boxes on the pallets that are within the marked area.