Cohere Aya Vision 8B

36.14% (30/83)

Passed

10.97s

Average Request Time

Summary

Ranked #1 on leaderboard for Object Detection, Sign Understanding.

Contribute a Prompt

Have an idea for a prompt? Open a pull request on the project repository!

Compare with another model

All Prompts

Annotations that do not fully enclose an object

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 6.84s

Model answer correct?: Yes

Prompt: How many annotations do not fully surround an object? Return only a number.

Ground Truth: 1

Model Output: There are 10 annotations that do not fully surround an object.

Background object identification

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 8.47s

Model answer correct?: Yes

Prompt: Is there a box in the background? Yes or no?

Ground Truth: Yes

Model Output: Yes, there is a box in the background of the image. It appears to be a cardboard box placed on a shelf or rack, likely in a storage or warehouse setting.

Bolts screwed in

Submitted by: Roboflow

Category: Defect Detection

Time taken: 10.23s

Model answer correct?: Yes

Prompt: Are all six bolts screwed in? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Bottle on wrong shelf

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 9.35s

Model answer correct?: Yes

Prompt: Are any bottles on the wrong shelf? If yes, say the name of the brand of the bottle on the wrong shelf like Coca-Cola.

Ground Truth: Fiji

Model Output: Yes, there are bottles on the wrong shelf. The brand of the bottle on the wrong shelf appears to be Fiji Water.

Colour of object closest to robotic arm grip

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.93s

Model answer correct?: Yes

Prompt: What is the colour of the object that, the robotic arm closest to picking up? Only return the colour.

Ground Truth: Black

Model Output: The object that the robotic arm is closest to picking up is black.

Conveyor belt empty

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.06s

Model answer correct?: Yes

Prompt: Is the conveyor belt empty? Return either yes or no.

Ground Truth: Yes

Model Output: Yes.

Count solar panels

Submitted by: Roboflow

Category: Object Counting

Time taken: 17.63s

Model answer correct?: Yes

Prompt: How many rows of solar panels are in the image? Return only a number.

Ground Truth: 4

Model Output: The image contains 4 rows of solar panels.

Date picker

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 3.40s

Model answer correct?: Yes

Prompt: What date is picked on the calendar? Answer like January 1 2020

Ground Truth: November 21 2017

Model Output: The date picked on the calendar is November 21, 2017.

Defect detection (no defect present)

Submitted by: Roboflow

Category: Defect Detection

Time taken: 6.71s

Model answer correct?: Yes

Prompt: Does the juice box have a defect? Yes or no.

Ground Truth: No

Model Output: Yes, the juice box appears to have a defect. The straw is bent and twisted, making it difficult to use properly. This could be a manufacturing issue or damage during transportation or handling.

Defect detection: bends

Submitted by: Roboflow

Category: Defect Detection

Time taken: 8.71s

Model answer correct?: Yes

Prompt: Are there any bends in the part? Return either yes or no.

Ground Truth: Yes

Model Output: Yes, there are bends in the part. The metal component has several curved sections and changes in direction, indicating the presence of bends.

Dent detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 7.52s

Model answer correct?: Yes

Prompt: How many dents are in the image? Return only the number.

Ground Truth: 1

Model Output: There is 1 dent in the image.

Digital workflow and process understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 1.05s

Model answer correct?: Yes

Prompt: List the steps in the workflow as a comma-separated list.

Ground Truth: Inputs, Llama 3.2 Vision, Outputs

Model Output: The steps in the workflow, as a comma-separated list, are: Inputs, llama3.2 Vision, Outputs

Item hanging off assembly line

Submitted by: Roboflow

Category: Object Detection

Time taken: 8.60s

Model answer correct?: Yes

Prompt: Is the candy bar hanging off the assembly line? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Leaf rust

Submitted by: Roboflow

Category: Defect Detection

Time taken: 10.91s

Model answer correct?: Yes

Prompt: Does the leaf show signs of leaf rust? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Log on train track

Submitted by: Roboflow

Category: Object Understanding

Time taken: 18.09s

Model answer correct?: Yes

Prompt: Is there a log on the track? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Missing pill in pack classification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 6.13s

Model answer correct?: Yes

Prompt: Are there any missing pills? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Moisture on cans

Submitted by: Roboflow

Category: Defect Detection

Time taken: 17.09s

Model answer correct?: Yes

Prompt: Is there moisture on the cans? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Object differentiation

Submitted by: Roboflow

Category: Object Detection

Time taken: 13.36s

Model answer correct?: Yes

Prompt: Does the container contain screws or bolts or nuts?

Ground Truth: bolts

Model Output: Based on the image provided, the container does not contain screws, bolts, or nuts. The image shows a green belt on a machine, with a blue component and some small metallic objects, but there are no visible screws, bolts, or nuts within the container. The metallic objects appear to be small components or parts of the machine, but they are not identified as screws, bolts, or nuts.

Object on top of another object

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 10.00s

Model answer correct?: Yes

Prompt: Is the shipping container on a truck? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Part number reading

Submitted by: Roboflow

Category: OCR

Time taken: 1.01s

Model answer correct?: Yes

Prompt: Return the part number.

Ground Truth: 10629101

Model Output: The part number is 10629101.

Proximity to a forklift

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 13.15s

Model answer correct?: Yes

Prompt: Is there a person next to the forklift? Yes or no.

Ground Truth: Yes

Model Output: Yes, there is a person next to the forklift. The image shows a forklift driver operating the vehicle, and another worker is standing nearby, likely assisting or overseeing the operation. The worker is wearing a high-visibility vest and a hard hat, indicating a focus on safety in the warehouse environment.

Read a serial number

Submitted by: Roboflow

Category: OCR

Time taken: 1.67s

Model answer correct?: Yes

Prompt: What is the serial number on the tire? Answer only the serial number.

Ground Truth: 3702692432

Model Output: 3702692432

Recycling sign understanding

Submitted by: Roboflow

Category: Sign Understanding

Time taken: 7.18s

Model answer correct?: Yes

Prompt: What plastic number is in the image? Return only the number with no leading 0s.

Ground Truth: 2

Model Output: The plastic number in the image is 02, which stands for HDPE (High-Density Polyethylene). HDPE is a type of plastic that is commonly used for making containers like milk jugs, detergent bottles, and some food packaging. It is known for its durability and resistance to chemicals, making it a popular choice for various applications.

Roast date on a label

Submitted by: Roboflow

Category: Document Understanding

Time taken: 3.00s

Model answer correct?: Yes

Prompt: When was the coffee roasted? Return only the roast date in DD/MM/YYYY, like 01/12/2019.

Ground Truth: 08/09/2020

Model Output: The coffee was roasted on 08/09/2020.

Robotic arm grip orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 9.30s

Model answer correct?: Yes

Prompt: Is the robotic arm grip pointing down, at a left angle, or at a right angle? Return down right or left.

Ground Truth: down

Model Output: The robotic arm grip is pointing down.

Scratch on metal

Submitted by: Roboflow

Category: Defect Detection

Time taken: 10.01s

Model answer correct?: Yes

Prompt: Is there a scratch on the metal? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Shipping manifest reading

Submitted by: Roboflow

Category: OCR

Time taken: 5.20s

Model answer correct?: Yes

Prompt: What is the dock number? Return like AAA12.

Ground Truth: D33

Model Output: The dock number is D33.

Solar panel detection and colour understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 17.86s

Model answer correct?: Yes

Prompt: Does the house with the terracotta roof have a solar panel? Yes or no.

Ground Truth: Yes

Model Output: Yes, the house with the terracotta roof has solar panels installed on its roof.

Tray in bag on assembly line

Submitted by: Roboflow

Category: Defect Detection

Time taken: 19.90s

Model answer correct?: Yes

Prompt: Is the tray in a blue bag? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

Worker in zone

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 24.24s

Model answer correct?: Yes

Prompt: Is the worker in the safe zone (marked in yellow)? Return only yes or no.

Ground Truth: Yes

Model Output: Yes.

(WIP) Table understanding from a web catalog

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.24s

Model answer correct?: No

Prompt: What is the package quantity of 90183A308? Return only a number. If SKU does not exist, return NULL.

Ground Truth: 50

Model Output: NULL

127

Submitted by: Object Counting

Category: Counting nodes

Time taken: 25.47s

Model answer correct?: No

Prompt: How many vertices (marked as red or orange dots) are visible in this diagram? Present your answer as an integer

Ground Truth: like 1.

Model Output: To determine the number of vertices (marked as red or orange dots) visible in the diagram, we need to carefully analyze the structure of the image. The diagram appears to be a complex network of interconnected nodes, forming a pattern that resembles a flower or a mandala. ### Step-by-Step Analysis: 1. **Identify the Central Node**: - The central node is marked with a yellow dot. This is the core of the pattern and is not counted as a vertex since it is not part of the red or orange dots. 2. **Count the Outer Layer**: - The outer layer consists of red dots arranged in a circular pattern. Each red dot is connected to its neighbors, forming a complete graph. - There are 12 red dots in the outer layer, forming a dodecagon (12-sided polygon). 3. **Count the Inner Layers**: - Inside the outer layer, there are additional layers of red dots. Each layer is smaller than the previous one, forming a pattern that resembles a flower. - The second layer has 10 red dots, the third layer has 8 red dots, and so on, until the innermost layer, which has 2 red dots. 4. **Calculate the Total Number of Red Dots**: - The total number of red dots is the sum of the dots in each layer: - Outer layer: 12 dots - Second layer: 10 dots - Third layer: 8 dots - Fourth layer: 6 dots - Fifth layer: 4 dots - Sixth layer: 2 dots - Adding these together: \[ 12 + 10 + 8 + 6 + 4 + 2 = 42 \] ### Final Answer: The number of vertices (marked as red or orange dots) visible in the diagram is **42**.

Chassis hole counting

Submitted by: Roboflow

Category: Object Understanding

Time taken: 23.34s

Model answer correct?: No

Prompt: How many holes are in the chassis? Return only a number.

Ground Truth: 4

Model Output: There are no visible holes in the chassis shown in the image.

Chessboard Analysis 1

Submitted by: Joshua

Category: Object Understanding

Time taken: 4.84s

Model answer correct?: No

Prompt: Analyse the chessboard image and answer the following questions. 1. How many pieces are on the board? Provide only the number. Then, list the count of each piece type on the board by colour (W for White, B for Black) and importance (points worth: K for King, Q for Queen, R for Rook, B for Bishop, N for Knight, P for Pawn). Use the format WQ: 0 (meaning no White Queen), with one piece type per line. Separate the number of pieces and the piece list with a blank line. 2. Are any pieces in positions that violate standard chess rules? If so, list ONLY their positions in the format where column 'a' is the leftmost, 'h' is the rightmost, row 1 is the bottom row, and row 8 is the top row (e.g., b7,h1,h2, with multiple positions separated by a comma without spaces). If no pieces are in illegal positions, write “None.” Please write the question number before giving an answer for each of the questions.

Ground Truth: 12 WK: 1 WQ: 1 WR: 1 WB: 1 WN: 1 WP: 1 BK: 1 BQ: 1 BR: 1 BB: 1 BN: 1 BP: 1 2. e3,e4

Model Output: 1. There are 32 pieces on the board. White pieces: - 1 Queen (Q) - 1 Rook (R) - 2 Bishops (B) - 2 Knights (N) - 8 Pawns (P) Black pieces: - 1 King (K) - 1 Rook (R) - 2 Bishops (B) - 2 Knights (N) - 8 Pawns (P) 2. None.

Circuit board understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 11.63s

Model answer correct?: No

Prompt: Read the number above the left transistor. Return only the number.

Ground Truth: 101

Model Output: The number above the left transistor is 35V/390UF.

Coffee plants ready to farm

Submitted by: Roboflow

Category: Object Understanding

Time taken: 14.04s

Model answer correct?: No

Prompt: Is the coffee plant ready to farm? It is only ready to farm if the majority of berries are red. Return only yes or no.

Ground Truth: No

Model Output: Yes.

Count bottles

Submitted by: Roboflow

Category: Object Counting

Time taken: 27.92s

Model answer correct?: No

Prompt: How many bottles are in the image? Answer only a number

Ground Truth: 27

Model Output: There are 21 bottles in the image.

Count number of biscuts

Submitted by: SnehitVaddi

Category: Object Counting

Time taken: 4.03s

Model answer correct?: No

Prompt: How many biscuts or cookies are there?

Ground Truth: 3

Model Output: There are three biscuits or cookies visible in the image. They appear to be rectangular in shape with a dark brown color, likely chocolate-flavored. The biscuits are partially unwrapped, revealing their texture and appearance. They are placed on a piece of aluminum foil, which is itself partially unwrapped, indicating that the biscuits were likely removed from a larger package. The foil has some wrinkles and creases, suggesting it has been used and handled. The biscuits are arranged in a slightly overlapping manner, with one biscuit partially covering another. The foil has a shiny, reflective surface, typical of aluminum, and there are some purple edges visible, possibly from the packaging material. The biscuits themselves have a slightly rough texture with visible ridges, indicating they may be homemade or artisanal. Overall, the image captures a small, intimate moment of enjoying a snack, with the biscuits being the main focus of the scene.

Count the shipping containers

Submitted by: Roboflow

Category: Object Counting

Time taken: 14.74s

Model answer correct?: No

Prompt: How many shipping containers are in the image?

Ground Truth: 62

Model Output: The image depicts a bustling port scene with a focus on shipping containers. There are approximately 100 shipping containers visible in the image, arranged in neat stacks on a large container ship. The containers are stacked in multiple layers, creating a visually striking pattern of colors and shapes. The ship is docked at a pier, with cranes and other port equipment visible in the background. The scene captures the efficiency and scale of global trade, showcasing the sheer number of containers that facilitate international commerce.

Crack detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 9.41s

Model answer correct?: No

Prompt: How many cracks are in the wood? Return a number.

Ground Truth: 1

Model Output: There are no visible cracks in the wood shown in the image. The wood appears to be smooth and intact, with a natural grain pattern and a central knot.

Defect localization

Submitted by: Roboflow

Category: Localization

Time taken: 4.80s

Model answer correct?: No

Prompt: Where is the defect in the product: top, left, bottom, right, or center? Return only the location, like left center.

Ground Truth: top center

Model Output: The defect in the product is located at the top left corner.

Find an author

Submitted by: User

Category: Document Understanding

Time taken: 1.88s

Model answer correct?: No

Prompt: Who transcribed this sheet music, according to the document? Answer as only the name is presented on the document.

Ground Truth: Otto Singer II

Model Output: Johannes Brahms

Glass rim crack

Submitted by: Roboflow

Category: Defect Detection

Time taken: 1.66s

Model answer correct?: No

Prompt: Is the glass rim cracked? Answer only yes or no.

Ground Truth: Yes

Model Output: No.

Label verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 13.51s

Model answer correct?: No

Prompt: Does the juice box package show an orange eat well label? Return only yes or no.

Ground Truth: Yes

Model Output: No.

Maximum gross weight label on shipping container

Submitted by: Roboflow

Category: OCR

Time taken: 2.21s

Model answer correct?: No

Prompt: What is the max gross weight?

Ground Truth: 32,500 kg

Model Output: The maximum gross weight of the container is **32,200 kg** or **70,946.33 lbs**.

Measure a sticker

Submitted by: Roboflow

Category: Object Measurement

Time taken: 4.48s

Model answer correct?: No

Prompt: How wide is the sticker in inches? Return only a real number.

Ground Truth: 3.5

Model Output: 3

Missing annotation in image

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 10.70s

Model answer correct?: No

Prompt: How many annotations are missing in the image? Return only a number.

Ground Truth: 1

Model Output: There are 6 annotations missing in the image.

Missing boxes on pallet

Submitted by: Roboflow

Category: Object Understanding

Time taken: 3.24s

Model answer correct?: No

Prompt: How many boxes are missing? Return only a number.

Ground Truth: 1

Model Output: 8

Missing chocolate location in grid

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.29s

Model answer correct?: No

Prompt: Return the position of the missing chocolate in the image if there is a missing chocolate. Positions should be as follows: R1C1, where R is the row and C is the column, like R2C3 would be Row 2 Column 3. Return only the R1C1 value and no other information. If no chocolates are missing, return R0C0.

Ground Truth: R1C4

Model Output: R0C0

Missing object on assembly line tray

Submitted by: Roboflow

Category: Object Understanding

Time taken: 13.76s

Model answer correct?: No

Prompt: Do any metal plates have no blue parts on top? If so, return the tray position. The position must be either 1, 2, 3, or 4. 1 is the leftmost tray. 4 is the rightmost tray. Otherwise, return NULL.

Ground Truth: 4

Model Output: The image shows a complex industrial machine with various components, including metal plates and blue parts. However, there are no visible blue parts on top of any of the metal plates in the image. Therefore, the answer is NULL.

Missing part in box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 11.79s

Model answer correct?: No

Prompt: How many parts are missing? Return only a number.

Ground Truth: 1

Model Output: There are 9 parts missing.

Name of item at a specified position in a list

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 0.69s

Model answer correct?: No

Prompt: What is the third item in the list? Return only the name of the item.

Ground Truth: null

Model Output: robot-arms

Number of bread buns on their sides

Submitted by: Roboflow

Category: Object Counting

Time taken: 15.89s

Model answer correct?: No

Prompt: How many buns are on their side? Return only a number.

Ground Truth: 1

Model Output: There are 24 buns on the side.

Object counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 5.69s

Model answer correct?: No

Prompt: How many empty spaces are on the bottom shelf? Return only a number.

Ground Truth: 3

Model Output: There are 2 empty spaces on the bottom shelf.

Object counting with a blurry image

Submitted by: Roboflow

Category: Object Counting

Time taken: 19.52s

Model answer correct?: No

Prompt: How many load carriers are in the image? Return only a number.

Ground Truth: 3

Model Output: There are 12 load carriers visible in the image. These include various types of pallets, crates, and boxes that are stacked and organized on the shelves within the warehouse. The load carriers are essential for storing and transporting goods within the facility, ensuring efficient logistics operations.

Object fallen off assembly line

Submitted by: Roboflow

Category: Object Understanding

Time taken: 17.72s

Model answer correct?: No

Prompt: Has a piece of crockery fallen off the assembly line? Yes or no.

Ground Truth: Yes

Model Output: No, there is no visible evidence of a piece of crockery falling off the assembly line in this image. The image shows a conveyor belt with various items, including what appears to be a green plastic container or box, but no crockery is visible. The conveyor belt is part of an industrial setting, likely a manufacturing or packaging facility, where items are being transported along a track. The presence of other objects like bowls and a towel suggests that this area is used for handling and preparing food items, but there is no indication of any crockery falling off the line.

Object orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 15.33s

Model answer correct?: No

Prompt: Is the item the right side up or upside down? Return only 'right side up' or 'upside down'.

Ground Truth: upside down

Model Output: right side up

Object overlapping

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 15.13s

Model answer correct?: No

Prompt: How many screws are overlapping? Return only a number, like 1 or 2.

Ground Truth: 4

Model Output: 2

Pallet stacking

Submitted by: Roboflow

Category: Object Counting

Time taken: 12.59s

Model answer correct?: No

Prompt: How many pallet are stacked?

Ground Truth: 13

Model Output: Based on the image provided, there are approximately 100 wooden pallets stacked in the warehouse. The pallets are arranged in various configurations, with some stacked vertically and others placed side by side. The image shows a variety of pallet types, including some that appear to be used and others that are newer or less worn. The overall scene suggests a busy storage area where pallets are being moved and stored efficiently.

Point a line flattens on a graph

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.14s

Model answer correct?: No

Prompt: At what epoch does the line start to flatten out? Return only the epoch number, like 99.

Ground Truth: 4

Model Output: The line starts to flatten out at epoch 99.

Read a receipt

Submitted by: Roboflow

Category: Receipt Reading

Time taken: 10.58s

Model answer correct?: No

Prompt: How much tax was paid? Only answer like $1.00

Ground Truth: $2.30

Model Output: The total amount paid on the receipt is $31.96. This includes the sub-surcharge of $0.68, which is 6% of the bill to offset the minimum wage in California. The total before taxes and fees is $28.00.

Read a screenshot of prose

Submitted by: Roboflow

Category: OCR

Time taken: 2.27s

Model answer correct?: No

Prompt: Read the text in the image. Return only unformatted text.

Ground Truth: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Model Output: I was thinking earlier today that I have gone through, to use the lingo, to each of Swift's Eras. Meta indeed. I started listening to Swift's music after hearing the Midnights album. A few weeks after hearing Ms. Swift's music for the first time, I found myself playing various songs from the album multiple times. I listened to the album in order multiple times.

Read barcode ID on circuit board

Submitted by: Roboflow

Category: OCR

Time taken: 17.89s

Model answer correct?: No

Prompt: What is the ID on the barcode? Return only the ID text.

Ground Truth: T074802630B2

Model Output: The ID on the barcode is T07480263.

Read dimensions on a floor plan

Submitted by: Roboflow

Category: Document Understanding

Time taken: 4.30s

Model answer correct?: No

Prompt: What are the dimensions of the living area? Return in format 11'1 X 11'1.

Ground Truth: 10'2" X 16'

Model Output: The living area in the provided floor plan measures 10'2" by 16'0". This space is designed to be a versatile and comfortable area for relaxation and entertainment. The dimensions allow for a spacious layout that can accommodate a variety of furniture arrangements, making it suitable for different lifestyles and preferences. The living area is located adjacent to the kitchen, creating a convenient and open-concept layout that promotes social interaction and ease of movement within the home.

Reading an easy CAPTCHA

Submitted by: Roboflow

Category: CAPTCHA

Time taken: 1.16s

Model answer correct?: No

Prompt: Please provide the string required to pass this CAPTCHA. Do not respond with anything else. Do not include whitespace besides spaces between words.

Ground Truth: Charybdis in-dubitable

Model Output: Charybdis

Rebar counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 15.27s

Model answer correct?: No

Prompt: How many rebar ends are visible? Answer only a number.

Ground Truth: 65

Model Output: There are 12 rebar ends visible in the image.

Relation identification: left or right

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 19.82s

Model answer correct?: No

Prompt: Is the dump truck to the left or right of the wheel loader? Answer only left or right.

Ground Truth: left

Model Output: right

Relation identification: top or bottom

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 11.51s

Model answer correct?: No

Prompt: Is the white box on top of or below the other box? Answer with either above or below.

Ground Truth: above

Model Output: The white box is on top of the other box.

Retrieve article tags in order

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 2.36s

Model answer correct?: No

Prompt: Return the article tags in order from left to right as a comma separated list, like: tag a, tag b, tag c.

Ground Truth: Coffee, Food, Health & wellbeing, Health, Fitness, features

Model Output: Coffee, Food, Health & Wellbeing

Robotic arm not over conveyor belt

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.01s

Model answer correct?: No

Prompt: Is the robotic arm over the conveyor belt? Return only yes or no.

Ground Truth: No

Model Output: Yes.

Seal verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 2.96s

Model answer correct?: No

Prompt: Is the bottle top properly sealed onto the bottle? Return only yes or no.

Ground Truth: No

Model Output: Yes.

Shipping container OCR

Submitted by: Roboflow

Category: OCR

Time taken: 2.62s

Model answer correct?: No

Prompt: What is the horizontal ID on the container? Return only the ID.

Ground Truth: JBHU282862

Model Output: JHBH 28862

Spatial relations

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 8.73s

Model answer correct?: No

Prompt: What colour are the bottles in the middle? Return only the colour name.

Ground Truth: Red

Model Output: The bottles in the middle are black.

Sudoku puzzle extraction

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.13s

Model answer correct?: No

Prompt: You are given an image of a Sudoku puzzle. Your task is to detect and extract the digits from the board and output them as a 9x9 grid in text format. Identify the numbers visible on the Sudoku grid. Use a dot (.) to represent empty cells (i.e., those without a digit). Return the result as a plain-text 9x9 grid, where each row is a string of 9 characters. Maintain the correct order from top-left to bottom-right of the board.

Ground Truth: ...4.8... .6..7..1. 7.2.9.5.4 .9.7.4.3. ..7.5.8.. .8.9.6.5. 9.4.1.7.8 .7..6..4. ...2.7...

Model Output: 4 7 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Supermarket shelf label reading with one item in focus

Submitted by: Roboflow

Category: OCR

Time taken: 3.84s

Model answer correct?: No

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: The wipes are priced at 58 GBP per pack.

Supermarket shelf label reading with two items in focus

Submitted by: Roboflow

Category: OCR

Time taken: 4.39s

Model answer correct?: No

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: The wipes are priced at 75p per pack, and there is an offer of 3 packs for £2.

Table understanding

Submitted by: Roboflow

Category: Document Understanding

Time taken: 2.45s

Model answer correct?: No

Prompt: What is the adjusted EBITDA in the nine months ended September 2023 (in millions)?

Ground Truth: 2,428

Model Output: The adjusted EBITDA for the nine months ended September 30, 2023, is $131,962 million. This figure represents a 40% increase compared to the same period in the previous year.

To check the part Outer diamter and inner diameter in mm

Submitted by: User

Category: Object Measurement

Time taken: 6.84s

Model answer correct?: No

Prompt: Please check and measure the inner diameter and outer diameter of the part in the attached images.

Ground Truth: The Outer Diameter of the part is 121 mm and Inner diameter is 32 mm

Model Output: Based on the image provided, I can provide the following measurements for the inner and outer diameters of the part: Inner diameter: The inner diameter appears to be approximately 0.5 inches (1.27 cm). This measurement is taken from the center of the hole to the edge of the inner circle. Outer diameter: The outer diameter appears to be approximately 2 inches (5.08 cm). This measurement is taken from the outer edge of the part to the opposite side, passing through the center. Please note that these measurements are approximate and may vary slightly depending on the specific image and measurement method used. If you require more precise measurements, I recommend using a ruler or measuring tool to obtain accurate values.

Torn receipt reading

Submitted by: Roboflow

Category: Document Understanding

Time taken: 13.56s

Model answer correct?: No

Prompt: What is the subtotal of the receipt? Return only the amount like $1.00 and no other text.

Ground Truth: $11.69

Model Output: $13.53

Total Credits in Fall 2015

Submitted by: Jayesh

Category: Document Understanding

Time taken: 2.49s

Model answer correct?: No

Prompt: Find the total credits in fall 2015/16 in the attached school transcript

Ground Truth: 12

Model Output: The total credits earned by the student in fall 2015/16 are 15. This is calculated by adding up the credits for each course: 1 credit for Physical Science, 1 credit for US History, 2 credits for English Composition, and 1 credit for English Language Appreciation.

Truck in bay

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 5.99s

Model answer correct?: No

Prompt: Is the Scania truck in the bay? Return either yes or no.

Ground Truth: No

Model Output: Yes.

Unsealed box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 156.16s

Model answer correct?: No

Prompt: Is the outer box sealed? Return yes or no.

Ground Truth: No

Model Output: Yes.

Web form understanding: autosuggestion

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.64s

Model answer correct?: No

Prompt: What is the autosuggested word in the form?

Ground Truth: forklift

Model Output: The autosuggested word in the form is "fork". This is evident from the highlighted text within the search bar, which shows "class: fork[]". The form is designed to search for images related to the term "fork", and the autosuggestion helps users quickly input the correct search term.