LFM 2.5 VL 1.6B

54.88% (45/82)

Passed

8.46s

Average Request Time

Summary

Ranked #1 on leaderboard for Object Measurement, Object Detection, Color Identification.

Contribute a Prompt

Have an idea for a prompt? Open a pull request on the project repository!

Compare with another model

All Prompts

(WIP) Table understanding from a web catalog

Submitted by: Roboflow

Category: Document Understanding

Time taken: 4.28s

Model answer correct?: Yes

Prompt: What is the package quantity of 90183A308? Return only a number. If SKU does not exist, return NULL.

Ground Truth: 50

Model Output: {"answer": 50}

Background object identification

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 8.43s

Model answer correct?: Yes

Prompt: Is there a box in the background? Yes or no?

Ground Truth: Yes

Model Output: {"answer": "yes"}

Bolts screwed in

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.93s

Model answer correct?: Yes

Prompt: Are all six bolts screwed in? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Chassis hole counting

Submitted by: Roboflow

Category: Object Understanding

Time taken: 6.08s

Model answer correct?: Yes

Prompt: How many holes are in the chassis? Return only a number.

Ground Truth: 4

Model Output: {"answer": 4}

Coffee plants ready to farm

Submitted by: Roboflow

Category: Object Understanding

Time taken: 3.07s

Model answer correct?: Yes

Prompt: Is the coffee plant ready to farm? It is only ready to farm if the majority of berries are red. Return only yes or no.

Ground Truth: No

Model Output: {"answer": "no"}

Conveyor belt empty

Submitted by: Roboflow

Category: Object Understanding

Time taken: 4.60s

Model answer correct?: Yes

Prompt: Is the conveyor belt empty? Return either yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Count number of biscuts

Submitted by: SnehitVaddi

Category: Object Counting

Time taken: 6.33s

Model answer correct?: Yes

Prompt: How many biscuts or cookies are there?

Ground Truth: 3

Model Output: {"answer": "3"}

Count solar panels

Submitted by: Roboflow

Category: Object Counting

Time taken: 3.37s

Model answer correct?: Yes

Prompt: How many rows of solar panels are in the image? Return only a number.

Ground Truth: 4

Model Output: {"answer": 4}

Crack detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.62s

Model answer correct?: Yes

Prompt: How many cracks are in the wood? Return a number.

Ground Truth: 1

Model Output: {"answer": 1}

Defect detection: bends

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.60s

Model answer correct?: Yes

Prompt: Are there any bends in the part? Return either yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Dent detection

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.14s

Model answer correct?: Yes

Prompt: How many dents are in the image? Return only the number.

Ground Truth: 1

Model Output: {"answer": 1}

Digital workflow and process understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 2.97s

Model answer correct?: Yes

Prompt: List the steps in the workflow as a comma-separated list.

Ground Truth: Inputs, Llama 3.2 Vision, Outputs

Model Output: {"answer": "Inputs, Llama 3.2 Vision, Outputs"}

Find an author

Submitted by: User

Category: Document Understanding

Time taken: 5.22s

Model answer correct?: Yes

Prompt: Who transcribed this sheet music, according to the document? Answer as only the name is presented on the document.

Ground Truth: Otto Singer II

Model Output: {"answer": "Otto Singer II"}

Item hanging off assembly line

Submitted by: Roboflow

Category: Object Detection

Time taken: 5.60s

Model answer correct?: Yes

Prompt: Is the candy bar hanging off the assembly line? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Label verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.62s

Model answer correct?: Yes

Prompt: Does the juice box package show an orange eat well label? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Leaf rust

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.85s

Model answer correct?: Yes

Prompt: Does the leaf show signs of leaf rust? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Log on train track

Submitted by: Roboflow

Category: Object Understanding

Time taken: 4.65s

Model answer correct?: Yes

Prompt: Is there a log on the track? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Maximum gross weight label on shipping container

Submitted by: Roboflow

Category: OCR

Time taken: 1.06s

Model answer correct?: Yes

Prompt: What is the max gross weight? Return only number (value in KG), without separators/decimals, like “1234”

Ground Truth: 32500

Model Output: {"answer": "32500"}

Measure a sticker

Submitted by: Roboflow

Category: Object Measurement

Time taken: 1.08s

Model answer correct?: Yes

Prompt: How wide is the sticker in inches? Return only a real number with 1 decimal value, like “12.3” for 12.3 inches

Ground Truth: 3.5

Model Output: {"answer": "3.5"}

Missing annotation in image

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 3.57s

Model answer correct?: Yes

Prompt: How many annotations are missing in the image? Return only a number.

Ground Truth: 1

Model Output: {"answer": 1}

Missing pill in pack classification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.30s

Model answer correct?: Yes

Prompt: Are there any missing pills? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Moisture on cans

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.24s

Model answer correct?: Yes

Prompt: Is there moisture on the cans? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Name of item at a specified position in a list

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 3.76s

Model answer correct?: Yes

Prompt: What is the third item in the list? Return only the name of the item.

Ground Truth: null

Model Output: {"answer": "null"}

Object counting with a blurry image

Submitted by: Roboflow

Category: Object Counting

Time taken: 4.16s

Model answer correct?: Yes

Prompt: How many load carriers are in the image? Return only a number.

Ground Truth: 3

Model Output: {"answer": 3}

Object differentiation

Submitted by: Roboflow

Category: Object Detection

Time taken: 4.60s

Model answer correct?: Yes

Prompt: Does the container contain screws or bolts or nuts?

Ground Truth: bolts

Model Output: {"answer": "bolts"}

Object on top of another object

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.27s

Model answer correct?: Yes

Prompt: Is the shipping container on a truck? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Object orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.16s

Model answer correct?: Yes

Prompt: Is the item the right side up or upside down? Return only 'right side up' or 'upside down'.

Ground Truth: upside down

Model Output: {"answer": "upside down"}

Part number reading

Submitted by: Roboflow

Category: OCR

Time taken: 254.54s

Model answer correct?: Yes

Prompt: Return the part number.

Ground Truth: 10629101

Model Output: {"answer": "10629101"}

Proximity to a forklift

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 4.11s

Model answer correct?: Yes

Prompt: Is there a person next to the forklift? Yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Read a screenshot of prose

Submitted by: Roboflow

Category: OCR

Time taken: 3.89s

Model answer correct?: Yes

Prompt: Read the text in the image. Return only unformatted text.

Ground Truth: I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times.

Model Output: {"answer": "I was thinking earlier today that I have gone through, to use the lingo, eras of listening to each of Swift's Eras. Meta indeed. I started listening to Ms. Swift's music after hearing the Midnights album. A few weeks after hearing the album for the first time, I found myself playing various songs on repeat. I listened to the album in order multiple times."}

Read a serial number

Submitted by: Roboflow

Category: OCR

Time taken: 1.35s

Model answer correct?: Yes

Prompt: What is the serial number on the tire? Answer only the serial number.

Ground Truth: 3702692432

Model Output: {"answer": "3702692432"}

Read barcode ID on circuit board

Submitted by: Roboflow

Category: OCR

Time taken: 3.69s

Model answer correct?: Yes

Prompt: What is the ID on the barcode? Return only the ID text.

Ground Truth: T074802630B2

Model Output: {"answer": "T074802630B2"}

Relation identification: left or right

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 7.15s

Model answer correct?: Yes

Prompt: Is the dump truck to the left or right of the wheel loader? Answer only left or right.

Ground Truth: left

Model Output: {"answer": "left"}

Relation identification: top or bottom

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.03s

Model answer correct?: Yes

Prompt: Is the white box on top of or below the other box? Answer with either above or below.

Ground Truth: above

Model Output: {"answer": "above"}

Retrieve article tags in order

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 7.12s

Model answer correct?: Yes

Prompt: Return the article tags in order from left to right as a comma separated list, like: tag a, tag b, tag c.

Ground Truth: Coffee, Food, Health & wellbeing, Health, Fitness, features

Model Output: {"answer": "Coffee, Food, Health & wellbeing, Health, Fitness, features"}

Roast date on a label

Submitted by: Roboflow

Category: Document Understanding

Time taken: 1.21s

Model answer correct?: Yes

Prompt: When was the coffee roasted? Return only the roast date in DD/MM/YYYY, like 01/12/2019.

Ground Truth: 08/09/2020

Model Output: {"answer": "08/09/2020"}

Scratch on metal

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.08s

Model answer correct?: Yes

Prompt: Is there a scratch on the metal? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Shipping manifest reading

Submitted by: Roboflow

Category: OCR

Time taken: 2.75s

Model answer correct?: Yes

Prompt: What is the dock number? Return like AAA12.

Ground Truth: D33

Model Output: {"answer": "D33"}

Solar panel detection and colour understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.98s

Model answer correct?: Yes

Prompt: Does the house with the terracotta roof have a solar panel? Yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Spatial relations

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 2.91s

Model answer correct?: Yes

Prompt: What colour are the bottles in the middle? Return only the colour name.

Ground Truth: Red

Model Output: {"answer": "red"}

Torn receipt reading

Submitted by: Roboflow

Category: Document Understanding

Time taken: 3.14s

Model answer correct?: Yes

Prompt: What is the subtotal of the receipt? Return only the amount like $1.00 and no other text.

Ground Truth: $11.69

Model Output: {"answer": "$11.69"}

Tray in bag on assembly line

Submitted by: Roboflow

Category: Defect Detection

Time taken: 6.01s

Model answer correct?: Yes

Prompt: Is the tray in a blue bag? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Unsealed box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.18s

Model answer correct?: Yes

Prompt: Is the outer box sealed? Return yes or no.

Ground Truth: No

Model Output: {"answer": "no"}

Web form understanding: autosuggestion

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 1.15s

Model answer correct?: Yes

Prompt: What is the autosuggested word in the form?

Ground Truth: forklift

Model Output: {"answer": "forklift"}

Worker in zone

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.30s

Model answer correct?: Yes

Prompt: Is the worker in the safe zone (marked in yellow)? Return only yes or no.

Ground Truth: Yes

Model Output: {"answer": "yes"}

Annotations that do not fully enclose an object

Submitted by: Roboflow

Category: Annotation Understanding

Time taken: 3.32s

Model answer correct?: No

Prompt: How many annotations do not fully surround an object? Return only a number.

Ground Truth: 1

Model Output: {"answer": 2}

Bottle on wrong shelf

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 2.00s

Model answer correct?: No

Prompt: Are any bottles on the wrong shelf? If yes, say the name of the brand of the bottle on the wrong shelf like Coca-Cola.

Ground Truth: Fiji

Model Output: {"answer": "no"}

Chessboard Analysis 1

Submitted by: Joshua

Category: Object Understanding

Time taken: 49.40s

Model answer correct?: No

Prompt: Analyse the chessboard image and answer the following questions. First task: how many pieces are on the board? Provide only the number. Second task: list the count of each piece type on the board by colour (W for White, B for Black) and importance (points worth: K for King, Q for Queen, R for Rook, B for Bishop, N for Knight, P for Pawn). Use the format “WQ”: 0 (meaning no White Queen). Third task: are any pieces in positions that violate standard chess rules? If so, list ONLY their positions in the format where column 'a' is the leftmost, 'h' is the rightmost, row 1 is the bottom row, and row 8 is the top row (e.g., b7,h1,h2, with multiple positions separated by a comma without spaces). If no pieces are in illegal positions, write “None.”. Output in JSON, in a format like this: {“total_pieces”:3, “pieces”:{“WK”:0, “WQ”:1},”illegal_pieces”:”a1,b2”}

Ground Truth: {“total_pieces":12,"pieces":{"WK":1,"WQ":1,"WR":1,"WB":1,"WN":1,"WP":1,"BK":1,"BQ":1,"BR":1,"BB":1,"BN":1,"BP":1},"illegal_pieces":"e3,e4"}

Model Output: {"answer": "3", "pieces":{"WQ":1, "WK":0, "BQ":0, "BR":0, "BK":0, "BN":0, "NP":0, "PQ":0, "PR":0, "RQ":0, "RR":0, "NB":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0, "NPB":0, "NQ":0,

Circuit board understanding

Submitted by: Roboflow

Category: Object Understanding

Time taken: 6.90s

Model answer correct?: No

Prompt: Read the number above the left transistor. Return only the component’s reference , like “C001” for some capacitor

Ground Truth: Q101

Model Output: {"answer": "C101"}

Colour of object closest to robotic arm grip

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.42s

Model answer correct?: No

Prompt: What is the colour of the object that, the robotic arm closest to picking up? Only return the colour.

Ground Truth: Black

Model Output: {"answer": "red"}

Count bottles

Submitted by: Roboflow

Category: Object Counting

Time taken: 4.37s

Model answer correct?: No

Prompt: How many bottles are in the image? Answer only a number

Ground Truth: 27

Model Output: {"answer": 30}

Count the shipping containers

Submitted by: Roboflow

Category: Object Counting

Time taken: 7.18s

Model answer correct?: No

Prompt: How many shipping containers are in the image?

Ground Truth: 62

Model Output: {"answer": "50"}

Counting nodes

Submitted by: User

Category: Object Counting

Time taken: 7.26s

Model answer correct?: No

Prompt: How many vertices (marked as red or orange dots) are visible in this diagram? Present your answer as an integer like 1.

Ground Truth: 127

Model Output: {"answer": 100}

Date picker

Submitted by: Roboflow

Category: Web Action Understanding

Time taken: 5.16s

Model answer correct?: No

Prompt: What date is picked on the calendar? Answer like January 1 2020

Ground Truth: November 21 2017

Model Output: {"answer": "2020-01-21"}

Defect detection (no defect present)

Submitted by: Roboflow

Category: Defect Detection

Time taken: 3.13s

Model answer correct?: No

Prompt: Does the juice box have a defect? Yes or no.

Ground Truth: No

Model Output: {"answer": "yes"}

Defect localization

Submitted by: Roboflow

Category: Localization

Time taken: 3.50s

Model answer correct?: No

Prompt: Where is the defect in the product: top, left, bottom, right, or center? Return only the location, like left center.

Ground Truth: top center

Model Output: {"answer": "top"}

Glass rim crack

Submitted by: Roboflow

Category: Defect Detection

Time taken: 0.94s

Model answer correct?: No

Prompt: Is the glass rim cracked? Answer only yes or no.

Ground Truth: Yes

Model Output: {"answer": "no"}

Missing boxes on pallet

Submitted by: Roboflow

Category: Object Understanding

Time taken: 0.90s

Model answer correct?: No

Prompt: How many boxes are missing? Return only a number.

Ground Truth: 1

Model Output: {"answer": 2}

Missing chocolate location in grid

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.82s

Model answer correct?: No

Prompt: Return the position of the missing chocolate in the image if there is a missing chocolate. Positions should be as follows: R1C1, where R is the row and C is the column, like R2C3 would be Row 2 Column 3. Return only the R1C1 value and no other information. If no chocolates are missing, return R0C0.

Ground Truth: R1C4

Model Output: {"answer": "R2C3"}

Missing object on assembly line tray

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.28s

Model answer correct?: No

Prompt: Do any metal plates have no blue parts on top? If so, return the tray position. The position must be either 1, 2, 3, or 4. 1 is the leftmost tray. 4 is the rightmost tray. Otherwise, return NULL.

Ground Truth: 4

Model Output: {"answer": 3}

Missing part in box

Submitted by: Roboflow

Category: Defect Detection

Time taken: 4.68s

Model answer correct?: No

Prompt: How many parts are missing? Return only a number.

Ground Truth: 1

Model Output: {"answer": 2}

Number of bread buns on their sides

Submitted by: Roboflow

Category: Object Counting

Time taken: 3.51s

Model answer correct?: No

Prompt: How many buns are on their side? Return only a number.

Ground Truth: 1

Model Output: {"answer": 3}

Object counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 2.60s

Model answer correct?: No

Prompt: How many empty spaces are on the bottom shelf? Return only a number.

Ground Truth: 3

Model Output: {"answer": 2}

Object fallen off assembly line

Submitted by: Roboflow

Category: Object Understanding

Time taken: 5.07s

Model answer correct?: No

Prompt: Has a piece of crockery fallen off the assembly line? Yes or no.

Ground Truth: Yes

Model Output: {"answer": "no"}

Object overlapping

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 6.00s

Model answer correct?: No

Prompt: How many screws are overlapping? Return only a number, like 1 or 2.

Ground Truth: 4

Model Output: {"answer": 2}

Pallet stacking

Submitted by: Roboflow

Category: Object Counting

Time taken: 3.05s

Model answer correct?: No

Prompt: How many pallet are stacked?

Ground Truth: 13

Model Output: {"answer": "9"}

Point a line flattens on a graph

Submitted by: Roboflow

Category: Document Understanding

Time taken: 4.51s

Model answer correct?: No

Prompt: At what epoch does the line start to flatten out? Return only the epoch number, like 99.

Ground Truth: 4

Model Output: {"answer": "99"}

Read a receipt

Submitted by: Roboflow

Category: Receipt Reading

Time taken: 4.66s

Model answer correct?: No

Prompt: How much tax was paid? Only answer like $1.00

Ground Truth: $2.30

Model Output: {"answer": 2.30}

Read dimensions on a floor plan

Submitted by: Roboflow

Category: Document Understanding

Time taken: 6.43s

Model answer correct?: No

Prompt: What are the dimensions of the living area? Return in format 11'1 X 11'1.

Ground Truth: 10'2" X 16'

Model Output: {"answer": "11'1 X 11'1"}

Reading an easy CAPTCHA

Submitted by: Roboflow

Category: CAPTCHA

Time taken: 1.14s

Model answer correct?: No

Prompt: Please provide the string required to pass this CAPTCHA. Do not respond with anything else. Do not include whitespace besides spaces between words.

Ground Truth: Charybdis in-dubitable

Model Output: {"answer": "Charybdis in-"}

Rebar counting

Submitted by: Roboflow

Category: Object Counting

Time taken: 4.77s

Model answer correct?: No

Prompt: How many rebar ends are visible? Answer only a number.

Ground Truth: 65

Model Output: {"answer": 60}

Recycling sign understanding

Submitted by: Roboflow

Category: Sign Understanding

Time taken: 3.41s

Model answer correct?: No

Prompt: What plastic number is in the image? Return only the number with no leading 0s.

Ground Truth: 2

Model Output: {"answer": "02"}

Robotic arm grip orientation

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 4.07s

Model answer correct?: No

Prompt: Is the robotic arm grip pointing down, at a left angle, or at a right angle? Return down right or left.

Ground Truth: down

Model Output: {"answer": "right"}

Robotic arm not over conveyor belt

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 3.92s

Model answer correct?: No

Prompt: Is the robotic arm over the conveyor belt? Return only yes or no.

Ground Truth: No

Model Output: {"answer": "yes"}

Seal verification

Submitted by: Roboflow

Category: Defect Detection

Time taken: 0.98s

Model answer correct?: No

Prompt: Is the bottle top properly sealed onto the bottle? Return only yes or no.

Ground Truth: No

Model Output: {"answer": "yes"}

Shipping container OCR

Submitted by: Roboflow

Category: OCR

Time taken: 4.53s

Model answer correct?: No

Prompt: What is the horizontal ID on the container? Return only the ID.

Ground Truth: JBHU282862

Model Output: {"answer": "282862"}

Sudoku puzzle extraction

Submitted by: Roboflow

Category: Document Understanding

Time taken: 61.05s

Model answer correct?: No

Prompt: You are given an image of a Sudoku puzzle. Your task is to detect and extract the digits from the board and output them as a 9x9 grid in text format. Identify the numbers visible on the Sudoku grid. Use a dot (.) to represent empty cells (i.e., those without a digit). Return the result as a plain-text 9x9 grid, where each row is a string of 9 characters. Maintain the correct order from top-left to bottom-right of the board.

Ground Truth: ...4.8... .6..7..1. 7.2.9.5.4 .9.7.4.3. ..7.5.8.. .8.9.6.5. 9.4.1.7.8 .7..6..4. ...2.7...

Model Output: {"answer": "3.5.2.1.4.6.7.8.9.1.2.5.6.8.9.4.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.2.7.8.9.4.

Supermarket shelf label reading with one item in focus

Submitted by: Roboflow

Category: OCR

Time taken: 3.04s

Model answer correct?: No

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: {"answer": "0.00"}

Supermarket shelf label reading with two items in focus

Submitted by: Roboflow

Category: OCR

Time taken: 4.56s

Model answer correct?: No

Prompt: How much are the wipes? Currency is GBP. Return in form 1.00, or 0.00.

Ground Truth: 0.58

Model Output: {"answer": "0.00"}

Table understanding

Submitted by: Roboflow

Category: Document Understanding

Time taken: 6.16s

Model answer correct?: No

Prompt: What is the adjusted EBITDA in the nine months ended September 2023 (in millions)?

Ground Truth: 2,428

Model Output: {"answer": 232598}

Total Credits in Fall 2015

Submitted by: Jayesh

Category: Document Understanding

Time taken: 3.92s

Model answer correct?: No

Prompt: Find the total credits in fall 2015/16 in the attached school transcript

Ground Truth: 12

Model Output: {"answer": "11"}

Truck in bay

Submitted by: Roboflow

Category: Spatial Relations

Time taken: 5.94s

Model answer correct?: No

Prompt: Is the Scania truck in the bay? Return either yes or no.

Ground Truth: No

Model Output: {"answer": "yes"}