opensandwich.ai
Model breakdown
Qwenqwen/qwen-2-vl-72b-instruct
Profile
Qwen 2 VL 72BQwen
Release
2024-09-18not on OpenRouter
Specs
72B32,768 tokens (base)
Capabilities
Text + Image + VideoOpen multimodal understanding, long video analysis, and visual agents
Training
Not publicly disclosedQwen
Links
Lab pageModel page

Under development: this benchmark and its published results are provisional, not final.

Back To ModelsOpen Best FitOpen Biggest Miss
Rank#28
-760.0alignment score
75.3%crowd match
Mean gap24.7%
Human match75.3%
Best fitBacon Lettuce Tomato
Average vote63.3%
63.3%model yes
62.8%human yes
Workload2K evals
2Kevals
100iterations
1.9Mtokens
Best fit3.7%
Bacon Lettuce Tomato

Bacon Lettuce Tomato

100.0%model yes
96.3%human yes
Biggest miss54.2%
Kitten in Bread

Kitten in Bread

0.0%model yes
54.2%human yes
Photo-by-photo

Model Results

Breaking down how close the model answered each question, compared to humans.

Dodge Van
Photo 02Dodge Van
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap7.0%
Model readLeans no
Sub Sandwich
Photo 03Sub Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap5.5%
Model readLeans yes
Sandwich Costume
Photo 04Sandwich Costume
Qwenqwen/qwen-2-vl-72b-instruct
4.0% yes96.0% no
Gap36.9%
Model readLeans no
Grilled Cheese
Photo 05Grilled Cheese
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap4.4%
Model readLeans yes
Kitten in Bread
Photo 07Kitten in Bread
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap54.2%
Model readLeans no
Hamburger
Photo 08Hamburger
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap27.0%
Model readLeans yes
Hashbrown Sandwich
Photo 09Hashbrown Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
99.0% yes1.0% no
Gap39.6%
Model readLeans yes
Hot Dog
Photo 10Hot Dog
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap39.8%
Model readLeans no
Pickle Sandwich
Photo 11Pickle Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
92.0% yes8.0% no
Gap26.4%
Model readLeans yes
Avocado Tea
Photo 12Avocado Tea
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap7.2%
Model readLeans yes
Panini
Photo 13Panini
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap7.6%
Model readLeans yes
Cookie PB
Photo 14Cookie PB
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap51.5%
Model readLeans no
Chicken Wrap
Photo 15Chicken Wrap
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap22.6%
Model readLeans no
Waffle Ice Cream
Photo 16Waffle Ice Cream
Qwenqwen/qwen-2-vl-72b-instruct
75.0% yes25.0% no
Gap8.7%
Model readLeans yes
Sloppy Joe
Photo 17Sloppy Joe
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
Gap20.6%
Model readLeans yes
Cigarette Sandwich
Photo 18Cigarette Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
Gap29.8%
Model readLeans no
KFC Double Down
Photo 19KFC Double Down
Qwenqwen/qwen-2-vl-72b-instruct
98.0% yes2.0% no
Gap42.3%
Model readLeans yes
Bagel PB&J
Photo 20Bagel PB&J
Qwenqwen/qwen-2-vl-72b-instruct
98.0% yes2.0% no
Gap51.4%
Model readLeans yes
PhotoVote SplitHuman responseGapRead
Bacon Lettuce Tomato
Photo 01Bacon Lettuce Tomato
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
3.7%absolute gap
Leans yesPeople mostly said yes
Dodge Van
Photo 02Dodge Van
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
7.0%absolute gap
Leans noPeople mostly said no
Sub Sandwich
Photo 03Sub Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
5.5%absolute gap
Leans yesPeople mostly said yes
Sandwich Costume
Photo 04Sandwich Costume
Qwenqwen/qwen-2-vl-72b-instruct
4.0% yes96.0% no
36.9%absolute gap
Leans noHuman knife-edge
Grilled Cheese
Photo 05Grilled Cheese
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
4.4%absolute gap
Leans yesPeople mostly said yes
Grilled Cheese Pineapple
Photo 06Grilled Cheese Pineapple
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
8.3%absolute gap
Leans yesPeople mostly said yes
Kitten in Bread
Photo 07Kitten in Bread
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
54.2%absolute gap
Leans noHuman knife-edge
Hamburger
Photo 08Hamburger
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
27.0%absolute gap
Leans yesSplit concept
Hashbrown Sandwich
Photo 09Hashbrown Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
99.0% yes1.0% no
39.6%absolute gap
Leans yesHuman knife-edge
Hot Dog
Photo 10Hot Dog
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
39.8%absolute gap
Leans noSplit concept
Pickle Sandwich
Photo 11Pickle Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
92.0% yes8.0% no
26.4%absolute gap
Leans yesSplit concept
Avocado Tea
Photo 12Avocado Tea
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
7.2%absolute gap
Leans yesPeople mostly said yes
Panini
Photo 13Panini
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
7.6%absolute gap
Leans yesPeople mostly said yes
Cookie PB
Photo 14Cookie PB
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
51.5%absolute gap
Leans noHuman knife-edge
Chicken Wrap
Photo 15Chicken Wrap
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
22.6%absolute gap
Leans noSplit concept
Waffle Ice Cream
Photo 16Waffle Ice Cream
Qwenqwen/qwen-2-vl-72b-instruct
75.0% yes25.0% no
8.7%absolute gap
Leans yesSplit concept
Sloppy Joe
Photo 17Sloppy Joe
Qwenqwen/qwen-2-vl-72b-instruct
100.0% yes0.0% no
20.6%absolute gap
Leans yesSplit concept
Cigarette Sandwich
Photo 18Cigarette Sandwich
Qwenqwen/qwen-2-vl-72b-instruct
0.0% yes100.0% no
29.8%absolute gap
Leans noSplit concept
KFC Double Down
Photo 19KFC Double Down
Qwenqwen/qwen-2-vl-72b-instruct
98.0% yes2.0% no
42.3%absolute gap
Leans yesHuman knife-edge
Bagel PB&J
Photo 20Bagel PB&J
Qwenqwen/qwen-2-vl-72b-instruct
98.0% yes2.0% no
51.4%absolute gap
Leans yesHuman knife-edge
qwen/qwen-2-vl-72b-instruct Sandwich Benchmark Breakdown | opensandwich.ai