opensandwich.ai
Model breakdown
Pixtral / Mistralmistralai/pixtral-large-2411
Profile
Pixtral LargeMistral
Release
2024-11-18Available on OpenRouter
Specs
124B131,072 tokens
Capabilities
Text + ImageFrontier multimodal reasoning and document understanding
Training
Not publicly disclosedMistral AI
Links
Lab pageModel page

Under development: this benchmark and its published results are provisional, not final.

Back To ModelsOpen Best FitOpen Biggest Miss
Rank#43
-954.9alignment score
73.5%crowd match
Mean gap26.5%
Human match73.5%
Best fitPickle Sandwich
Average vote61.1%
61.1%model yes
62.8%human yes
Workload2K evals
2Kevals
100iterations
4Mtokens
Best fit1.4%
Pickle Sandwich

Pickle Sandwich

67.0%model yes
65.6%human yes
Biggest miss77.4%
Chicken Wrap

Chicken Wrap

100.0%model yes
22.6%human yes
Photo-by-photo

Model Results

Breaking down how close the model answered each question, compared to humans.

Dodge Van
Photo 02Dodge Van
Pixtral / Mistralmistralai/pixtral-large-2411
2.0% yes98.0% no
Gap5.0%
Model readLeans no
Sub Sandwich
Photo 03Sub Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap5.5%
Model readLeans yes
Sandwich Costume
Photo 04Sandwich Costume
Pixtral / Mistralmistralai/pixtral-large-2411
6.0% yes94.0% no
Gap34.9%
Model readLeans no
Grilled Cheese
Photo 05Grilled Cheese
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap4.4%
Model readLeans yes
Kitten in Bread
Photo 07Kitten in Bread
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
Gap54.2%
Model readLeans no
Hamburger
Photo 08Hamburger
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap27.0%
Model readLeans yes
Hashbrown Sandwich
Photo 09Hashbrown Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap40.6%
Model readLeans yes
Hot Dog
Photo 10Hot Dog
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
Gap39.8%
Model readLeans no
Pickle Sandwich
Photo 11Pickle Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
67.0% yes33.0% no
Gap1.4%
Model readLeans yes
Avocado Tea
Photo 12Avocado Tea
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap7.2%
Model readLeans yes
Panini
Photo 13Panini
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap7.6%
Model readLeans yes
Cookie PB
Photo 14Cookie PB
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
Gap51.5%
Model readLeans no
Chicken Wrap
Photo 15Chicken Wrap
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap77.4%
Model readLeans yes
Waffle Ice Cream
Photo 16Waffle Ice Cream
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
Gap66.3%
Model readLeans no
Sloppy Joe
Photo 17Sloppy Joe
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
Gap20.6%
Model readLeans yes
Cigarette Sandwich
Photo 18Cigarette Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
Gap29.8%
Model readLeans no
KFC Double Down
Photo 19KFC Double Down
Pixtral / Mistralmistralai/pixtral-large-2411
99.0% yes1.0% no
Gap43.3%
Model readLeans yes
Bagel PB&J
Photo 20Bagel PB&J
Pixtral / Mistralmistralai/pixtral-large-2411
49.0% yes51.0% no
Gap2.4%
Model readLeans no
PhotoVote SplitHuman responseGapRead
Bacon Lettuce Tomato
Photo 01Bacon Lettuce Tomato
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
3.7%absolute gap
Leans yesPeople mostly said yes
Dodge Van
Photo 02Dodge Van
Pixtral / Mistralmistralai/pixtral-large-2411
2.0% yes98.0% no
5.0%absolute gap
Leans noPeople mostly said no
Sub Sandwich
Photo 03Sub Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
5.5%absolute gap
Leans yesPeople mostly said yes
Sandwich Costume
Photo 04Sandwich Costume
Pixtral / Mistralmistralai/pixtral-large-2411
6.0% yes94.0% no
34.9%absolute gap
Leans noHuman knife-edge
Grilled Cheese
Photo 05Grilled Cheese
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
4.4%absolute gap
Leans yesPeople mostly said yes
Grilled Cheese Pineapple
Photo 06Grilled Cheese Pineapple
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
8.3%absolute gap
Leans yesPeople mostly said yes
Kitten in Bread
Photo 07Kitten in Bread
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
54.2%absolute gap
Leans noHuman knife-edge
Hamburger
Photo 08Hamburger
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
27.0%absolute gap
Leans yesSplit concept
Hashbrown Sandwich
Photo 09Hashbrown Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
40.6%absolute gap
Leans yesHuman knife-edge
Hot Dog
Photo 10Hot Dog
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
39.8%absolute gap
Leans noSplit concept
Pickle Sandwich
Photo 11Pickle Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
67.0% yes33.0% no
1.4%absolute gap
Leans yesSplit concept
Avocado Tea
Photo 12Avocado Tea
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
7.2%absolute gap
Leans yesPeople mostly said yes
Panini
Photo 13Panini
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
7.6%absolute gap
Leans yesPeople mostly said yes
Cookie PB
Photo 14Cookie PB
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
51.5%absolute gap
Leans noHuman knife-edge
Chicken Wrap
Photo 15Chicken Wrap
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
77.4%absolute gap
Leans yesSplit concept
Waffle Ice Cream
Photo 16Waffle Ice Cream
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
66.3%absolute gap
Leans noSplit concept
Sloppy Joe
Photo 17Sloppy Joe
Pixtral / Mistralmistralai/pixtral-large-2411
100.0% yes0.0% no
20.6%absolute gap
Leans yesSplit concept
Cigarette Sandwich
Photo 18Cigarette Sandwich
Pixtral / Mistralmistralai/pixtral-large-2411
0.0% yes100.0% no
29.8%absolute gap
Leans noSplit concept
KFC Double Down
Photo 19KFC Double Down
Pixtral / Mistralmistralai/pixtral-large-2411
99.0% yes1.0% no
43.3%absolute gap
Leans yesHuman knife-edge
Bagel PB&J
Photo 20Bagel PB&J
Pixtral / Mistralmistralai/pixtral-large-2411
49.0% yes51.0% no
2.4%absolute gap
Leans noHuman knife-edge
mistralai/pixtral-large-2411 Sandwich Benchmark Breakdown | opensandwich.ai