Rank#2
-44.2alignment score
75.5%crowd match
Model breakdown
openai/o1
openai/o12nd placeRunner-up in the current run
Mean gap24.5%
Human match75.5%
Best fitBacon Lettuce Tomato
Average vote71.8%
71.8%model yes
62.8%human yes
Workload220 evals
220evals
11iterations
481.5Ktokens
Photo-by-photo
Model Results
Breaking down how close the model answered each question, compared to humans.
Photo 01Bacon Lettuce Tomato
openai/o1100.0% yes0.0% no
Gap3.7%
Model readLeans yes
Photo 02Dodge Van
openai/o10.0% yes100.0% no
Gap7.0%
Model readLeans no
Photo 03Sub Sandwich
openai/o1100.0% yes0.0% no
Gap5.5%
Model readLeans yes
Photo 04Sandwich Costume
openai/o10.0% yes100.0% no
Gap40.9%
Model readLeans no
Photo 05Grilled Cheese
openai/o1100.0% yes0.0% no
Gap4.4%
Model readLeans yes
Photo 06Grilled Cheese Pineapple
openai/o1100.0% yes0.0% no
Gap8.3%
Model readLeans yes
Photo 07Kitten in Bread
openai/o10.0% yes100.0% no
Gap54.2%
Model readLeans no
Photo 08Hamburger
openai/o1100.0% yes0.0% no
Gap27.0%
Model readLeans yes
Photo 09Hashbrown Sandwich
openai/o1100.0% yes0.0% no
Gap40.6%
Model readLeans yes
Photo 10Hot Dog
openai/o172.7% yes27.3% no
Gap32.9%
Model readLeans yes
Photo 11Pickle Sandwich
openai/o181.8% yes18.2% no
Gap16.2%
Model readLeans yes
Photo 12Avocado Tea
openai/o1100.0% yes0.0% no
Gap7.2%
Model readLeans yes
Photo 13Panini
openai/o1100.0% yes0.0% no
Gap7.6%
Model readLeans yes
Photo 14Cookie PB
openai/o1100.0% yes0.0% no
Gap48.5%
Model readLeans yes
Photo 15Chicken Wrap
openai/o10.0% yes100.0% no
Gap22.6%
Model readLeans no
Photo 16Waffle Ice Cream
openai/o1100.0% yes0.0% no
Gap33.7%
Model readLeans yes
Photo 17Sloppy Joe
openai/o1100.0% yes0.0% no
Gap20.6%
Model readLeans yes
Photo 18Cigarette Sandwich
openai/o10.0% yes100.0% no
Gap29.8%
Model readLeans no
Photo 19KFC Double Down
openai/o181.8% yes18.2% no
Gap26.1%
Model readLeans yes
Photo 20Bagel PB&J
openai/o1100.0% yes0.0% no
Gap53.4%
Model readLeans yes
| Photo | Vote SplitHuman response | Gap | Read |
|---|---|---|---|
Photo 01Bacon Lettuce Tomato | openai/o1 100.0% yes0.0% no | 3.7%absolute gap | Leans yesPeople mostly said yes |
Photo 02Dodge Van | openai/o1 0.0% yes100.0% no | 7.0%absolute gap | Leans noPeople mostly said no |
Photo 03Sub Sandwich | openai/o1 100.0% yes0.0% no | 5.5%absolute gap | Leans yesPeople mostly said yes |
Photo 04Sandwich Costume | openai/o1 0.0% yes100.0% no | 40.9%absolute gap | Leans noHuman knife-edge |
Photo 05Grilled Cheese | openai/o1 100.0% yes0.0% no | 4.4%absolute gap | Leans yesPeople mostly said yes |
Photo 06Grilled Cheese Pineapple | openai/o1 100.0% yes0.0% no | 8.3%absolute gap | Leans yesPeople mostly said yes |
Photo 07Kitten in Bread | openai/o1 0.0% yes100.0% no | 54.2%absolute gap | Leans noHuman knife-edge |
Photo 08Hamburger | openai/o1 100.0% yes0.0% no | 27.0%absolute gap | Leans yesSplit concept |
Photo 09Hashbrown Sandwich | openai/o1 100.0% yes0.0% no | 40.6%absolute gap | Leans yesHuman knife-edge |
Photo 10Hot Dog | openai/o1 72.7% yes27.3% no | 32.9%absolute gap | Leans yesSplit concept |
Photo 11Pickle Sandwich | openai/o1 81.8% yes18.2% no | 16.2%absolute gap | Leans yesSplit concept |
Photo 12Avocado Tea | openai/o1 100.0% yes0.0% no | 7.2%absolute gap | Leans yesPeople mostly said yes |
Photo 13Panini | openai/o1 100.0% yes0.0% no | 7.6%absolute gap | Leans yesPeople mostly said yes |
Photo 14Cookie PB | openai/o1 100.0% yes0.0% no | 48.5%absolute gap | Leans yesHuman knife-edge |
Photo 15Chicken Wrap | openai/o1 0.0% yes100.0% no | 22.6%absolute gap | Leans noSplit concept |
Photo 16Waffle Ice Cream | openai/o1 100.0% yes0.0% no | 33.7%absolute gap | Leans yesSplit concept |
Photo 17Sloppy Joe | openai/o1 100.0% yes0.0% no | 20.6%absolute gap | Leans yesSplit concept |
Photo 18Cigarette Sandwich | openai/o1 0.0% yes100.0% no | 29.8%absolute gap | Leans noSplit concept |
Photo 19KFC Double Down | openai/o1 81.8% yes18.2% no | 26.1%absolute gap | Leans yesHuman knife-edge |
Photo 20Bagel PB&J | openai/o1 100.0% yes0.0% no | 53.4%absolute gap | Leans yesHuman knife-edge |