Not solved by any model

There are 613 examples not solved by any model. Solving some of these can be a good signal that your model is indeed better than leading models if these are good problems.
hellaswag/10030, hellaswag/10032, hellaswag/1005, hellaswag/1015, hellaswag/1029, hellaswag/1055, hellaswag/1063, hellaswag/1074, hellaswag/1075, hellaswag/1078, hellaswag/1082, hellaswag/1085, hellaswag/109, hellaswag/110, hellaswag/1112, hellaswag/1115, hellaswag/1140, hellaswag/1165, hellaswag/1168, hellaswag/118, hellaswag/12, hellaswag/1200, hellaswag/1215, hellaswag/1216, hellaswag/1237, hellaswag/1247, hellaswag/1249, hellaswag/125, hellaswag/1261, hellaswag/1265, hellaswag/1266, hellaswag/1271, hellaswag/1285, hellaswag/1287, hellaswag/1294, hellaswag/1304, hellaswag/1315, hellaswag/1337, hellaswag/1358, hellaswag/1375, hellaswag/1389, hellaswag/1397, hellaswag/1400, hellaswag/142, hellaswag/1420, hellaswag/1421, hellaswag/1446, hellaswag/1449, hellaswag/145, hellaswag/1452, hellaswag/1458, hellaswag/1460, hellaswag/1468, hellaswag/1469, hellaswag/1476, hellaswag/1477, hellaswag/148, hellaswag/1482, hellaswag/1494, hellaswag/1497, hellaswag/1508, hellaswag/1509, hellaswag/1512, hellaswag/1516, hellaswag/1525, hellaswag/1540, hellaswag/1542, hellaswag/1544, hellaswag/1545, hellaswag/1547, hellaswag/1550, hellaswag/1555, hellaswag/1556, hellaswag/1565, hellaswag/1567, hellaswag/1577, hellaswag/1579, hellaswag/1584, hellaswag/1585, hellaswag/1604, hellaswag/1610, hellaswag/163, hellaswag/1633, hellaswag/1642, hellaswag/1648, hellaswag/1650, hellaswag/1652, hellaswag/166, hellaswag/1661, hellaswag/1662, hellaswag/1666, hellaswag/1670, hellaswag/1672, hellaswag/1677, hellaswag/169, hellaswag/1711, hellaswag/1713, hellaswag/1741, hellaswag/1743, hellaswag/1774, hellaswag/1775, hellaswag/1779, hellaswag/1807, hellaswag/1812, hellaswag/1823, hellaswag/1828, hellaswag/183, hellaswag/1832, hellaswag/1840, hellaswag/1845, hellaswag/1854, hellaswag/1870, hellaswag/1875, hellaswag/1877, hellaswag/1878, hellaswag/190, hellaswag/1916, hellaswag/1917, hellaswag/192, hellaswag/1947, hellaswag/195, hellaswag/1970, hellaswag/1971, hellaswag/1976, hellaswag/1984, hellaswag/1997, hellaswag/2012, hellaswag/202, hellaswag/2055, hellaswag/2066, hellaswag/2074, hellaswag/2075, hellaswag/2083, hellaswag/2086, hellaswag/2087, hellaswag/2105, hellaswag/2107, hellaswag/2138, hellaswag/2143, hellaswag/2162, hellaswag/2165, hellaswag/2176, hellaswag/218, hellaswag/2182, hellaswag/2187, hellaswag/2189, hellaswag/2190, hellaswag/2204, hellaswag/2208, hellaswag/2209, hellaswag/221, hellaswag/2211, hellaswag/2216, hellaswag/2237, hellaswag/224, hellaswag/2246, hellaswag/2251, hellaswag/2259, hellaswag/2262, hellaswag/2287, hellaswag/2292, hellaswag/2295, hellaswag/230, hellaswag/2306, hellaswag/2321, hellaswag/2323, hellaswag/2340, hellaswag/2346, hellaswag/2347, hellaswag/2349, hellaswag/235, hellaswag/2354, hellaswag/2355, hellaswag/239, hellaswag/2393, hellaswag/2396, hellaswag/2397, hellaswag/240, hellaswag/2404, hellaswag/2406, hellaswag/2409, hellaswag/2419, hellaswag/2425, hellaswag/2427, hellaswag/2428, hellaswag/2467, hellaswag/2497, hellaswag/250, hellaswag/251, hellaswag/2510, hellaswag/252, hellaswag/2521, hellaswag/2559, hellaswag/2575, hellaswag/2598, hellaswag/2599, hellaswag/2609, hellaswag/261, hellaswag/2617, hellaswag/2622, hellaswag/2624, hellaswag/2625, hellaswag/2632, hellaswag/2641, hellaswag/2646, hellaswag/2661, hellaswag/2662, hellaswag/2677, hellaswag/269, hellaswag/270, hellaswag/2722, hellaswag/2738, hellaswag/2746, hellaswag/2749, hellaswag/2754, hellaswag/2759, hellaswag/2765, hellaswag/2770, hellaswag/2776, hellaswag/2777, hellaswag/2783, hellaswag/2787, hellaswag/2789, hellaswag/279, hellaswag/2792, hellaswag/2807, hellaswag/2809, hellaswag/2817, hellaswag/2821, hellaswag/2822, hellaswag/2833, hellaswag/2848, hellaswag/2852, hellaswag/2859, hellaswag/2860, hellaswag/2878, hellaswag/2895, hellaswag/2906, hellaswag/2918, hellaswag/2921, hellaswag/2934, hellaswag/2942, hellaswag/2954, hellaswag/296, hellaswag/297, hellaswag/2981, hellaswag/2986, hellaswag/2990, hellaswag/30, hellaswag/3005, hellaswag/3015, hellaswag/3035, hellaswag/3040, hellaswag/3042, hellaswag/3059, hellaswag/3062, hellaswag/3083, hellaswag/3086, hellaswag/3101, hellaswag/3106, hellaswag/3117, hellaswag/3122, hellaswag/3125, hellaswag/3126, hellaswag/3133, hellaswag/3136, hellaswag/3137, hellaswag/3138, hellaswag/3140, hellaswag/3155, hellaswag/3171, hellaswag/3176, hellaswag/3179, hellaswag/3185, hellaswag/3189, hellaswag/3204, hellaswag/3207, hellaswag/3209, hellaswag/3212, hellaswag/323, hellaswag/3233, hellaswag/3242, hellaswag/327, hellaswag/3291, hellaswag/3298, hellaswag/3310, hellaswag/3331, hellaswag/3349, hellaswag/3402, hellaswag/3434, hellaswag/345, hellaswag/3457, hellaswag/3474, hellaswag/3487, hellaswag/349, hellaswag/3498, hellaswag/3513, hellaswag/354, hellaswag/3541, hellaswag/3567, hellaswag/358, hellaswag/3592, hellaswag/3597, hellaswag/3617, hellaswag/3627, hellaswag/365, hellaswag/3666, hellaswag/368, hellaswag/3682, hellaswag/3693, hellaswag/371, hellaswag/3719, hellaswag/3727, hellaswag/3756, hellaswag/3766, hellaswag/380, hellaswag/3802, hellaswag/3810, hellaswag/3818, hellaswag/3819, hellaswag/3830, hellaswag/3844, hellaswag/389, hellaswag/3897, hellaswag/3904, hellaswag/3915, hellaswag/3926, hellaswag/3939, hellaswag/3965, hellaswag/3969, hellaswag/401, hellaswag/4038, hellaswag/4041, hellaswag/4056, hellaswag/4077, hellaswag/4112, hellaswag/4180, hellaswag/4195, hellaswag/4202, hellaswag/4236, hellaswag/4237, hellaswag/4278, hellaswag/4285, hellaswag/4308, hellaswag/434, hellaswag/4340, hellaswag/4345, hellaswag/436, hellaswag/4361, hellaswag/4375, hellaswag/440, hellaswag/4492, hellaswag/4493, hellaswag/452, hellaswag/4520, hellaswag/4592, hellaswag/4642, hellaswag/465, hellaswag/4706, hellaswag/4735, hellaswag/474, hellaswag/4744, hellaswag/4749, hellaswag/4750, hellaswag/4752, hellaswag/4753, hellaswag/4767, hellaswag/4797, hellaswag/48, hellaswag/4804, hellaswag/4827, hellaswag/4836, hellaswag/4848, hellaswag/4891, hellaswag/4899, hellaswag/50, hellaswag/5006, hellaswag/5023, hellaswag/5038, hellaswag/505, hellaswag/5065, hellaswag/5078, hellaswag/5115, hellaswag/5126, hellaswag/514, hellaswag/5169, hellaswag/5292, hellaswag/53, hellaswag/530, hellaswag/5300, hellaswag/5301, hellaswag/5317, hellaswag/5326, hellaswag/5330, hellaswag/534, hellaswag/536, hellaswag/5383, hellaswag/5397, hellaswag/5412, hellaswag/5434, hellaswag/545, hellaswag/546, hellaswag/5461, hellaswag/5464, hellaswag/5481, hellaswag/5493, hellaswag/5524, hellaswag/558, hellaswag/5617, hellaswag/5619, hellaswag/5629, hellaswag/5707, hellaswag/5744, hellaswag/579, hellaswag/5794, hellaswag/583, hellaswag/5832, hellaswag/5833, hellaswag/5872, hellaswag/5883, hellaswag/5898, hellaswag/5916, hellaswag/5959, hellaswag/5961, hellaswag/6, hellaswag/6053, hellaswag/6060, hellaswag/609, hellaswag/610, hellaswag/6130, hellaswag/6135, hellaswag/614, hellaswag/616, hellaswag/6161, hellaswag/6230, hellaswag/6250, hellaswag/6273, hellaswag/630, hellaswag/6323, hellaswag/6363, hellaswag/6381, hellaswag/6400, hellaswag/642, hellaswag/6430, hellaswag/6432, hellaswag/645, hellaswag/6483, hellaswag/6530, hellaswag/6578, hellaswag/6605, hellaswag/661, hellaswag/6626, hellaswag/6714, hellaswag/6739, hellaswag/674, hellaswag/6741, hellaswag/6743, hellaswag/6750, hellaswag/6758, hellaswag/6761, hellaswag/6762, hellaswag/6766, hellaswag/6768, hellaswag/677, hellaswag/6784, hellaswag/68, hellaswag/6807, hellaswag/6825, hellaswag/6883, hellaswag/6887, hellaswag/6970, hellaswag/6991, hellaswag/6996, hellaswag/7, hellaswag/70, hellaswag/7000, hellaswag/7029, hellaswag/7048, hellaswag/705, hellaswag/708, hellaswag/709, hellaswag/71, hellaswag/712, hellaswag/7130, hellaswag/7196, hellaswag/7246, hellaswag/7269, hellaswag/727, hellaswag/7271, hellaswag/7285, hellaswag/732, hellaswag/7348, hellaswag/7354, hellaswag/7355, hellaswag/7395, hellaswag/7405, hellaswag/7451, hellaswag/747, hellaswag/7501, hellaswag/7510, hellaswag/7521, hellaswag/7527, hellaswag/754, hellaswag/7548, hellaswag/7556, hellaswag/7568, hellaswag/7595, hellaswag/7624, hellaswag/7651, hellaswag/7653, hellaswag/7674, hellaswag/7781, hellaswag/7813, hellaswag/7818, hellaswag/7823, hellaswag/7827, hellaswag/7884, hellaswag/789, hellaswag/7910, hellaswag/7915, hellaswag/7916, hellaswag/7917, hellaswag/7947, hellaswag/80, hellaswag/806, hellaswag/8095, hellaswag/8127, hellaswag/8133, hellaswag/8145, hellaswag/8195, hellaswag/820, hellaswag/821, hellaswag/8218, hellaswag/8220, hellaswag/8221, hellaswag/826, hellaswag/8271, hellaswag/8315, hellaswag/833, hellaswag/836, hellaswag/8382, hellaswag/839, hellaswag/845, hellaswag/8462, hellaswag/8487, hellaswag/8511, hellaswag/8540, hellaswag/8561, hellaswag/857, hellaswag/8575, hellaswag/8594, hellaswag/8607, hellaswag/862, hellaswag/8682, hellaswag/8692, hellaswag/8693, hellaswag/8728, hellaswag/8756, hellaswag/8766, hellaswag/884, hellaswag/8864, hellaswag/8894, hellaswag/8896, hellaswag/8899, hellaswag/8928, hellaswag/895, hellaswag/8950, hellaswag/8957, hellaswag/899, hellaswag/8990, hellaswag/9006, hellaswag/9033, hellaswag/9044, hellaswag/9057, hellaswag/9058, hellaswag/9130, hellaswag/9159, hellaswag/9166, hellaswag/9173, hellaswag/9198, hellaswag/921, hellaswag/9226, hellaswag/927, hellaswag/9279, hellaswag/928, hellaswag/9329, hellaswag/9339, hellaswag/9377, hellaswag/9400, hellaswag/9401, hellaswag/9410, hellaswag/9412, hellaswag/9413, hellaswag/942, hellaswag/9437, hellaswag/948, hellaswag/9481, hellaswag/9512, hellaswag/9524, hellaswag/9540, hellaswag/9630, hellaswag/9647, hellaswag/9653, hellaswag/9657, hellaswag/967, hellaswag/9771, hellaswag/9834, hellaswag/987, hellaswag/9879, hellaswag/9886, hellaswag/9889, hellaswag/9912, hellaswag/9976