🔎 CRUXEval Sample Explorer 🔎

CRUXEval is a benchmark complementary to HumanEval and MBPP measuring code reasoning, understanding, and execution capabilities!

Homepage Paper Code HF Dataset Leaderboard

model: temp:

def f(num):
    letter = 1
    for i in '1234567890':
        num = num.replace(i,'')
        if len(num) == 0: break
        num = num[letter:] + num[:letter]
        letter += 1
    return num
assert f('bwmm7h') == 'mhbwm'

CRUXEval-I assert f(??) == 'mhbwm'

CRUXEval-O assert f('bwmm7h') == ??

The site is inspired by and based on Minerva sample explorer and Llemma. Thanks!