🔎 CRUXEval Sample Explorer 🔎

CRUXEval is a benchmark complementary to HumanEval and MBPP measuring code reasoning, understanding, and execution capabilities!

Homepage Paper Code HF Dataset Leaderboard

model: temp:

def f(text, repl):
    trans = str.maketrans(text.lower(), repl.lower())
    return text.translate(trans)
assert f('upper case', 'lower case') == 'lwwer case'

CRUXEval-I assert f(??) == 'lwwer case'

CRUXEval-O assert f('upper case', 'lower case') == ??

The site is inspired by and based on Minerva sample explorer and Llemma. Thanks!