🔎 CRUXEval Sample Explorer 🔎

CRUXEval is a benchmark complementary to HumanEval and MBPP measuring code reasoning, understanding, and execution capabilities!

Homepage Paper Code HF Dataset Leaderboard

model: temp:

def f(a_str, prefix):
    if a_str.removeprefix(prefix):
        return a_str
    else:
        return prefix + a_str
assert f('abc', 'abcd') == 'abc'

CRUXEval-I assert f(??) == 'abc'

CRUXEval-O assert f('abc', 'abcd') == ??

The site is inspired by and based on Minerva sample explorer and Llemma. Thanks!