The eval stage is extremely slow.

[xming521]

eval = {
    "partitioner": {
        "type": "NaivePartitioner",
    },
    "runner": {
        "type": "LocalRunner",
        "max_num_workers": 100,
        "task": {"type": "OpenICLEvalTask"},
    },
}

inference takes only half an hour, while evaluation requires two hours.
All datasets consist of multiple-choice questions.

[peter-xbs]

really slow indeed!

[xming521]

fix in #2409