Scientists design new 'AGI benchmark' that indicates whether any future AI model could cause 'catastrophic harm' - Keumars Afifi-Sabet, Live Science

OpenAI scientists have designed MLE-bench — a compilation of 75 extremely difficult tests that can assess whether a future advanced AI agent is capable of modifying its own code and improving itself. Scientists have designed a new set of tests that measure whether artificial intelligence (AI) agents can modify their own code and improve its capabilities without human instruction. The benchmark, dubbed "MLE-bench," is a compilation of 75 Kaggle tests, each one a challenge that tests machine learning engineering. This work involves training AI models, preparing datasets, and running scientific experiments, and the Kaggle tests measure how well the machine learning algorithms perform at specific tasks.

https://www.livescience.com/technology/artificial-intelligence/scientists-design-new-agi-benchmark-that-may-say-whether-any-future-ai-model-could-cause-catastrophic-harm