179 questions
No questions match those filters.
← All questions
EvaluationMedium· Reasoning· Generalization· Benchmarking
What is ARC-AGI trying to measure that GPQA is not?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansGPQA relies on knowledge retrieval and expert-level reasoning that a model may have memorized during training. In contrast, ARC-AGI deliberately strips away language and factual knowledge to test a model’s ability to infer rules from novel spatial patterns and apply them compositionally. This makes ARC-AGI resistant to the memorization-based performance seen in knowledge-heavy benchmarks.