Artificial intelligence: Performance on knowledge tests vs. training computation
Performance on knowledge tests is measured with the MMLU benchmark, here with 5-shot learning, which gauges a model’s accuracy after receiving only fiveexamples for each task. Training computation is measured in total petaFLOP, which is 10¹⁵ floating-point operations.
Interactive visualization requires JavaScript
Related research and data
- Artificial Intelligence
- Scaling up: how increasing inputs has made artificial intelligence more capable
Charts
- Affiliation of research teams building notable AI systems, by year of publication
- Annual attendance at major artificial intelligence conferences
- Annual global corporate investment in artificial intelligence, by type
- Annual granted patents related to artificial intelligence, by industry
- Annual industrial robots installed in top 5 countries
- Annual patent applications related to AI per million people
- Annual patent applications related to AI, by status
- Annual patent applications related to artificial intelligence
- Annual private investment in artificial intelligenceCSET
- Annual private investment in artificial intelligenceNetBase Quid
- Annual private investment in artificial intelligence, by focus areaNetBase Quid
- Annual professional service robots installed globally, by application area
- Annual scholarly publications on artificial intelligence
- Artificial intelligence: Performance on knowledge tests vs. dataset size
- Artificial intelligence: Performance on knowledge tests vs. number of parameters
- Computation used to train notable AI systems, by affiliation of researchers
- Computation used to train notable artificial intelligence systems, by domain
- Countries with national artificial intelligence strategies
- Cumulative AI-related bills passed into law since 2016, as of 2023
- Cumulative number of large-scale AI models by domain since 2017
- Cumulative number of large-scale AI systems by country since 2017
- Cumulative number of notable AI systems by domain
- Datapoints used to train notable artificial intelligence systems
- Domain of notable artificial intelligence systems, by year of publication
- Employer of new AI PhDs in the United States and Canada
- Exponential growth of computation in the training of notable AI systems
- Exponential growth of datapoints used to train notable AI systems
- Exponential growth of parameters in notable AI systems
- GPU computational performance per dollar
- Global annual number of reported artificial intelligence incidents and controversies
- Global investment in generative AI
- Global views about AI's impact on society in the next 20 years, by demographic group
- Global views about the safety of riding in a self-driving car, by demographic group
- Hardware and energy cost to train notable AI systems
- Highest chess rating ever achieved by computers
- How worried are Americans about their work being automated?
- ImageNet: Top-performing AI systems in labeling images
- Industrial robots: Annual installations and total in operation
- Market share for logic chip production, by manufacturing stage
- Newly-funded artificial intelligence companies
- Number of large-scale AI systems released per year
- Parameters in notable artificial intelligence systems
- Parameters vs. training dataset size in notable AI systems, by researcher affiliation
- Protein folding prediction accuracy
- Scholarly publications on artificial intelligence per million people
- Share of artificial intelligence jobs among all job postings
- Share of companies using artificial intelligence technology
- Share of women among new computer science PhDs, United States and Canada
- Test scores of AI systems on various capabilities relative to human performance
- Top performing AI systems in coding, math, and language-based knowledge tests
- Total number of AI PhD graduates in US and Canada
- Training computation vs. dataset size in notable AI systems, by researcher affiliation
- Training computation vs. parameters in notable AI systems, by domain
- Training computation vs. parameters in notable AI systems, by researcher affiliation
- Views about AI's impact on society in the next 20 years
- Views about the safety of riding in a self-driving car
- Views of Americans about robot vs. human intelligence