The second annual AI Index report pulls together data and expert findings on the field’s progress and acceleration.

The rate of progress in the field of artificial intelligence is one of the most hotly contested aspects of the ongoing boom in teaching computers and robots how to see the world, make sense of it, and eventually perform complex tasks both in the physical realm and the virtual one. And just how fast the industry is moving, and to what end, is typically measured not just by actual product advancements and research milestones, but also by the prognostications and voiced concerns of AI leaders, futurists, academics, economists, and policymakers. AI is going to change the world — but how and when are still open questions.

Today, findings from a group of experts were published in an ongoing effort to help answer those questions. The experts include members of Harvard, MIT, Stanford, the nonprofit OpenAI, and the Partnership on AI industry consortium, among others, and they were put together as part of the second annual AI Index. The goal is to measure the field’s progress using hard data and to try and make sense of that progress as it relates to thorny subjects like workplace automation and the overarching quest for artificial general intelligence, or the type of intelligence that could let a machine perform any task a human could.


The first report, published last December, found that investment and work in AI was accelerated at an unprecedented rate and that, while progress in certain fields like limited game-playing and vision has been extraordinary, AI remains far behind in general intelligence tasks that would result in, say, total automation of more than a limited variety of jobs. Still, the report was lacking in what the authors call a “global perspective,” and this second edition set out to answer many of the same questions with new, more granular data and a more international scope.

“There is no AI story without global perspective. The 2017 report was heavily skewed towards North American activities. This reflected a limited number of global partnerships, not an intrinsic bias,” reads the 2018 report’s introduction. “This year, we begin to close the global gap. We recognize that there is a long journey ahead — one that involves collaboration and outside participation — to make this report truly comprehensive.”

In that spirit of global analysis, the second AI Index report finds that commercial and research work in AI, as well as funding, is exploding pretty much everywhere on the planet. There’s an especially high concentration in Europe and Asia, with China, Japan, and South Korea leading Eastern countries in AI research paper publication, university enrollment, and patent applications. In fact, Europe is the largest publisher of AI papers, with 28 percent of all AI-related publications last year. China is close behind with 25 percent, while North America is responsible for 17 percent.


When it comes to the type of AI activity, the report finds that machine learning and so-called probabilistic reasoning — or the type of cognition-related performance that lets a game-playing AI outsmart a human opponent — is far and away the leading research category by a number of published papers.

Not far behind, however, is work on computer vision, which is the foundational sub-discipline of AI that’s helping to develop self-driving cars and power augmented reality and object recognition, and neural networks, which, like machine learning, are instrumental in training those algorithms to improve over time. Less important, at least in the current moment, are areas like natural language processing, which is what lets your smart speaker understand what you’re saying and respond in kind, and general planning and decision making, which is what will be required of robots when automated machines are inevitably more integral facets of daily life.


A fascinating element of the report is how research in those categories breaks down by global region. China is heavily focused on agricultural science, engineering, and technology, while Europe and North America are focused more on the humanities and medical and health sciences, though Europe is generally more well-rounded in its approach to research.

Some other interesting tidbits from the report include US AI research papers, which, despite being lower in volume, outpace China and Europe in citations. Government-related organizations and research outfits also account for far more papers in China and Europe than corporations or the medical field, while the US’s AI research efforts are largely dominated by corporate efforts, which makes sense given the immense investment in the field from Apple, Amazon, Google, Facebook, and Microsoft.


As far as performance goes, AI continues to skyrocket, especially in fields like computer vision. By measuring benchmark performance for the widely used image training database ImageNet, the report finds that the time it takes to spin up a model that can classify pictures at state-of-the-art accuracy fell “from around on hour to around 4 minutes” in just 18 months. That equates to a roughly 16x jump in training speed. Other areas like object segmentation, which is what lets software differentiate between an image’s background and its subject, has increased in precision by 72 percent in just three years.


For areas like machine translation and parsing, which is what lets software understand syntactic structures and more easily answer questions, accuracy and proficiency is getting more and more refined, but with diminishing returns as algorithms get ever closer human-level understanding of language.

In a separate “human-level milestones” section, the report breaks down some big 2018 milestones in fields like game-playing and medical diagnostics where progress is accelerating at surprising rates. Those include progress from Google-owned DeepMind in playing the classic first-person shooter Quake in objective-oriented game modes like capture the flag, as well as landmark performances against amateur and then former professional players of the online battle arena game Dota 2.


All of this hard data is fantastic in understanding where the AI field stands right now and how it’s been growing over the years and is projected to grow in the future. Yet, we’re still stuck in murky territory when it comes to harder questions around automation and the ways that AI could be implemented in areas like criminal justice, border patrol screenings, warfare, and other areas where performance is less important than the underlying governmental policy at play. AI will only continue to get more sophisticated, but there are a number of hurdles, both technological and with regard to bias and safety, before such software could be reliably used without error in hospitals, education systems, airports, and police departments.

Unfortunately, that hasn’t stopped corporations and governments from continuing to plow forward in deploying AI in the real world. This year, we discovered that Amazon was selling its Recognition facial recognition software to law enforcement, while Google found itself embroiled in controversy after it was discovered it was contributing computer vision expertise to a Department of Defense drone program known as Project Maven.


Google said it would pull out of the project once its contract expired, and it also published a wide-ranging set of AI ethics principles that included a pledge never to develop AI weaponry surveillance systems or to contribute to any project that violated “widely accepted principles of international law and human rights.” But it’s clear that the leaders of Silicon Valley see AI as a prime business opportunity and such projects and contracts as the financial reward for participating in the AI research arms race.

Elsewhere in the world, AI is helping governments pioneer systems of surveillance and law enforcement that constantly track citizens as they move about society. According to The New York Times, China is using millions of cameras and AI-assisted technologies like facial recognition to create the world’s most comprehensive surveillance system for its nearly 1.4 billion-person populace. Such a system is expected to link with the country’s new social credit system for scoring citizens and stratifying society into layers of access and privilege based on education, financial background, and other metrics, all of which will be informed by a day-to-day data collection and analysis of people’s real-world and online behaviors.

With automation, we’ve come to an understanding that mass unemployment isn’t coming anytime soon, and the bigger concern is whether we as a society are prepared for the nature of work to transition toward less stable, lower-paid jobs without safety nets like health insurance.

Not everyone is going to lose their job right away. Rather, certain jobs will be eliminated over time, while others will become semi-automated. And some jobs will always require a human being. The fate of workers will depend on certain employer constraints, labor laws and regulations, and whether there’s a good enough system in place to transition people into new roles or industries. For instance, a McKinsey Global Institute report from November of last year found that 800 million jobs could be lost to worldwide automation by 2030, but only about 6 percent of all jobs are at risk of complete automation. How that process of moving from a human-only job to an AI- or robot-assisted one is developed could mean the difference between a full-blown crisis and a historical paradigm shift.


A paper from US think tank, the Center for Global Development, that was published back in July centered on the potential effects of AI and robotic automation on global labor markets. Researchers found that there is not nearly enough work being done to prepare for the overall automation fallout, and we’re spending too much time debating the general ethics and viability of complete automation in a narrow set of markets. “Questions like profitability, labor regulations, unionization, and corporate-social expectations will be at least as important as technical constraints in determining which jobs get automated,” the paper concluded.

Not everything is all doom and gloom. Part of the philosophy behind the AI Index report is about asking the right questions and making sure that the people making policy, the public, and the leaders of the AI industry have data to make informed decisions. It may be too early to reliably measure the impact of AI on society — the industry is only just getting started — but preparing ourselves for what it all means and how it will affect daily life, work, and public institutions like health care, education, and law enforcement is perhaps just as important as the research and product development itself. Only by investing in both can we avoid the risk of creating technologies that change the world for the worse.

Via The Verge