The Computational Limits of Deep Learning

Ben Lorica

The Data Exchange Podcast: Neil Thompson on the computational demands, economic costs, and environmental impact of AI.

Subscribe: Apple • Android • Spotify • Stitcher • Google • RSS.

In this episode of the Data Exchange I speak with Neil Thompson, Research Scientist at Computer Science and Artificial Intelligence Lab (CSAIL) and the Initiative on the Digital Economy, both at MIT.  I wanted Neil on the podcast to discuss a recent paper he co-wrote entitled “The Computational Limits of Deep Learning” (summary version here). This paper provides estimates of the amount of computation, economic costs, and environmental impact that come with increasingly large and more accurate deep learning models.

Download the 2020 NLP Survey Report and learn how companies are using and implementing natural language technologies.


The authors present estimates for computational, economic, energy metrics needed to achieve three set of error rates across some standard benchmarks used by deep learning researchers.  The results suggest that the path we are on is currently unsustainable. As an example, let’s look at some cost estimates (I previously created charts based on tables from this paper):



I asked Neil about the many hardware startups that offer accelerators for deep learning. While he acknowledges that specialized hardware can lead to amazing speedups, he believes these new hardware initiatives alone won’t suffice:

    The problem with hardware specialization is that it’s an example of decreasing marginal returns. … When you first specialize you take the thing that gets you the most advantage, then you take the next one, then the next one. Incrementally they become less and less impactful on your overall outcome. So I am very skeptical that these improvements will lead us to the kind of massive increases that we will need.
But the paper is not all gloom and doom. We discussed probable future scenarios, including the possibility of new, more efficient machine learning models that draw on domain knowledge.

Subscribe to our Newsletter:
We also publish a popular newsletter where we share highlights from recent episodes, trends in AI / machine learning / data, and a collection of recommendations.

Related content:
[Image: MareNostrum 4 supercomputer at Barcelona Supercomputing Center from Wikimedia]