Towards Green AI: How to Make Deep Learning Models More Efficient in Production | by Leonie Monigatti | Aug, 2023


From Academia to Industry: Finding the best trade-off between predictive performance and inference runtime for sustainability in Machine Learning practices

A man sitting at a bonfire made from GPUs making s’mores that look like the planet Earth.
Making s’mEARTHs at the GPU bonfire (Image hand-drawn by the author)

This article was originally published on Kaggle as an entry to the “2023 Kaggle AI Report” competition on July 5th, 2023, in which it won 1st place in the category “Kaggle competitions”. As it reviews Kaggle competition writeups, it is a special edition of “The Kaggle Blueprints” series.

“I think we’re at the end of the era where it’s going to be these, like, giant, giant models. […] We’ll make them better in other ways.”, said Sam Altman, CEO of OpenAI, shortly after their release of GPT-4. This statement surprised many, as GPT-4 is estimated to be ten times larger (1.76 trillion parameters) than its predecessor, GPT-3 (175 billion parameters).

“I think we’re at the end of the era where it’s going to be these, like, giant, giant models. […] We’ll make them better in other ways.” — Sam Altman

In 2019, Strubell et al. [1] estimated that training a natural language processing (NLP) pipeline, including tuning and experimentation, produces around 35 tonnes of carbon dioxide equivalent, more than twice the average U.S. citizen’s annual consumption.

Let’s put it more into perspective: It was reported that information technologies produced 3.7% of global CO2 emissions in 2019. That’s more than global aviation (1.9%) and shipping (1.7%) combined!

Deep Learning models have been pushing state-of-the-art performances across different fields. These performance gains are often the results of larger models. But building bigger models requires more computations in both the training and the inference stage. And more computations require bigger hardware and more electricity and thus emit more CO2 and lead to a bigger carbon footprint, which is bad for the environment.



Source link

Leave a Comment