MacMusic  |  PcMusic  |  440 Software  |  440 Forums  |  440TV  |  Zicos
training
Search

Training a Single AI Model Can Emit As Much Carbon As Five Cars In Their Lifetimes

Friday June 7, 2019. 04:02 AM , from Slashdot
In a new paper, researchers at the University of Massachusetts, Amherst, performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 pounds of carbon dioxide equivalent -- nearly five times the lifetime emissions of the average American car (and that includes manufacture of the car itself). MIT Technology Review reports: The researchers looked at four models in the field that have been responsible for the biggest leaps in performance: the Transformer, ELMo, BERT, and GPT-2. They trained each on a single GPU for up to a day to measure its power draw. They then used the number of training hours listed in the model's original papers to calculate the total energy consumed over the complete training process. That number was converted into pounds of carbon dioxide equivalent based on the average energy mix in the US, which closely matches the energy mix used by Amazon's AWS, the largest cloud services provider.

They found that the computational and environmental costs of training grew proportionally to model size and then exploded when additional tuning steps were used to increase the model's final accuracy. In particular, they found that a tuning process known as neural architecture search, which tries to optimize a model by incrementally tweaking a neural network's design through exhaustive trial and error, had extraordinarily high associated costs for little performance benefit. Without it, the most costly model, BERT, had a carbon footprint of roughly 1,400 pounds of carbon dioxide equivalent, close to a round-trip trans-American flight. What's more, the researchers note that the figures should only be considered as baselines. Using a model they'd produced in a previous paper as a case study, the researchers 'found that the process of building and testing a final paper-worthy model required training 4,789 models over a six-month period,' the report states. 'Converted to CO2 equivalent, it emitted more than 78,000 pounds and is likely representative of typical work in the field.'

Read more of this story at Slashdot.
rss.slashdot.org/~r/Slashdot/slashdot/~3/i9aO-X_VpQg/training-a-single-ai-model-can-emit-as-much-car...
News copyright owned by their original publishers | Copyright © 2004 - 2024 Zicos / 440Network
Current Date
Mar, Thu 28 - 10:11 CET