AI technologies, including large language models (LLMs), have rapidly become integral to everyday life. However, the computational power required to support these technologies is driven by data centers, which consume massive amounts of energy. In Germany alone, data centers consumed around 16 billion kilowatt-hours (kWh) of electricity in 2020—approximately 1% of the nation’s total energy consumption. By 2025, this figure is expected to rise to 22 billion kWh, highlighting the growing energy demands of AI systems.
As AI applications become increasingly complex, their energy needs are expected to grow, especially in the realm of training neural networks, which demand significant computational resources. In response to this challenge, researchers have pioneered a revolutionary training method that is 100 times faster than traditional approaches, all while maintaining the same level of accuracy. This innovation has the potential to substantially reduce the energy required for AI training, offering a promising solution to the industry’s sustainability concerns.
Neural networks, the backbone of AI tasks like image recognition and natural language processing, are designed to mimic the human brain. These networks consist of interconnected artificial neurons that process information by assigning weighted values to input signals. Once the signal surpasses a certain threshold, it is passed along to the next layer of nodes.
Training these networks is a resource-intensive task. Initially, the parameters are assigned randomly, often following a normal distribution. Through numerous iterations, these values are adjusted to enhance prediction accuracy. Given the vast number of calculations involved, training neural networks consumes a considerable amount of energy.
Felix Dietrich, a professor specializing in Physics-enhanced Machine Learning, and his research team have developed an innovative training method to address this issue. Rather than relying on iterative processes to adjust the parameters between nodes, their approach uses probabilities. This probabilistic method focuses on strategically utilizing values at key points in the training data, particularly where rapid and significant changes are occurring.
The goal of their study is to use this new approach to uncover energy-efficient dynamic systems from data. These systems evolve over time according to specific rules and are commonly found in fields such as climate modeling and financial markets.
“Our method allows us to determine the necessary parameters using significantly less computational power, making the training of neural networks faster and more energy-efficient,” says Felix Dietrich. “Furthermore, we’ve found that the accuracy of our method is on par with that of traditional iterative training methods.”
By dramatically reducing the energy consumption required for training while maintaining high accuracy, this breakthrough opens up new possibilities for the future of AI—helping the field advance sustainably and efficiently.
By Impact Lab
