Navigation
Search
|
Researchers Claim New Technique Slashes AI Energy Use By 95%
Wednesday October 9, 2024. 05:30 AM , from Slashdot
The algorithm's impact extends beyond energy savings. L-Mul outperforms current 8-bit standards in some cases, achieving higher precision while using significantly less bit-level computation. Tests across natural language processing, vision tasks, and symbolic reasoning showed an average performance drop of just 0.07% -- a negligible tradeoff for the potential energy savings. Transformer-based models, the backbone of large language models like GPT, could benefit greatly from L-Mul. The algorithm seamlessly integrates into the attention mechanism, a computationally intensive part of these models. Tests on popular models such as Llama, Mistral, and Gemma even revealed some accuracy gain on certain vision tasks. At an operational level, L-Mul's advantages become even clearer. The research shows that multiplying two float8 numbers (the way AI models would operate today) requires 325 operations, while L-Mul uses only 157 -- less than half. 'To summarize the error and complexity analysis, L-Mul is both more efficient and more accurate than fp8 multiplication,' the study concludes. But nothing is perfect and this technique has a major achilles heel: It requires a special type of hardware, so the current hardware isn't optimized to take full advantage of it. Plans for specialized hardware that natively supports L-Mul calculations may be already in motion. 'To unlock the full potential of our proposed method, we will implement the L-Mul and L-Matmul kernel algorithms on hardware level and develop programming APIs for high-level model design,' the researchers say. Read more of this story at Slashdot.
https://science.slashdot.org/story/24/10/08/2035247/researchers-claim-new-technique-slashes-ai-energ...
Related News |
25 sources
Current Date
Oct, Wed 16 - 13:24 CEST
|