Behold the Megatron: Microsoft and Nvidia build massive language processor

MT-NLG is a beast that fed on over 4,000 GPUs
Nvidia and Microsoft announced their largest monolithic transformer language model to date, an AI model with a whopping 530 billion parameters they developed together, named the Megatron-Turing Natural Language Generation model.…