What you need to know
- Megatron-Turing NLG 530B is a language model.
- Microsoft and NVIDIA teamed up to train it and make it the largest, most powerful AI language model.
- The companies admit their work is nowhere near finished as issues such as systemic bias remain embedded in the model's core.
Megatron-Turing NLG 530B (MT-NLG), the AI language model succeeding the Turing NLG 17B and Megatron-LM, has been described by NVIDIA and Microsoft as the "world's largest and most powerful generative language model."
To give a very brief primer on what language models are: They're tools used to anticipate word choice. This sort of tech can help the services you use to identify spam in an email inbox or figure out what word was spoken in a video, all so that you don't have to. As you can imagine, crafting these AI language models is no small feat given the complexity of the task at hand. That's why NVIDIA and Microsoft are proud to present Megatron-Turing NLG 530B. The companies say that MT-NLG has unmatched accuracy in the following areas:
- Completion prediction
- Reading comprehension
- Commonsense reasoning
- Natural language inferences
- Word sense disambiguation
With that being said, it's not perfect. The language model isn't devoid of bias, which is derived from the data it uses to learn how to perform its tasks. Similarly, the model is also influenced by toxicity, according to NVIDIA's report on the creation.
If you can put those pitfalls aside, it's still worth noting that MT-NLG remains "the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters," as stated by the NVIDIA report. Think about Megatron the next time you let an email write itself for you. And don't mistake it with Pegatron.
