Skip to main content

In The News

Turing-NLG: A 17-billion-parameter language model by Microsoft

February 10, 2020

Turing Natural Language Generation (T-NLG) is a 17 billion parameter language model by Microsoft that outperforms the state of the art on many downstream NLP tasks. We present a demo of the model, including its freeform generation, question answering, and summarization capabilities, to academics for feedback and research purposes. - This summary was generated by the Turing-NLG language model itself.

ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters

February 10, 2020

The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly advances large model training by improving scale, speed, cost, and usability, unlocking the ability to train 100-billion-parameter models. DeepSpeed is compatible with PyTorch.