Turing-NLG: A 17-billion-parameter language model by Microsoft
February 10, 2020
Turing Natural Language Generation (T-NLG) is a 17 billion parameter language model by Microsoft that outperforms the state of the art on many downstream NLP tasks. We present a demo of the model, including its freeform generation, question answering, and summarization capabilities, to academics for feedback and research purposes. - This summary was generated by the Turing-NLG language model itself.
ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters
February 10, 2020
The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly advances large model training by improving scale, speed, cost, and usability, unlocking the ability to train 100-billion-parameter models. DeepSpeed is compatible with PyTorch.
Learning web search intent representations from massive web search logs
Published July 21, 2019
Have you ever wondered what happens when you ask a search engine to search for something as seemingly simple as “how do you grill salmon”? Have you found yourself entering multiple searches before arriving at a webpage with a satisfying answer? Perhaps it was only after finally entering “how to cook salmon on a grill” that you found the webpage you wanted in the first place, leaving you wishing search engines simply had the intelligence to understand that when you entered your initial search, your intent was to cook the salmon on a grill.
Microsoft makes it easier to build popular language representation model BERT at large scale
Published July 17, 2019
The open sourcing of our recipe to pre-train BERT (Bidirectional Encoder Representations from Transformers) built by the Bing team, including code that works on Azure Machine Learning, so that customers can unlock the power of training custom versions of BERT-large models using their own data. This will enable developers and data scientists to build their own general-purpose language representation beyond BERT.
As search needs evolve, Microsoft makes AI tools for better search available to researchers and developers
Published May 15, 2019
Only a few years ago, web search was simple. Users typed a few words and waded through pages of results. Today, those same users may instead snap a picture on a phone and drop it into a search box or use an intelligent assistant to ask a question without physically touching a device at all. They may also type a question and expect an actual reply, not a list of pages with likely answers. These tasks challenge traditional search engines, which are based around an inverted index system that relies on keyword matches to produce results.