Episode 11
Moin Nadeem (MIT): The extraordinary future of natural language models
Moin Nadeem is a masters student at MIT, where he studies natural language generation. His research interests broadly include natural language processing, information retrieval, and software systems for machine learning.
Learn more about Moin:
https://twitter.com/moinnadeem
Want to level-up your skills in machine learning and software engineering? Join the ML Engineered Newsletter: http://bit.ly/mle-newsletter
Comments? Questions? Submit them here: http://bit.ly/mle-survey
Follow Charlie on Twitter: https://twitter.com/CharlieYouAI
Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/
Subscribe to ML Engineered: https://mlengineered.com/listen
Timestamps:
01:35 Follow Charlie on Twitter (https://twitter.com/CharlieYouAI)
03:10 How Moin got started in computer science
05:50 Using ML to identify depression on Twitter in high school
11:00 Building a system to track phone locations on MIT’s campus
14:35 Specializing in NLP
17:20 Building an end-to-end fact-checking system (https://www.aclweb.org/anthology/N19-4014/)
25:15 Predicting statement stance with neural multi-task learning (https://www.aclweb.org/anthology/D19-6603/)
27:20 Is feature engineering in NLP dead?
29:40 Reconciling language models with existing knowledge graphs
35:20 How advances in AI hardware will affect NLP research (crazy!)
47:25 Moin’s research into sampling algorithms for natural language generation (https://arxiv.org/abs/2009.07243)
57:10 Under-rated areas of ML research
01:00:10 How research works at MIT CSAIL
01:04:35 How Moin keeps up in such a fast-moving field
01:11:30 Starting the MIT Machine Intelligence Community
01:16:30 Rapid Fire Questions
Links:
FAKTA: An Automatic End-to-End Fact Checking System
StereoSet: Measuring stereotypical bias in pretrained language models
Neural Multi-Task Learning for Stance Prediction
Rich Sutton - The Bitter Lesson
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
Strategies for Pre-training Graph Neural Networks
Transformers For Image Recognition at Scale
