module_planning:ml:nlp:start
This is an old revision of the document!
Table of Contents
Notes on NLP
Papers / Websites
- The Annotated Transformer (Harvard): http://nlp.seas.harvard.edu/2018/04/03/attention.html
- Attention Is All You Need (original paper): https://arxiv.org/abs/1706.03762
- Distilling the Knowledge in a Neural Network: https://arxiv.org/abs/1503.02531
- Well-Read Students Learn Better: On the Importance of Pre-training Compact Models: https://arxiv.org/abs/1908.08962
- TensorFlow Hub: https://tfhub.dev/
- Google Research, BERT - Smaller Models on Git: https://github.com/google-research/bert
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: https://arxiv.org/pdf/1810.04805.pdf
Websites
Getting meaning from text: self-attention step-by-step video (Romain Futrzynski):
https://peltarion.com/blog/data-science/self-attention-video- Paper Dissected: “Attention is All You Need” Explained:
https://mlexplained.com/2017/12/29/attention-is-all-you-need-explained/ - Speech and Language Processing, Dan Jurafsky and James H. Martin (3rd ed. draft):
https://web.stanford.edu/~jurafsky/slp3/
Videos
- RNN W3L08 - Attention Model (Andrew Ng): https://www.youtube.com/watch?v=FMXUkEbjf9k&feature=youtu.be
- Attention is all you need; Attentional Neural Network Models | Łukasz Kaiser | Masterclass: https://www.youtube.com/watch?v=rBCqOTEfxvg&feature=youtu.be
- Transformer Neural Networks - EXPLAINED! (Attention is all you need) (CodeEmporium): https://www.youtube.com/watch?v=TQQlZhbC5ps&feature=youtu.be
- Attention in Neural Networks (CodeEmporium):
https://www.youtube.com/watch?v=W2rWgXJBZhU&t - [Transformer] Attention Is All You Need | AISC Foundational (Joseph Palermo (Dessa)):
https://www.youtube.com/watch?v=S0KakHcj_rs&feature=youtu.be - Ivan Bilan: Understanding and Applying Self-Attention for NLP | PyData Berlin 2018: https://www.youtube.com/watch?v=OYygPG4d9H0&feature=youtu.be
- Transformer (Attention is all you need)(Minsuk Heo):
https://www.youtube.com/watch?v=z1xs9jdZnuY&feature=youtu.be - Self-attention step-by-step | How to get meaning from text | Peltarion Platform (Romain Futrzynski):
https://www.youtube.com/watch?v=-9vVhYEXeyQ&feature=emb_logo
module_planning/ml/nlp/start.1628849240.txt.gz · Last modified: 2021/08/13 10:07 by rolf