====== Notes on NLP ======

===== Papers / Websites =====

  * The Annotated Transformer (Harvard): http://nlp.seas.harvard.edu/2018/04/03/attention.html

  * Attention Is All You Need (original paper): https://arxiv.org/abs/1706.03762
  * Distilling the Knowledge in a Neural Network: https://arxiv.org/abs/1503.02531
  * Well-Read Students Learn Better: On the Importance of Pre-training Compact Models: https://arxiv.org/abs/1908.08962

  * TensorFlow Hub: https://tfhub.dev/
  * Google Research, BERT - Smaller Models on Git: https://github.com/google-research/bert
  * BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: https://arxiv.org/pdf/1810.04805.pdf

===== Websites =====

  * :!: Getting meaning from text: self-attention step-by-step video (Romain Futrzynski): \\ https://peltarion.com/blog/data-science/self-attention-video
  * :!: The Illustrated Transformer (Jay Alammar): \\ http://jalammar.github.io/illustrated-transformer/
  * :!: Paper Dissected: “Attention is All You Need” Explained: \\ https://mlexplained.com/2017/12/29/attention-is-all-you-need-explained/
  * Speech and Language Processing, Dan Jurafsky and James H. Martin (3rd ed. draft): \\ https://web.stanford.edu/~jurafsky/slp3/
===== Videos =====

  * RNN W3L08 - Attention Model (Andrew Ng): https://www.youtube.com/watch?v=FMXUkEbjf9k&feature=youtu.be
  * Attention is all you need; Attentional Neural Network Models | Łukasz Kaiser | Masterclass: https://www.youtube.com/watch?v=rBCqOTEfxvg&feature=youtu.be
  * Transformer Neural Networks - EXPLAINED! (Attention is all you need) (CodeEmporium): https://www.youtube.com/watch?v=TQQlZhbC5ps&feature=youtu.be
  * Attention in Neural Networks (CodeEmporium): \\ https://www.youtube.com/watch?v=W2rWgXJBZhU&t
  * :!: [Transformer] Attention Is All You Need | AISC Foundational (Joseph Palermo (Dessa)): \\ https://www.youtube.com/watch?v=S0KakHcj_rs&feature=youtu.be
  * Ivan Bilan: Understanding and Applying Self-Attention for NLP | PyData Berlin 2018: https://www.youtube.com/watch?v=OYygPG4d9H0&feature=youtu.be
  * :!: Transformer (Attention is all you need)(Minsuk Heo): \\ https://www.youtube.com/watch?v=z1xs9jdZnuY&feature=youtu.be
  * :!: Self-attention step-by-step | How to get meaning from text | Peltarion Platform (Romain Futrzynski): \\ https://www.youtube.com/watch?v=-9vVhYEXeyQ&feature=emb_logo

===== Tutorials =====

  * [[https://www.kaggle.com/pavansanagapati/knowledge-graph-nlp-tutorial-bert-spacy-nltk|Knowledge Graph & NLP Tutorial-(BERT,spaCy,NLTK)]] on Kaggle by Pavan Anagapati