GPT

This project is an project to store notes and codes during the study of the Generative Pre-trained Transformer (GPT) language model.

Contents:

• Model Design
  • Parameters
  • Workflow Chart
  • Reference
• Input Encoding
  • Unicode Encoding
  • Tokenization
  • Token Embedding (WTE)
  • Positional Encoding (WPE)
  • Encoding Combination
  • Reference
• Neural Networks
  • Layer Normalization
  • Feed-Forward
  • Attention Layer
  • Reference
• Training and Predicting
  • Pre-Training
  • Meta-Learning
  • Examples
• Review
  • Paper Review
  • Critics
  • Reference
• Reference
  • Papers
  • Implementation
  • APIs
  • Fine-Tuning
  • Other Sources
  • Reference

Back to NLP.