Statistical Learning

Contents:

• Mean Squared Error Decomposition
  • TL;DR
  • Derivation
  • Comments
• Bayes Theorem
  • Joint, Marginal and Conditional Distributions
  • Likelihood
  • Bayes Theorem
• Bayes Example: Biased Coin
  • Maximum Likelihood Estimate
  • Bayes Estimate
• Bayes Example: Classification
• Multivariate Gaussian Distribution
  • Parameter Estimation
  • Properties of the Covariance Matrix
• LDA: Classification
  • Probability Density Function
  • Decision Boundary
• LDA: Dimensionality Reduction
  • Within / Between Class Variance
  • Optimal Projection
  • Lagrangian Function
• Quadratic Discriminant Analysis
  • Probability Density Function
  • Classification
• Linear Regression: Formula Derivation
  • Functional Representation
  • Least Squares Solution
• Linear Regression: Signal-to-Noise Ratio (SNR)
  • Signal Energy
  • Signal-to-Noise Ratio
• Linear Regression: Ordinary Least Squares
  • TL;DR
  • Hypothesis
  • Zero Mean Error and Unbiased Estimator
  • Homoscedasticity and BLUE
  • Normally Distributed Error and MLE
• Linear Regression: OLS Formula
  • Originally Derived Formula
  • Factored Form
  • Slope Weighted Average Form
  • Correlation Coefficient Form
  • Expectation and Variance
• Linear Regression: Ridge Regression
  • Projection Matrix
  • Shrinkage Factor
  • Interpretation
• SVM: Kernel Method
  • Empirical Risk
  • Derivation of Solution
  • Appendix: Proof of Riesz Representation Theorem
• Gradient in Backpropagation
  • TL;DR
  • Fully Connected Layer
  • Gradient
• Gradient Descent
  • Empirical Risk Minimization
  • Stochastic vs. Batch
• Policy Gradient
  • Trust Region Policy Optimization (TRPO)
  • Proximal Policy Optimization (PPO)
• Appendix A: Setup R Environment
  • Download and Install R
  • RStudio IDE
  • Reference
• Appendix B: Setup Jupyter Lab Environment
  • Prerequisites
  • Install Jupyter Lab and Kernels
  • Known Issues

---

Back to Mathematics.