ML Foundations Hub

• Studies kernel density estimation for high-dimensional distribution ρ(x)
• Traditional approaches focus on limit of a large sample regime

Introducing a kernel-based two-sample test for comparing probability distributions up to group actions, yielding invariant kernels for robust testing.

Explore practical exercises in the ARPM framework for machine learning:

• Probabilistic Framework equips tools for the Probabilistic Ecosystem,...

Essential lecture notes on high-dimensional covariance estimation:

• Introduction & Survey: 1 hour, 12 minutes overview
• Course Focus: Feature...

Quick Recap

• Shared a PDF of the Mathematics For Machine Learning book, which notes difficulties learners face in understanding algorithms like...

• Geometric core: Video details maximum margin classifier, support vectors, and margin derivation 2 / |w| in SVM primal formulation.
• Hard/soft...

• Perfect separability causes logistic regression to have no maximum likelihood solution.
• MLE fails without bounds, lacking a defined optimum.
• L2 regularization restores a well-defined classifier and decision boundary.

Key insights from Statistical Rethinking Lecture A10:

• Explores hidden confounds and sensitivity analysis in Bayesian regression workflows
• 1-hour...

Key highlights from these rigorous MIT lecture notes:

• Discrete random walks: Xt as sum of iid zero-mean Yi, CLT gives Xt/√t → N(0, Var(Y)) for large...

Paper derives generalization bounds directly by plugging stability bounds into Lemma 3.4, plus optimization error bounds via Eq. (7.21). Covers bias, stability, generalization in stochastic gradient methods.

This well-respected text is designed for the first course in probability and statistics taken by students majoring in engineering and computing sciences.

Fundamentals of Statistical Signal Processing Volume 1 equips engineers, scientists, and enthusiasts with a deep understanding of how to model, analyze, and design systems.

Model Selection and Multimodel Inference provides a practical information-theoretic approach, highlighting how data conclusions inform frameworks and yield actionable strategies for statistical modeling.

Kernel methods shine as flexible nonparametric tools for density and regression estimation, making minimal assumptions about data generation unlike...

Quick Recap

• Repost of eLife paper on the influence of sample size and covariate distributions using a simple regression framework.
• PDF of The...

• Simple regression framework enables clear scalability analysis
• Excels for understanding but limits multi-site generalization
• Highlights gaps vs. hierarchical models in complex settings
• Focuses on sample size and covariate distributions impacts

• Searle’s classical approach: Focuses on parameter estimation in linear models using OLS and MLE.
• Hastie et al.’s learning view: Defines...

• Classical statistical learning theory quantifies generalization performance via upper bounds on generalization error (test-training gap)
• Conjugate...

• InterXshift method introduces semiparametric discovery and estimation of interactions
• Provides nonparametric definition of statistical...

Key theoretical lectures from CUSO winter school on adaptivity in ML:

• Lecture 1: Generalization bounds, the SGD way
• Lecture 2: Adaptivity of kernel methods to smoothness
• Lecture 3: Adaptivity of neural networks