AI Notes Directory
Comprehensive guides, mathematical derivations, interactive visualizers, and implementation references across deep neural networks and generative modeling.
AI Notes & Deep Learning
A centralized technical reference and interactive playground covering foundational and modern neural architectures—from convolutional spatial hierarchies to multi-head self-attention, rotary embeddings, and latent diffusion pipelines.
Interactive Learning: Every major architecture includes interactive simulations to help you visualize tensor flows and build mathematical intuition before formalizing your code.
🕹️ Interactive Visualizer
Attention is the engine of modern AI. Watch each token query every other token and pull weighted information from their values — the core operation behind GPT, BERT, and modern frontier models.
Scaled Dot-Product Attention
Querying context for: "The"
Waiting to process next sequence step...
Key Insight: Each token produces a Query, Key, and Value vector. The dot product of and determines how much it attends to each other token before pulling from .
Deep Neural Networks (DNN)
Foundational neural building blocks, backpropagation dynamics, sequence processing, and embedding spaces.
CNN Guide
Convolutional layers, pooling strategies, receptive fields, stride arithmetic, and spatial hierarchy learning.
Optimizers & Dynamics
SGD, momentum, RMSprop, Adam, and AdamW optimizer update rules with loss landscape visualizations.
Dropout & Regularization
Stochastic zeroing, ensemble interpretation, inverted dropout scaling, and overfitting prevention.
Recurrent Networks (RNN, LSTM, GRU)
Unrolled computation graphs, gating mechanisms, cell states, and vanishing gradient solutions.
Positional Embeddings & RoPE
Sinusoidal positions, learned encodings, rotary position embeddings (RoPE), and relative distance preservation.
Tokenization & Vocabularies
Byte-Pair Encoding (BPE), WordPiece, SentencePiece, subword splitting, and special token handling.
Vector Embeddings
Word2Vec, token lookup matrices, vector spaces, cosine similarity, and high-dimensional projection.
Generative Modeling
Probabilistic density modeling, latent spaces, score-based diffusion, and adversarial convergence.
Diffusion Models
Forward noise injection, reverse score estimation, DDPM formulations, and classifier-free guidance.
Variational Autoencoders (VAE)
Evidence Lower Bound (ELBO), reparameterization trick, Gaussian KL divergence, and latent sampling.
Generative Adversarial Networks (GAN)
Minimax games, generator vs discriminator dynamics, mode collapse mitigation, and Wasserstein loss.
Autoregressive Models
Causal masking, next-token prediction distributions, temperature scaling, top-k, and nucleus (top-p) sampling.
Normalizing Flows
Invertible transformations, change of variables formula, exact log-likelihood tracking, and coupling layers.