Demystifying deep learning, with interactive intuition.
Every concept is deconstructed from foundational mathematics into clean, interactive visual simulations and reproducible PyTorch code. Built to make the math click before you formalise it.
Neural Architecture Hub is a compendium of first-principles deep learning designed for engineers and researchers. No skipped steps, no black boxes. Every mathematical derivation is grounded with interactive visual intuition and clean, reproducible tensor code.
Conceptual First-Principles
Derive why architectures work—from linear algebra, optimization, and probability theory.
Interactive Visual Labs
Manipulate attention heads, convolution kernels, and latent trajectories in real time.
Zero-Jargon Explanations
Clear geometric explanations replacing opaque terminology with tangible insights.
Engineered for deep comprehension.
Combine live spatial simulations, rigorous mathematical proofs, and clean code to build an unshakable mental model of deep learning.
Interactive Spatial Math Playground
Compute Conv2D receptive fields and output dimensions live
Mathematical Rigor
Interactive breakdowns of governing equations
Click any term in the Attention equation below to inspect its mathematical and geometric necessity:
Prevents dot-product magnitudes from exploding into saturated softmax regions with zero gradients.
Deep Neural Networks Track
Foundational representations, kernels & optimization
Generative Models Track
Probabilistic modeling, diffusion & synthesis
Reverse SDEs & noise schedules
ELBO derivation & reparameterization
Minimax game & Wasserstein divergence
Causal masking & sequence likelihood
Exact log-likelihood via invertible maps
import torch
import torch.nn as nn
import math
class ScaledDotProductAttention(nn.Module):
def __init__(self, d_k: int):
super().__init__()
self.scale = 1.0 / math.sqrt(d_k)
def forward(self, q, k, v, mask=None):
# [batch, heads, seq_len, seq_len]
scores = torch.matmul(q, k.transpose(-2, -1)) * self.scale
if mask is not None:
scores = scores.masked_fill(mask == 0, -1e9)
attn_weights = torch.softmax(scores, dim=-1)
return torch.matmul(attn_weights, v), attn_weightsEvery architecture, deconstructed.
Switch between foundational neural paradigms to see the mathematical mechanics, visual representations, and key theoretical insights.
Dynamic routing where tokens actively construct their context
Unlike fixed convolution kernels, self-attention computes dynamic, data-dependent weights between all token pairs in a sequence—yielding global receptive field in a single layer.
- Global receptive field in a single $\mathcal{O}(1)$ depth step
- Permutation equivariance preserved until positional encodings are injected
- Multi-head splitting projects representations into orthogonal semantic subspaces
First-Principles Rigor.
No hand-waving or skipped derivations. Every formula is proved from linear algebra, multivariable calculus, and probability theory.
Interactive Intuition.
Explore convolution kernels, attention heads, and diffusion trajectories through real-time interactive widgets before formalizing the math.
Production PyTorch.
Clean, self-contained reference implementations showing exactly how tensors are shaped, indexed, and backpropagated in real codebases.
Master Neural Architectures,
from the ground up.
Jump directly into full-length derivations, manipulate interactive tensors, or bookmark our quick-reference formulas for your next research project.