Videos · GenAI
GenAI
From attention math to production agents, evals, safety and strategy: what you should be able to explain from first principles.
151 topics in 17 sections · 3 with hand-picked videos
01Transformer fundamentals
- L1Tokenization & BPE8 videos
- L1Token embeddings & the residual stream8 videos
- L1Next-token prediction & cross-entropySearch ↗
- L1Self-attention (Q, K, V)Search ↗
- L1Causal (masked) self-attentionSearch ↗
- L1Multi-head attentionSearch ↗
- L2Positional encoding & RoPESearch ↗
- L2LayerNorm, RMSNorm & pre-normSearch ↗
- L2Feed-forward blocks & SwiGLUSearch ↗
- L2Decoder-only vs encoder-decoderSearch ↗
02Modern LLM architectures
- L1MQA, GQA & MLASearch ↗
- L2FlashAttentionSearch ↗
- L1Mixture of Experts & routingSearch ↗
- L2Long-context extension (YaRN)Search ↗
- L3Sparse & linear attentionSearch ↗
- L3State space models & hybrids (Mamba)Search ↗
- L3Multi-token predictionSearch ↗
- L3Diffusion language modelsSearch ↗
- L2Case study: DeepSeek-V3Search ↗
03Pretraining & scaling
04Distributed training & hardware
- L2GPU architecture for deep learningSearch ↗
- L3Roofline & arithmetic intensitySearch ↗
- L2Mixed precision: FP16, BF16, FP8Search ↗
- L3Activation checkpointingSearch ↗
- L3Collectives: all-reduce & NCCLSearch ↗
- L2Data parallelism, ZeRO & FSDPSearch ↗
- L2Tensor & pipeline parallelismSearch ↗
- L3Sequence & context parallelismSearch ↗
- L3MFU & training throughputSearch ↗
- L3Checkpointing & fault tolerance at scaleSearch ↗
05Fine-tuning & preference tuning
- L2In-context learning & few-shot promptingSearch ↗
- L1Supervised fine-tuning & instruction dataSearch ↗
- L1LoRA & QLoRA8 videos
- L2Catastrophic forgettingSearch ↗
- L1Reward modelsSearch ↗
- L1RLHF with PPOSearch ↗
- L1DPO & preference optimizationSearch ↗
- L3Constitutional AI & RLAIFSearch ↗
- L3Model mergingSearch ↗
- L1Prompting vs RAG vs fine-tuningSearch ↗
06RL & reasoning models
- L1Chain-of-thought & self-consistencySearch ↗
- L1Reasoning models & test-time computeSearch ↗
- L2Best-of-N, verifiers & searchSearch ↗
- L3Process vs outcome reward modelsSearch ↗
- L2GRPO & RL with verifiable rewardsSearch ↗
- L3Case study: DeepSeek-R1 recipeSearch ↗
- L2Reward hackingSearch ↗
- L3RL environments & verifiersSearch ↗
- L3Multi-turn agentic RLSearch ↗
07Multimodal & generative media
- L2Vision Transformers (ViT)Search ↗
- L2CLIP & contrastive learningSearch ↗
- L2Vision-language models (VLMs)Search ↗
- L3Diffusion modelsSearch ↗
- L3Flow matchingSearch ↗
- L3Diffusion transformers (DiT)Search ↗
- L3Video generation modelsSearch ↗
- L3Speech models (ASR & TTS)Search ↗
- L3Real-time speech-to-speechSearch ↗
- L3World modelsSearch ↗
08Inference fundamentals
- L1Sampling: temperature, top-k & top-pSearch ↗
- L1Structured outputs & constrained decodingSearch ↗
- L1KV cacheSearch ↗
- L1Prefill vs decode & memory-bound inferenceSearch ↗
- L1GPU memory math for LLMsSearch ↗
- L1TTFT, TPOT & throughput trade-offsSearch ↗
- L1Quantization (GPTQ, AWQ, FP8)Search ↗
- L2KV cache compression & evictionSearch ↗
09Serving at scale
- L1Continuous batchingSearch ↗
- L1PagedAttention & vLLMSearch ↗
- L1Prefix caching & RadixAttentionSearch ↗
- L2Speculative decoding (EAGLE, Medusa)Search ↗
- L2Disaggregated prefill/decode servingSearch ↗
- L3MoE serving & expert parallelismSearch ↗
- L2vLLM vs SGLang vs TensorRT-LLMSearch ↗
- L3Autoscaling & cold startsSearch ↗
- L3On-device & edge LLMsSearch ↗
10Retrieval & RAG
- L1Embedding models & similaritySearch ↗
- L3Fine-tuning embedding modelsSearch ↗
- L1Chunking strategiesSearch ↗
- L1Vector indexes (HNSW, IVF, PQ)Search ↗
- L1Hybrid search (BM25 + dense)Search ↗
- L1Rerankers & cross-encodersSearch ↗
- L3Late interaction (ColBERT)Search ↗
- L2Query rewriting & HyDESearch ↗
- L2Contextual retrievalSearch ↗
- L3Multimodal RAG & ColPaliSearch ↗
- L3GraphRAGSearch ↗
- L2Agentic RAGSearch ↗
- L1Evaluating RAGSearch ↗
- L2Long context vs RAGSearch ↗
11Agent foundations
- L1Workflows vs agentsSearch ↗
- L1Function & tool callingSearch ↗
- L1ReAct & agent loopsSearch ↗
- L2Planning & reflection patternsSearch ↗
- L2Designing tools for agentsSearch ↗
- L1Model Context Protocol (MCP)Search ↗
- L3Agent-to-agent protocol (A2A)Search ↗
- L2Agent memorySearch ↗
- L1Context engineeringSearch ↗
- L2Multi-agent & orchestrator-workerSearch ↗
12Agent harnesses & long-horizon
13Evaluation
- L2LLM benchmarksSearch ↗
- L3Benchmark contaminationSearch ↗
- L1Building an eval suiteSearch ↗
- L1LLM-as-a-judge & its biasesSearch ↗
- L3Human preference evals & Arena EloSearch ↗
- L1Hallucination: causes & detectionSearch ↗
- L2Eval statistics: error bars & significanceSearch ↗
- L1Evaluating agentsSearch ↗
- L2Online evals & A/B testing LLM featuresSearch ↗
14Security & guardrails
15Alignment & interpretability
- L2SycophancySearch ↗
- L3Scheming & alignment fakingSearch ↗
- L3CoT monitoring & faithfulnessSearch ↗
- L3Scalable oversightSearch ↗
- L3Mechanistic interpretabilitySearch ↗
- L3Sparse autoencoders & featuresSearch ↗
- L3Circuit tracing & attribution graphsSearch ↗
- L3Probes & steering vectorsSearch ↗
- L3Dangerous-capability evals & RSPsSearch ↗