Skip to content

AiBook

devchan64/AiBook

Start
Start
- Introduction
- Table of Contents
Part 1. Introduction to AI and the Landscape
Part 1. Introduction to AI and the Landscape
- Overview
- Module 1. Scope and History of AI
  Module 1. Scope and History of AI
  - Chapter 1. What Is AI
    Chapter 1. What Is AI
    
    P1-1.1 Scope of the Word AI
    
    P1-1.2 Problems AI Deals With
    
    P1-1.3 Relationship Among AI, Machine Learning, Deep Learning, and Generative AI
  - Chapter 2. History of AI and Shifts in Paradigm
    Chapter 2. History of AI and Shifts in Paradigm
    
    P1-2.1 Symbolic AI and Rule-Based Approaches
    
    P1-2.2 Search, Knowledge Representation, and Probabilistic Inference
    
    P1-2.3 The Flow Toward Machine Learning, Deep Learning, and Generative AI
- Module 2. Rules, Models, and Learning
  Module 2. Rules, Models, and Learning
  - Chapter 3. From Rules to Learning
    Chapter 3. From Rules to Learning
    
    P1-3.1 Strengths and Limits of Rule-Based Systems
    
    P1-3.2 What It Means to Learn Patterns from Data
    
    P1-3.3 Rule-Based Approaches and Representation Learning
  - Chapter 4. Turning a Problem into a Model
    Chapter 4. Turning a Problem into a Model
    
    P1-4.1 Becoming Comfortable with the Word Model
    
    P1-4.2 Input, Output, and Data
    
    P1-4.3 Features, Representations, and Parameters
    
    P1-4.4 How Problem Definition Determines the Model
  - Chapter 5. Learning and Inference
    Chapter 5. Learning and Inference
    
    P1-5.1 What Learning Changes
    
    P1-5.2 What Inference Executes
    
    P1-5.3 Distinguishing Inference-Related Terms
- Module 3. Uncertainty and Problem Solving
  Module 3. Uncertainty and Problem Solving
  - Chapter 6. Uncertainty and Probabilistic Judgment
    Chapter 6. Uncertainty and Probabilistic Judgment
    
    P1-6.1 Problems with Incomplete Information and Many Exceptions
    
    P1-6.2 Distinguishing Probability, Uncertainty, and Stochastic Processes
    
    P1-6.3 Where Probabilistic Judgment Appears in AI
  - Chapter 7. Search and Heuristics
    Chapter 7. Search and Heuristics
    
    P1-7.1 Search Space and Computational Limits
    
    P1-7.2 What Heuristics Reduce
    
    P1-7.3 Difference Between Heuristics and Probabilistic Models
    
    P1-7.4 Supplementary Learning: From Route Finding to Autonomous Driving Path Planning
  - Chapter 8. Basic Distinctions Among Learning Types
    Chapter 8. Basic Distinctions Among Learning Types
    
    P1-8.1 Supervised Learning: Input and Label
    
    P1-8.2 Unsupervised Learning: Structure and Representation
    
    P1-8.3 Reinforcement Learning: Action and Reward
- Module 4. Transition to Deep Learning and Generative AI
  Module 4. Transition to Deep Learning and Generative AI
  - Chapter 9. Spread of the Deep-Learning Paradigm
    Chapter 9. Spread of the Deep-Learning Paradigm
    
    P1-9.1 Image Recognition and Representation Learning
    
    P1-9.2 Cases of Object Detection and Speech Generation
    
    P1-9.3 Direct Lineage and Surrounding Evidence for LLMs
  - Chapter 10. Introduction to Generative AI
    Chapter 10. Introduction to Generative AI
    
    P1-10.1 Difference Among Classification, Prediction, and Generation
    
    P1-10.2 Intuition for Next-Output Generation
    
    P1-10.3 Quality and Risk of Generated Outputs
  - Chapter 11. Where Did LLMs Come From?
    Chapter 11. Where Did LLMs Come From?
    
    P1-11.1 Statistical Language Models and Embeddings
    
    P1-11.2 RNN, Seq2Seq, and Attention
    
    P1-11.3 Transformers and Pretrained LLMs
- Module 5. Core Elements of the LLM User Experience
  Module 5. Core Elements of the LLM User Experience
  - Chapter 12. Prompt Basics and the Feel of Using LLMs
    Chapter 12. Prompt Basics and the Feel of Using LLMs
    
    P1-12.1 What a Prompt Specifies
    
    P1-12.2 Instruction, Context, and Example
    
    P1-12.3 Limits of Prompts and First Validation Criteria
  - Chapter 13. Embeddings and Vector Search
    Chapter 13. Embeddings and Vector Search
    
    P1-13.1 What It Means to Represent Text as a Vector
    
    P1-13.2 Intuition for Similarity Search
    
    P1-13.3 The Problem Awareness That Leads to RAG
    
    P1-13.4 Intuition for Implementing Vector Search
  - Chapter 14. AI Service Architecture and Execution Environments
    Chapter 14. AI Service Architecture and Execution Environments
    
    P1-14.1 Model, Application, Data, and Tool
    
    P1-14.2 The Place of RAG and Tool Use
    
    P1-14.3 The Place of the Word Agent
    
    P1-14.4 The Conceptual Place of MCP
    
    P1-14.5 Harnesses and Evaluation Execution Environments
    
    P1-14.6 Overview of Real-World Constraints on AI Services
- Module 6. Social Issues and Application
  Module 6. Social Issues and Application
  - Chapter 15. AI Ethics, Copyright, and Security
    Chapter 15. AI Ethics, Copyright, and Security
    
    P1-15.1 Bias, Safety, and Accountability
    
    P1-15.2 Copyright and Training Data
    
    P1-15.3 Security and Privacy
  - Chapter 16. Cases of Practical Application
    Chapter 16. Cases of Practical Application
    
    P1-16.1 Personal Learning and Documentation
    
    P1-16.2 Work Automation and Search
    
    P1-16.3 How to Verify Understanding Through a Project
  - Chapter 17. The Future of AI
    Chapter 17. The Future of AI
    
    P1-17.1 What Kinds of Evidence Should Be Used for Forecasts
    
    P1-17.2 Reading News, Columns, and Reports
    
    P1-17.3 Distinguishing Prediction from a Working Hypothesis
- Summary
Part 2. Foundational Recovery
Part 2. Foundational Recovery
- Overview
- Module 1. Reading Mathematics Again as a Computational Language
  Module 1. Reading Mathematics Again as a Computational Language
  - Chapter 1. Why Mathematics Is Needed in AI
    Chapter 1. Why Mathematics Is Needed in AI
    
    P2-1.1 What Mathematics Does in AI Computation
    
    P2-1.2 Where Formulas, Code, and Data Meet
  - Chapter 2. Reading Mathematical Notation Again
    Chapter 2. Reading Mathematical Notation Again
    
    P2-2.1 Reading Variables, Functions, and Expressions Again
    
    P2-2.2 Sigma and Repeated Computation
    
    P2-2.3 Limit and the Intuition of Change
    
    P2-2.4 Why Log and Exp Keep Appearing
- Module 2. Core Mathematics for Reading Model Computation
  Module 2. Core Mathematics for Reading Model Computation
  - Chapter 3. What Linear Algebra Models
    Chapter 3. What Linear Algebra Models
    
    P2-3.1 Scalars, Vectors, and Matrices
    
    P2-3.2 Vector Space and the Intuition of Position
    
    P2-3.3 What Matrix Multiplication Reuses
    
    P2-3.4 Dot Product, Norm, Distance, and Similarity
    
    P2-3.5 Python Execution Environments: Colab and Local PC
    
    P2-3.6 Checking Linear Algebra Through NumPy
  - Chapter 4. What Derivatives Are a Tool for Finding
    Chapter 4. What Derivatives Are a Tool for Finding
    
    P2-4.1 Looking Back at How Derivatives Were First Learned
    
    P2-4.2 Rate of Change and Slope
    
    P2-4.3 Derivative and Gradient
    
    P2-4.4 Why Learning Needs Derivatives
    
    P2-4.5 Supplementary Learning: From High-School Derivatives to Multivariable Differentiation
    
    P2-4.6 Composite Functions and the Chain Rule
  - Chapter 5. Why Probability and Statistics Are Needed
    Chapter 5. Why Probability and Statistics Are Needed
    
    P2-5.1 How Probability Expresses Uncertainty Numerically
    
    P2-5.2 Distribution, Mean, and Variance
    
    P2-5.3 Sample, Estimation, and Error
    
    P2-5.4 Checking Probability and Statistics with Small Data
    
    P2-5.5 Supplementary Learning: How to Read Standard Deviation, Correlation, and Confidence Intervals for the First Time
  - Chapter 6. What Kind of Problem Optimization Solves
    Chapter 6. What Kind of Problem Optimization Solves
    
    P2-6.1 What Optimization Searches For
    
    P2-6.2 Loss Functions and Objective Functions
    
    P2-6.3 Intuition for Gradient Descent
- Module 3. Runtime Environment and Python Fundamentals
  Module 3. Runtime Environment and Python Fundamentals
  - Chapter 7. Setting Up the Learning Environment
    Chapter 7. Setting Up the Learning Environment
    
    P2-7.1 Local Environment and Runtime
    
    P2-7.2 Terminal, Shell, and Working Directory
    
    P2-7.3 Python Interpreter and Script
    
    P2-7.4 Virtual Environment and Package
    
    P2-7.5 Dependency and Reproducibility
    
    P2-7.6 Supplemental Learning: Opening Terminals by Operating System
    
    P2-7.7 Supplementary Learning: When Python Installation Becomes Necessary
    
    P2-7.8 Supplemental Learning: Reading Shell Execution Flow
    
    P2-7.9 Supplemental Learning: Checking Local Python Environment Problems
  - Chapter 8. Review of Core Python Syntax
    Chapter 8. Review of Core Python Syntax
    
    P2-8.1 Values, Variables, and Types
    
    P2-8.2 Lists: Ordered Groups of Values
    
    P2-8.3 Dictionaries: Structures That Find Values by Key
    
    P2-8.4 Loops: Processing an Iterable One Item at a Time
    
    P2-8.5 Functions and Small Reuse
    
    P2-8.6 Supplementary Learning: Meeting Classes and Objects for the First Time
    
    P2-8.7 Supplemental Learning: Distinguishing References and Copies
  - Chapter 9. Intuition for Data Structures and Graphs
    Chapter 9. Intuition for Data Structures and Graphs
    
    P2-9.1 Why Data Structures Are Needed
    
    P2-9.2 Intuition for Arrays, Tables, Trees, and Graphs
    
    P2-9.3 How Graphs Represent Relations
    
    P2-9.4 Supplementary Learning: How to Read Traditional Data Structures for the First Time
  - Chapter 10. Jupyter and Colab Notebooks
    Chapter 10. Jupyter and Colab Notebooks
    
    P2-10.1 Why Notebooks Are Useful for Learning
    
    P2-10.2 Difference Among Jupyter, Colab, and Local Execution
    
    P2-10.3 Organizing Notebooks as Re-runnable Records
- Module 4. Data Computation and Visualization Tools
  Module 4. Data Computation and Visualization Tools
  - Chapter 11. Reusing Vectors and Matrices with NumPy
    Chapter 11. Reusing Vectors and Matrices with NumPy
    
    P2-11.1 Building Vectors and Matrices with NumPy Arrays
    
    P2-11.2 Indexing, Slicing, and Axis
    
    P2-11.3 Broadcasting and Vectorization
    
    P2-11.4 Supplementary Learning: How to Read Shape and Shared Underlying Objects Together in NumPy
  - Chapter 12. Handling Data Structure with Pandas
    Chapter 12. Handling Data Structure with Pandas
    
    P2-12.1 What a Pandas DataFrame Represents
    
    P2-12.2 Selection, Filtering, and Aggregation
    
    P2-12.3 Intuition for Preparing a Dataset for Learning
  - Chapter 13. Visualizing Concepts with Matplotlib
    Chapter 13. Visualizing Concepts with Matplotlib
    
    P2-13.1 What a Plot Reveals
    
    P2-13.2 Checking Basic Charts and the Shape of Formulas
    
    P2-13.3 Comparing and Saving Several Plots
- Module 5. Reproducibility and Integrated Review
  Module 5. Reproducibility and Integrated Review
  - Chapter 14. Git and Document Management
    Chapter 14. Git and Document Management
    
    P2-14.1 Git as a Tool for Managing Change History
    
    P2-14.2 Branches, Commits, and Document Reproducibility
  - Chapter 15. Reading Mathematics Again Through Software
    Chapter 15. Reading Mathematics Again Through Software
    
    P2-15.1 A Small Procedure for Turning Formulas into Code
    
    P2-15.2 Checkpoint Before Moving to Part 3
- Summary
Part 3. Data Modeling
Part 3. Data Modeling
- Overview
- Module 1. Roles and Problem Scenes
  Module 1. Roles and Problem Scenes
  - Chapter 1. What Data Modeling Tries to Achieve and How It Proceeds
    Chapter 1. What Data Modeling Tries to Achieve and How It Proceeds
    
    P3-1.1 What Data Modeling Tries to Achieve
    
    P3-1.2 In What Order Data Modeling Proceeds
    
    P3-1.3 How to Write a Data Question so the Problem Structure Appears Before the Model Name
  - Chapter 2. Reading Storage Structure and Design Axes First
    Chapter 2. Reading Storage Structure and Design Axes First
    
    P3-2.1 Why Stored Records Are Not Yet a Dataset by Themselves
    
    P3-2.2 What Structure Exists Inside a Dataset Candidate
    
    P3-2.3 What to Write Down First When Receiving a New Table
  - Chapter 3. A Dataset Is Not Given from the Beginning
    Chapter 3. A Dataset Is Not Given from the Beginning
    
    P3-3.1 Why Raw Data Should Not Be Read Immediately as a Learning Problem
    
    P3-3.2 How a Dataset Is Redesigned to Fit the Question
    
    P3-3.3 Which Columns Should Be Sketched First When Turning a Question into a First Table Draft
- Module 2. Samples and Table Structure
  Module 2. Samples and Table Structure
  - Chapter 4. How to Determine the Sample Unit
    Chapter 4. How to Determine the Sample Unit
    
    P3-4.1 How to Determine One Comparable Sample
    
    P3-4.2 What Else Becomes Unstable When the Sample Unit Wobbles
    
    P3-4.3 How One Row, One Sample, and One Recent Window Differ
    
    P3-4.4 What Signals That the Sample Unit Has Been Chosen Poorly
    
    P3-4.5 How Representative the Collected Samples Are of the Overall Operating Situation
  - Chapter 5. How to Turn Raw Logs into Summary Tables
    Chapter 5. How to Turn Raw Logs into Summary Tables
    
    P3-5.1 How to Turn Raw Logs into Comparable Tables
    
    P3-5.2 How Summary Tables Preserve Patterns Beyond the Mean
    
    P3-5.3 Why Raw Time Series Cannot Yet Be Called Learning Inputs Directly
    
    P3-5.4 Where to Cut an Input Window and How to Match Its Length
    
    P3-5.5 How to Handle Samples with Missing Values or Empty Segments
    
    P3-5.6 Overlapping Input Windows and Sample Counts
    
    P3-5.7 Rules for Folding Multiple Follow-Up Events
- Module 3. Features and Comparison Structure
  Module 3. Features and Comparison Structure
  - Chapter 6. How to Design Features and Intermediate Representations
    Chapter 6. How to Design Features and Intermediate Representations
    
    P3-6.1 What Kinds of Features Preserve the Structure to Be Compared
    
    P3-6.2 What Intermediate Representation Can Be Added When Features Alone Are Not Enough
    
    P3-6.3 How Human-Designed Features Connect to Later Learned Representations
    
    P3-6.4 Why Not Every Column in a Summary Table Is a Feature
    
    P3-6.5 How to Read and Keep Features Together When Their Units and Scales Differ
    
    P3-6.6 Same Column Name, Different Feature
  - Chapter 7. How to Read Baseline Comparison Structure
    Chapter 7. How to Read Baseline Comparison Structure
    
    P3-7.1 What the Preserved Structure Must Be Compared Against for Change to Become Visible
    
    P3-7.2 How to Turn a Comparison Table into a Human Review Sentence
    
    P3-7.3 Why Baseline Comparison and a Baseline Model Are Different Yet Both Needed
    
    P3-7.4 What Window and Conditions Should Be Used to Build a Baseline
    
    P3-7.5 Whether a Baseline Should Stay Fixed or Be Refreshed as a Recent-Normal Baseline
- Module 4. Boundaries of Interpretation
  Module 4. Boundaries of Interpretation
  - Chapter 8. How to Read Sample Size and Repeatability
    Chapter 8. How to Read Sample Size and Repeatability
    
    P3-8.1 What Controls the Strength of Interpretation
    
    P3-8.2 How Far to State a Change Signal and Where to Stop Before Claiming Cause
    
    P3-8.3 In What Order and Vocabulary Conservative Interpretation Sentences Should Be Written
    
    P3-8.4 Conservative Interpretation and Operational Columns
    
    P3-8.5 How to Combine Several Comparison Columns into One Candidate Review Priority
    
    P3-8.6 Confirmed Labels Left Only on Some Cases
    
    P3-8.7 Data Interpretation Changed by Operational Intervention
- Module 5. Problem Type and Connection to Later Parts
  Module 5. Problem Type and Connection to Later Parts
  - Chapter 9. Which Problems Should Be Turned into Prediction Problems
    Chapter 9. Which Problems Should Be Turned into Prediction Problems
    
    P3-9.1 How Far the Current Problem Should Be Elevated
    
    P3-9.2 Why Some Problems Should Remain Comparison Reports to the End
    
    P3-9.3 Differences Among Three Operational Tables
    
    P3-9.4 How Review Results Become Target-Label Candidates Out of Operational Notes
    
    P3-9.5 What Keeps the Same Event Traceable Across Several Outputs
    
    P3-9.6 Checking Label Consistency
    
    P3-9.7 What Must Be Closed First Before Sending Inputs and Results to Later Parts
    
    P3-9.8 What One Prediction Actually Decides and Why Score and Policy Are Different
    
    P3-9.9 How to Distinguish the Real Goal from a Proxy Target
    
    P3-9.10 Delayed Label Confirmation and Incomplete Negatives
    
    P3-9.11 Target Candidates and Changing Criteria
    
    P3-9.12 Target Names and Error Costs
    
    P3-9.13 Problem Boundaries to Hand Off to Part 4
- Summary
Part 4. Machine Learning
Part 4. Machine Learning
- Overview
- Module 1. The Place of Machine Learning
  Module 1. The Place of Machine Learning
  - Chapter 1. Distinguishing AI, Machine Learning, and Deep Learning
    Chapter 1. Distinguishing AI, Machine Learning, and Deep Learning
    
    P4-1.1 Relationship Among AI, Machine Learning, and Deep Learning
    
    P4-1.2 What It Means to Learn Rules from Data
  - Chapter 2. Supervised, Unsupervised, and Reinforcement Learning
    Chapter 2. Supervised, Unsupervised, and Reinforcement Learning
    
    P4-2.1 Supervised Learning
    
    P4-2.2 Unsupervised Learning
    
    P4-2.3 Reinforcement Learning
  - Chapter 3. Why Heuristics Are Needed
    Chapter 3. Why Heuristics Are Needed
    
    P4-3.1 Why Heuristics Are Needed
    
    P4-3.2 Heuristics and Model Selection
- Module 2. Data and Evaluation
  Module 2. Data and Evaluation
  - Chapter 4. Data Splitting and Validation
    Chapter 4. Data Splitting and Validation
    
    P4-4.1 Training Data and Evaluation Data
    
    P4-4.2 Validation and Test
  - Chapter 5. Overfitting and Generalization
    Chapter 5. Overfitting and Generalization
    
    P4-5.1 Overfitting and Underfitting
    
    P4-5.2 Generalization
  - Chapter 6. Evaluation Metrics
    Chapter 6. Evaluation Metrics
    
    P4-6.1 The Role of a Metric
    
    P4-6.2 Evaluation Standards by Problem Type
    
    P4-6.3 Supplementary Learning: How to Read Metrics in Site Reliability Engineering
    
    P4-6.4 Supplementary Learning: A Question Map for Evaluation Metrics
- Module 3. Practical Heuristics
  Module 3. Practical Heuristics
  - Chapter 7. Heuristics for Feature Selection and Preprocessing
    Chapter 7. Heuristics for Feature Selection and Preprocessing
    
    P4-7.1 Feature Selection
    
    P4-7.2 Preprocessing
    
    P4-7.3 Supplementary Learning: Separating Preprocessing Input Problems
    
    P4-7.4 Supplementary Learning: Distinguishing Feature Selection Methods
  - Chapter 8. Heuristics for Model Selection
    Chapter 8. Heuristics for Model Selection
    
    P4-8.1 Model Selection
    
    P4-8.2 Baseline Models
    
    P4-8.3 Supplementary Learning: How to Set Up a Baseline for the First Time by Problem Type
  - Chapter 9. Hyperparameter Tuning
    Chapter 9. Hyperparameter Tuning
    
    P4-9.1 Hyperparameters
    
    P4-9.2 Tuning and Validation Cost
    
    P4-9.3 Supplementary Learning: Tool Map After Model Selection
- Module 4. Basic Algorithms
  Module 4. Basic Algorithms
  - Chapter 10. Linear Regression
    Chapter 10. Linear Regression
    
    P4-10.1 Intuition for Linear Regression
    
    P4-10.2 Evaluation and Limits of Linear Regression
    
    P4-10.3 Supplementary Learning: How to Read Regression Diagnostics for the First Time
  - Chapter 11. Logistic Regression
    Chapter 11. Logistic Regression
    
    P4-11.1 Intuition for Logistic Regression
    
    P4-11.2 Decision Boundaries
    
    P4-11.3 Supplementary Learning: How to Read Log-Odds and MLE for the First Time
    
    P4-11.4 Supplementary Learning: How to Read Multiclass Logistic Regression
    
    P4-11.5 Supplementary Learning: How to Read Solvers and Regularization for the First Time
  - Chapter 12. k-NN
    Chapter 12. k-NN
    
    P4-12.1 Intuition for k-NN
    
    P4-12.2 Distance and Scale
    
    P4-12.3 What Should Be Checked First When Using k-NN
  - Chapter 13. SVM
    Chapter 13. SVM
    
    P4-13.1 Intuition for SVM
    
    P4-13.2 Introductory Meaning of the Kernel
- Module 5. Trees and Ensembles
  Module 5. Trees and Ensembles
  - Chapter 14. Decision Trees
    Chapter 14. Decision Trees
    
    P4-14.1 Decision Trees
    
    P4-14.2 Overfitting in Trees
  - Chapter 15. Random Forests
    Chapter 15. Random Forests
    
    P4-15.1 Random Forests
    
    P4-15.2 Feature Importance
    
    P4-15.3 OOB and Random-Forest Inspection
    
    P4-15.4 Supplementary Learning: Comparing Extra Trees and Random Forest
  - Chapter 16. Gradient Boosting
    Chapter 16. Gradient Boosting
    
    P4-16.1 Gradient Boosting
    
    P4-16.2 Performance and Risk of Boosting
    
    P4-16.3 Supplementary Learning: Seeing Boosting Libraries Together from the First Implementation and Operations View
- Module 6. Finding Structure
  Module 6. Finding Structure
  - Chapter 17. Clustering
    Chapter 17. Clustering
    
    P4-17.1 Intuition for Clustering
    
    P4-17.2 Caution in Interpreting Clusters
    
    P4-17.3 Supplementary Learning: Hierarchical and Spectral Clustering
    
    P4-17.4 Supplementary Learning: How to Connect Clustering to Semi-Supervised Learning for the First Time
  - Chapter 18. Dimensionality Reduction
    Chapter 18. Dimensionality Reduction
    
    P4-18.1 Dimensionality Reduction
    
    P4-18.2 Visualization and Information Loss
  - Chapter 19. Reinforcement-Learning Algorithms
    Chapter 19. Reinforcement-Learning Algorithms
    
    P4-19.1 Value-Based Reinforcement Learning
    
    P4-19.2 Policy-Based Reinforcement Learning
    
    P4-19.3 Cautions in Applying Reinforcement Learning
    
    P4-19.4 Supplementary Learning: A Map of Later Reinforcement-Learning Branches
    
    P4-19.5 Supplementary Learning: First Reading of Reinforcement-Learning Formula Names
    
    P4-19.6 Supplementary Learning: First Reading of Policy Gradient
- Summary
Part 5. Deep Learning
Part 5. Deep Learning
- Overview
- Module 1. Starting Point of Neural Networks
  Module 1. Starting Point of Neural Networks
  - Chapter 1. The Perceptron
    Chapter 1. The Perceptron
    
    P5-1.1 Intuition for the Perceptron
    
    P5-1.2 Linear Combination and Activation
  - Chapter 2. Multilayer Neural Networks
    Chapter 2. Multilayer Neural Networks
    
    P5-2.1 Multilayer Neural Networks
    
    P5-2.2 Hidden Layers and Representation
- Module 2. Output, Loss, and Backpropagation
  Module 2. Output, Loss, and Backpropagation
  - Chapter 3. Activation Functions
    Chapter 3. Activation Functions
    
    P5-3.1 Activation Functions
    
    P5-3.2 Sigmoid
    
    P5-3.3 Tanh
    
    P5-3.4 ReLU
    
    P5-3.5 Formula Comparison of Representative Activation Functions
    
    P5-3.6 Choosing the Output Layer and Activation
  - Chapter 4. Loss Functions
    Chapter 4. Loss Functions
    
    P5-4.1 Loss Functions
    
    P5-4.2 Loss by Problem Type
  - Chapter 5. Backpropagation
    Chapter 5. Backpropagation
    
    P5-5.1 Intuition for Backpropagation
    
    P5-5.2 Computation Graphs
- Module 3. Learning Loops and Stabilization
  Module 3. Learning Loops and Stabilization
  - Chapter 6. Separating Learning and Inference
    Chapter 6. Separating Learning and Inference
    
    P5-6.1 The Four Steps of the Training Loop
    
    P5-6.2 Learning Steps, Batches, and Epochs
    
    P5-6.3 Learning and Inference
    
    P5-6.4 Training Mode and Evaluation Mode
  - Chapter 7. Optimizers
    Chapter 7. Optimizers
    
    P5-7.1 The Role of the Optimizer
    
    P5-7.2 Learning Rate and Update Step Size
    
    P5-7.3 Intuition for Adam: Adaptive Updates
    
    P5-7.4 Supplementary Reading: Distinguishing Adaptive Optimization Claims
  - Chapter 8. Regularization and Dropout
    Chapter 8. Regularization and Dropout
    
    P5-8.1 Regularization
    
    P5-8.2 Dropout
    
    P5-8.3 Supplementary Learning: Stabilizing Deep Computation
    
    P5-8.4 Supplementary Reading: How A Large Initialization Scale Shakes The Computation Range
- Module 4. Computation Environment and Representation Learning
  Module 4. Computation Environment and Representation Learning
  - Chapter 9. GPU and Parallel Processing
    Chapter 9. GPU and Parallel Processing
    
    P5-9.1 GPU and Parallel Processing
    
    P5-9.2 Batch and Tensor Computation
  - Chapter 10. Representation Learning
    Chapter 10. Representation Learning
    
    P5-10.1 Representation Learning
    
    P5-10.2 Representations in Deeper Layers
- Module 5. Spatial Structure and CNNs
  Module 5. Spatial Structure and CNNs
  - Chapter 11. CNNs
    Chapter 11. CNNs
    
    P5-11.1 Intuition for CNNs
    
    P5-11.2 Convolution and Pooling
    
    P5-11.3 Supplementary Reading: Comparing CNNs and Vision Transformers
- Module 6. Sequential Structures and the RNN Family
  Module 6. Sequential Structures and the RNN Family
  - Chapter 12. RNN, LSTM, and GRU
    Chapter 12. RNN, LSTM, and GRU
    
    P5-12.1 Why RNN, LSTM, and GRU Are Needed
    
    P5-12.2 Long-Term Dependency
- Module 7. From Attention to Transformers
  Module 7. From Attention to Transformers
  - Chapter 13. Attention
    Chapter 13. Attention
    
    P5-13.1 Intuition for Attention
    
    P5-13.2 The Flow Toward Self-Attention
    
    P5-13.3 Supplementary Reading: QKV and Multi-Head Attention
  - Chapter 14. Transformers
    Chapter 14. Transformers
    
    P5-14.1 Why Attention Alone Does Not Close the Transformer
    
    P5-14.2 What Does Each Of The Four Transformer Block Components Do?
    
    P5-14.3 Two Devices That Stabilize Deep Repetition
    
    P5-14.4 RNN State Passing and Transformer Parallel Computation
    
    P5-14.5 How Do Sequential State And Direct Re-Reference Split In Long Context?
    
    P5-14.6 Supplementary Learning: Position-Wise Representation Processing
    
    P5-14.7 Supplementary Learning: A Path That Preserves the Original Representation
    
    P5-14.8 Supplementary Learning: Normalization That Aligns the Value Baseline
- Module 8. Generative Models and Sampling
  Module 8. Generative Models and Sampling
  - Chapter 15. Intuition for Generative Models
    Chapter 15. Intuition for Generative Models
    
    P5-15.1 What Changes When We Read Generative AI Through Deep Learning
    
    P5-15.2 Candidate Distributions in Generative Models
    
    P5-15.3 How Sampling Pulls Actual Outputs From Candidate Distributions
- Summary
Part 6. LLMs and Generative AI
Part 6. LLMs and Generative AI
- Overview
- Summary
Part 7. Projects
Part 7. Projects
- Overview
- Summary
Appendix
Appendix
- Concept Glossary

Concept Glossary: X¶

This page lists English glossary entries for this letter directly. Terms are written as entries on this page instead of linking out to a separate glossary page.