AI Engineering Mastery
Curriculum overview

Learn AI the way professionals actually use it.

This roadmap starts with basic vocabulary and grows into the topics that matter for real AI jobs: paper reading, Transformer basics, LLM behavior, retrieval systems, agents, data pipelines, training, fine-tuning, and infrastructure. It is intentionally selective, so beginners can focus on useful professional skills instead of unnecessary detours.

Topic modules

Start with foundations, then move across the AI system stack.

Basics of Transformers

Foundation

Learn tokens, vectors, embeddings, attention, positional information, and why Transformer architecture changed sequence modeling.

LLM engineeringML researchAI product work
  • Vocabulary
  • Transformer intuition
  • Attention math
  • Paper reading

LLM Foundations

Models

Understand next-token prediction, context windows, decoding, prompting, instruction tuning, model limits, and evaluation basics.

AI applicationsModel evaluationPrompt systems
  • Token prediction
  • Prompt behavior
  • Context limits
  • Evaluation basics

RAG Systems

Applications

Build systems that retrieve outside knowledge, rank evidence, cite sources, and reduce hallucinations with measurable grounding.

SearchSupport assistantsEnterprise knowledge
  • Embeddings
  • Chunking
  • Retrieval
  • Grounded answers

Agents and Tool Use

Workflows

Design model workflows that plan, call tools, inspect results, recover from errors, and stop safely.

AutomationResearch workflowsOperations tools
  • Tool schemas
  • Agent loops
  • Tracing
  • Safety boundaries

Data Pipelines

Data

Prepare, clean, version, and evaluate the data that powers retrieval, training, fine-tuning, and production feedback loops.

RAGFine-tuningEvaluation
  • Data cleaning
  • Versioning
  • Labeling
  • Quality checks

Training Runs

Training

Understand datasets, loss, optimization, checkpoints, validation, regression tests, and how model behavior changes during training.

ML engineeringFine-tuningModel research
  • Loss curves
  • Checkpoints
  • Validation
  • Regression tests

Fine-Tuning

Adaptation

Learn when to adapt a model with examples, how to format training data, and how to compare the result against prompting or RAG.

Domain assistantsStyle controlTask specialization
  • Training data
  • Eval sets
  • Regression risk
  • Deployment checks

AI Systems

Infrastructure

Operate AI features with serving, latency, caching, monitoring, tracing, cost controls, and incident response.

Production appsPlatform teamsReliability work
  • Serving design
  • Caching
  • Monitoring
  • Cost model
Project track

Small runnable projects that prove the concepts.

Use these as the bridge between reading lessons and building professional AI systems. Each project includes starter files, a run command, eval checks, failure modes, and primary sources.

First LLM Eval Project

First build

Create a tiny evaluation harness that sends the same task examples to two prompts or models, scores the outputs, and prints a pass/fail report.

npm run eval:first-llm
  • All starter examples run without changing the evaluation code, and the harness is ready to expand toward 10 or more cases.
  • The report shows baseline score, candidate score, and every failed case.

First RAG App

First build

Build a local question-answering app over a small document folder, retrieve source chunks, and answer only when evidence is available.

npm run rag:dev
  • Questions with known answers cite the right source document.
  • Out-of-scope questions return a refusal instead of an invented answer.

First Tool-Using Agent

First build

Build a step-limited agent with one read-only tool, strict argument validation, structured observations, and a trace of every decision.

npm run agent:first
  • The agent calls the tool when the answer requires external information.
  • Malformed or out-of-scope tool arguments are rejected before execution.

First Fine-Tuning Dataset and Eval Plan

Builder

Prepare a small supervised fine-tuning dataset plan, split it correctly, define quality checks, and decide whether fine-tuning is justified.

npm run dataset:check
  • Every example has an input, ideal output, category, and review status.
  • No duplicate or near-duplicate examples cross train/test boundaries.

First Production AI Observability and Eval Gate

Production gate

Add tracing, metrics, and a pre-release evaluation gate to one AI workflow so production changes are measurable and reversible.

npm run release:gate
  • Every eval run records model, prompt, data, and tool versions.
  • The gate fails when quality drops, cost exceeds budget, or unsupported answers increase.
First deep module

Attention Is All You Need

The Transformer lesson teaches tokens, embeddings, self-attention, multi-head attention, positional encoding, encoder-decoder flow, and why the architecture became central to modern AI.

  • Read the Transformer paper section by section and explain the authors' argument.
  • Define tokens, embeddings, hidden states, attention scores, softmax, residual connections, masking, dropout, and label smoothing from first principles.
  • Derive the scaled dot-product attention equation and explain every symbol.
  • Trace how the encoder, decoder, multi-head attention, feed-forward layers, positional encodings, and output softmax move information through the architecture.
Open lesson Continue to LLM Foundations Continue to RAG Systems Continue to Agents and Tool Use Continue to Data Pipelines Continue to Training Runs Continue to Fine-Tuning Continue to AI Systems