Backend Engineering Journey

A complete, self-paced Mid-to-Senior Backend Engineering Curriculum built for the AI era. It takes a learner with rusty fundamentals (~10% retention) all the way to designing, building, verifying, and operating production-grade backend systems.

Instead of memorizing syntax that AI can generate in seconds, this repository trains you in engineering judgment: systems thinking, debugging subtle concurrency and database failure modes, reviewing AI-generated code for security and performance traps, and architecting resilient services.

New here? Read `docs/START-HERE.md` first

It is a ten-minute walkthrough of how the repository works, how to run the local browser UI, how to work through the 5-rung exercise ladder, and how to build the incremental capstone project (PulseForge).

How this repository is organized

docs/ — Start here:
docs/START-HERE.md: Onboarding and study workflow.
docs/reading-list.md: Curated chapter-by-chapter mapping to industry-standard books (Fluent Python, Grokking Algorithms, SQL Performance Explained, Cosmic Python, Designing Data-Intensive Applications, HTTP: The Definitive Guide, System Design Interview).
docs/lesson-standard.md: The strict specification every lesson follows, enforced by lesson-lint.
docs/concept-map.toml: Machine-readable concept graph powering the web UI's mastery grid.
docs/glossary.md: Plain-English dictionary of backend, database, networking, and distributed systems terminology.
phase0-foundations/ → phase5-devops-and-observability/ — The six core phases of the curriculum.
pulseforge/ — The incremental flagship portfolio project (PulseForge: a distributed event ingestion, workflow orchestration, and analytics platform) that evolves from v0.1 to v1.0 across phases.
PROGRESS.md — The master curriculum checklist across all phases and milestones.
web_ui/ — Zero-dependency local web server (python3 -m web_ui.server or uv run course-ui) for reading lessons, tracking exercise rungs, and visualizing concept mastery in your browser.
tools/ — Repository verification and curriculum linting (python3 -m tools.lesson_lint).

The Path, Phase by Phase

Phase	Theme & Stack	Capstone Milestone (`pulseforge/`)
Phase 0	Linux, Git, Python Internals & Testing — OS processes, I/O descriptors, Git internals, Python memory/object model, strict typing, `pytest` fixtures & mocking	`v0.1` — CLI log parser, structured event validator & test suite
Phase 1	CS Fundamentals, Algorithms & Software Architecture — Big-O, hash tables, heaps, trees/graphs, sliding window, SOLID, Clean/Hexagonal Architecture, Code Review	`v0.2` — In-memory sliding-window rate limiter, ring buffer & hexagonal domain core
Phase 2	Networking, Protocols & Concurrency — TCP/IP, DNS, TLS, HTTP/1.1–3, WebSockets, SSE, `asyncio` event loop internals, threads vs processes, backpressure	`v0.3` — Async TCP/HTTP ingestion server with SSE live tailing & graceful shutdown
Phase 3	Relational Databases, SQL, FastAPI & Django — PostgreSQL internals, ACID, isolation levels, B-Tree/GIN indexing, `EXPLAIN ANALYZE`, N+1, SQLAlchemy 2.0, FastAPI, Django ORM/Admin, JWT/OAuth2/RBAC	`v0.4` — Multi-tenant FastAPI ingestion API + Django management portal + PostgreSQL storage
Phase 4	Caching, Queues, Workflows, Search & gRPC — Redis caching & Lua locks, RabbitMQ, Celery, Temporal durable workflows, Elasticsearch full-text search, gRPC, Outbox & Idempotency	`v0.5` — Redis deduplication, RabbitMQ/Celery workers, Temporal alert workflows, Elasticsearch & gRPC
Phase 5	Containers, CI/CD, IaC, Observability & System Design — Multi-stage Docker, GitHub Actions, Kubernetes, Ansible, Prometheus, Grafana, Sentry APM, Load Testing & System Design	`v1.0` — Production Docker/K8s stack, Ansible provisioning, Prometheus/Grafana/Sentry observability

Quick Start

cd backend-engineering-journey

# Verify every written lesson: headings, examples, reference solutions, PROGRESS.md, PulseForge
python3 -m tools.lesson_lint

# Work on one lesson (the starter tests fail until you solve the ladder)
python3 -m unittest phase0-foundations/01-linux-and-shell/01-processes-and-streams/test_exercises.py -v

Requires Python 3.11 or newer. The linter and the web UI use only the standard library.

Current status

Written and verified (Phases 0 & 1): 21 lessons, plus PulseForge v0.1.
Planned, not written yet (Phases 2-5): 40 lesson stubs. They hold a title and an outline only, are marked as stubs in the UI, and cannot be ticked off.
python3 -m tools.lesson_lint verifies the written lessons and prints the stub count separately. It never counts a stub as verified.

Reading & Tracking in Your Browser

python3 -m web_ui.server --no-open

Open http://127.0.0.1:5050. The server binds to loopback only and rejects foreign Host and Origin headers.

Sidebar with search filter, full phase/module/lesson tree, and completion counters.
Todo Manager (/todos): add, toggle, prioritize, and track your active study tasks directly in the repo.
Reader for lessons, docs and module indexes, with a click-to-reveal reference solution.
Self-check panel on every written lesson: tick the 5 exercise rungs, rate confidence (1-5), and keep notes.
/progress: overall and per-phase bars, commit streak, and the concept-mastery grid built from docs/concept-map.toml.

Where progress and tasks live

All progress and tasks are stored in the repository, so git log is your study log:

File	Written by the UI when you...
`PROGRESS.md`	mark a lesson complete (ticks its checkbox, updates "Currently working on")
`progress/rungs.json`	save rungs, confidence and notes
`progress/todos.json`	create, toggle, or delete custom study tasks

Commit both with the lesson. See docs/adr/0001-progress-tracking-state.md.