PulseForge — Production Event & Workflow Engine

PulseForge is the flagship incremental capstone project of the Backend Engineering Journey.

Instead of building 10 disconnected "to-do list" tutorials, you build and evolve a single, production-grade distributed backend platform across all six phases of the curriculum:

• v0.1 (Phase 0): High-throughput CLI structured log stream parser, schema validation, and diagnostics.
• v0.2 (Phase 1): Hexagonal domain core, in-memory sliding-window rate limiter per tenant, ring-buffer event batcher, and DAG alert-rule scheduler.
• v0.3 (Phase 2): Asynchronous HTTP/TCP ingestion server (asyncio), bounded queue backpressure, Server-Sent Events (SSE) live tailing, and zero-drop graceful shutdown.
• v0.4 (Phase 3): PostgreSQL persistence (SQLAlchemy 2.0 ORM & Core), B-Tree/GIN indexing, keyset pagination, FastAPI ingestion APIs, Django operations portal & DRF API, and Argon2 + JWT RBAC authentication.
• v0.5 (Phase 4): Redis caching & distributed Lua locks, RabbitMQ + Celery enrichment workers, Temporal durable incident escalation workflows, Elasticsearch full-text search, and high-performance gRPC RPC ingestion.
• v1.0 (Phase 5): Multi-stage Docker build, GitHub Actions matrix CI/CD, Kubernetes manifests (HPA, probes), Ansible host provisioning, Prometheus/Grafana dashboards, Sentry APM tracing, and locust/k6 load-test benchmarking.

Directory Structure

pulseforge/
├── docs/
│   └── adr/                  # Architectural Decision Records
├── src/
│   └── pulseforge/           # Core Python package
│       ├── __init__.py
│       ├── domain/           # Pure business logic (entities, value objects)
│       ├── adapters/         # I/O drivers (DB, Redis, HTTP, CLI)
│       └── entrypoints/      # FastAPI, Django, Celery, CLI commands
├── tests/
│   ├── unit/                 # Fast, isolated domain tests
│   └── integration/          # Tests with Postgres, Redis, RabbitMQ
├── pyproject.toml
└── README.md

Milestone: v0.1 Specification (Phase 0)

In this milestone, you implement the core streaming engine for PulseForge:

1. Stream gigabytes of JSON events from stdin or files in $O(1)$ memory.
2. Validate against strict dataclass schema (EventBatch).
3. Classify error spikes and emit structured diagnostics to stderr while passing clean events to stdout.
4. 100% unit test coverage using in-memory Fakes and deterministic fixtures.