Architectural Brief: Peak Demand Planning Engine
Commercial energy planning breaks when the recommendation engine treats physics as a pricing detail. This build puts tenant scope, forecast uncertainty, and asset feasibility ahead of savings math, then stores enough snapshot data for an operator to replay the same decision later.
System Topology
Infrastructure Decisions
- Compute: A single Nim binary runs web or worker mode through
APP_MODE. I chose this over separate framework services because the PRD requires one self-hosted deployment path, anddocker-compose.prod.ymlcan run the same image aswebandworkerwithout duplicating release artifacts. - Data Layer: PostgreSQL 16 with TimescaleDB stores tenants, sites, assets, tariffs, forecasts, plans, feedback, jobs, outbox events, and interval readings. I chose it over a separate time-series service because the schema needs tenant-scoped foreign keys between operational records and meter history, not a detached metrics store.
- Replay Layer: DuckDB handles replay scans through a configured
DUCKDB_CLI_PATH. I chose DuckDB over a second OLAP database because replay is a local backtest workload, and Batch 016 showed the runtime path had to be declared rather than hidden inside the source file. - Core Language and Web: Nim 2.x with Mummy HTTP and server-rendered templates. I chose Nim over a heavier web stack because the codebase is mostly deterministic planning, import validation, and job execution. The web layer stays thin while the planner remains readable enough to test at function level.
- Jobs: Postgres-backed job rows and advisory-lock workers replace a mandatory queue service. I chose this over Redis-first jobs because replay, outbox dispatch, peak-risk scans, and expiry checks already depend on the same tenant-scoped database state.
- Integrations: Notification Hub, Workflow Engine, and IoT sensor input sit behind
*_ENABLED=falsedefaults. I chose optional adapters over hard dependencies because the core product must run from CSV, JSON, and local forecasts with no ecosystem services available.
Constraints That Shaped the Design
- Tenant boundary: Every data-bearing table has
tenant_id, and cross-table relationships such as assets to sites and plans to tariffs use tenant-aware constraints. Batch 005 exposed why plainsite_idforeign keys were not enough: Tenant A rows could reference Tenant B sites until composite constraints closed the hole. - Input: The product accepts interval readings, tariffs, assets, forecasts, replay windows, and operator decisions through API and UI paths. Imports persist valid rows while keeping line-level errors in the import report.
- Output: A plan stores selected actions, rejected actions, savings estimates, risk summary, confidence band, and frozen input snapshots. Replay outputs compare planner, no-action, and threshold policies against the same historical input snapshot.
- Scale Handled: The evidence pack verifies a 35,000-row deterministic import fixture, a 24-hour planning contract with 96 intervals, and a 12-month replay fixture. Those numbers are fixture evidence, not a live customer performance promise.
- Forecast uncertainty: Forecasts store p10, p50, and p90. The planner records which band drove the action, and tests prove p90 can produce a higher savings estimate than p50 for the same asset set.
- Operational failure: Optional integration failure cannot roll back core planning. Outbox rows move through queued, sent, retrying, and dead-letter states, while local UI banners report disabled or degraded integrations.
Decision Log
| Decision | Alternative Rejected | Why |
|---|---|---|
| Tenant-aware composite foreign keys | tenant_id columns plus plain foreign keys |
Batch 005 proved plain site references allowed cross-tenant child rows. The database now rejects the relationship, not just the service layer. |
| Physical feasibility before economics | Score every candidate first, reject later | Battery SOC, cycle limits, comfort limits, telemetry gaps, and tariff matrix exclusions must be visible as rejected actions before expected savings mean anything. |
| Stale forecasts persist failed plans | Let planner fall back to do_nothing |
Batch 012 found stale forecasts could become ready no-op plans. Invalid input now fails, while real infeasible windows can still produce do-nothing. |
| Explicit tariff and asset matrices | Implicit branching inside planner code | The PRD required every allowed and rejected cell to have a reachable symbol, test name, and explanation. This makes enum expansion auditable. |
| DuckDB CLI path in config | Hardcoded duckdb executable |
Batch 016 showed replay could pass source checks while failing in a container. DUCKDB_CLI_PATH makes the runtime dependency visible. |
| Outbox dispatch from persisted payload | Re-query mutable plan state during retries | Retry-safe events need immutable payloads. The outbox record stores the rendered event so a later retry cannot drift from the accepted plan. |
| Server-rendered HTMX UI | React SPA | Operators need dense tables, forms, and audit views. HTMX keeps route behavior close to the API contracts and keeps first-load JavaScript small. |
Data Contracts
The core schema has fourteen operational tables: tenants, users, sites, assets, interval readings, tariffs, imports, forecasts, curtailment plans, operator feedback, replay runs, job runs, outbox events, and integration settings. interval_readings is the Timescale hypertable. curtailment_plans.input_snapshot, operator_feedback.original_snapshot, and replay_runs.input_snapshot are the audit spine.
The important contract is not the table count. It is the direction of authority. Imports produce qualified interval rows. Forecasts produce confidence bands. Plans consume a chosen band plus tariff and asset constraints. Feedback stores the original recommendation before changing status. Replay uses frozen historical inputs rather than the latest tariff or asset settings.
Scaling Limits
The current design is intentionally self-hostable. One PostgreSQL instance carries jobs, events, time-series rows, tenants, and audit state. That is the right boundary for a single commercial operator or a small multi-tenant deployment.
The first pressure point will be replay concurrency. A 12-month replay can run through DuckDB locally, but many parallel replay jobs would need a worker pool policy, stored progress, and possibly partitioned interval reads. The second pressure point is multi-site optimization, which the PRD excludes from MVP. The schema can hold multiple sites, but the planner reasons about one site per plan.
The architectural line stays clear: make physics and auditability true before adding portfolio-scale optimization.