Architectural Brief: Cloud Commitment Portfolio Optimizer
A cloud commitment decision can remain open for days while prices, forecasts, and account data continue to move. This architecture freezes the evidence first, then produces a recommendation, replay, approval packet, and report from that fixed record.
System Topology
The API owns authenticated commands and server-rendered views. PostgreSQL holds tenant state, policies, run status, recommendations, approvals, and audit records. The object store holds the larger immutable inputs and outputs: normalized source objects, forecast distributions, frontier artifacts, replay results, and rendered reports.
The worker is a separate process. It polls PostgreSQL for forecast, optimizer, approval-expiry, backtest, and ecosystem work, then writes bounded results back to the same stores. Redis supports shared admission controls and queue health, but a Redis payload is not the source of truth for a financial run. The domain workers claim their database rows, which leaves recovery tied to durable state.
Infrastructure Decisions
-
Compute: Two Node.js process types, one Fastify API and one polling worker, run from the same image. I chose this over running financial jobs inside request handlers because forecasts, replays, report rendering, and adapter retries need their own lifecycle. A failed cycle is logged without taking down the HTTP process, and a stopped worker leaves queued rows visible for recovery.
-
Data layer: PostgreSQL 16 owns the 25-migration relational model, while a guarded object store owns immutable artifacts. I chose this split over putting every forecast point and rendered file into relational rows because transactions matter for approvals and tenant boundaries, while large append-only artifacts need cheap byte storage and stable object references.
-
Coordination: Redis 7 is used for shared rate admission, idempotency reservations, and queue readiness. I chose database row claims as the authority for domain work instead of treating Redis as the financial ledger. A queue message can be retried or lost; an
optimizer_runsrow still records its frozen input URI, policy, price versions, seed, and terminal state. -
Core languages: TypeScript 5 handles the API, services, repositories, and workers. Zig 0.14.1 handles the pinned economic kernel contract. I chose this split over running the economic formulas through TypeScript floating-point numbers because money enters as canonical decimal strings and must stay in integer cents. The Zig CLI rejects JSON numbers and pins five golden cases to one evaluator; TypeScript remains responsible for tenant state, HTTP, and database orchestration.
-
Presentation: HTMX and server-rendered HTML were chosen over a client-side application. The release audit records no dashboard JavaScript and uses semantic tables as the accessible alternative to frontier charts. That keeps approval review usable at the 390-pixel target without shipping a second state model to the browser.
-
Deployment: Docker Compose runs PostgreSQL, Redis, a one-shot migration gate, the API, and the worker. I chose an explicit self-hosted stack over a managed cloud dependency because billing exports and recommendation evidence can remain under the operator's control. The tradeoff is clear: backup and restore must include both PostgreSQL and the object-storage volume.
Operational Contracts
An optimizer request does not tell a worker to "use the latest data." The API resolves a forecast run, scenario, policy, provider, instrument, price table versions, and deterministic seed. It writes optimizer-run-input-snapshot/v1 before creating the queued row. The worker reads that object and the referenced forecast artifact, then writes a frontier and either one recommendation or an infeasible result.
processNextOptimizerRun() stores optimizer-frontier:v1 before marking a row completed or infeasible. Only the completed path inserts a candidate. An infeasible run gets zero recommendations and ranked relaxations. If artifact storage fails, the run ends with OPTIMIZER_WORKER_FAILED, not a terminal success pointing at a missing object.
Money crosses JSON boundaries as canonical decimal strings and is calculated as integer minor units. The first policy API build exposed why exact text matters. I was wrong to treat numeric equality as contract equality. PostgreSQL returned 12.50, but response formatting collapsed it to 12.5. The repository projection now preserves the two-decimal policy value, while cents remain integer strings through Zig, TypeScript, snapshots, and reports.
The application is advisory. It imports billing evidence and produces buy, renew, resize, exchange, no-action, or manual-review recommendations, but it does not purchase commitments from a provider. That boundary keeps an optimizer defect from becoming an automatic cash commitment.
Constraints That Shaped the Design
-
Input: Recorded AWS, Azure, GCP, and synthetic billing exports arrive as CSV, Parquet, JSON snapshots, or the implemented native AWS CUR boundary. Missing required fields, schema drift, or failed control totals quarantine a batch instead of creating partial usage history.
-
Output: Frontier and failure artifacts have a disclosure boundary. The integration suite scans stored state and serialized output for credentials, raw rows, stack traces, and internal candidate IDs. An infeasible result can explain which policy field needs to move without publishing the input rows used in the calculation.
-
Scale handled: The 2026-08-26 release receipt measured a 12-month replay over 1,000,000 line items in 383.38 ms. It measured optimizer p95 at 63.51 ms across 10,000 candidates and 25 iterations on the local validation host. These are kernel and replay receipts, not end-to-end concurrent-user guarantees.
-
Hard constraints: Price data can become stale, demand forecasts can carry low confidence, and a policy can make every candidate infeasible. The architecture represents each condition directly. It does not swap in fallback pricing or turn an infeasible run into a low-risk recommendation.
-
Tenancy: Every data-bearing domain table is tenant-owned, and protected routes resolve a database-confirmed actor before business logic. Cross-tenant resources return the same not-found shape. Background work carries the tenant ID through row claims, artifacts, logs, and events.
-
External services: The worker calls adapters after forecast, optimizer, expiry, and backtest work. Adapter failures retain retry state and cannot rewrite a core run result. The invoice reconciliation adapter remains inactive until its endpoint contract is verified.
Decision Log
| Decision | Alternative Rejected | Why |
|---|---|---|
| Freeze run inputs before queueing | Resolve current forecast and prices inside the worker | Queue delay could change the economic identity of a run before it starts. |
| Persist infeasible runs with relaxation hints | Return an error or choose the least bad candidate | "No acceptable portfolio" is a valid financial result and must not become a purchase recommendation. |
| Write the frontier object before terminal state | Mark the run complete before artifact storage | A completed row must not point at a frontier object that was never written. |
| Dispatch by provider and instrument | Infer one generic calculation from price rows | optimizeCommitmentRun() names five paths, and requireInstrument() rejects a mismatched pair before evaluation. |
| Quarantine a bad import as one batch | Insert valid rows and skip malformed rows | Partial history can make an unsafe commitment look well utilized. |
| Keep execution advisory | Call provider purchase APIs after approval | The repository proves analysis and control, not delegated authority over cloud spend. |
| Claim domain work from PostgreSQL rows | Treat Redis jobs as the only record of work | Recovery needs durable run state, frozen input references, and explicit terminal outcomes. |
| Use explicit decimal strings at boundaries | Send cents as JSON numbers | Provider bills and long-horizon totals can exceed safe JavaScript integer handling. |
Scaling Limits
The first pressure point is orchestration, not the economic kernel. runCycle() awaits one forecast, optimizer, expiry, backtest, and adapter operation in a fixed sequence, while cycleOperation blocks overlapping cycles. The 120-second queue-lag alert is the cutover signal: if lag stays above it for five minutes under normal load, split job types into separate worker pools and partition row claims by job type or tenant.
The next data-plane gate is 10,000,000 line items, ten times the measured replay set. That test should prove monthly fact partitions and separate analytical workers before the limit rises. Import configuration already permits a 1,024 MB file, so concurrent near-cap artifacts should move to S3-compatible storage before import concurrency increases. PostgreSQL can continue to own run state, approvals, and audit records.