454 lines
18 KiB
Markdown
454 lines
18 KiB
Markdown
# Trading TDD Plan — Progress Tracker
|
||
|
||
Companion to [TRADING_DEVELOPMENT_PLAN.md](./TRADING_DEVELOPMENT_PLAN.md). Each step follows **Red → Green → Confirm** before moving on.
|
||
|
||
**How agents should use this file:**
|
||
|
||
1. Pick the first unchecked step in order.
|
||
2. Write the failing test (Red), implement minimal code (Green), run the Confirm gate.
|
||
3. Check off boxes and add a one-line note under **Progress log** with date and result.
|
||
4. Do not skip Confirm gates or start Phase 2 until Gate B passes.
|
||
|
||
---
|
||
|
||
## Overall status
|
||
|
||
| Milestone | Status | Notes |
|
||
|-----------|--------|-------|
|
||
| Step 0 — Test harness | ✅ Done | 2026-05-23 |
|
||
| Steps 1–11 — Phase 1 MVP | 🔄 In progress | Day 7: Steps 9–10 done; Gate A scripted test green |
|
||
| Gate A — Local loop (test mode) | ✅ Done | 2026-05-26 — `trading_orchestrator_gate_a_test.dart` |
|
||
| Gate B — Paper Alpaca E2E | ⬜ Not started | |
|
||
| Steps 12–15 — Phase 2 | ⬜ Not started | |
|
||
| Steps 16–19 — Phase 3 | ⬜ Deferred | |
|
||
|
||
Legend: ⬜ Not started · 🔄 In progress · ✅ Done · ⏸ Blocked
|
||
|
||
---
|
||
|
||
## Progress log
|
||
|
||
<!-- Agents: append newest entries at the top -->
|
||
|
||
| Date | Step | Result |
|
||
|------|------|--------|
|
||
| 2026-05-26 | 10 + Gate A | `dart test` green (57 tests); `TradingOrchestrator` (ingest → evaluate → actuate per user) + `QuestionBackgroundWorker` integration; Gate A scripted test (seed → tick → question → +10 → tick → trade_orders row → tick → cooldown) passing in test mode |
|
||
| 2026-05-26 | 9 | `dart test` green (56 tests); AlpacaTradingClient (POST /v2/orders, GET by_client_order_id, dup detection) + TradeActuator (test mode + mocked Alpaca, guardrails, idempotency); tagged `alpaca` live roundtrip script |
|
||
| 2026-05-25 | 7–8 | `dart test` green (47 tests); TradingPipeline evaluate/handleAnswer + QuestionPipeline branch |
|
||
| 2026-05-25 | 5–6 | `dart test` green (39 tests); rule engine + guardrails (pure logic) |
|
||
| 2026-05-23 | 3–4 | `dart test` green (21 tests); Alpaca MD client, ingest, poll interval; live Alpaca OK |
|
||
| 2026-05-23 | 1.3–1.4, 2 | `dart test` green (17 tests); config merge, trade orders, Alpaca env/models |
|
||
| 2026-05-23 | 0, 1.1–1.2 | `dart test` green (4 tests); migration 004 + MarketDataDb |
|
||
|
||
---
|
||
|
||
## Principles
|
||
|
||
| Principle | How it applies |
|
||
|-----------|----------------|
|
||
| **One behavior per cycle** | One failing test → minimal code → gate check |
|
||
| **No Alpaca in unit tests** | Mock HTTP clients; inject fixture JSON |
|
||
| **DB tests are integration** | Real Postgres test DB (`cyberhybridhub_test`) |
|
||
| **E2E gates are manual/scripted** | Flutter swipe + Alpaca paper dashboard at defined checkpoints |
|
||
| **Feature flags off until wired** | `TRADING_ENABLED=false` until Step 10 complete |
|
||
|
||
---
|
||
|
||
## Step 0 — Test harness
|
||
|
||
### 0.1 Server test setup
|
||
|
||
- [x] Add `dev_dependencies: test: ^1.25.0` to `server/pubspec.yaml`
|
||
- [x] Create `server/test/` smoke test
|
||
- [x] Create `server/test/helpers/fixture_loader.dart`
|
||
- [x] Create `server/test/helpers/mock_http_client.dart`
|
||
- [x] Create `server/test/helpers/test_env.dart`
|
||
- [x] Create fixture files:
|
||
- [x] `server/test/fixtures/alpaca_latest_trade.json`
|
||
- [x] `server/test/fixtures/alpaca_daily_bars.json`
|
||
- [x] `server/test/fixtures/trading_config_default.json`
|
||
- [x] `server/test/fixtures/market_snapshots_spy_dip.json`
|
||
|
||
**Red:** Smoke test fails until layout exists.
|
||
|
||
**Green:** Minimal test directory and helpers.
|
||
|
||
**Confirm:** `cd server && dart test` passes with 1 smoke test.
|
||
|
||
---
|
||
|
||
### 0.2 Integration test DB
|
||
|
||
- [x] Test helper applies migrations 001–004 on `cyberhybridhub_test`
|
||
- [x] `server/test/integration/migration_test.dart` — migration applies cleanly
|
||
|
||
**Red:** Migration test fails (no `004_trading.sql` yet).
|
||
|
||
**Green:** Migration runner + empty-schema apply.
|
||
|
||
**Confirm:** `dart test test/integration/migration_test.dart` passes locally and in CI.
|
||
|
||
---
|
||
|
||
### 0.3 Flutter (minimal)
|
||
|
||
- [ ] No new Flutter tests until Step 9 — existing `SwipeQuestionTile` + SignalR contract unchanged
|
||
|
||
---
|
||
|
||
## Phase 1 — Foundation (MVP)
|
||
|
||
Maps to TRADING_DEVELOPMENT_PLAN §5, §6, §7, §8, §9, §10, §14 Phase 1.
|
||
|
||
---
|
||
|
||
### Step 1 — Schema & DB accessors
|
||
|
||
**Refs:** §5 Postgres schema · `market_data_db.dart` · `trading_config_db.dart` · `trade_orders_db.dart`
|
||
|
||
#### 1.1 Migration
|
||
|
||
- [x] **Red:** Integration test — INSERT into `market_data_snapshots`, `user_trading_config`, `trade_orders`; FK to `users` enforced
|
||
- [x] **Green:** `server/migrations/004_trading.sql`
|
||
- [x] **Green:** Seed `trading_config_templates` row `default_paper_watchlist`
|
||
- [x] **Confirm:** `\d market_data_snapshots` in psql; seed template queryable
|
||
|
||
#### 1.2 MarketDataDb
|
||
|
||
- [x] **Red:** `insertSnapshot()` then `latestForSymbol(symbol, metric)` returns newest row by `as_of`
|
||
- [x] **Green:** `server/lib/trading/market_data_db.dart`
|
||
- [x] **Confirm:** Two SPY/`last_trade` inserts → latest returns newer price
|
||
|
||
#### 1.3 TradingConfigDb
|
||
|
||
- [x] **Red:** `resolveEffectiveConfig(uid)` merges template + user JSONB; user override wins
|
||
- [x] **Green:** `server/lib/trading/trading_config_db.dart`
|
||
- [x] **Green:** `server/lib/trading/trading_config.dart` parser
|
||
- [x] **Confirm:** Partial user override yields merged `enabled`, `data_inputs`, `rules`
|
||
|
||
#### 1.4 TradeOrdersDb
|
||
|
||
- [x] **Red:** Duplicate `client_order_id` raises unique violation; `findByClientOrderId` returns existing row
|
||
- [x] **Green:** `server/lib/trading/trade_orders_db.dart`
|
||
- [x] **Confirm:** Idempotency check prevents double-insert
|
||
|
||
---
|
||
|
||
### Step 2 — Alpaca env & models
|
||
|
||
**Refs:** §3 · §11 · `alpaca_env.dart` · `alpaca_models.dart`
|
||
|
||
- [x] **Red:** `AlpacaEnv.fromMap()` reads keys; `assertPaperOnly()` throws when live URL + `ALPACA_ALLOW_LIVE=false`
|
||
- [x] **Green:** `server/lib/alpaca/alpaca_env.dart`
|
||
- [x] **Green:** `server/lib/alpaca/alpaca_models.dart` (Trade, Bar, OrderRequest)
|
||
- [x] **Green:** Extend `server/lib/env.dart` with trading flags (after env tests pass)
|
||
- [x] **Confirm:** Unit tests — paper default, live blocked, missing keys
|
||
|
||
---
|
||
|
||
### Step 3 — Alpaca Market Data client (REST)
|
||
|
||
**Refs:** §9.1 · `alpaca_market_data_client.dart`
|
||
|
||
- [x] **Red:** Mock HTTP — `getLatestTrade('SPY')` parses fixture; sends `APCA-API-KEY-ID` headers
|
||
- [x] **Green:** `server/lib/alpaca/alpaca_market_data_client.dart` with injectable `http.Client`
|
||
- [x] **Confirm:** Optional tagged test `@Tags(['alpaca'])` — one real call (skipped in CI)
|
||
|
||
---
|
||
|
||
### Step 4 — Snapshot normalization & ingest
|
||
|
||
**Refs:** §9.3 · `market_data_ingest.dart`
|
||
|
||
- [x] **Red:** Config `data_inputs` with `[last_trade, daily_bar, prev_close]` → 3 snapshot rows with correct `metric`, `price`, `as_of`
|
||
- [x] **Green:** `server/lib/trading/market_data_ingest.dart` — `runIfDue()` writes via `MarketDataDb`
|
||
- [x] **Confirm:** Integration — ingest for seeded user; `SELECT * FROM market_data_snapshots WHERE symbol='SPY'`
|
||
|
||
#### 4.1 Poll interval
|
||
|
||
- [x] **Red:** Second call within `poll_interval_seconds` does not fetch (uses `user_trading_state.context`)
|
||
- [x] **Green:** Update `user_trading_state` on ingest
|
||
- [x] **Confirm:** Two rapid `runIfDue` calls → mock HTTP call count = 1
|
||
|
||
---
|
||
|
||
### Step 5 — Rule engine (pure logic)
|
||
|
||
**Refs:** §4.2 · §8 · `rule_engine.dart`
|
||
|
||
- [x] **Red:** `price_below_pct_of_ref` — SPY `last_trade=492`, `prev_close=500`, threshold `-1.5` → fires with `pct ≈ -1.6`
|
||
- [x] **Green:** `server/lib/trading/rule_engine.dart` — `RuleEngine.evaluate(rule, snapshots) → RuleEvaluation?`
|
||
- [x] **Confirm:** Table-driven unit tests:
|
||
|
||
| Case | Expected |
|
||
|------|----------|
|
||
| Above threshold (-0.5%) | No fire |
|
||
| Missing metric | No fire |
|
||
| Stale `as_of` (> max_staleness) | No fire |
|
||
| Cooldown: rule fired today | No fire |
|
||
| Template `{{pct}}`, `{{symbol}}`, `{{price}}` | Substituted |
|
||
|
||
---
|
||
|
||
### Step 6 — Guardrails
|
||
|
||
**Refs:** §8.3
|
||
|
||
- [x] **Red:** `Guardrails.check()` rejects when `max_orders_per_day` exceeded
|
||
- [x] **Red:** Rejects blocklisted symbol
|
||
- [x] **Red:** Rejects when `require_question_before_order` violated
|
||
- [x] **Green:** `server/lib/trading/guardrails.dart`
|
||
- [x] **Confirm:** Server-side max notional ceiling enforced even if config allows more
|
||
|
||
---
|
||
|
||
### Step 7 — Trading pipeline — question creation
|
||
|
||
**Refs:** §8.1 · `trading_pipeline.dart` · `QuestionService`
|
||
|
||
- [x] **Red:** Rule fires + guardrails pass + queue room → `createAndDeliverQuestion` with `pipeline_key=trading`, `pipeline_step=dip_confirm:await_confirm`, `correct_answer=10`, substituted text
|
||
- [x] **Green:** `server/lib/trading/trading_pipeline.dart` — `evaluate()` with mocked deps
|
||
- [x] **Confirm:** Integration — seeded snapshots + config → row in `questions` with correct tags
|
||
- [x] **Green:** `user_trading_state.context` records pending rule id and phase
|
||
|
||
---
|
||
|
||
### Step 8 — Answer handling → order proposal
|
||
|
||
**Refs:** §8.2 · extend `QuestionPipeline.onAnswerSubmitted`
|
||
|
||
- [x] **Red:** `pipeline_key=trading`, user `+10`, `BranchDecision.yesNo` → match → stages order (`submit_order`), no Alpaca yet
|
||
- [x] **Red:** User `-10` → skip logged, no order
|
||
- [x] **Green:** `PipelineKeys.trading` in `question_pipeline.dart`
|
||
- [x] **Green:** `_handleTradingAnswer` in `question_pipeline.dart` (delegates to `TradingPipeline.handleAnswer`)
|
||
- [x] **Confirm:** Integration test exercises the `QuestionPipeline.onAnswerSubmitted` switch for `pipeline_key=trading`
|
||
|
||
---
|
||
|
||
### Step 9 — Trade actuator
|
||
|
||
**Refs:** §10 · `alpaca_trading_client.dart` · `TradeActuator`
|
||
|
||
- [x] **Red:** Mock HTTP — `submitOrder(notional: 10, side: buy)` POSTs paper URL with `client_order_id`; `trade_orders` row `status=accepted`
|
||
- [x] **Red:** Duplicate `client_order_id` → 422 → resolved via `getOrderByClientOrderId`
|
||
- [x] **Green:** `server/lib/alpaca/alpaca_trading_client.dart`
|
||
- [x] **Green:** `server/lib/trading/trade_actuator.dart` (test mode short-circuit + Alpaca path)
|
||
- [x] **Confirm:** Tagged integration test (`server/test/alpaca/alpaca_trading_live_test.dart`) — paper roundtrip, skipped without credentials
|
||
- [x] **Confirm:** `client_order_id` format `{firebase_uid}-{rule_id}-{question_id}` unique (set by `TradingPipeline.handleAnswer`, enforced by `trade_orders.client_order_id UNIQUE`)
|
||
|
||
---
|
||
|
||
### Step 10 — Worker integration
|
||
|
||
**Refs:** §7 · `question_background_worker.dart` · `trading_orchestrator.dart`
|
||
|
||
- [x] **Red:** `TRADING_ENABLED=true`, test mode (no Alpaca client) — one orchestrator tick runs evaluate + actuate against fixture snapshots
|
||
- [x] **Green:** `server/lib/trading/trading_orchestrator.dart` (ingest → evaluate → actuate, per-stage failure isolation)
|
||
- [x] **Green:** `QuestionBackgroundWorker` accepts optional `TradingOrchestrator`, runs it after `QuestionPipeline.runMaintenanceCycle`
|
||
- [x] **Green:** `bin/server.dart` builds the trading stack only when `TRADING_ENABLED=true`; uses real Alpaca clients only when `QUESTION_PIPELINE_TEST_MODE=false` and credentials present
|
||
- [x] **Confirm:** **Gate A** below — `test/integration/trading_orchestrator_gate_a_test.dart` green
|
||
|
||
---
|
||
|
||
### Step 11 — End-to-end with real Alpaca paper
|
||
|
||
**Refs:** §17 · §15 integration
|
||
|
||
- [ ] **Red:** Manual/scripted E2E checklist (or tagged automated test)
|
||
- [ ] **Green:** Real keys in `.env`, `QUESTION_PIPELINE_TEST_MODE=false`
|
||
- [ ] **Confirm:** **Gate B** (see below)
|
||
|
||
---
|
||
|
||
## Gate A — Local loop (test mode)
|
||
|
||
**Prerequisite:** Steps 0–10 complete.
|
||
|
||
Automated as `server/test/integration/trading_orchestrator_gate_a_test.dart`.
|
||
|
||
- [x] Seed user with `user_trading_config.enabled=true` and dip rule
|
||
- [x] Insert fixture snapshots (or test-mode ingest)
|
||
- [x] Run one worker tick → question queued
|
||
- [x] Submit answer +10 via API
|
||
- [x] Run next tick → `trade_orders` row (fake Alpaca id in test mode)
|
||
- [x] Third tick → rule does not re-fire (cooldown)
|
||
- [x] Set `TRADING_ENABLED=true` in dev `.env` only after Gate A passes (manual — flip the flag when ready to start using the live stack against paper Alpaca)
|
||
|
||
---
|
||
|
||
## Gate B — Paper Alpaca E2E
|
||
|
||
**Prerequisite:** Gate A complete.
|
||
|
||
- [ ] Ingest real SPY prices
|
||
- [ ] Rule fires (threshold or market conditions)
|
||
- [ ] Flutter: question via SignalR
|
||
- [ ] User swipes +10
|
||
- [ ] Order visible in Alpaca paper UI
|
||
- [ ] `trade_orders.question_id` and `rule_id` populated
|
||
|
||
---
|
||
|
||
## Phase 2 — Configuration API & hardening
|
||
|
||
**Start only after Gate B passes.** Maps to TRADING_DEVELOPMENT_PLAN §13, §14 Phase 2.
|
||
|
||
---
|
||
|
||
### Step 12 — Config validation
|
||
|
||
- [ ] **Red:** Validator rejects >30 symbols, missing `rules[].id`, invalid `mode`
|
||
- [ ] **Green:** `TradingConfigValidator`
|
||
- [ ] **Confirm:** Invalid JSON → 400 on PUT
|
||
|
||
---
|
||
|
||
### Step 13 — HTTP endpoints
|
||
|
||
- [ ] **Red:** Handler tests — `GET/PUT /v1/me/trading/config`
|
||
- [ ] **Red:** `GET /v1/me/trading/orders`
|
||
- [ ] **Red:** `GET /v1/me/trading/snapshots?symbol=SPY`
|
||
- [ ] **Green:** `server/lib/handlers/trading_handler.dart`
|
||
- [ ] **Confirm:** curl with Firebase token; sanitized config (no secrets)
|
||
|
||
---
|
||
|
||
### Step 14 — Daily guardrail counters
|
||
|
||
- [ ] **Red:** After order submit, `context.daily_order_count` increments; resets UTC midnight
|
||
- [ ] **Green:** Counter in `user_trading_state.context`
|
||
- [ ] **Confirm:** Third order same day blocked when `max_orders_per_day=3`
|
||
|
||
---
|
||
|
||
### Step 15 — Order status polling
|
||
|
||
- [ ] **Red:** Mock Alpaca status `filled` → `trade_orders.status` and `filled_at` updated
|
||
- [ ] **Green:** Poll on worker tick or post-submit
|
||
- [ ] **Confirm:** Failed order → follow-up question (optional)
|
||
|
||
---
|
||
|
||
## Phase 3 — Streaming & observability
|
||
|
||
**Deferred until Phase 2 stable.** Maps to TRADING_DEVELOPMENT_PLAN §9.2, §14 Phase 3.
|
||
|
||
| Step | Red test | Green impl | Done |
|
||
|------|----------|------------|------|
|
||
| 16 WS ingest | Mock WS message → snapshot upsert | `alpaca_ws_ingest.dart` | ⬜ |
|
||
| 17 Symbol cap | 31st symbol rejected | Config validator + WS manager | ⬜ |
|
||
| 18 SignalR `ReceiveTradeUpdate` | Hub mock receives on fill | Optional Flutter listener | ⬜ |
|
||
| 19 Metrics | Log ingest lag, rule fires, order latency | Structured logging | ⬜ |
|
||
|
||
---
|
||
|
||
## Test pyramid (target)
|
||
|
||
```text
|
||
┌─────────────────┐
|
||
│ Gate B (manual) │ 1 scenario / release
|
||
├─────────────────┤
|
||
│ Gate A (worker) │ 1 scripted integration
|
||
├─────────────────┤
|
||
│ HTTP handlers │ ~8 tests
|
||
├─────────────────┤
|
||
│ DB integration │ ~12 tests
|
||
├─────────────────┤
|
||
│ Unit (engine, │ ~40 tests
|
||
│ clients, config)│
|
||
└─────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## Suggested schedule (first 2 weeks)
|
||
|
||
| Day | Steps | Deliverable |
|
||
|-----|-------|-------------|
|
||
| 1 | 0, 1.1–1.2 | Test harness + migration + snapshot DB |
|
||
| 2 | 1.3–1.4, 2 | Config merge + env |
|
||
| 3 | 3–4 | Alpaca MD client + ingest |
|
||
| 4 | 5–6 | Rule engine + guardrails |
|
||
| 5 | 7–8 | Questions from rules + answer branch |
|
||
| 6 | 9 | Trade actuator |
|
||
| 7 | 10 | Worker wire-up → Gate A |
|
||
| 8 | 11 | Gate B with paper account |
|
||
|
||
---
|
||
|
||
## CI configuration
|
||
|
||
```yaml
|
||
# Suggested CI jobs
|
||
- dart test # unit + DB integration (no Alpaca)
|
||
- dart test --tags=alpaca # optional; secrets required; nightly/manual
|
||
```
|
||
|
||
**CI env:**
|
||
|
||
```bash
|
||
TRADING_ENABLED=true
|
||
QUESTION_PIPELINE_TEST_MODE=true
|
||
DATABASE_URL=postgres://.../cyberhybridhub_test
|
||
# No ALPACA_* keys in default CI job
|
||
```
|
||
|
||
---
|
||
|
||
## Safety tests (must pass before live)
|
||
|
||
These should **fail the build** if removed:
|
||
|
||
- [ ] `AlpacaEnv` refuses live URL when `ALPACA_ALLOW_LIVE=false`
|
||
- [ ] Server-side max notional ceiling enforced regardless of config
|
||
- [ ] Every order has non-null `question_id` when `require_question_before_order=true`
|
||
- [ ] `client_order_id` UNIQUE prevents duplicate Alpaca POST
|
||
|
||
---
|
||
|
||
## Mapping to existing code
|
||
|
||
| New piece | Extends |
|
||
|-----------|---------|
|
||
| `TradingPipeline.evaluate` | `QuestionPipeline._canEnqueue` queue limits |
|
||
| `onAnswerSubmitted` trading branch | geography/weather switch in `question_pipeline.dart` |
|
||
| `BranchDecision.yesNo` | +10/-10 swipe (already used for weather) |
|
||
| `ExternalDataFetcher` pattern | Alpaca clients with injectable `http.Client` |
|
||
| `QUESTION_PIPELINE_TEST_MODE` | Skip Alpaca; fixture snapshots |
|
||
|
||
**Existing touchpoints:**
|
||
|
||
| Component | Path |
|
||
|-----------|------|
|
||
| Interval worker | `server/lib/workers/question_background_worker.dart` |
|
||
| Question pipeline | `server/lib/pipeline/question_pipeline.dart` |
|
||
| Branch decisions | `server/lib/pipeline/branch_decision.dart` |
|
||
| External fetcher | `server/lib/pipeline/external_data_fetcher.dart` |
|
||
| Env | `server/lib/env.dart` |
|
||
| Flutter hub | `lib/services/questions_hub_service.dart` |
|
||
| Flutter swipe UI | `lib/widgets/swipe_question_tile.dart` |
|
||
|
||
---
|
||
|
||
## MVP definition of done
|
||
|
||
- [ ] `dart test` in `server/` green (~40+ tests, no network in default job)
|
||
- [ ] Gate A passes with test mode
|
||
- [ ] Gate B passes on Alpaca paper
|
||
- [ ] TRADING_DEVELOPMENT_PLAN §17 scenario reproducible from seed SQL
|
||
- [ ] `TRADING_ENABLED=false` default; no secrets in Flutter
|
||
- [ ] Every order traceable via `trade_orders.question_id` + `rule_id`
|
||
|
||
---
|
||
|
||
## References
|
||
|
||
- [TRADING_DEVELOPMENT_PLAN.md](./TRADING_DEVELOPMENT_PLAN.md)
|
||
- [server/README.md](./server/README.md)
|
||
- [Alpaca Market Data API](https://docs.alpaca.markets/docs/market-data-api)
|
||
- [Alpaca Trading API](https://docs.alpaca.markets/docs/trading-api)
|
||
|
||
---
|
||
|
||
*Document version: 1.0 — TDD progress tracker for Alpaca trading MVP.*
|