cyberhybridhub/server/README.md

Cyber Hybrid Hub API

Postgres-backed profile API for the Flutter app.

Setup

1. Create the database (once):

   createdb cyberhybridhub
   

2. Copy and edit environment variables:

   cp .env.example .env
   

3. Install dependencies and run (from this server/ directory):

   dart pub get
   dart run bin/server.dart
   

The API listens on http://localhost:3000 by default (PORT in .env).

Tests

From the repo root (loads server/.env automatically):

./scripts/test-server.sh          # unit + DB integration (no live Alpaca)
./scripts/test-server-alpaca.sh   # live SPY quote — requires keys in server/.env

Or from server/:

# Uses DATABASE_URL from the environment or server/.env
export DATABASE_URL=postgresql://postgres:PASSWORD@localhost:5432/cyberhybridhub
dart pub get
dart test

Integration tests apply migrations 001–009 on cyberhybridhub_test and truncate trading tables between cases. Optional override: TEST_DATABASE_URL.

Endpoints

Method	Path	Auth
GET	/v1/me/profile	Authorization: Bearer <Firebase ID token>
PUT	/v1/me/profile	same
POST	/v1/me/incoming-question	same — pushes a question to the client via SignalR
POST	/v1/me/questions/bootstrap	ensure starter question at login
GET	/v1/me/questions	list unanswered questions (queue order)
POST	/v1/me/questions/{id}/answer	submit answer ({"answer": 0} default)
POST	/v1/me/questions/{id}/defer	move question to end of queue

SignalR — incoming questions

Hub URL: http://localhost:3000/hubs/questions

The Flutter app calls POST /v1/me/questions/bootstrap once at login to ensure a starter question exists (random correct answer from -10 to 10) when the user has none. After sign-in it connects to SignalR and listens for ReceiveQuestion. On each new WebSocket connection the API only delivers an existing unanswered question — it does not create new rows.

Client payload (correct answer is not sent):

{
  "id": "uuid",
  "assignedUserId": "firebase-uid",
  "text": "...",
  "sentAt": "...",
  "unansweredCount": 2
}

unansweredCount is the number of unanswered rows for that user (shown in the app when greater than 1).

questions table: id (UUID), assigned_user_id, question_text, user_response (nullable), correct_answer, created_at, modified_at.

Background question pipeline

A background worker runs inside the API process (enabled by default). On each interval it walks registered users, fetches data from public web APIs, and enqueues pipeline questions when the user's queue is not full.

Env var	Default	Purpose
QUESTION_WORKER_ENABLED	true	Set to false to disable the worker
QUESTION_WORKER_INTERVAL_SECONDS	60	Seconds between maintenance cycles
QUESTION_PIPELINE_TEST_MODE	false	Use random -10..10 starter-style questions instead of API copy

External APIs used

• REST Countries — population and capital facts
• Open-Meteo — current temperature by city

Branching flow

1. Track choice — user swipes toward +10 (weather) or -10 (geography).
2. Geography — yes/no population threshold, then capital confirmation; wrong population guess triggers a recovery question.
3. Weather — yes/no warm/cool for a random European city, then a follow-up to continue weather or switch to geography.

When a user submits an answer (POST .../answer), the pipeline evaluates the response and may immediately create the next branched question and push it over SignalR.

Pipeline state is stored in user_pipeline_state; questions may include source_tag, pipeline_key, and pipeline_step columns (migration 003_question_pipeline.sql).

Test from the shell (replace ID_TOKEN):

curl -s -X POST http://localhost:3000/v1/me/incoming-question \
  -H "Authorization: Bearer ID_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text":"What is your preferred contact method?"}'

Market history

Alpaca 1Min bars aggregated into two US regular-session half-days per trading day (morning 9:30–12:45 ET, afternoon 12:45–16:00 ET, ~195 minutes each). Stored as metric=bar, timeframe=sessionHalf. Rolling window: MARKET_HISTORY_WINDOW_DAYS (default 5 US trading days, two session halves each; weekends/holidays skipped).

Backfill (kind=backfill): fetches each ended slot still missing in DB; skips the open slot. Throttled to MARKET_HISTORY_API_REQUESTS_PER_MINUTE (default 200/min). On HTTP 429: wait 1 minute, retry once; if still limited, save partial rows and resume next tick. Runs when hasPendingSlots is true (worker tick). One bars request per slot × symbol batch. Before each worker or admin pipeline, orphaned market_data_sync_runs rows with finished_at IS NULL are closed (stale rows first, then any remaining orphans) so a crashed prior sync cannot block new work. Universe / cleanup keep hour-based cadence. guess_weekly_move reads these bars.

Requires TRADING_ENABLED=true when MARKET_HISTORY_SYNC_ENABLED=true.

Migration 008: legacy 1Day / 4Hour history cleanup and slots_synced on sync runs.

Migration 010: deletes 4Hour bars, adds sessionHalf timeframe + partial index.

Migration 009: adds market_data_sync_runs.backfill_items JSONB — per-slot UTC start + symbol list for each backfill run.

Env var	Default	Purpose
MARKET_HISTORY_SYNC_ENABLED	false	Worker: universe → backfill → cleanup
MARKET_HISTORY_WINDOW_DAYS	5	Oldest slot start to retain / backfill
MARKET_HISTORY_RETENTION_DAYS	5	Delete as_of older than this
MARKET_HISTORY_ARCHIVE_ENABLED	false	Archive before delete
MARKET_UNIVERSE_REFRESH_HOURS	24	Min hours between universe syncs
MARKET_HISTORY_SYNC_HOURS	24	Unused for backfill (slot-gated); fallback if no slot gate
MARKET_HISTORY_CLEANUP_HOURS	24	Min hours between cleanups
MARKET_HISTORY_SYNC_HOUR_UTC	(unset)	UTC hour floor; same-day cap applies to universe/cleanup only
HISTORY_SYNC_BATCH_SIZE	50	Symbols per bars request
HISTORY_SYNC_MAX_SYMBOLS	2000	Max symbols per backfill run
MARKET_HISTORY_API_REQUESTS_PER_MINUTE	200	Max Alpaca HTTP calls per rolling minute during backfill
MARKET_HISTORY_SYNC_STALE_MINUTES	30	Abort in-progress sync rows older than this before a new pipeline
MIN_BARS_FOR_GUESS	5	Min 4-hour bars for guess eligibility
GUESS_COOLDOWN_HOURS	24	Per-symbol guess cooldown

Admin portal (market history log)

Read-only audit log and on-demand triggers for universe sync, backfill, and retention cleanup. Mounted when ADMIN_PORTAL_ENABLED=true.

Env var	Default	Purpose
ADMIN_PORTAL_ENABLED	false	Mount /v1/admin/market-history/* and /v1/admin/market-data/*
ADMIN_FIREBASE_UIDS	(empty)	Comma-separated Firebase UIDs allowed to call admin routes

Requires TRADING_ENABLED=true, Alpaca credentials, and QUESTION_PIPELINE_TEST_MODE=false for on-demand resync/cleanup when MARKET_HISTORY_SYNC_ENABLED=true. When sync is disabled, the admin log and week-coverage calendar remain available read-only; resync/cleanup return 503. Scheduled worker sync also requires MARKET_HISTORY_SYNC_ENABLED=true.

Method	Path	Auth
GET	/v1/admin/market-history/sync-runs	Admin Firebase UID
POST	/v1/admin/market-data/resync	Admin Firebase UID
POST	/v1/admin/market-data/cleanup?archive=true|false	Admin Firebase UID

sync-runs includes optional config when the server has market-history env loaded:

{ "archiveEnabled": true, "windowDays": 5, "retentionDays": 5, "syncEnabled": true }

Flutter uses config.archiveEnabled to show the archive checkbox in the cleanup confirm dialog.

Admin portal tests

From the repo root:

./scripts/test-admin-portal.sh           # server + Flutter admin suites
./scripts/check-admin-portal-coverage.sh # coverage + Section 2 thresholds

CI: .github/workflows/admin-portal.yml runs both on admin-related changes.

Flutter client

Run the app with the API URL (defaults to http://localhost:3000):

flutter run --dart-define=API_BASE_URL=http://localhost:3000