cyberhybridhub/server

Cyber Hybrid Hub API

Postgres-backed profile API for the Flutter app.

Setup

1. Create the database (once):

   createdb cyberhybridhub
   

2. Copy and edit environment variables:

   cp .env.example .env
   

3. Install dependencies and run (from this server/ directory):

   dart pub get
   dart run bin/server.dart
   

The API listens on http://localhost:3000 by default (PORT in .env).

Tests

From the repo root (loads server/.env automatically):

./scripts/test-server.sh          # unit + DB integration (no live Alpaca)
./scripts/test-server-alpaca.sh   # live SPY quote — requires keys in server/.env

Or from server/:

# Uses DATABASE_URL from the environment or server/.env
export DATABASE_URL=postgresql://postgres:PASSWORD@localhost:5432/cyberhybridhub
dart pub get
dart test

Integration tests apply migrations 001–009 on cyberhybridhub_test and truncate trading tables between cases. Optional override: TEST_DATABASE_URL.

Endpoints

Method	Path	Auth
GET	/v1/me/profile	Authorization: Bearer <Firebase ID token>
PUT	/v1/me/profile	same
POST	/v1/me/incoming-question	same — pushes a question to the client via SignalR
POST	/v1/me/questions/bootstrap	create one random prospective question on login (fallback to oldest unanswered)
GET	/v1/me/questions	list unanswered questions (queue order)
GET	/v1/me/questions/score	current cumulative prospective-question score
POST	/v1/me/questions/score/reset	reset guess score and answer statistics to zero
POST	/v1/me/questions/{id}/answer	submit answer ({"answer": 0} default)
POST	/v1/me/questions/{id}/defer	move question to end of queue

SignalR — incoming questions

Hub URL: http://localhost:3000/hubs/questions

The Flutter app calls POST /v1/me/questions/bootstrap once at login. The API ensures a users row exists, picks a random row from market_history_prospective_questions, and creates a user question linked by metadata.prospective_question_id (falls back to oldest unanswered when no prospective rows exist). After sign-in it connects to SignalR and listens for ReceiveQuestion. On each new WebSocket connection the API only delivers an existing unanswered question — it does not create new rows.

Client payload (correct answer is not sent):

{
  "id": "uuid",
  "assignedUserId": "firebase-uid",
  "text": "...",
  "sentAt": "...",
  "unansweredCount": 2
}

unansweredCount is the number of unanswered rows for that user (shown in the app when greater than 1).

questions table: id (UUID), assigned_user_id, question_text, user_response (nullable), correct_answer, created_at, modified_at.

market_history_prospective_answers table snapshots answered prospective-question context: question_id, prospective_question_id, symbol, older_slot_start, newer_slot_start, expected_percent_increase_price, and user_slider_value.

Guess score (user_trading_state.context.guess_score, keyed by Firebase UID):

• Persists across devices and logins for the same firebase_uid.
• GET /v1/me/questions/score and login bootstrap score repair counters/total from market_history_prospective_answers when stored JSON is missing or stale.

Prospective guess progression (user_trading_state.context.guess_score):

• slot_start — older session-half edge for the active pair; advances when every top-50% volume symbol in (slot_start → next slot) has been answered.
• Reset sets slot_start to the earliest slot in the rolling history window and clears that user's market_history_prospective_answers and market_history_prospective_assignments rows.
• Assignments (market_history_prospective_assignments, migration 014): one row per (user, older_slot_start, newer_slot_start) written when the question is created (pending until answered). Survives logins; unique constraint blocks a second asset for the same slot pair before answer.
• Question pick uses bar audit data for that slot pair (not a global random row).

Prospective answer scoring (user_trading_state.context.guess_score):

PROSPECTIVE_ANSWER_CLOSENESS_ENABLED	Correct sign	Wrong sign
false (default)	+1	-1
true	+1 plus up to +1 closeness (full bonus within ±1 of expected %, else fractional)	-2

Background question pipeline

A background worker runs inside the API process (enabled by default). On each interval it walks registered users, fetches data from public web APIs, and enqueues pipeline questions when the user's queue is not full.

Env var	Default	Purpose
QUESTION_WORKER_ENABLED	true	Set to false to disable the worker
QUESTION_WORKER_INTERVAL_SECONDS	60	Seconds between maintenance cycles
QUESTION_PIPELINE_TEST_MODE	false	Use random -10..10 starter-style questions instead of API copy
PROSPECTIVE_ANSWER_CLOSENESS_ENABLED	false	Enable closeness bonus scoring (+1/-2 with magnitude bonus) instead of simple +1/-1

External APIs used

• REST Countries — population and capital facts
• Open-Meteo — current temperature by city

Branching flow

1. Track choice — user swipes toward +10 (weather) or -10 (geography).
2. Geography — yes/no population threshold, then capital confirmation; wrong population guess triggers a recovery question.
3. Weather — yes/no warm/cool for a random European city, then a follow-up to continue weather or switch to geography.

When a user submits an answer (POST .../answer), the pipeline evaluates the response and may immediately create the next branched question and push it over SignalR.

Pipeline state is stored in user_pipeline_state; questions may include source_tag, pipeline_key, and pipeline_step columns (migration 003_question_pipeline.sql).

Test from the shell (replace ID_TOKEN):

curl -s -X POST http://localhost:3000/v1/me/incoming-question \
  -H "Authorization: Bearer ID_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text":"What is your preferred contact method?"}'

Market history

Alpaca 1Min bars aggregated into two US regular-session half-days per trading day (morning 9:30–12:45 ET, afternoon 12:45–16:00 ET, ~195 minutes each). Stored as metric=bar, timeframe=sessionHalf. Rolling window: MARKET_HISTORY_WINDOW_DAYS (default 5 US trading days, two session halves each; weekends/holidays skipped).

Backfill (kind=backfill): fetches each ended slot still missing in DB; skips the open slot. Throttled to MARKET_HISTORY_API_REQUESTS_PER_MINUTE (default 200/min). On HTTP 429: wait 1 minute, retry once; if still limited, save partial rows and resume next tick. Runs when hasPendingSlots is true (worker tick). One bars request per slot × symbol batch. Before each worker or admin pipeline, orphaned market_data_sync_runs rows with finished_at IS NULL are closed (stale rows first, then any remaining orphans) so a crashed prior sync cannot block new work. Universe / cleanup keep hour-based cadence. guess_weekly_move reads these bars.

Worker pipeline (each tick when sync enabled): universe → backfill → cleanup → prospective-question refresh (silent; not logged to market_data_sync_runs).

Prospective questions (market_history_prospective_questions, migration 012):

• Identity: UNIQUE (symbol, older_slot_start, newer_slot_start) — one row per asset per canonical slot pair (slot starts snapped to session-half boundaries).
• Refresh: top 50% by mean dollar volume for the latest completed pair; INSERT … ON CONFLICT DO UPDATE; prune symbols that dropped out of the top half for that pair; pg_advisory_xact_lock so overlapping ticks cannot duplicate rows.
• Answer: correct_answer = percent change in average OHLC price (older → newer).
• Cleanup: cleanup deletes rows with older_slot_start before the retention cutoff (MARKET_HISTORY_RETENTION_DAYS, same as market_data_snapshots). Refresh also purges expired rows before upserting.

Requires TRADING_ENABLED=true when MARKET_HISTORY_SYNC_ENABLED=true.

Migration 008: legacy 1Day / 4Hour history cleanup and slots_synced on sync runs.

Migration 010: deletes 4Hour bars, adds sessionHalf timeframe + partial index.

Migration 009: adds market_data_sync_runs.backfill_items JSONB — per-slot UTC start + symbol list for each backfill run.

Env var	Default	Purpose
MARKET_HISTORY_SYNC_ENABLED	false	Worker: universe → backfill → cleanup
MARKET_HISTORY_WINDOW_DAYS	5	Oldest slot start to retain / backfill
MARKET_HISTORY_RETENTION_DAYS	5	Delete as_of older than this
MARKET_HISTORY_ARCHIVE_ENABLED	false	Archive before delete
MARKET_UNIVERSE_REFRESH_HOURS	24	Min hours between universe syncs
MARKET_HISTORY_SYNC_HOURS	24	Unused for backfill (slot-gated); fallback if no slot gate
MARKET_HISTORY_CLEANUP_HOURS	24	Min hours between cleanups
MARKET_HISTORY_SYNC_HOUR_UTC	(unset)	UTC hour floor; same-day cap applies to universe/cleanup only
HISTORY_SYNC_BATCH_SIZE	50	Symbols per bars request
HISTORY_SYNC_MAX_SYMBOLS	2000	Max symbols per backfill run
MARKET_HISTORY_API_REQUESTS_PER_MINUTE	200	Max Alpaca HTTP calls per rolling minute during backfill
MARKET_HISTORY_SYNC_STALE_MINUTES	30	Abort in-progress sync rows older than this before a new pipeline
MIN_BARS_FOR_GUESS	5	Min 4-hour bars for guess eligibility
GUESS_COOLDOWN_HOURS	24	Per-symbol guess cooldown

Admin portal (market history log)

Read-only audit log and on-demand triggers for universe sync, backfill, and retention cleanup. Mounted when ADMIN_PORTAL_ENABLED=true.

Env var	Default	Purpose
ADMIN_PORTAL_ENABLED	false	Mount /v1/admin/market-history/* and /v1/admin/market-data/*
ADMIN_FIREBASE_UIDS	(empty)	Comma-separated Firebase UIDs allowed to call admin routes

Requires TRADING_ENABLED=true, Alpaca credentials, and QUESTION_PIPELINE_TEST_MODE=false for on-demand resync/cleanup when MARKET_HISTORY_SYNC_ENABLED=true. When sync is disabled, the admin log and week-coverage calendar remain available read-only; resync/cleanup return 503. Scheduled worker sync also requires MARKET_HISTORY_SYNC_ENABLED=true.

Method	Path	Auth
GET	/v1/admin/market-history/sync-runs	Admin Firebase UID
POST	/v1/admin/market-data/resync	Admin Firebase UID
POST	/v1/admin/market-data/cleanup?archive=true|false	Admin Firebase UID

sync-runs includes optional config when the server has market-history env loaded:

{ "archiveEnabled": true, "windowDays": 5, "retentionDays": 5, "syncEnabled": true }

Flutter uses config.archiveEnabled to show the archive checkbox in the cleanup confirm dialog.

Admin portal tests

From the repo root:

./scripts/test-admin-portal.sh           # server + Flutter admin suites
./scripts/check-admin-portal-coverage.sh # coverage + Section 2 thresholds

CI: .github/workflows/admin-portal.yml runs both on admin-related changes.

Flutter client

Run the app with the API URL (defaults to http://localhost:3000):

flutter run --dart-define=API_BASE_URL=http://localhost:3000