# Cyber Hybrid Hub API

Postgres-backed profile API for the Flutter app.

## Setup

1. Create the database (once):

   ```bash
   createdb cyberhybridhub
   ```

2. Copy and edit environment variables:

   ```bash
   cp .env.example .env
   ```

3. Install dependencies and run (from this `server/` directory):

   ```bash
   dart pub get
   dart run bin/server.dart
   ```

The API listens on `http://localhost:3000` by default (`PORT` in `.env`).

## Tests

From the repo root (loads `server/.env` automatically):

```bash
./scripts/test-server.sh          # unit + DB integration (no live Alpaca)
./scripts/test-server-alpaca.sh   # live SPY quote — requires keys in server/.env
```

Or from `server/`:

```bash
# Uses DATABASE_URL from the environment or server/.env
export DATABASE_URL=postgresql://postgres:PASSWORD@localhost:5432/cyberhybridhub
dart pub get
dart test
```

Integration tests apply migrations `001`–`009` on `cyberhybridhub_test` and truncate
trading tables between cases. Optional override: `TEST_DATABASE_URL`.

## Endpoints

| Method | Path | Auth |
|--------|------|------|
| `GET` | `/v1/me/profile` | `Authorization: Bearer <Firebase ID token>` |
| `PUT` | `/v1/me/profile` | same |
| `POST` | `/v1/me/incoming-question` | same — pushes a question to the client via SignalR |
| `POST` | `/v1/me/questions/bootstrap` | create one random prospective question on login (fallback to oldest unanswered) |
| `GET` | `/v1/me/questions` | list unanswered questions (queue order) |
| `GET` | `/v1/me/questions/score` | current cumulative prospective-question score |
| `POST` | `/v1/me/questions/score/reset` | reset guess score and answer statistics to zero |
| `POST` | `/v1/me/questions/{id}/answer` | submit answer (`{"answer": 0}` default) |
| `POST` | `/v1/me/questions/{id}/defer` | move question to end of queue |

## SignalR — incoming questions

Hub URL: `http://localhost:3000/hubs/questions`

The Flutter app calls `POST /v1/me/questions/bootstrap` once at login. The API ensures a
`users` row exists, picks a random row from `market_history_prospective_questions`, and
creates a user question linked by `metadata.prospective_question_id` (falls back to
oldest unanswered when no prospective rows exist). After sign-in it connects to SignalR
and listens for `ReceiveQuestion`. On each new WebSocket connection the API only delivers
an existing unanswered question — it does not create new rows.

Client payload (correct answer is not sent):

```json
{
  "id": "uuid",
  "assignedUserId": "firebase-uid",
  "text": "...",
  "sentAt": "...",
  "unansweredCount": 2
}
```

`unansweredCount` is the number of unanswered rows for that user (shown in the app when greater than 1).

`questions` table: `id` (UUID), `assigned_user_id`, `question_text`, `user_response`
(nullable), `correct_answer`, `created_at`, `modified_at`.

`market_history_prospective_answers` table snapshots answered prospective-question context:
`question_id`, `prospective_question_id`, `symbol`, `older_slot_start`,
`newer_slot_start`, `expected_percent_increase_price`, and `user_slider_value`.

Guess score (`user_trading_state.context.guess_score`, keyed by Firebase UID):

- Persists across devices and logins for the same `firebase_uid`.
- `GET /v1/me/questions/score` and login bootstrap `score` repair counters/total
  from `market_history_prospective_answers` when stored JSON is missing or stale.

Prospective guess progression (`user_trading_state.context.guess_score`):

- `slot_start` — older session-half edge for the active pair; advances when every
  top-50% volume symbol in `(slot_start → next slot)` has been answered.
- Reset sets `slot_start` to the earliest slot in the rolling history window and
  clears that user's `market_history_prospective_answers` and
  `market_history_prospective_assignments` rows.
- **Assignments** (`market_history_prospective_assignments`, migration `014`):
  one row per `(user, older_slot_start, newer_slot_start)` written when the
  question is created (`pending` until answered). Survives logins; unique
  constraint blocks a second asset for the same slot pair before answer.
- Question pick uses bar audit data for that slot pair (not a global random row).

Prospective answer scoring (`user_trading_state.context.guess_score`):

| `PROSPECTIVE_ANSWER_CLOSENESS_ENABLED` | Correct sign | Wrong sign |
|----------------------------------------|--------------|------------|
| `false` (default) | `+1` | `-1` |
| `true` | `+1` plus up to `+1` closeness (full bonus within ±1 of expected %, else fractional) | `-2` |

## Background question pipeline

A background worker runs inside the API process (enabled by default). On each
interval it walks registered users, fetches data from public web APIs, and
enqueues pipeline questions when the user's queue is not full.

| Env var | Default | Purpose |
|---------|---------|---------|
| `QUESTION_WORKER_ENABLED` | `true` | Set to `false` to disable the worker |
| `QUESTION_WORKER_INTERVAL_SECONDS` | `60` | Seconds between maintenance cycles |
| `QUESTION_PIPELINE_TEST_MODE` | `false` | Use random -10..10 starter-style questions instead of API copy |
| `PROSPECTIVE_ANSWER_CLOSENESS_ENABLED` | `false` | Enable closeness bonus scoring (`+1`/`-2` with magnitude bonus) instead of simple `+1`/`-1` |

**External APIs used**

- [REST Countries](https://restcountries.com/) — population and capital facts
- [Open-Meteo](https://open-meteo.com/) — current temperature by city

**Branching flow**

1. **Track choice** — user swipes toward +10 (weather) or -10 (geography).
2. **Geography** — yes/no population threshold, then capital confirmation;
   wrong population guess triggers a recovery question.
3. **Weather** — yes/no warm/cool for a random European city, then a follow-up
   to continue weather or switch to geography.

When a user submits an answer (`POST .../answer`), the pipeline evaluates the
response and may immediately create the next branched question and push it over
SignalR.

Pipeline state is stored in `user_pipeline_state`; questions may include
`source_tag`, `pipeline_key`, and `pipeline_step` columns (migration
`003_question_pipeline.sql`).

Test from the shell (replace `ID_TOKEN`):

```bash
curl -s -X POST http://localhost:3000/v1/me/incoming-question \
  -H "Authorization: Bearer ID_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text":"What is your preferred contact method?"}'
```

## Market history

Alpaca **`1Min`** bars aggregated into two **US regular-session** half-days per trading day
(morning **9:30–12:45 ET**, afternoon **12:45–16:00 ET**, ~195 minutes each).
Stored as `metric=bar`, `timeframe=sessionHalf`. Rolling window: `MARKET_HISTORY_WINDOW_DAYS` (default **5 US trading days**, two session halves each; weekends/holidays skipped).

**Backfill** (`kind=backfill`): fetches each **ended** slot still missing in DB; skips the open slot.
Throttled to `MARKET_HISTORY_API_REQUESTS_PER_MINUTE` (default 200/min). On HTTP 429: wait 1 minute, retry once; if still limited, save partial rows and resume next tick.
Runs when `hasPendingSlots` is true (worker tick). One bars request per slot × symbol batch.
Before each worker or admin pipeline, orphaned `market_data_sync_runs` rows with
`finished_at IS NULL` are closed (stale rows first, then any remaining orphans) so a
crashed prior sync cannot block new work.
**Universe** / **cleanup** keep hour-based cadence. `guess_weekly_move` reads these bars.

**Worker pipeline** (each tick when sync enabled): `universe` → `backfill` → `cleanup` →
prospective-question refresh (silent; not logged to `market_data_sync_runs`).

**Prospective questions** (`market_history_prospective_questions`, migration `012`):
- **Identity:** `UNIQUE (symbol, older_slot_start, newer_slot_start)` — one row per
  asset per canonical slot pair (slot starts snapped to session-half boundaries).
- **Refresh:** top 50% by mean dollar volume for the latest completed pair; `INSERT …
  ON CONFLICT DO UPDATE`; prune symbols that dropped out of the top half for that pair;
  `pg_advisory_xact_lock` so overlapping ticks cannot duplicate rows.
- **Answer:** `correct_answer` = percent change in average OHLC price (older → newer).
- **Cleanup:** `cleanup` deletes rows with `older_slot_start` before the retention
  cutoff (`MARKET_HISTORY_RETENTION_DAYS`, same as `market_data_snapshots`). Refresh
  also purges expired rows before upserting.

Requires `TRADING_ENABLED=true` when `MARKET_HISTORY_SYNC_ENABLED=true`.

**Migration `008`:** legacy `1Day` / `4Hour` history cleanup and `slots_synced` on sync runs.

**Migration `010`:** deletes `4Hour` bars, adds `sessionHalf` timeframe + partial index.

**Migration `009`:** adds `market_data_sync_runs.backfill_items` JSONB — per-slot UTC start + symbol list for each backfill run.

| Env var | Default | Purpose |
|---------|---------|---------|
| `MARKET_HISTORY_SYNC_ENABLED` | `false` | Worker: universe → backfill → cleanup |
| `MARKET_HISTORY_WINDOW_DAYS` | `5` | Oldest slot start to retain / backfill |
| `MARKET_HISTORY_RETENTION_DAYS` | `5` | Delete `as_of` older than this |
| `MARKET_HISTORY_ARCHIVE_ENABLED` | `false` | Archive before delete |
| `MARKET_UNIVERSE_REFRESH_HOURS` | `24` | Min hours between universe syncs |
| `MARKET_HISTORY_SYNC_HOURS` | `24` | Unused for backfill (slot-gated); fallback if no slot gate |
| `MARKET_HISTORY_CLEANUP_HOURS` | `24` | Min hours between cleanups |
| `MARKET_HISTORY_SYNC_HOUR_UTC` | *(unset)* | UTC hour floor; same-day cap applies to universe/cleanup only |
| `HISTORY_SYNC_BATCH_SIZE` | `50` | Symbols per bars request |
| `HISTORY_SYNC_MAX_SYMBOLS` | `2000` | Max symbols per backfill run |
| `MARKET_HISTORY_API_REQUESTS_PER_MINUTE` | `200` | Max Alpaca HTTP calls per rolling minute during backfill |
| `MARKET_HISTORY_SYNC_STALE_MINUTES` | `30` | Abort in-progress sync rows older than this before a new pipeline |
| `MIN_BARS_FOR_GUESS` | `5` | Min 4-hour bars for guess eligibility |
| `GUESS_COOLDOWN_HOURS` | `24` | Per-symbol guess cooldown |

## Admin portal (market history log)

Read-only audit log and on-demand triggers for universe sync, backfill, and
retention cleanup. Mounted when `ADMIN_PORTAL_ENABLED=true`.

| Env var | Default | Purpose |
|---------|---------|---------|
| `ADMIN_PORTAL_ENABLED` | `false` | Mount `/v1/admin/market-history/*` and `/v1/admin/market-data/*` |
| `ADMIN_FIREBASE_UIDS` | *(empty)* | Comma-separated Firebase UIDs allowed to call admin routes |

Requires `TRADING_ENABLED=true`, Alpaca credentials, and
`QUESTION_PIPELINE_TEST_MODE=false` for on-demand resync/cleanup when
`MARKET_HISTORY_SYNC_ENABLED=true`. When sync is disabled, the admin log and
week-coverage calendar remain available read-only; resync/cleanup return `503`.
Scheduled worker sync also requires `MARKET_HISTORY_SYNC_ENABLED=true`.

| Method | Path | Auth |
|--------|------|------|
| `GET` | `/v1/admin/market-history/sync-runs` | Admin Firebase UID |
| `POST` | `/v1/admin/market-data/resync` | Admin Firebase UID |
| `POST` | `/v1/admin/market-data/cleanup?archive=true\|false` | Admin Firebase UID |

`sync-runs` includes optional `config` when the server has market-history env loaded:

```json
{ "archiveEnabled": true, "windowDays": 5, "retentionDays": 5, "syncEnabled": true }
```

Flutter uses `config.archiveEnabled` to show the archive checkbox in the cleanup
confirm dialog.

### Admin portal tests

From the repo root:

```bash
./scripts/test-admin-portal.sh           # server + Flutter admin suites
./scripts/check-admin-portal-coverage.sh # coverage + Section 2 thresholds
```

CI: `.github/workflows/admin-portal.yml` runs both on admin-related changes.

## Flutter client

Run the app with the API URL (defaults to `http://localhost:3000`):

```bash
flutter run --dart-define=API_BASE_URL=http://localhost:3000
```