# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Repo layout

```
.
├── cms/            Go service: video ingestion/CMS (implemented)
├── discovery/      Go service: catalogue discovery/read side (skeleton)
├── infrastructure/ Pulumi (Go) program provisioning all AWS resources
├── tests/          Go module: Gherkin/gobdd scenarios run against a live cms
└── .gitea/workflows/  Gitea Actions CI/CD pipelines
```

`discovery` is the read side. It owns no ingestion: it learns what exists by
draining `discovery-catalogue-events`, the queue subscribed to the catalogue
topic `cms` announces ready videos on, and keeps its own copy in its own
database. Neither service reads the other's tables and nothing polls `cms`.

It serves `GET /health`, `GET /api/videos/{id}` and the Swagger UI. Migration
`0001_init` still creates nothing (it predates the domain and exists only
because `internal/db/migrate.go` embeds `migrations/*.sql`, which will not
compile against an empty directory); `0002_create_videos_table` is where the
schema actually starts. It now has an `internal/services` (SQS only) and an
`internal/consumers`, and a task role scoped to that one queue — receive,
delete, get-attributes, and nothing else. It cannot publish back onto the
topic: it is a subscriber, not a participant.

## Commands

### cms (Go 1.25, module `thamanyah/cms/v2`)

```bash
cd cms
go run .                    # serves on :8081 (requires DB_*/S3_*/MEDIACONVERT_* env vars — see below)
go run . migrate            # applies pending DB migrations, then exits (no HTTP server)
go build ./...
go vet ./...
```

### discovery (Go 1.25, module `thamanyah/discovery`)

Same commands as `cms`, and the same two entry paths (`runServer`,
`runMigrate`) in `discovery/main.go` — only the port and the dependencies
differ. Note the module path carries no `/v2`: unlike `cms`, this module has
never had a v1.

```bash
cd discovery
go run .                    # serves on :8080 (requires DB_* plus AWS_REGION/CATALOGUE_EVENTS_QUEUE_URL — no S3/MediaConvert)
go run . migrate            # applies pending DB migrations, then exits (no HTTP server)
go build ./...
go vet ./...
```

Its OpenAPI spec is generated into `discovery/docs` by the same `swag init`
invocation as `cms`, run from `discovery/`. In `docker-compose.yml` it is a
service of its own on `127.0.0.1:8080`, wired to the `discovery` database and
role LocalStack provisions, plus the AWS_* vars its consumer needs — and, like
`cms`, started in server mode only, so a freshly created local database needs
`docker exec discovery ./discovery migrate` once. Until that runs, the consumer
logs `relation "videos" does not exist` per announcement and leaves them on the
queue; they are picked up once the table exists.

`discovery`'s data model is one table, `videos`: the catalogue's copy of an
announced video, keyed by **the id `cms` issued** (no `DEFAULT` — the id
arrives in the announcement and is reused verbatim, so both services name the
same video the same way). `categories` is a denormalized `TEXT[]` of names:
`cms` owns the vocabulary and announces names, so there is nothing to join to.
`repositories.SaveVideo` is an **upsert** — the topic delivers at-least-once,
so the same announcement can arrive twice, and a plain `INSERT` would fail on
the second, wedging every message queued behind it. A later announcement wins.

`cms`, `discovery` and `infrastructure` hold no test files of their own. The
only tests in the repo are the black-box BDD scenarios in `tests/` — see below.

`cms` is a JSON API only — it serves no HTML and has no static assets. The
OpenAPI spec is generated from swaggo annotations on the handlers into
`cms/docs`, a committed, compiled-in Go package. If you add or change a
handler, its annotation comments, or a request/response struct, regenerate it:

```bash
go install github.com/swaggo/swag/cmd/swag@v1.16.6   # match go.mod; not installed by default
swag init --generalInfo main.go --dir ./ --parseInternal --output ./docs
```

`--parseInternal` is required — the handlers live under `internal/`, which
swag skips without it. Keep the `swaggo/swag` version in `go.mod` and the CLI
in lockstep: `http-swagger/v2` transitively pulls a much older `swag` whose
`swag.Spec` struct lacks the `LeftDelim`/`RightDelim` fields newer generators
emit, and the build breaks outright if the two drift. (The CLI's `--version`
misreports itself as v1.16.4; `go version -m $(go env GOPATH)/bin/swag` gives
the real one.)

### tests (Go, module `thamanyah/tests`)

```bash
cd tests
go test ./...               # runs features/*.feature against CMS_BASE_URL (default http://localhost:8081)
go test -v ./...            # -v prints the Gherkin: gobdd nests a subtest per feature/scenario/step
CMS_BASE_URL=… go test ./...
```

Black-box BDD covering the video upload feature: its own Go module importing
nothing from `cms/`, talking to a running service over HTTP only, so the same
scenarios run against compose and against a deployed environment. Skips (does
not fail) when nothing is serving. Uploading is a plain `PUT` to the presigned
URL, no AWS SDK. Scenarios can be tagged `@known-gap` to pin current behaviour
that differs from the documented contract — see `tests/README.md`.

Note compose starts `cms` in server mode only: the migrate container exists in
the ECS task definition, not in `docker-compose.yml`, so a freshly created
local database needs `docker exec cms ./cms migrate` once or every scenario
fails on `relation "categories" does not exist`.

### infrastructure (Go, Pulumi, module `thamanyah`)

```bash
cd infrastructure
pulumi preview               # plan changes against stack "main"
pulumi up                    # apply — this touches real AWS resources, confirm with the user first
pulumi stack output          # e.g. ecsClusterArn, cmsServiceArn
```

Deploys run in CI (`.gitea/workflows/infrastructure-deploy.yml`) on push to
`main` touching `infrastructure/**`. Treat local `pulumi up` as something to
confirm with the user, not a routine dev command — it mutates shared cloud
state and Pulumi state isn't safe to update concurrently with CI.

## Architecture

### cms package layout

Four `internal` packages, split by the kind of thing they hold — keep new code
on the same seams:

| package | holds |
|---|---|
| `internal/api` | the **wire contract**: request/response structs with their JSON + swaggo tags. No logic. |
| `internal/models` | the **domain types** the rest of the code passes around: `Video`, `Category`, `VideoStatus`. No JSON tags, no SQL. |
| `internal/handlers` | HTTP: decode `api.*`, validate, call repositories/services, encode `api.*`. |
| `internal/db` | the `*sql.DB` connection lifecycle and migrations; `internal/db/repositories` holds all SQL. |
| `internal/services` | **AWS only** — S3, MediaConvert and SQS. Nothing database-related lives here. |

`api` and `models` are deliberately separate types even where the fields look
alike: `api.CompleteResponse` is the published schema, `models.Video` is the
row. Handlers translate between them field by field, so renaming a `models`
field doesn't move the published API and vice versa.

### cms service

Plain `net/http` (Go 1.22+ pattern-based `ServeMux`), no framework. Entry
point `cms/main.go` builds the dependencies and assigns them to package-level
interface vars that the handlers call through:

- `services.S3Client` (`services.S3`) ← `*services.S3Concrete`
- `services.MediaConvertClient` (`services.MediaConvert`) ← `*services.MediaConvertConcrete`
- `repositories.VideoRepo` (`repositories.VideoRepository`) ← `*repositories.ConcreteVideoRepository`
- `repositories.CatagoriesRepo` (`repositories.CatagoriesRepository`) ← `*repositories.ConcreteCatagoriesRepository`

Each interface has exactly one implementation; the indirection is what makes
the handlers substitutable in tests, even though no tests exist yet. Note the
spelling: the categories repository is `Catagories`/`CatagoriesRepo` in
`repositories/catagories.go` (and `handlers.validateCatagoryIDS`) — the
misspelling is load-bearing for compilation, so match it rather than "fixing"
it piecemeal.

On boot, `S3Concrete.AssertSuccessfulConnection` proactively exercises
head-bucket/put/get/presign against the bucket (writing a throwaway
`.s3-connectivity-check` object), and `db.AssertSuccessfulConnection` pings
Postgres — both panic on failure rather than letting the service come up in a
broken state.

`cms/main.go` has two entry paths, dispatched on `os.Args[1]`: the default
path (`runServer`) boots the HTTP server; `./cms migrate` (`runMigrate`)
only opens the DB connection, applies pending migrations via
`cms/internal/db.Migrate` (golang-migrate, `iofs` source, SQL files embedded
from `cms/internal/db/migrations/*.sql`), and exits — it does not touch
S3/MediaConvert or start the server. This is run as its own ECS container
before the main container starts (see infrastructure below), so `runServer`
never runs migrations itself, only `AssertSuccessfulConnection`.

The DB connection is opened once for the process lifetime by
`db.CreateDBConnection(connString)` and closed by `db.CloseConnection` — both
plain functions over `*sql.DB` in `internal/db/client.go`, not methods on a
wrapper type. Repositories take that `*sql.DB` as their `SQLDB` field.

Routes (`cms/main.go`): `GET /health`, `GET /api/categories`,
`POST /api/videos/presign`, `POST /api/videos`, plus Swagger UI at
`GET /swagger/` (`/swagger/doc.json` serves the spec). The UI assets are
embedded in the binary by `swaggo/files`, so nothing is read from disk and
nothing is fetched from a CDN at runtime.

Handlers live in `cms/internal/handlers` — `handlers.go` holds `Health` and
the shared response writers, `videos.go` the categories and upload endpoints.
The wire structs they serve live in `cms/internal/api` (`api.go` for
`HealthResponse` and `ProblemDetails`, `videos.go` for `Category`,
`CategoriesResponse`, `PresignRequest`/`PresignResponse` and
`CompleteRequest`/`CompleteResponse`), referenced by the handlers' swaggo
annotations as `api.CompleteResponse` and so on — so the generated spec's
definition names track that package, and renaming a type there changes the
published schema names.

There is no view layer: the `internal/views` templ package, the `static/`
directory, and the htmx frontend were all removed when the service became a
JSON API, along with the `templ` dependency.

Error responses are RFC 9457 Problem Details objects
(`application/problem+json`), written by
`writeProblem(w, status, title, detail)`. `type` is always `"about:blank"`;
`title` is a short summary held identical across every occurrence of a given
problem, so clients can branch on it; `detail` is the only member that varies
with request data. Success responses go through `writeJSON`
(`application/json`). Both share `writeJSONContent`. Note this deviates
slightly from RFC 9457, which pairs an `about:blank` type with a title that is
just the HTTP status phrase — meaningful titles like these are supposed to
carry a real `type` URI. Adding per-problem type URIs is the conforming fix if
it ever matters.

### Data model (Postgres, `cms/internal/db/migrations/`, two migrations: `0001`, `0002`)

- `videos` — one row per uploaded video: `title`, `description`, `tags`
  (free-text, comma-separated — not normalized), `file_name`, `storage_key`
  (the S3 key, unique), `mediaconvert_job_id`, `status` (written once as
  `"processing"` on insert — see Known gaps), `size_bytes`, timestamps.
  `id` is a `UUID` filled by the column's own `DEFAULT gen_random_uuid()`
  (v4) and read back through `RETURNING id` — the application does not
  generate it. (An earlier revision generated UUIDv7 application-side; that
  was reverted in `785154c`, so primary keys are random, not time-ordered.)
  `status` is a plain `TEXT NOT NULL DEFAULT 'processing'` column with **no
  CHECK constraint** — the closed set exists only in Go, as the
  `models.VideoStatus` string type in `cms/internal/models/video.go`
  (`processing`, `ready`, `failed`). Extending it means adding a constant
  there and extending the `enums` annotation on `api.CompleteResponse.Status`
  before regenerating the spec. (The doc comment on `VideoStatus` claims a
  `videos_status_check` constraint enforces it in the database — that
  constraint does not exist; nothing has ever created it.)
- `categories` — a small fixed lookup table (`documentary`, `news`,
  `entertainment`, `podcast`, `other`), seeded by migration `0001`. Its
  `SMALLSERIAL` ids are part of the public API: `GET /api/categories` returns
  `{id, name}` pairs (ordered by name, not id) and `POST /api/videos` takes
  `categoryIds`, so the seed order in migration `0001` is what fixes which id
  means which name — never renumber it.
- `video_categories` — join table (`video_id`, `category_id`, composite PK,
  `ON DELETE CASCADE`) added in migration `0002`: a video can belong to
  *multiple* categories, not just one. `ConcreteVideoRepository.CreateVideo`
  inserts the `videos` row and its `video_categories` links inside a single
  transaction, taking the category ids as given — `handlers.validateCatagoryIDS`
  is what checks them against `CatagoriesRepo.ListCategoriesIDs` before the
  transcode job is queued, leaving the foreign key and the composite PK as
  backstops (see Known gaps for how those failures surface).

### Video upload → transcode pipeline

1. Client calls `POST /api/videos/presign` → cms returns a presigned S3 `PUT`
   URL for `raw-uploads-bucket`, key `videos/<random-hex>.<ext>`. The
   submitted `contentType` (only `video/mp4` or `video/quicktime`) is signed
   into the URL, so the client's `PUT` must send the identical header.
2. Client `PUT`s the file directly to S3 (from a browser this requires the
   bucket's CORS rule, set up in `infrastructure/main.go`,
   its allowed origin is now stale). The file never passes through cms.
3. Client calls `POST /api/videos` with the metadata (categories given as
   `categoryIds` from `GET /api/categories`) + key → cms validates the
   category ids, calls `MediaConvertClient.QueueEncodingJob(key)`, submitting
   a MediaConvert job `s3://raw-uploads-bucket/<key>` →
   `s3://encoded-bucket/<key>` (H.264/AAC → MP4, QVBR rate control — QVBR
   requires `MaxBitrate` to be set explicitly), then
   `repositories.VideoRepo.CreateVideo` persists the `videos` row (status
   `"processing"`) and its `video_categories` links.
4. Finished output lands in `encoded-bucket`, served via CloudFront.
5. MediaConvert reports the job's state changes to an SNS topic that fans out
   to `cms-mediaconvert-events`, which `internal/consumers` reads for the
   lifetime of the process: it records the outcome (`ready`/`failed`) and the
   playback URL, rewriting the `s3://` playlist path onto `PLAYBACK_BASE_URL`.
   A job that came out **ready** is then announced on the `catalogue-events`
   SNS topic via `services.CatalogueClient` — `{videoId, title, playbackUrl,
   categories}`, categories by **name** so a subscriber needs nothing from
   cms to interpret them. Failed jobs are recorded but never announced. A
   topic, not a queue, so the announcement fans out: `discovery-catalogue-events`
   subscribes today (raw delivery, so there is no SNS envelope to unwrap), and
   a second reader can subscribe its own queue without cms changing. Note a
   topic only delivers to subscriptions that exist when it publishes, so a new
   subscriber's queue has to be in place before the announcement, not after.
   Delivery is at-least-once: a failed publish leaves the *job* event on the
   consumer's queue, and the outcome update is idempotent, so a redelivery
   re-announces and a subscriber can see a video twice.

Config wiring: cms reads `S3_BUCKET`, `MEDIACONVERT_INPUT_BUCKET`,
`MEDIACONVERT_OUTPUT_BUCKET`, `MEDIACONVERT_ROLE_ARN`, `AWS_REGION`,
`MEDIACONVERT_EVENTS_QUEUE_URL`, `CATALOGUE_EVENTS_TOPIC_ARN`,
`PLAYBACK_BASE_URL`,
`DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` from env vars
injected by the ECS task definition (`extraEnv` and the unconditional DB_*
vars in `deployFargateService`, `infrastructure/main.go`) — `main.go` panics
on boot if any required var is empty.

### infrastructure (`infrastructure/main.go`, single Pulumi Go program, region `us-east-1`, stack `main`)

- **Postgres**: one shared RDS instance (`db.t3.micro`, single-AZ, no
  backups — intentionally minimal). Each app (`cms`, `discovery`) gets its
  own login role and same-named database via the `postgresql` provider
  (`newServiceDatabase`), so services never share DB credentials.
- **ECS Fargate**: one cluster (`app-cluster`), one ALB *per service* (each
  gets its own DNS name rather than sharing a load balancer on different
  ports). `deployFargateService(...)` is the shared helper building a
  service's ECR repo, CloudWatch log group, task definition, ECS service,
  and ALB. `taskRole` is optional (nil = no AWS identity beyond the shared
  execution role); `extraEnv` appends container env vars beyond the DB_* set;
  `runMigrations` (true for both services) adds a second,
  non-essential `<name>-migrate` container to the task — same image,
  `command: ["migrate"]` — with the main container's `dependsOn` set to
  `condition: "COMPLETE"` on it. This is ECS's container-dependency
  mechanism, the Fargate equivalent of a Kubernetes init container: ECS runs
  the migrate container to completion (exit 0) before starting the main
  container, so schema migrations always finish before the service accepts
  traffic. No separate CI/Docker migration step exists — `Dockerfile`'s
  `ENTRYPOINT ["./cms"]` plus the container's `command` override composes to
  `./cms migrate`.
- **S3 + CloudFront**: `encoded-bucket` holds finished transcoded output,
  served publicly via CloudFront using Origin Access Control (OAC) — the
  bucket itself blocks all public access; only CloudFront's OAC principal
  can read it.
- **S3 raw uploads**: `raw-uploads-bucket` is a separate, private bucket for
  pre-transcode uploads — deliberately kept apart from `encoded-bucket` so
  raw source video is never reachable through the public CDN. CORS is
  scoped to `PUT` only, from the `cms` ALB's own origin — correct back when
  cms served the upload page itself, but stale now that it serves no UI (see
  Known gaps).
- **IAM roles** — five distinct roles/users, each scoped narrowly, don't
  conflate them:
  - `ecs-task-execution-role` — shared by both services' ECS *agent* (image
    pull, log write, Secrets Manager read for DB password). Not usable by
    application code inside the container.
  - `cms-task-role` — the `cms` container's own AWS identity: S3
    `ListBucket`/`PutObject`/`GetObject` on `raw-uploads-bucket` only,
    `mediaconvert:CreateJob`, `iam:PassRole` scoped to the MediaConvert
    service role (`iam:PassedToService` condition), receive/delete on
    `cms-mediaconvert-events`, and `sns:Publish` on the `catalogue-events`
    topic — the only thing it writes to. It has no access to
    `discovery-catalogue-events`, the queue subscribed to that topic: cms
    publishes, it does not reach into a subscriber.
  - `discovery-task-role` — the `discovery` container's own AWS identity, and
    its only one: receive/delete/get-attributes on
    `discovery-catalogue-events`. No S3, no MediaConvert, and no `sns:Publish`
    — it consumes the catalogue, it does not add to it.
  - `mediaconvert-service-role` — trusted by `mediaconvert.amazonaws.com`,
    not by ECS; the role MediaConvert itself assumes (passed as
    `CreateJobInput.Role`) to read `raw-uploads-bucket` and write
    `encoded-bucket`. Distinct from `cms-task-role` by design: one is "cms
    calling AWS", the other is "AWS calling AWS on cms's behalf".
  - `gitea-ci-user` — an IAM **user** (static access keys, not OIDC — the
    Gitea Actions runner doesn't support instance-profile auth) scoped to
    just ECR push (`cms`/`discovery` repos) and
    `ecs:UpdateService`/`ecs:DescribeServices` on the two ECS services.
    Never broaden this to `ecr:*`/`ecs:*`.

### CI/CD (`.gitea/workflows/`)

- **`cms-deploy.yml`**: push to `main` touching `cms/**`. Builds/pushes the
  Docker image to ECR using `gitea-ci-user`, installs the AWS CLI (not
  preinstalled on the runner image — via AWS's official install script, not
  a third-party action), then `aws ecs update-service --force-new-deployment`.
  Cluster/service ARNs come from `pulumi stack output` and are set as repo
  *variables* (not secrets — ARNs aren't sensitive).
- **`discovery-deploy.yml`**: the same pipeline for `discovery`, on pushes
  touching `discovery/**`. One difference: the ECS service ARN is read from
  the `DISCOVERY_ECS_SERVICE_ARN` repo variable rather than pinned in the
  workflow, so it has to be set (from `pulumi stack output
  discoveryServiceArn`) before the first deploy can succeed.
- **`infrastructure-deploy.yml`**: push to `main` touching
  `infrastructure/**`. Runs `pulumi up` using a separate, broader AWS
  credential (`PULUMI_AWS_ACCESS_KEY_ID`/`SECRET`) than `gitea-ci-user`,
  since provisioning IAM/RDS/ECS/CloudFront needs wider permissions than
  pushing images and forcing deployments. Guarded with a concurrency group
  since Pulumi state isn't safe to update concurrently.
- The runner's `ubuntu-latest` label maps to a docker image configured on
  the runner host (outside this repo) — currently minimal, lacking the AWS
  CLI, hence the manual install step in `cms-deploy.yml`.