FEAT: Load Catagories from backend
Build, Push and Deploy CMS / build-push-deploy (push) Successful in 2m11s
Build, Push and Deploy CMS / build-push-deploy (push) Successful in 2m11s
This commit is contained in:
@@ -21,7 +21,8 @@ deployable.
|
||||
### cms (Go 1.25, module `thamanyah/cms/v2`)
|
||||
```bash
|
||||
cd cms
|
||||
go run . # serves on :8081
|
||||
go run . # serves on :8081 (requires DB_*/S3_*/MEDIACONVERT_* env vars — see below)
|
||||
go run . migrate # applies pending DB migrations, then exits (no HTTP server)
|
||||
go build ./...
|
||||
go vet ./...
|
||||
```
|
||||
@@ -49,14 +50,25 @@ state and Pulumi state isn't safe to update concurrently with CI.
|
||||
|
||||
### cms service
|
||||
Plain `net/http` (Go 1.22+ pattern-based `ServeMux`), no framework. Entry
|
||||
point `cms/main.go` wires up AWS SDK v2 clients (S3, MediaConvert) from env
|
||||
vars and assigns them to package-level interface vars in `internal/services`
|
||||
(`services.S3Client`, `services.MediaConvertClient`) — handlers call through
|
||||
these interfaces, and `S3Concrete`/`MediaConvertConcrete` are the only
|
||||
implementations, which is what makes the handlers testable even though no
|
||||
tests exist yet. On boot, `S3Concrete.AssertSuccessfulConnection` proactively
|
||||
exercises head/put/get/presign against the bucket and panics on failure,
|
||||
rather than letting the service come up in a broken state.
|
||||
point `cms/main.go` wires up AWS SDK v2 clients (S3, MediaConvert) and a
|
||||
Postgres connection from env vars and assigns them to package-level interface
|
||||
vars in `internal/services` (`services.S3Client`, `services.MediaConvertClient`,
|
||||
`services.DB`) — handlers call through these interfaces, and
|
||||
`S3Concrete`/`MediaConvertConcrete`/`DBConcrete` are the only implementations,
|
||||
which is what makes the handlers testable even though no tests exist yet. On
|
||||
boot, `S3Concrete.AssertSuccessfulConnection` proactively exercises
|
||||
head/put/get/presign against the bucket, and `DBConcrete.AssertSuccessfulConnection`
|
||||
pings Postgres — both panic on failure rather than letting the service come
|
||||
up in a broken state.
|
||||
|
||||
`cms/main.go` has two entry paths, dispatched on `os.Args[1]`: the default
|
||||
path (`runServer`) boots the HTTP server; `./cms migrate` (`runMigrate`)
|
||||
only opens the DB connection, applies pending migrations via
|
||||
`cms/internal/db.Migrate` (golang-migrate, `iofs` source, SQL files embedded
|
||||
from `cms/internal/db/migrations/*.sql`), and exits — it does not touch
|
||||
S3/MediaConvert or start the server. This is run as its own ECS container
|
||||
before the main container starts (see infrastructure below), so `runServer`
|
||||
never runs migrations itself, only `AssertSuccessfulConnection`.
|
||||
|
||||
Routes (`cms/main.go`): `GET /`, `GET /health`, `GET /videos/new`,
|
||||
`POST /videos/presign`, `POST /videos`, static files under `/static/`.
|
||||
@@ -65,6 +77,23 @@ Views live in `cms/internal/views` (templ components) with a shared
|
||||
`layouts.Layout` wrapper; `types.go` holds view-model structs like
|
||||
`VideoMetadata` used by the upload-success page.
|
||||
|
||||
### Data model (Postgres, `cms/internal/db/migrations/`)
|
||||
- `videos` — one row per uploaded video: `title`, `description`, `tags`
|
||||
(free-text, comma-separated — not normalized), `file_name`, `storage_key`
|
||||
(the S3 key, unique), `mediaconvert_job_id`, `status` (written once as
|
||||
`"processing"` on insert — see Known gaps), `size_bytes`, timestamps.
|
||||
`id` is a `UUID` generated application-side as a UUIDv7
|
||||
(`uuid.NewV7()` in `services.DBConcrete.CreateVideo`) rather than via the
|
||||
column's own `DEFAULT gen_random_uuid()` (which generates v4 and is never
|
||||
actually relied on) — v7 keeps primary-key inserts roughly time-ordered,
|
||||
avoiding B-tree fragmentation as the table grows.
|
||||
- `categories` — a small fixed lookup table (`documentary`, `news`,
|
||||
`entertainment`, `podcast`, `other`), seeded by migration `0001`.
|
||||
- `video_categories` — join table (`video_id`, `category_id`, composite PK,
|
||||
`ON DELETE CASCADE`) added in migration `0002`: a video can belong to
|
||||
*multiple* categories, not just one. `CreateVideo` inserts the `videos` row
|
||||
and its `video_categories` links inside a single transaction.
|
||||
|
||||
### Video upload → transcode pipeline
|
||||
1. Browser calls `POST /videos/presign` → cms returns a presigned S3 `PUT`
|
||||
URL for `raw-uploads-bucket`, key `videos/<random-hex>.<ext>`.
|
||||
@@ -73,13 +102,17 @@ Views live in `cms/internal/views` (templ components) with a shared
|
||||
3. Browser calls `POST /videos` with the metadata + key → cms calls
|
||||
`MediaConvertClient.QueueEncodingJob(key)`, submitting a MediaConvert job
|
||||
`s3://raw-uploads-bucket/<key>` → `s3://encoded-bucket/<key>` (H.264/AAC →
|
||||
MP4, QVBR rate control — QVBR requires `MaxBitrate` to be set explicitly).
|
||||
MP4, QVBR rate control — QVBR requires `MaxBitrate` to be set explicitly),
|
||||
then `services.DB.CreateVideo` persists the `videos` row (status
|
||||
`"processing"`) and its `video_categories` links.
|
||||
4. Finished output lands in `encoded-bucket`, served via CloudFront.
|
||||
|
||||
Config wiring: cms reads `S3_BUCKET`, `MEDIACONVERT_INPUT_BUCKET`,
|
||||
`MEDIACONVERT_OUTPUT_BUCKET`, `MEDIACONVERT_ROLE_ARN`, `AWS_REGION` from env
|
||||
vars injected by the ECS task definition (`extraEnv` in
|
||||
`deployFargateService`, `infrastructure/main.go`).
|
||||
`MEDIACONVERT_OUTPUT_BUCKET`, `MEDIACONVERT_ROLE_ARN`, `AWS_REGION`,
|
||||
`DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` from env vars
|
||||
injected by the ECS task definition (`extraEnv` and the unconditional DB_*
|
||||
vars in `deployFargateService`, `infrastructure/main.go`) — `main.go` panics
|
||||
on boot if any required var is empty.
|
||||
|
||||
### infrastructure (`infrastructure/main.go`, single Pulumi Go program, region `us-east-1`, stack `main`)
|
||||
- **Postgres**: one shared RDS instance (`db.t3.micro`, single-AZ, no
|
||||
@@ -91,7 +124,17 @@ vars injected by the ECS task definition (`extraEnv` in
|
||||
ports). `deployFargateService(...)` is the shared helper building a
|
||||
service's ECR repo, CloudWatch log group, task definition, ECS service,
|
||||
and ALB. `taskRole` is optional (nil = no AWS identity beyond the shared
|
||||
execution role); `extraEnv` appends container env vars beyond the DB_* set.
|
||||
execution role); `extraEnv` appends container env vars beyond the DB_* set;
|
||||
`runMigrations` (true for `cms`, false for `discovery`) adds a second,
|
||||
non-essential `<name>-migrate` container to the task — same image,
|
||||
`command: ["migrate"]` — with the main container's `dependsOn` set to
|
||||
`condition: "COMPLETE"` on it. This is ECS's container-dependency
|
||||
mechanism, the Fargate equivalent of a Kubernetes init container: ECS runs
|
||||
the migrate container to completion (exit 0) before starting the main
|
||||
container, so schema migrations always finish before the service accepts
|
||||
traffic. No separate CI/Docker migration step exists — `Dockerfile`'s
|
||||
`ENTRYPOINT ["./cms"]` plus the container's `command` override composes to
|
||||
`./cms migrate`.
|
||||
- **S3 + CloudFront**: `encoded-bucket` holds finished transcoded output,
|
||||
served publicly via CloudFront using Origin Access Control (OAC) — the
|
||||
bucket itself blocks all public access; only CloudFront's OAC principal
|
||||
@@ -144,6 +187,9 @@ vars injected by the ECS task definition (`extraEnv` in
|
||||
- `cms/internal/services/mediaconvert.go` never calls `DescribeEndpoints`
|
||||
and configures no custom MediaConvert endpoint — relies on the SDK's
|
||||
default regional endpoint.
|
||||
- No MediaConvert completion webhook or poller exists — a `videos.status`
|
||||
row is written once as `"processing"` in `CreateVideo` and never updated,
|
||||
even after the transcode job actually finishes or fails.
|
||||
- `discovery` has infra provisioned (ECR repo, ECS service, ALB, Postgres
|
||||
DB/role) but no application code.
|
||||
DB/role) but no application code — it doesn't touch its database at all.
|
||||
- No automated tests exist for `cms`, `discovery`, or `infrastructure`.
|
||||
|
||||
Reference in New Issue
Block a user