FEAT: Added Database To CMS Serbove
Build, Push and Deploy CMS / build-push-deploy (push) Successful in 2m17s
Build, Push and Deploy CMS / build-push-deploy (push) Successful in 2m17s
This commit is contained in:
@@ -0,0 +1,149 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Repo layout
|
||||
|
||||
```
|
||||
.
|
||||
├── cms/ Go service: video ingestion/CMS (implemented)
|
||||
├── discovery/ Go service: empty placeholder, no code yet
|
||||
├── infrastructure/ Pulumi (Go) program provisioning all AWS resources
|
||||
└── .gitea/workflows/ Gitea Actions CI/CD pipelines
|
||||
```
|
||||
|
||||
`discovery` is provisioned in infra (its own ECR repo, ECS service, ALB,
|
||||
Postgres database/role) but has no application code yet — don't assume it's
|
||||
deployable.
|
||||
|
||||
## Commands
|
||||
|
||||
### cms (Go 1.25, module `thamanyah/cms/v2`)
|
||||
```bash
|
||||
cd cms
|
||||
go run . # serves on :8081
|
||||
go build ./...
|
||||
go vet ./...
|
||||
```
|
||||
There are no test files in this repo (`cms`, `discovery`, or `infrastructure`) — don't assume a test suite exists.
|
||||
|
||||
Views are written as `.templ` files (github.com/a-h/templ) and compiled to
|
||||
`*_templ.go`. If you edit a `.templ` file, regenerate its Go code with the
|
||||
`templ generate` CLI before building (not installed in this environment by
|
||||
default — install via `go install github.com/a-h/templ/cmd/templ@v0.3.1020`
|
||||
to match `go.mod`, or check for an existing binary first).
|
||||
|
||||
### infrastructure (Go, Pulumi, module `thamanyah`)
|
||||
```bash
|
||||
cd infrastructure
|
||||
pulumi preview # plan changes against stack "main"
|
||||
pulumi up # apply — this touches real AWS resources, confirm with the user first
|
||||
pulumi stack output # e.g. ecsClusterArn, cmsServiceArn
|
||||
```
|
||||
Deploys run in CI (`.gitea/workflows/infrastructure-deploy.yml`) on push to
|
||||
`main` touching `infrastructure/**`. Treat local `pulumi up` as something to
|
||||
confirm with the user, not a routine dev command — it mutates shared cloud
|
||||
state and Pulumi state isn't safe to update concurrently with CI.
|
||||
|
||||
## Architecture
|
||||
|
||||
### cms service
|
||||
Plain `net/http` (Go 1.22+ pattern-based `ServeMux`), no framework. Entry
|
||||
point `cms/main.go` wires up AWS SDK v2 clients (S3, MediaConvert) from env
|
||||
vars and assigns them to package-level interface vars in `internal/services`
|
||||
(`services.S3Client`, `services.MediaConvertClient`) — handlers call through
|
||||
these interfaces, and `S3Concrete`/`MediaConvertConcrete` are the only
|
||||
implementations, which is what makes the handlers testable even though no
|
||||
tests exist yet. On boot, `S3Concrete.AssertSuccessfulConnection` proactively
|
||||
exercises head/put/get/presign against the bucket and panics on failure,
|
||||
rather than letting the service come up in a broken state.
|
||||
|
||||
Routes (`cms/main.go`): `GET /`, `GET /health`, `GET /videos/new`,
|
||||
`POST /videos/presign`, `POST /videos`, static files under `/static/`.
|
||||
|
||||
Views live in `cms/internal/views` (templ components) with a shared
|
||||
`layouts.Layout` wrapper; `types.go` holds view-model structs like
|
||||
`VideoMetadata` used by the upload-success page.
|
||||
|
||||
### Video upload → transcode pipeline
|
||||
1. Browser calls `POST /videos/presign` → cms returns a presigned S3 `PUT`
|
||||
URL for `raw-uploads-bucket`, key `videos/<random-hex>.<ext>`.
|
||||
2. Browser `PUT`s the file directly to S3 (requires the bucket's CORS rule,
|
||||
set up in `infrastructure/main.go`).
|
||||
3. Browser calls `POST /videos` with the metadata + key → cms calls
|
||||
`MediaConvertClient.QueueEncodingJob(key)`, submitting a MediaConvert job
|
||||
`s3://raw-uploads-bucket/<key>` → `s3://encoded-bucket/<key>` (H.264/AAC →
|
||||
MP4, QVBR rate control — QVBR requires `MaxBitrate` to be set explicitly).
|
||||
4. Finished output lands in `encoded-bucket`, served via CloudFront.
|
||||
|
||||
Config wiring: cms reads `S3_BUCKET`, `MEDIACONVERT_INPUT_BUCKET`,
|
||||
`MEDIACONVERT_OUTPUT_BUCKET`, `MEDIACONVERT_ROLE_ARN`, `AWS_REGION` from env
|
||||
vars injected by the ECS task definition (`extraEnv` in
|
||||
`deployFargateService`, `infrastructure/main.go`).
|
||||
|
||||
### infrastructure (`infrastructure/main.go`, single Pulumi Go program, region `us-east-1`, stack `main`)
|
||||
- **Postgres**: one shared RDS instance (`db.t3.micro`, single-AZ, no
|
||||
backups — intentionally minimal). Each app (`cms`, `discovery`) gets its
|
||||
own login role and same-named database via the `postgresql` provider
|
||||
(`newServiceDatabase`), so services never share DB credentials.
|
||||
- **ECS Fargate**: one cluster (`app-cluster`), one ALB *per service* (each
|
||||
gets its own DNS name rather than sharing a load balancer on different
|
||||
ports). `deployFargateService(...)` is the shared helper building a
|
||||
service's ECR repo, CloudWatch log group, task definition, ECS service,
|
||||
and ALB. `taskRole` is optional (nil = no AWS identity beyond the shared
|
||||
execution role); `extraEnv` appends container env vars beyond the DB_* set.
|
||||
- **S3 + CloudFront**: `encoded-bucket` holds finished transcoded output,
|
||||
served publicly via CloudFront using Origin Access Control (OAC) — the
|
||||
bucket itself blocks all public access; only CloudFront's OAC principal
|
||||
can read it.
|
||||
- **S3 raw uploads**: `raw-uploads-bucket` is a separate, private bucket for
|
||||
pre-transcode uploads — deliberately kept apart from `encoded-bucket` so
|
||||
raw source video is never reachable through the public CDN. CORS is
|
||||
scoped to `PUT` only, from the `cms` ALB's own origin (browser uploads
|
||||
directly via presigned URL).
|
||||
- **IAM roles** — four distinct roles/users, each scoped narrowly, don't
|
||||
conflate them:
|
||||
- `ecs-task-execution-role` — shared by both services' ECS *agent* (image
|
||||
pull, log write, Secrets Manager read for DB password). Not usable by
|
||||
application code inside the container.
|
||||
- `cms-task-role` — the `cms` container's own AWS identity: S3
|
||||
`ListBucket`/`PutObject`/`GetObject` on `raw-uploads-bucket` only,
|
||||
`mediaconvert:CreateJob`, and `iam:PassRole` scoped to the MediaConvert
|
||||
service role (`iam:PassedToService` condition). `discovery` has no task
|
||||
role — it doesn't touch S3 or MediaConvert.
|
||||
- `mediaconvert-service-role` — trusted by `mediaconvert.amazonaws.com`,
|
||||
not by ECS; the role MediaConvert itself assumes (passed as
|
||||
`CreateJobInput.Role`) to read `raw-uploads-bucket` and write
|
||||
`encoded-bucket`. Distinct from `cms-task-role` by design: one is "cms
|
||||
calling AWS", the other is "AWS calling AWS on cms's behalf".
|
||||
- `gitea-ci-user` — an IAM **user** (static access keys, not OIDC — the
|
||||
Gitea Actions runner doesn't support instance-profile auth) scoped to
|
||||
just ECR push (`cms`/`discovery` repos) and
|
||||
`ecs:UpdateService`/`ecs:DescribeServices` on the two ECS services.
|
||||
Never broaden this to `ecr:*`/`ecs:*`.
|
||||
|
||||
### CI/CD (`.gitea/workflows/`)
|
||||
- **`cms-deploy.yml`**: push to `main` touching `cms/**`. Builds/pushes the
|
||||
Docker image to ECR using `gitea-ci-user`, installs the AWS CLI (not
|
||||
preinstalled on the runner image — via AWS's official install script, not
|
||||
a third-party action), then `aws ecs update-service --force-new-deployment`.
|
||||
Cluster/service ARNs come from `pulumi stack output` and are set as repo
|
||||
*variables* (not secrets — ARNs aren't sensitive).
|
||||
- **`infrastructure-deploy.yml`**: push to `main` touching
|
||||
`infrastructure/**`. Runs `pulumi up` using a separate, broader AWS
|
||||
credential (`PULUMI_AWS_ACCESS_KEY_ID`/`SECRET`) than `gitea-ci-user`,
|
||||
since provisioning IAM/RDS/ECS/CloudFront needs wider permissions than
|
||||
pushing images and forcing deployments. Guarded with a concurrency group
|
||||
since Pulumi state isn't safe to update concurrently.
|
||||
- The runner's `ubuntu-latest` label maps to a docker image configured on
|
||||
the runner host (outside this repo) — currently minimal, lacking the AWS
|
||||
CLI, hence the manual install step in `cms-deploy.yml`.
|
||||
|
||||
## Known gaps
|
||||
|
||||
- `cms/internal/services/mediaconvert.go` never calls `DescribeEndpoints`
|
||||
and configures no custom MediaConvert endpoint — relies on the SDK's
|
||||
default regional endpoint.
|
||||
- `discovery` has infra provisioned (ECR repo, ECS service, ALB, Postgres
|
||||
DB/role) but no application code.
|
||||
- No automated tests exist for `cms`, `discovery`, or `infrastructure`.
|
||||
@@ -8,6 +8,9 @@ require (
|
||||
github.com/aws/aws-sdk-go-v2/config v1.32.36
|
||||
github.com/aws/aws-sdk-go-v2/service/mediaconvert v1.97.2
|
||||
github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1
|
||||
github.com/golang-migrate/migrate/v4 v4.19.1
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/lib/pq v1.12.3
|
||||
)
|
||||
|
||||
require (
|
||||
|
||||
+64
@@ -1,3 +1,7 @@
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
|
||||
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
|
||||
github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
|
||||
github.com/aws/aws-sdk-go-v2 v1.43.5 h1:yKT5GYnFWhuDo+DqKvE5ZPwVn3RjC4MAeBtZGlh6AVM=
|
||||
@@ -38,5 +42,65 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 h1:eQ5BtXDrPg2wK0AjtVPzeBhUpYPe
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.45.5/go.mod h1:f9ImhnOISY7BuTZLM8qHepCYnglHBVLk5wVzatmP++w=
|
||||
github.com/aws/smithy-go v1.27.7 h1:Zgj5z4LfcDYoQIVk+n/yGdTkP/2y6ZT5vYxe0fp7bqE=
|
||||
github.com/aws/smithy-go v1.27.7/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
|
||||
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
|
||||
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
|
||||
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
|
||||
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dhui/dktest v0.4.6 h1:+DPKyScKSEp3VLtbMDHcUq6V5Lm5zfZZVb0Sk7Ahom4=
|
||||
github.com/dhui/dktest v0.4.6/go.mod h1:JHTSYDtKkvFNFHJKqCzVzqXecyv+tKt8EzceOmQOgbU=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
|
||||
github.com/docker/docker v28.3.3+incompatible h1:Dypm25kh4rmk49v1eiVbsAtpAsYURjYkaKubwuBdxEI=
|
||||
github.com/docker/docker v28.3.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
|
||||
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
|
||||
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang-migrate/migrate/v4 v4.19.1 h1:OCyb44lFuQfYXYLx1SCxPZQGU7mcaZ7gH9yH4jSFbBA=
|
||||
github.com/golang-migrate/migrate/v4 v4.19.1/go.mod h1:CTcgfjxhaUtsLipnLoQRWCrjYXycRz/g5+RWDuYgPrE=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
|
||||
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
|
||||
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
|
||||
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
|
||||
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
|
||||
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
|
||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
|
||||
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
|
||||
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
|
||||
go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
|
||||
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
|
||||
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
|
||||
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
|
||||
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
|
||||
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"embed"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
"github.com/golang-migrate/migrate/v4/database/postgres"
|
||||
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrationsFS embed.FS
|
||||
|
||||
func Migrate(sqlDB *sql.DB) error {
|
||||
driver, err := postgres.WithInstance(sqlDB, &postgres.Config{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("db: creating migration driver: %w", err)
|
||||
}
|
||||
|
||||
source, err := iofs.New(migrationsFS, "migrations")
|
||||
if err != nil {
|
||||
return fmt.Errorf("db: loading embedded migrations: %w", err)
|
||||
}
|
||||
|
||||
m, err := migrate.NewWithInstance("iofs", source, "postgres", driver)
|
||||
if err != nil {
|
||||
return fmt.Errorf("db: initializing migrator: %w", err)
|
||||
}
|
||||
|
||||
if err := m.Up(); err != nil && !errors.Is(err, migrate.ErrNoChange) {
|
||||
return fmt.Errorf("db: applying migrations: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
DROP TABLE videos;
|
||||
DROP TABLE categories;
|
||||
@@ -0,0 +1,26 @@
|
||||
CREATE TABLE categories (
|
||||
id SMALLSERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE
|
||||
);
|
||||
|
||||
INSERT INTO categories (name) VALUES
|
||||
('documentary'),
|
||||
('news'),
|
||||
('entertainment'),
|
||||
('podcast'),
|
||||
('other');
|
||||
|
||||
CREATE TABLE videos (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
title TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
category_id SMALLINT NOT NULL REFERENCES categories(id),
|
||||
tags TEXT NOT NULL DEFAULT '',
|
||||
file_name TEXT NOT NULL,
|
||||
storage_key TEXT NOT NULL UNIQUE,
|
||||
mediaconvert_job_id TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'processing',
|
||||
size_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
@@ -102,6 +102,23 @@ func CompleteVideoUpload(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
_, err = services.DB.CreateVideo(r.Context(), services.Video{
|
||||
Title: title,
|
||||
Description: strings.TrimSpace(req.Description),
|
||||
Category: strings.TrimSpace(req.Category),
|
||||
Tags: strings.TrimSpace(req.Tags),
|
||||
FileName: strings.TrimSpace(req.FileName),
|
||||
StorageKey: key,
|
||||
MediaConvertJobID: jobID,
|
||||
Status: "processing",
|
||||
SizeBytes: 60,
|
||||
})
|
||||
if err != nil {
|
||||
log.Printf("Something Went Wrong Saving The Video Record: %s", err)
|
||||
renderUploadError(w, r, "Something Went Wrong Saving The Video Record")
|
||||
return
|
||||
}
|
||||
|
||||
views.VideoUploadSuccess(views.VideoMetadata{
|
||||
Title: title,
|
||||
Description: strings.TrimSpace(req.Description),
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"thamanyah/cms/v2/internal/db"
|
||||
"time"
|
||||
)
|
||||
|
||||
var DB DBClient
|
||||
|
||||
type Video struct {
|
||||
ID string
|
||||
Title string
|
||||
Description string
|
||||
Category string
|
||||
Tags string
|
||||
FileName string
|
||||
StorageKey string
|
||||
MediaConvertJobID string
|
||||
Status string
|
||||
SizeBytes int64
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
type DBClient interface {
|
||||
CreateVideo(ctx context.Context, v Video) (Video, error)
|
||||
}
|
||||
|
||||
type DBConcrete struct {
|
||||
ConnectionString string
|
||||
}
|
||||
|
||||
func (svc DBConcrete) CreateVideo(ctx context.Context, v Video) (Video, error) {
|
||||
db, err := sql.Open("postgres", svc.ConnectionString)
|
||||
if err != nil {
|
||||
return Video{}, err
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
var categoryID int64
|
||||
if err := db.QueryRowContext(ctx, `SELECT id FROM categories WHERE name = $1`, v.Category).Scan(&categoryID); err != nil {
|
||||
return Video{}, fmt.Errorf("looking up category %q: %w", v.Category, err)
|
||||
}
|
||||
|
||||
row := db.QueryRowContext(ctx, `
|
||||
INSERT INTO videos (title, description, category_id, tags, file_name, storage_key, mediaconvert_job_id, status, size_bytes)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
|
||||
RETURNING created_at, updated_at
|
||||
`, v.Title, v.Description, categoryID, v.Tags, v.FileName, v.StorageKey, v.MediaConvertJobID, v.Status, v.SizeBytes)
|
||||
|
||||
if err := row.Scan(&v.CreatedAt, &v.UpdatedAt); err != nil {
|
||||
return Video{}, err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
func (svc DBConcrete) Migrate() {
|
||||
sqlDB, err := sql.Open("postgres", svc.ConnectionString)
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("failed to open db connection: %w", err))
|
||||
}
|
||||
if err := db.Migrate(sqlDB); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
sqlDB.Close()
|
||||
}
|
||||
|
||||
// AssertSuccessfulConnection verifies the database is reachable, mirroring
|
||||
// S3Concrete's boot-time check — panics rather than let the service come up broken.
|
||||
func (svc DBConcrete) AssertSuccessfulConnection(ctx context.Context) {
|
||||
db, err := sql.Open("postgres", svc.ConnectionString)
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("db: cannot open connection: %w", err))
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
if err := db.PingContext(ctx); err != nil {
|
||||
panic(fmt.Errorf("db: cannot connect: %w", err))
|
||||
}
|
||||
}
|
||||
+78
-8
@@ -12,30 +12,68 @@ import (
|
||||
"github.com/aws/aws-sdk-go-v2/config"
|
||||
"github.com/aws/aws-sdk-go-v2/service/mediaconvert"
|
||||
"github.com/aws/aws-sdk-go-v2/service/s3"
|
||||
_ "github.com/lib/pq"
|
||||
)
|
||||
|
||||
func main() {
|
||||
config, err := config.LoadDefaultConfig(context.TODO())
|
||||
if len(os.Args) > 1 && os.Args[1] == "migrate" {
|
||||
runMigrate()
|
||||
return
|
||||
}
|
||||
runServer()
|
||||
}
|
||||
|
||||
func runMigrate() {
|
||||
concreteDBClient := &services.DBConcrete{ConnectionString: requireDBConnectionString()}
|
||||
concreteDBClient.Migrate()
|
||||
log.Println("migrations applied successfully")
|
||||
}
|
||||
|
||||
func runServer() {
|
||||
awsConfig, err := config.LoadDefaultConfig(context.Background())
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("failed To Load S3 Config: %s", err))
|
||||
}
|
||||
|
||||
concreteS3Client := &services.S3Concrete{
|
||||
S3Client: s3.NewFromConfig(config),
|
||||
Bucket: os.Getenv("S3_BUCKET"),
|
||||
bucketName := os.Getenv("S3_BUCKET")
|
||||
if bucketName == "" {
|
||||
panic(fmt.Errorf("missing required env var: S3_BUCKET"))
|
||||
}
|
||||
|
||||
mediaConvertRole := os.Getenv("MEDIACONVERT_ROLE_ARN")
|
||||
if mediaConvertRole == "" {
|
||||
panic(fmt.Errorf("missing required env var: MEDIACONVERT_ROLE_ARN"))
|
||||
}
|
||||
|
||||
mediaConvertInputBucket := os.Getenv("MEDIACONVERT_INPUT_BUCKET")
|
||||
if mediaConvertInputBucket == "" {
|
||||
panic(fmt.Errorf("missing required env var: MEDIACONVERT_INPUT_BUCKET"))
|
||||
}
|
||||
|
||||
mediaConvertOutputBucket := os.Getenv("MEDIACONVERT_OUTPUT_BUCKET")
|
||||
if mediaConvertOutputBucket == "" {
|
||||
panic(fmt.Errorf("missing required env var: MEDIACONVERT_OUTPUT_BUCKET"))
|
||||
}
|
||||
|
||||
concreteS3Client := &services.S3Concrete{
|
||||
S3Client: s3.NewFromConfig(awsConfig),
|
||||
Bucket: bucketName,
|
||||
}
|
||||
concreteS3Client.AssertSuccessfulConnection(context.Background())
|
||||
|
||||
services.S3Client = concreteS3Client
|
||||
|
||||
services.MediaConvertClient = &services.MediaConvertConcrete{
|
||||
MediaConvertClient: mediaconvert.NewFromConfig(config),
|
||||
Role: os.Getenv("MEDIACONVERT_ROLE_ARN"),
|
||||
InputBucket: os.Getenv("MEDIACONVERT_INPUT_BUCKET"),
|
||||
OutputBucket: os.Getenv("MEDIACONVERT_OUTPUT_BUCKET"),
|
||||
MediaConvertClient: mediaconvert.NewFromConfig(awsConfig),
|
||||
Role: mediaConvertRole,
|
||||
InputBucket: mediaConvertInputBucket,
|
||||
OutputBucket: mediaConvertOutputBucket,
|
||||
}
|
||||
|
||||
concreteDBClient := &services.DBConcrete{ConnectionString: requireDBConnectionString()}
|
||||
concreteDBClient.AssertSuccessfulConnection(context.Background())
|
||||
services.DB = concreteDBClient
|
||||
|
||||
mux := http.NewServeMux()
|
||||
|
||||
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.Dir("static"))))
|
||||
@@ -52,3 +90,35 @@ func main() {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func requireDBConnectionString() string {
|
||||
dbHost := os.Getenv("DB_HOST")
|
||||
if dbHost == "" {
|
||||
panic(fmt.Errorf("missing required env var: DB_HOST"))
|
||||
}
|
||||
|
||||
dbPort := os.Getenv("DB_PORT")
|
||||
if dbPort == "" {
|
||||
panic(fmt.Errorf("missing required env var: DB_PORT"))
|
||||
}
|
||||
|
||||
dbUser := os.Getenv("DB_USER")
|
||||
if dbUser == "" {
|
||||
panic(fmt.Errorf("missing required env var: DB_USER"))
|
||||
}
|
||||
|
||||
dbPassword := os.Getenv("DB_PASSWORD")
|
||||
if dbPassword == "" {
|
||||
panic(fmt.Errorf("missing required env var: DB_PASSWORD"))
|
||||
}
|
||||
|
||||
dbName := os.Getenv("DB_NAME")
|
||||
if dbName == "" {
|
||||
panic(fmt.Errorf("missing required env var: DB_NAME"))
|
||||
}
|
||||
|
||||
return fmt.Sprintf(
|
||||
"host=%s port=%s user=%s password=%s dbname=%s sslmode=require",
|
||||
dbHost, dbPort, dbUser, dbPassword, dbName,
|
||||
)
|
||||
}
|
||||
|
||||
+50
-15
@@ -95,6 +95,11 @@ type envVar struct {
|
||||
// taskRole is optional (nil means no TaskRoleArn is set, i.e. the container
|
||||
// gets no AWS identity of its own beyond execRole's pull/logs permissions);
|
||||
// extraEnv is appended to the container's environment on top of the DB_* vars.
|
||||
// When runMigrations is true, the task also gets a non-essential
|
||||
// "<name>-migrate" container running the same image with `command: ["migrate"]`,
|
||||
// and the main container depends on it with condition COMPLETE — the ECS
|
||||
// equivalent of a Kubernetes init container, ensuring DB migrations finish
|
||||
// before the service starts accepting traffic.
|
||||
func deployFargateService(
|
||||
ctx *pulumi.Context,
|
||||
name string,
|
||||
@@ -110,6 +115,7 @@ func deployFargateService(
|
||||
dbAddress pulumi.StringOutput,
|
||||
dbPort pulumi.IntOutput,
|
||||
dbPassword *random.RandomPassword,
|
||||
runMigrations bool,
|
||||
) (*ecr.Repository, *ecs.Service, *lb.LoadBalancer, error) {
|
||||
alb, err := lb.NewLoadBalancer(ctx, name+"-alb", &lb.LoadBalancerArgs{
|
||||
LoadBalancerType: pulumi.String("application"),
|
||||
@@ -219,37 +225,66 @@ func deployFargateService(
|
||||
dbHost := args[3].(string)
|
||||
dbPort := args[4].(int)
|
||||
|
||||
environment := []map[string]string{
|
||||
dbEnvironment := []map[string]string{
|
||||
{"name": "DB_HOST", "value": dbHost},
|
||||
{"name": "DB_PORT", "value": fmt.Sprintf("%d", dbPort)},
|
||||
{"name": "DB_NAME", "value": name},
|
||||
{"name": "DB_USER", "value": name},
|
||||
}
|
||||
dbSecrets := []map[string]string{
|
||||
{"name": "DB_PASSWORD", "valueFrom": secretArn},
|
||||
}
|
||||
|
||||
environment := append([]map[string]string{}, dbEnvironment...)
|
||||
for i, ev := range extraEnv {
|
||||
environment = append(environment, map[string]string{"name": ev.Name, "value": args[5+i].(string)})
|
||||
}
|
||||
|
||||
def := []map[string]any{
|
||||
{
|
||||
"name": name,
|
||||
"image": image,
|
||||
"portMappings": []map[string]any{
|
||||
{"containerPort": containerPort, "protocol": "tcp"},
|
||||
},
|
||||
"environment": environment,
|
||||
"secrets": []map[string]string{
|
||||
{"name": "DB_PASSWORD", "valueFrom": secretArn},
|
||||
mainContainer := map[string]any{
|
||||
"name": name,
|
||||
"image": image,
|
||||
"portMappings": []map[string]any{
|
||||
{"containerPort": containerPort, "protocol": "tcp"},
|
||||
},
|
||||
"environment": environment,
|
||||
"secrets": dbSecrets,
|
||||
"logConfiguration": map[string]any{
|
||||
"logDriver": "awslogs",
|
||||
"options": map[string]string{
|
||||
"awslogs-group": logGroupName,
|
||||
"awslogs-region": awsRegion,
|
||||
"awslogs-stream-prefix": name,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
def := []map[string]any{}
|
||||
|
||||
if runMigrations {
|
||||
migrateContainerName := name + "-migrate"
|
||||
def = append(def, map[string]any{
|
||||
"name": migrateContainerName,
|
||||
"image": image,
|
||||
"essential": false,
|
||||
"command": []string{"migrate"},
|
||||
"environment": dbEnvironment,
|
||||
"secrets": dbSecrets,
|
||||
"logConfiguration": map[string]any{
|
||||
"logDriver": "awslogs",
|
||||
"options": map[string]string{
|
||||
"awslogs-group": logGroupName,
|
||||
"awslogs-region": awsRegion,
|
||||
"awslogs-stream-prefix": name,
|
||||
"awslogs-stream-prefix": migrateContainerName,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
mainContainer["dependsOn"] = []map[string]string{
|
||||
{"containerName": migrateContainerName, "condition": "COMPLETE"},
|
||||
}
|
||||
}
|
||||
|
||||
def = append(def, mainContainer)
|
||||
|
||||
b, err := json.Marshal(def)
|
||||
return string(b), err
|
||||
},
|
||||
@@ -761,7 +796,7 @@ func main() {
|
||||
|
||||
cmsRepo, cmsService, cmsAlb, err := deployFargateService(ctx, "cms", 8081,
|
||||
cluster, execRole, cmsTaskRole, cmsExtraEnv, vpc.Id, subnets.Ids, albSecurityGroup, serviceSecurityGroup,
|
||||
db.Address, db.Port, cmsPassword)
|
||||
db.Address, db.Port, cmsPassword, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -788,7 +823,7 @@ func main() {
|
||||
|
||||
discoveryRepo, discoveryService, discoveryAlb, err := deployFargateService(ctx, "discovery", 8080,
|
||||
cluster, execRole, nil, nil, vpc.Id, subnets.Ids, albSecurityGroup, serviceSecurityGroup,
|
||||
db.Address, db.Port, discoveryPassword)
|
||||
db.Address, db.Port, discoveryPassword, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user