FEAT: Added Database To CMS Serbove
Build, Push and Deploy CMS / build-push-deploy (push) Successful in 2m17s

This commit is contained in:
FahdShalhoub
2026-08-16 22:19:03 +03:00
parent c0d4401f4a
commit 96cabc5716
10 changed files with 510 additions and 23 deletions
+149
View File
@@ -0,0 +1,149 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Repo layout
```
.
├── cms/ Go service: video ingestion/CMS (implemented)
├── discovery/ Go service: empty placeholder, no code yet
├── infrastructure/ Pulumi (Go) program provisioning all AWS resources
└── .gitea/workflows/ Gitea Actions CI/CD pipelines
```
`discovery` is provisioned in infra (its own ECR repo, ECS service, ALB,
Postgres database/role) but has no application code yet — don't assume it's
deployable.
## Commands
### cms (Go 1.25, module `thamanyah/cms/v2`)
```bash
cd cms
go run . # serves on :8081
go build ./...
go vet ./...
```
There are no test files in this repo (`cms`, `discovery`, or `infrastructure`) — don't assume a test suite exists.
Views are written as `.templ` files (github.com/a-h/templ) and compiled to
`*_templ.go`. If you edit a `.templ` file, regenerate its Go code with the
`templ generate` CLI before building (not installed in this environment by
default — install via `go install github.com/a-h/templ/cmd/templ@v0.3.1020`
to match `go.mod`, or check for an existing binary first).
### infrastructure (Go, Pulumi, module `thamanyah`)
```bash
cd infrastructure
pulumi preview # plan changes against stack "main"
pulumi up # apply — this touches real AWS resources, confirm with the user first
pulumi stack output # e.g. ecsClusterArn, cmsServiceArn
```
Deploys run in CI (`.gitea/workflows/infrastructure-deploy.yml`) on push to
`main` touching `infrastructure/**`. Treat local `pulumi up` as something to
confirm with the user, not a routine dev command — it mutates shared cloud
state and Pulumi state isn't safe to update concurrently with CI.
## Architecture
### cms service
Plain `net/http` (Go 1.22+ pattern-based `ServeMux`), no framework. Entry
point `cms/main.go` wires up AWS SDK v2 clients (S3, MediaConvert) from env
vars and assigns them to package-level interface vars in `internal/services`
(`services.S3Client`, `services.MediaConvertClient`) — handlers call through
these interfaces, and `S3Concrete`/`MediaConvertConcrete` are the only
implementations, which is what makes the handlers testable even though no
tests exist yet. On boot, `S3Concrete.AssertSuccessfulConnection` proactively
exercises head/put/get/presign against the bucket and panics on failure,
rather than letting the service come up in a broken state.
Routes (`cms/main.go`): `GET /`, `GET /health`, `GET /videos/new`,
`POST /videos/presign`, `POST /videos`, static files under `/static/`.
Views live in `cms/internal/views` (templ components) with a shared
`layouts.Layout` wrapper; `types.go` holds view-model structs like
`VideoMetadata` used by the upload-success page.
### Video upload → transcode pipeline
1. Browser calls `POST /videos/presign` → cms returns a presigned S3 `PUT`
URL for `raw-uploads-bucket`, key `videos/<random-hex>.<ext>`.
2. Browser `PUT`s the file directly to S3 (requires the bucket's CORS rule,
set up in `infrastructure/main.go`).
3. Browser calls `POST /videos` with the metadata + key → cms calls
`MediaConvertClient.QueueEncodingJob(key)`, submitting a MediaConvert job
`s3://raw-uploads-bucket/<key>``s3://encoded-bucket/<key>` (H.264/AAC →
MP4, QVBR rate control — QVBR requires `MaxBitrate` to be set explicitly).
4. Finished output lands in `encoded-bucket`, served via CloudFront.
Config wiring: cms reads `S3_BUCKET`, `MEDIACONVERT_INPUT_BUCKET`,
`MEDIACONVERT_OUTPUT_BUCKET`, `MEDIACONVERT_ROLE_ARN`, `AWS_REGION` from env
vars injected by the ECS task definition (`extraEnv` in
`deployFargateService`, `infrastructure/main.go`).
### infrastructure (`infrastructure/main.go`, single Pulumi Go program, region `us-east-1`, stack `main`)
- **Postgres**: one shared RDS instance (`db.t3.micro`, single-AZ, no
backups — intentionally minimal). Each app (`cms`, `discovery`) gets its
own login role and same-named database via the `postgresql` provider
(`newServiceDatabase`), so services never share DB credentials.
- **ECS Fargate**: one cluster (`app-cluster`), one ALB *per service* (each
gets its own DNS name rather than sharing a load balancer on different
ports). `deployFargateService(...)` is the shared helper building a
service's ECR repo, CloudWatch log group, task definition, ECS service,
and ALB. `taskRole` is optional (nil = no AWS identity beyond the shared
execution role); `extraEnv` appends container env vars beyond the DB_* set.
- **S3 + CloudFront**: `encoded-bucket` holds finished transcoded output,
served publicly via CloudFront using Origin Access Control (OAC) — the
bucket itself blocks all public access; only CloudFront's OAC principal
can read it.
- **S3 raw uploads**: `raw-uploads-bucket` is a separate, private bucket for
pre-transcode uploads — deliberately kept apart from `encoded-bucket` so
raw source video is never reachable through the public CDN. CORS is
scoped to `PUT` only, from the `cms` ALB's own origin (browser uploads
directly via presigned URL).
- **IAM roles** — four distinct roles/users, each scoped narrowly, don't
conflate them:
- `ecs-task-execution-role` — shared by both services' ECS *agent* (image
pull, log write, Secrets Manager read for DB password). Not usable by
application code inside the container.
- `cms-task-role` — the `cms` container's own AWS identity: S3
`ListBucket`/`PutObject`/`GetObject` on `raw-uploads-bucket` only,
`mediaconvert:CreateJob`, and `iam:PassRole` scoped to the MediaConvert
service role (`iam:PassedToService` condition). `discovery` has no task
role — it doesn't touch S3 or MediaConvert.
- `mediaconvert-service-role` — trusted by `mediaconvert.amazonaws.com`,
not by ECS; the role MediaConvert itself assumes (passed as
`CreateJobInput.Role`) to read `raw-uploads-bucket` and write
`encoded-bucket`. Distinct from `cms-task-role` by design: one is "cms
calling AWS", the other is "AWS calling AWS on cms's behalf".
- `gitea-ci-user` — an IAM **user** (static access keys, not OIDC — the
Gitea Actions runner doesn't support instance-profile auth) scoped to
just ECR push (`cms`/`discovery` repos) and
`ecs:UpdateService`/`ecs:DescribeServices` on the two ECS services.
Never broaden this to `ecr:*`/`ecs:*`.
### CI/CD (`.gitea/workflows/`)
- **`cms-deploy.yml`**: push to `main` touching `cms/**`. Builds/pushes the
Docker image to ECR using `gitea-ci-user`, installs the AWS CLI (not
preinstalled on the runner image — via AWS's official install script, not
a third-party action), then `aws ecs update-service --force-new-deployment`.
Cluster/service ARNs come from `pulumi stack output` and are set as repo
*variables* (not secrets — ARNs aren't sensitive).
- **`infrastructure-deploy.yml`**: push to `main` touching
`infrastructure/**`. Runs `pulumi up` using a separate, broader AWS
credential (`PULUMI_AWS_ACCESS_KEY_ID`/`SECRET`) than `gitea-ci-user`,
since provisioning IAM/RDS/ECS/CloudFront needs wider permissions than
pushing images and forcing deployments. Guarded with a concurrency group
since Pulumi state isn't safe to update concurrently.
- The runner's `ubuntu-latest` label maps to a docker image configured on
the runner host (outside this repo) — currently minimal, lacking the AWS
CLI, hence the manual install step in `cms-deploy.yml`.
## Known gaps
- `cms/internal/services/mediaconvert.go` never calls `DescribeEndpoints`
and configures no custom MediaConvert endpoint — relies on the SDK's
default regional endpoint.
- `discovery` has infra provisioned (ECR repo, ECS service, ALB, Postgres
DB/role) but no application code.
- No automated tests exist for `cms`, `discovery`, or `infrastructure`.
+3
View File
@@ -8,6 +8,9 @@ require (
github.com/aws/aws-sdk-go-v2/config v1.32.36
github.com/aws/aws-sdk-go-v2/service/mediaconvert v1.97.2
github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1
github.com/golang-migrate/migrate/v4 v4.19.1
github.com/google/uuid v1.6.0
github.com/lib/pq v1.12.3
)
require (
+64
View File
@@ -1,3 +1,7 @@
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
github.com/aws/aws-sdk-go-v2 v1.43.5 h1:yKT5GYnFWhuDo+DqKvE5ZPwVn3RjC4MAeBtZGlh6AVM=
@@ -38,5 +42,65 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 h1:eQ5BtXDrPg2wK0AjtVPzeBhUpYPe
github.com/aws/aws-sdk-go-v2/service/sts v1.45.5/go.mod h1:f9ImhnOISY7BuTZLM8qHepCYnglHBVLk5wVzatmP++w=
github.com/aws/smithy-go v1.27.7 h1:Zgj5z4LfcDYoQIVk+n/yGdTkP/2y6ZT5vYxe0fp7bqE=
github.com/aws/smithy-go v1.27.7/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dhui/dktest v0.4.6 h1:+DPKyScKSEp3VLtbMDHcUq6V5Lm5zfZZVb0Sk7Ahom4=
github.com/dhui/dktest v0.4.6/go.mod h1:JHTSYDtKkvFNFHJKqCzVzqXecyv+tKt8EzceOmQOgbU=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/docker v28.3.3+incompatible h1:Dypm25kh4rmk49v1eiVbsAtpAsYURjYkaKubwuBdxEI=
github.com/docker/docker v28.3.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-migrate/migrate/v4 v4.19.1 h1:OCyb44lFuQfYXYLx1SCxPZQGU7mcaZ7gH9yH4jSFbBA=
github.com/golang-migrate/migrate/v4 v4.19.1/go.mod h1:CTcgfjxhaUtsLipnLoQRWCrjYXycRz/g5+RWDuYgPrE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+38
View File
@@ -0,0 +1,38 @@
package db
import (
"database/sql"
"embed"
"errors"
"fmt"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/postgres"
"github.com/golang-migrate/migrate/v4/source/iofs"
)
//go:embed migrations/*.sql
var migrationsFS embed.FS
func Migrate(sqlDB *sql.DB) error {
driver, err := postgres.WithInstance(sqlDB, &postgres.Config{})
if err != nil {
return fmt.Errorf("db: creating migration driver: %w", err)
}
source, err := iofs.New(migrationsFS, "migrations")
if err != nil {
return fmt.Errorf("db: loading embedded migrations: %w", err)
}
m, err := migrate.NewWithInstance("iofs", source, "postgres", driver)
if err != nil {
return fmt.Errorf("db: initializing migrator: %w", err)
}
if err := m.Up(); err != nil && !errors.Is(err, migrate.ErrNoChange) {
return fmt.Errorf("db: applying migrations: %w", err)
}
return nil
}
@@ -0,0 +1,2 @@
DROP TABLE videos;
DROP TABLE categories;
@@ -0,0 +1,26 @@
CREATE TABLE categories (
id SMALLSERIAL PRIMARY KEY,
name TEXT NOT NULL UNIQUE
);
INSERT INTO categories (name) VALUES
('documentary'),
('news'),
('entertainment'),
('podcast'),
('other');
CREATE TABLE videos (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
title TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
category_id SMALLINT NOT NULL REFERENCES categories(id),
tags TEXT NOT NULL DEFAULT '',
file_name TEXT NOT NULL,
storage_key TEXT NOT NULL UNIQUE,
mediaconvert_job_id TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'processing',
size_bytes BIGINT NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
+17
View File
@@ -102,6 +102,23 @@ func CompleteVideoUpload(w http.ResponseWriter, r *http.Request) {
return
}
_, err = services.DB.CreateVideo(r.Context(), services.Video{
Title: title,
Description: strings.TrimSpace(req.Description),
Category: strings.TrimSpace(req.Category),
Tags: strings.TrimSpace(req.Tags),
FileName: strings.TrimSpace(req.FileName),
StorageKey: key,
MediaConvertJobID: jobID,
Status: "processing",
SizeBytes: 60,
})
if err != nil {
log.Printf("Something Went Wrong Saving The Video Record: %s", err)
renderUploadError(w, r, "Something Went Wrong Saving The Video Record")
return
}
views.VideoUploadSuccess(views.VideoMetadata{
Title: title,
Description: strings.TrimSpace(req.Description),
+83
View File
@@ -0,0 +1,83 @@
package services
import (
"context"
"database/sql"
"fmt"
"thamanyah/cms/v2/internal/db"
"time"
)
var DB DBClient
type Video struct {
ID string
Title string
Description string
Category string
Tags string
FileName string
StorageKey string
MediaConvertJobID string
Status string
SizeBytes int64
CreatedAt time.Time
UpdatedAt time.Time
}
type DBClient interface {
CreateVideo(ctx context.Context, v Video) (Video, error)
}
type DBConcrete struct {
ConnectionString string
}
func (svc DBConcrete) CreateVideo(ctx context.Context, v Video) (Video, error) {
db, err := sql.Open("postgres", svc.ConnectionString)
if err != nil {
return Video{}, err
}
defer db.Close()
var categoryID int64
if err := db.QueryRowContext(ctx, `SELECT id FROM categories WHERE name = $1`, v.Category).Scan(&categoryID); err != nil {
return Video{}, fmt.Errorf("looking up category %q: %w", v.Category, err)
}
row := db.QueryRowContext(ctx, `
INSERT INTO videos (title, description, category_id, tags, file_name, storage_key, mediaconvert_job_id, status, size_bytes)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
RETURNING created_at, updated_at
`, v.Title, v.Description, categoryID, v.Tags, v.FileName, v.StorageKey, v.MediaConvertJobID, v.Status, v.SizeBytes)
if err := row.Scan(&v.CreatedAt, &v.UpdatedAt); err != nil {
return Video{}, err
}
return v, nil
}
func (svc DBConcrete) Migrate() {
sqlDB, err := sql.Open("postgres", svc.ConnectionString)
if err != nil {
panic(fmt.Errorf("failed to open db connection: %w", err))
}
if err := db.Migrate(sqlDB); err != nil {
panic(err)
}
sqlDB.Close()
}
// AssertSuccessfulConnection verifies the database is reachable, mirroring
// S3Concrete's boot-time check — panics rather than let the service come up broken.
func (svc DBConcrete) AssertSuccessfulConnection(ctx context.Context) {
db, err := sql.Open("postgres", svc.ConnectionString)
if err != nil {
panic(fmt.Errorf("db: cannot open connection: %w", err))
}
defer db.Close()
if err := db.PingContext(ctx); err != nil {
panic(fmt.Errorf("db: cannot connect: %w", err))
}
}
+78 -8
View File
@@ -12,30 +12,68 @@ import (
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/mediaconvert"
"github.com/aws/aws-sdk-go-v2/service/s3"
_ "github.com/lib/pq"
)
func main() {
config, err := config.LoadDefaultConfig(context.TODO())
if len(os.Args) > 1 && os.Args[1] == "migrate" {
runMigrate()
return
}
runServer()
}
func runMigrate() {
concreteDBClient := &services.DBConcrete{ConnectionString: requireDBConnectionString()}
concreteDBClient.Migrate()
log.Println("migrations applied successfully")
}
func runServer() {
awsConfig, err := config.LoadDefaultConfig(context.Background())
if err != nil {
panic(fmt.Errorf("failed To Load S3 Config: %s", err))
}
concreteS3Client := &services.S3Concrete{
S3Client: s3.NewFromConfig(config),
Bucket: os.Getenv("S3_BUCKET"),
bucketName := os.Getenv("S3_BUCKET")
if bucketName == "" {
panic(fmt.Errorf("missing required env var: S3_BUCKET"))
}
mediaConvertRole := os.Getenv("MEDIACONVERT_ROLE_ARN")
if mediaConvertRole == "" {
panic(fmt.Errorf("missing required env var: MEDIACONVERT_ROLE_ARN"))
}
mediaConvertInputBucket := os.Getenv("MEDIACONVERT_INPUT_BUCKET")
if mediaConvertInputBucket == "" {
panic(fmt.Errorf("missing required env var: MEDIACONVERT_INPUT_BUCKET"))
}
mediaConvertOutputBucket := os.Getenv("MEDIACONVERT_OUTPUT_BUCKET")
if mediaConvertOutputBucket == "" {
panic(fmt.Errorf("missing required env var: MEDIACONVERT_OUTPUT_BUCKET"))
}
concreteS3Client := &services.S3Concrete{
S3Client: s3.NewFromConfig(awsConfig),
Bucket: bucketName,
}
concreteS3Client.AssertSuccessfulConnection(context.Background())
services.S3Client = concreteS3Client
services.MediaConvertClient = &services.MediaConvertConcrete{
MediaConvertClient: mediaconvert.NewFromConfig(config),
Role: os.Getenv("MEDIACONVERT_ROLE_ARN"),
InputBucket: os.Getenv("MEDIACONVERT_INPUT_BUCKET"),
OutputBucket: os.Getenv("MEDIACONVERT_OUTPUT_BUCKET"),
MediaConvertClient: mediaconvert.NewFromConfig(awsConfig),
Role: mediaConvertRole,
InputBucket: mediaConvertInputBucket,
OutputBucket: mediaConvertOutputBucket,
}
concreteDBClient := &services.DBConcrete{ConnectionString: requireDBConnectionString()}
concreteDBClient.AssertSuccessfulConnection(context.Background())
services.DB = concreteDBClient
mux := http.NewServeMux()
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.Dir("static"))))
@@ -52,3 +90,35 @@ func main() {
log.Fatal(err)
}
}
func requireDBConnectionString() string {
dbHost := os.Getenv("DB_HOST")
if dbHost == "" {
panic(fmt.Errorf("missing required env var: DB_HOST"))
}
dbPort := os.Getenv("DB_PORT")
if dbPort == "" {
panic(fmt.Errorf("missing required env var: DB_PORT"))
}
dbUser := os.Getenv("DB_USER")
if dbUser == "" {
panic(fmt.Errorf("missing required env var: DB_USER"))
}
dbPassword := os.Getenv("DB_PASSWORD")
if dbPassword == "" {
panic(fmt.Errorf("missing required env var: DB_PASSWORD"))
}
dbName := os.Getenv("DB_NAME")
if dbName == "" {
panic(fmt.Errorf("missing required env var: DB_NAME"))
}
return fmt.Sprintf(
"host=%s port=%s user=%s password=%s dbname=%s sslmode=require",
dbHost, dbPort, dbUser, dbPassword, dbName,
)
}
+50 -15
View File
@@ -95,6 +95,11 @@ type envVar struct {
// taskRole is optional (nil means no TaskRoleArn is set, i.e. the container
// gets no AWS identity of its own beyond execRole's pull/logs permissions);
// extraEnv is appended to the container's environment on top of the DB_* vars.
// When runMigrations is true, the task also gets a non-essential
// "<name>-migrate" container running the same image with `command: ["migrate"]`,
// and the main container depends on it with condition COMPLETE — the ECS
// equivalent of a Kubernetes init container, ensuring DB migrations finish
// before the service starts accepting traffic.
func deployFargateService(
ctx *pulumi.Context,
name string,
@@ -110,6 +115,7 @@ func deployFargateService(
dbAddress pulumi.StringOutput,
dbPort pulumi.IntOutput,
dbPassword *random.RandomPassword,
runMigrations bool,
) (*ecr.Repository, *ecs.Service, *lb.LoadBalancer, error) {
alb, err := lb.NewLoadBalancer(ctx, name+"-alb", &lb.LoadBalancerArgs{
LoadBalancerType: pulumi.String("application"),
@@ -219,37 +225,66 @@ func deployFargateService(
dbHost := args[3].(string)
dbPort := args[4].(int)
environment := []map[string]string{
dbEnvironment := []map[string]string{
{"name": "DB_HOST", "value": dbHost},
{"name": "DB_PORT", "value": fmt.Sprintf("%d", dbPort)},
{"name": "DB_NAME", "value": name},
{"name": "DB_USER", "value": name},
}
dbSecrets := []map[string]string{
{"name": "DB_PASSWORD", "valueFrom": secretArn},
}
environment := append([]map[string]string{}, dbEnvironment...)
for i, ev := range extraEnv {
environment = append(environment, map[string]string{"name": ev.Name, "value": args[5+i].(string)})
}
def := []map[string]any{
{
"name": name,
"image": image,
"portMappings": []map[string]any{
{"containerPort": containerPort, "protocol": "tcp"},
},
"environment": environment,
"secrets": []map[string]string{
{"name": "DB_PASSWORD", "valueFrom": secretArn},
mainContainer := map[string]any{
"name": name,
"image": image,
"portMappings": []map[string]any{
{"containerPort": containerPort, "protocol": "tcp"},
},
"environment": environment,
"secrets": dbSecrets,
"logConfiguration": map[string]any{
"logDriver": "awslogs",
"options": map[string]string{
"awslogs-group": logGroupName,
"awslogs-region": awsRegion,
"awslogs-stream-prefix": name,
},
},
}
def := []map[string]any{}
if runMigrations {
migrateContainerName := name + "-migrate"
def = append(def, map[string]any{
"name": migrateContainerName,
"image": image,
"essential": false,
"command": []string{"migrate"},
"environment": dbEnvironment,
"secrets": dbSecrets,
"logConfiguration": map[string]any{
"logDriver": "awslogs",
"options": map[string]string{
"awslogs-group": logGroupName,
"awslogs-region": awsRegion,
"awslogs-stream-prefix": name,
"awslogs-stream-prefix": migrateContainerName,
},
},
},
})
mainContainer["dependsOn"] = []map[string]string{
{"containerName": migrateContainerName, "condition": "COMPLETE"},
}
}
def = append(def, mainContainer)
b, err := json.Marshal(def)
return string(b), err
},
@@ -761,7 +796,7 @@ func main() {
cmsRepo, cmsService, cmsAlb, err := deployFargateService(ctx, "cms", 8081,
cluster, execRole, cmsTaskRole, cmsExtraEnv, vpc.Id, subnets.Ids, albSecurityGroup, serviceSecurityGroup,
db.Address, db.Port, cmsPassword)
db.Address, db.Port, cmsPassword, true)
if err != nil {
return err
}
@@ -788,7 +823,7 @@ func main() {
discoveryRepo, discoveryService, discoveryAlb, err := deployFargateService(ctx, "discovery", 8080,
cluster, execRole, nil, nil, vpc.Id, subnets.Ids, albSecurityGroup, serviceSecurityGroup,
db.Address, db.Port, discoveryPassword)
db.Address, db.Port, discoveryPassword, false)
if err != nil {
return err
}