FEAT: Swicthed QUERY Method To GET
Build, Push and Deploy Discovery / build-push-deploy (push) Successful in 5m2s
Build, Push and Deploy Discovery / build-push-deploy (push) Successful in 5m2s
This commit is contained in:
@@ -18,7 +18,7 @@ draining `discovery-catalogue-events`, the queue subscribed to the catalogue
|
|||||||
topic `cms` announces ready videos on, and keeps its own copy in its own
|
topic `cms` announces ready videos on, and keeps its own copy in its own
|
||||||
database. Neither service reads the other's tables and nothing polls `cms`.
|
database. Neither service reads the other's tables and nothing polls `cms`.
|
||||||
|
|
||||||
It serves `GET /health`, `GET /api/videos/{id}`, `QUERY /api/videos` (the
|
It serves `GET /health`, `GET /api/videos/{id}`, `GET /api/videos` (the
|
||||||
catalogue search — see below) and the Swagger UI. Migration `0001_init` still
|
catalogue search — see below) and the Swagger UI. Migration `0001_init` still
|
||||||
creates nothing (it predates the domain and exists only because
|
creates nothing (it predates the domain and exists only because
|
||||||
`internal/db/migrate.go` embeds `migrations/*.sql`, which will not compile
|
`internal/db/migrate.go` embeds `migrations/*.sql`, which will not compile
|
||||||
@@ -87,24 +87,34 @@ not find "documentary". It is named explicitly in the column *and* in every
|
|||||||
query — a query built under a different configuration stems its terms
|
query — a query built under a different configuration stems its terms
|
||||||
differently and silently matches nothing.
|
differently and silently matches nothing.
|
||||||
|
|
||||||
### Catalogue search (`QUERY /api/videos`)
|
### Catalogue search (`GET /api/videos`)
|
||||||
|
|
||||||
Served on **QUERY**, not GET or POST: a search is safe and idempotent (which
|
Served on **GET**, not POST: a search is safe and idempotent, which POST is not,
|
||||||
POST is not), but its parameters are a structured document that does not belong
|
and its parameters fit a query string. It shares the path with
|
||||||
in a query string. Go's `ServeMux` routes the method fine. Two consequences to
|
`GET /api/videos/{id}`; `ServeMux` keeps them apart, the more specific pattern
|
||||||
know before touching it:
|
winning. Two things to know before touching it:
|
||||||
|
|
||||||
- It **cannot appear in the OpenAPI spec**. Swagger 2.0 and OpenAPI 3.x define
|
- This endpoint was on **QUERY** until the method was changed to GET. The
|
||||||
`PathItem` with a fixed field per method and there is no slot for QUERY;
|
parameters are now a query string, not a JSON body, so there is no
|
||||||
`swag` rejects the annotation outright with `invalid method: QUERY`. So the
|
`api.SearchRequest` any more — `handlers.parseSearchQuery` reads
|
||||||
handler carries **no `@Router` annotation** and the endpoint is described in
|
`r.URL.Query()` into an unexported `searchQuery`. Anything still sending a
|
||||||
prose in `main.go`'s `@description` instead. Adding a `@Router … [query]`
|
JSON body gets the whole catalogue, since every parameter is optional and it
|
||||||
line breaks `swag init` for the whole service.
|
supplied none of them.
|
||||||
- QUERY is still an IETF draft (`draft-ietf-httpbis-safe-method-w-body`), so
|
- It **does** appear in the OpenAPI spec now, as `@Router /api/videos [get]`
|
||||||
expect intermediaries that have never heard of it, and no HTTP caching of the
|
with one `@Param … query` per parameter (`categories` carries
|
||||||
kind a `GET` with a query string would get.
|
`collectionFormat(multi)`). That is the reason the change is worth anything
|
||||||
|
beyond method purity: QUERY had no `PathItem` slot in Swagger 2.0 or
|
||||||
|
OpenAPI 3.x, so `swag` rejected the annotation with `invalid method: QUERY`
|
||||||
|
and the endpoint had to be described in prose instead.
|
||||||
|
|
||||||
Request `{title, categories, limit, cursor}`, response `{videos, nextCursor}`.
|
`categories` is **repeated**, not comma-separated
|
||||||
|
(`?categories=news&categories=documentary`), so a category name containing a
|
||||||
|
comma stays one name; an empty value is dropped rather than treated as a
|
||||||
|
category named `""`. `limit` is the only parameter that can be malformed — a
|
||||||
|
non-numeric one is a 400 `Malformed Search`; zero or negative reads as "no
|
||||||
|
preference" and takes the default, as omitting it does.
|
||||||
|
|
||||||
|
Response `{videos, nextCursor}`.
|
||||||
`title` is matched lexically, ANDing the words with the **last one as a prefix**
|
`title` is matched lexically, ANDing the words with the **last one as a prefix**
|
||||||
(`desert & fal:*`) so a part-typed word still matches; `categories` narrows to
|
(`desert & fal:*`) so a part-typed word still matches; `categories` narrows to
|
||||||
videos filed under **any** of the names (`&&`, not `@>`); `limit` defaults to 20
|
videos filed under **any** of the names (`&&`, not `@>`); `limit` defaults to 20
|
||||||
|
|||||||
+82
-1
@@ -15,6 +15,71 @@ const docTemplate = `{
|
|||||||
"host": "{{.Host}}",
|
"host": "{{.Host}}",
|
||||||
"basePath": "{{.BasePath}}",
|
"basePath": "{{.BasePath}}",
|
||||||
"paths": {
|
"paths": {
|
||||||
|
"/api/videos": {
|
||||||
|
"get": {
|
||||||
|
"description": "Finds catalogued videos by the words of their title and the categories they are filed under, most relevant first. Every parameter is optional — a request with none of them browses the whole catalogue, newest first. Paging is by cursor rather than by offset, so a page stays the same page while videos are being announced into the catalogue around it.",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"videos"
|
||||||
|
],
|
||||||
|
"summary": "Search the catalogue",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"example": "desert fal",
|
||||||
|
"description": "Matched lexically against video titles: the words are all required, and the last is treated as a prefix so a part-typed word still finds something.",
|
||||||
|
"name": "title",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"collectionFormat": "multi",
|
||||||
|
"example": "documentary",
|
||||||
|
"description": "Narrows the search to videos filed under any one of these names — not all of them. Repeat the parameter per category. Omitted means every category.",
|
||||||
|
"name": "categories",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"default": 20,
|
||||||
|
"description": "How many videos to return. Absent, zero or negative takes the default of 20; anything above the maximum of 100 is capped to it rather than refused, and the cursor still reaches the rest.",
|
||||||
|
"name": "limit",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"description": "Asks for the page after the one a previous search ended at. Send back the nextCursor from that search unchanged; it is opaque, and the only thing to do with it is return it.",
|
||||||
|
"name": "cursor",
|
||||||
|
"in": "query"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "OK",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.SearchResults"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "Bad Request",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.ProblemDetails"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "Internal Server Error",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.ProblemDetails"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"/api/videos/{id}": {
|
"/api/videos/{id}": {
|
||||||
"get": {
|
"get": {
|
||||||
"description": "Returns the catalogue's copy of a video: what a reader needs to show and play it. A video appears here only after the CMS has announced it as ready, so a video that is still transcoding — or one the CMS never made ready — is a 404.",
|
"description": "Returns the catalogue's copy of a video: what a reader needs to show and play it. A video appears here only after the CMS has announced it as ready, so a video that is still transcoding — or one the CMS never made ready — is a 404.",
|
||||||
@@ -108,6 +173,22 @@ const docTemplate = `{
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"api.SearchResults": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"nextCursor": {
|
||||||
|
"description": "NextCursor reaches the page after this one. Empty on the last page,\nwhich is how a reader knows there is no more to ask for.",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"videos": {
|
||||||
|
"description": "Videos are the matches, most relevant first. Never null: a search that\nmatches nothing is an empty list.",
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/definitions/api.Video"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"api.Video": {
|
"api.Video": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
@@ -145,7 +226,7 @@ var SwaggerInfo = &swag.Spec{
|
|||||||
BasePath: "/",
|
BasePath: "/",
|
||||||
Schemes: []string{},
|
Schemes: []string{},
|
||||||
Title: "Thamanyah Discovery API",
|
Title: "Thamanyah Discovery API",
|
||||||
Description: "JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.\n\nNot listed below: `QUERY /api/videos`, the catalogue search. It takes a JSON body `{title, categories, limit, cursor}` and answers `{videos, nextCursor}` — title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search. It is absent here because OpenAPI has no slot for the QUERY method, not because it is unsupported; see handlers.SearchVideos.",
|
Description: "JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.\n\nThe catalogue search is `GET /api/videos`, listed below: title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search.",
|
||||||
InfoInstanceName: "swagger",
|
InfoInstanceName: "swagger",
|
||||||
SwaggerTemplate: docTemplate,
|
SwaggerTemplate: docTemplate,
|
||||||
LeftDelim: "{{",
|
LeftDelim: "{{",
|
||||||
|
|||||||
@@ -1,13 +1,78 @@
|
|||||||
{
|
{
|
||||||
"swagger": "2.0",
|
"swagger": "2.0",
|
||||||
"info": {
|
"info": {
|
||||||
"description": "JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.\n\nNot listed below: `QUERY /api/videos`, the catalogue search. It takes a JSON body `{title, categories, limit, cursor}` and answers `{videos, nextCursor}` — title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search. It is absent here because OpenAPI has no slot for the QUERY method, not because it is unsupported; see handlers.SearchVideos.",
|
"description": "JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.\n\nThe catalogue search is `GET /api/videos`, listed below: title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search.",
|
||||||
"title": "Thamanyah Discovery API",
|
"title": "Thamanyah Discovery API",
|
||||||
"contact": {},
|
"contact": {},
|
||||||
"version": "1.0"
|
"version": "1.0"
|
||||||
},
|
},
|
||||||
"basePath": "/",
|
"basePath": "/",
|
||||||
"paths": {
|
"paths": {
|
||||||
|
"/api/videos": {
|
||||||
|
"get": {
|
||||||
|
"description": "Finds catalogued videos by the words of their title and the categories they are filed under, most relevant first. Every parameter is optional — a request with none of them browses the whole catalogue, newest first. Paging is by cursor rather than by offset, so a page stays the same page while videos are being announced into the catalogue around it.",
|
||||||
|
"produces": [
|
||||||
|
"application/json"
|
||||||
|
],
|
||||||
|
"tags": [
|
||||||
|
"videos"
|
||||||
|
],
|
||||||
|
"summary": "Search the catalogue",
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"example": "desert fal",
|
||||||
|
"description": "Matched lexically against video titles: the words are all required, and the last is treated as a prefix so a part-typed word still finds something.",
|
||||||
|
"name": "title",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"collectionFormat": "multi",
|
||||||
|
"example": "documentary",
|
||||||
|
"description": "Narrows the search to videos filed under any one of these names — not all of them. Repeat the parameter per category. Omitted means every category.",
|
||||||
|
"name": "categories",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "integer",
|
||||||
|
"default": 20,
|
||||||
|
"description": "How many videos to return. Absent, zero or negative takes the default of 20; anything above the maximum of 100 is capped to it rather than refused, and the cursor still reaches the rest.",
|
||||||
|
"name": "limit",
|
||||||
|
"in": "query"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"description": "Asks for the page after the one a previous search ended at. Send back the nextCursor from that search unchanged; it is opaque, and the only thing to do with it is return it.",
|
||||||
|
"name": "cursor",
|
||||||
|
"in": "query"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "OK",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.SearchResults"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"400": {
|
||||||
|
"description": "Bad Request",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.ProblemDetails"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"500": {
|
||||||
|
"description": "Internal Server Error",
|
||||||
|
"schema": {
|
||||||
|
"$ref": "#/definitions/api.ProblemDetails"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"/api/videos/{id}": {
|
"/api/videos/{id}": {
|
||||||
"get": {
|
"get": {
|
||||||
"description": "Returns the catalogue's copy of a video: what a reader needs to show and play it. A video appears here only after the CMS has announced it as ready, so a video that is still transcoding — or one the CMS never made ready — is a 404.",
|
"description": "Returns the catalogue's copy of a video: what a reader needs to show and play it. A video appears here only after the CMS has announced it as ready, so a video that is still transcoding — or one the CMS never made ready — is a 404.",
|
||||||
@@ -101,6 +166,22 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"api.SearchResults": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"nextCursor": {
|
||||||
|
"description": "NextCursor reaches the page after this one. Empty on the last page,\nwhich is how a reader knows there is no more to ask for.",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"videos": {
|
||||||
|
"description": "Videos are the matches, most relevant first. Never null: a search that\nmatches nothing is an empty list.",
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/definitions/api.Video"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"api.Video": {
|
"api.Video": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
|
|||||||
@@ -22,6 +22,21 @@ definitions:
|
|||||||
example: about:blank
|
example: about:blank
|
||||||
type: string
|
type: string
|
||||||
type: object
|
type: object
|
||||||
|
api.SearchResults:
|
||||||
|
properties:
|
||||||
|
nextCursor:
|
||||||
|
description: |-
|
||||||
|
NextCursor reaches the page after this one. Empty on the last page,
|
||||||
|
which is how a reader knows there is no more to ask for.
|
||||||
|
type: string
|
||||||
|
videos:
|
||||||
|
description: |-
|
||||||
|
Videos are the matches, most relevant first. Never null: a search that
|
||||||
|
matches nothing is an empty list.
|
||||||
|
items:
|
||||||
|
$ref: '#/definitions/api.Video'
|
||||||
|
type: array
|
||||||
|
type: object
|
||||||
api.Video:
|
api.Video:
|
||||||
properties:
|
properties:
|
||||||
categories:
|
categories:
|
||||||
@@ -46,10 +61,65 @@ info:
|
|||||||
description: |-
|
description: |-
|
||||||
JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.
|
JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.
|
||||||
|
|
||||||
Not listed below: `QUERY /api/videos`, the catalogue search. It takes a JSON body `{title, categories, limit, cursor}` and answers `{videos, nextCursor}` — title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search. It is absent here because OpenAPI has no slot for the QUERY method, not because it is unsupported; see handlers.SearchVideos.
|
The catalogue search is `GET /api/videos`, listed below: title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search.
|
||||||
title: Thamanyah Discovery API
|
title: Thamanyah Discovery API
|
||||||
version: "1.0"
|
version: "1.0"
|
||||||
paths:
|
paths:
|
||||||
|
/api/videos:
|
||||||
|
get:
|
||||||
|
description: Finds catalogued videos by the words of their title and the categories
|
||||||
|
they are filed under, most relevant first. Every parameter is optional — a
|
||||||
|
request with none of them browses the whole catalogue, newest first. Paging
|
||||||
|
is by cursor rather than by offset, so a page stays the same page while videos
|
||||||
|
are being announced into the catalogue around it.
|
||||||
|
parameters:
|
||||||
|
- description: 'Matched lexically against video titles: the words are all required,
|
||||||
|
and the last is treated as a prefix so a part-typed word still finds something.'
|
||||||
|
example: desert fal
|
||||||
|
in: query
|
||||||
|
name: title
|
||||||
|
type: string
|
||||||
|
- collectionFormat: multi
|
||||||
|
description: Narrows the search to videos filed under any one of these names
|
||||||
|
— not all of them. Repeat the parameter per category. Omitted means every
|
||||||
|
category.
|
||||||
|
example: documentary
|
||||||
|
in: query
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
name: categories
|
||||||
|
type: array
|
||||||
|
- default: 20
|
||||||
|
description: How many videos to return. Absent, zero or negative takes the
|
||||||
|
default of 20; anything above the maximum of 100 is capped to it rather
|
||||||
|
than refused, and the cursor still reaches the rest.
|
||||||
|
in: query
|
||||||
|
name: limit
|
||||||
|
type: integer
|
||||||
|
- description: Asks for the page after the one a previous search ended at. Send
|
||||||
|
back the nextCursor from that search unchanged; it is opaque, and the only
|
||||||
|
thing to do with it is return it.
|
||||||
|
in: query
|
||||||
|
name: cursor
|
||||||
|
type: string
|
||||||
|
produces:
|
||||||
|
- application/json
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: OK
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/api.SearchResults'
|
||||||
|
"400":
|
||||||
|
description: Bad Request
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/api.ProblemDetails'
|
||||||
|
"500":
|
||||||
|
description: Internal Server Error
|
||||||
|
schema:
|
||||||
|
$ref: '#/definitions/api.ProblemDetails'
|
||||||
|
summary: Search the catalogue
|
||||||
|
tags:
|
||||||
|
- videos
|
||||||
/api/videos/{id}:
|
/api/videos/{id}:
|
||||||
get:
|
get:
|
||||||
description: 'Returns the catalogue''s copy of a video: what a reader needs
|
description: 'Returns the catalogue''s copy of a video: what a reader needs
|
||||||
|
|||||||
@@ -12,26 +12,10 @@ type Video struct {
|
|||||||
Categories []string `json:"categories" example:"documentary,news"`
|
Categories []string `json:"categories" example:"documentary,news"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// SearchRequest is the body of QUERY /api/videos: what a reader is looking
|
// The search itself takes no body type: GET /api/videos reads its parameters
|
||||||
// for. Every member is optional — an empty body browses the whole catalogue.
|
// — title, categories, limit and cursor — from the query string, so there is
|
||||||
type SearchRequest struct {
|
// no request document to publish a schema for. They are documented as
|
||||||
// Title is matched lexically against video titles: the words are all
|
// parameters on handlers.SearchVideos, and appear in the generated spec there.
|
||||||
// required, and the last is treated as a prefix so a part-typed word still
|
|
||||||
// finds something.
|
|
||||||
Title string `json:"title" example:"desert fal"`
|
|
||||||
// Categories narrows the search to videos filed under any one of these
|
|
||||||
// names — not all of them, so it widens the choice of category while
|
|
||||||
// narrowing the results. Empty means every category.
|
|
||||||
Categories []string `json:"categories" example:"documentary,news"`
|
|
||||||
// Limit is how many videos to return. Absent or zero takes the default of
|
|
||||||
// 20; anything above the maximum of 100 is capped to it rather than
|
|
||||||
// refused, and the cursor still reaches the rest.
|
|
||||||
Limit int `json:"limit" example:"20"`
|
|
||||||
// Cursor asks for the page after the one a previous search ended at. Send
|
|
||||||
// back the nextCursor from that search, unchanged; it is opaque, and the
|
|
||||||
// only thing to do with it is return it.
|
|
||||||
Cursor string `json:"cursor"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// SearchResults is one page of search results.
|
// SearchResults is one page of search results.
|
||||||
type SearchResults struct {
|
type SearchResults struct {
|
||||||
|
|||||||
@@ -4,9 +4,12 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
"fmt"
|
||||||
"log"
|
"log"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/url"
|
||||||
"slices"
|
"slices"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"thamanyah/discovery/internal/api"
|
"thamanyah/discovery/internal/api"
|
||||||
"thamanyah/discovery/internal/db/repositories"
|
"thamanyah/discovery/internal/db/repositories"
|
||||||
@@ -111,9 +114,9 @@ func searchCacheKey(title string, categories []string, limit int, cursor string)
|
|||||||
// readers naming the same categories in a different order now share one
|
// readers naming the same categories in a different order now share one
|
||||||
// entry instead of building the same page twice.
|
// entry instead of building the same page twice.
|
||||||
//
|
//
|
||||||
// Never nil, so that a client sending "categories": [] and one omitting
|
// Never nil, so that a client sending no categories at all and one whose
|
||||||
// the member entirely — the same search, since both mean "every category"
|
// categories parsed away to none — the same search, since both mean "every
|
||||||
// — land on one entry rather than on `null` and `[]`.
|
// category" — land on one entry rather than on `null` and `[]`.
|
||||||
sorted := append([]string{}, categories...)
|
sorted := append([]string{}, categories...)
|
||||||
slices.Sort(sorted)
|
slices.Sort(sorted)
|
||||||
sorted = slices.Compact(sorted)
|
sorted = slices.Compact(sorted)
|
||||||
@@ -175,55 +178,110 @@ func cacheSearch(ctx context.Context, key string, results api.SearchResults) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// searchQuery is a search as the query string spells it out. Every parameter
|
||||||
|
// is optional: a request carrying none of them browses the whole catalogue,
|
||||||
|
// newest first.
|
||||||
|
type searchQuery struct {
|
||||||
|
title string
|
||||||
|
categories []string
|
||||||
|
limit int
|
||||||
|
cursor string
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseSearchQuery reads the search out of the URL, applying the default and
|
||||||
|
// the cap to the page size so that the rest of the handler works with a limit
|
||||||
|
// it can use as given.
|
||||||
|
//
|
||||||
|
// The only parameter that can be malformed is the limit: every other one is a
|
||||||
|
// string the search takes as written, so there is nothing to spell wrong.
|
||||||
|
func parseSearchQuery(values url.Values) (searchQuery, error) {
|
||||||
|
parsed := searchQuery{
|
||||||
|
title: values.Get("title"),
|
||||||
|
// Repeated — ?categories=news&categories=documentary — rather than one
|
||||||
|
// comma-separated value, for the reason searchCacheKey encodes them as
|
||||||
|
// JSON rather than joining them: a category name is a name, and one
|
||||||
|
// containing a comma must not read as two.
|
||||||
|
//
|
||||||
|
// Empty values are dropped so that a client sending ?categories= means
|
||||||
|
// "every category", as omitting it does, rather than narrowing the
|
||||||
|
// search to a category named "".
|
||||||
|
categories: nonEmpty(values["categories"]),
|
||||||
|
cursor: values.Get("cursor"),
|
||||||
|
limit: searchPageSize,
|
||||||
|
}
|
||||||
|
|
||||||
|
if raw := values.Get("limit"); raw != "" {
|
||||||
|
limit, err := strconv.Atoi(raw)
|
||||||
|
if err != nil {
|
||||||
|
return searchQuery{}, fmt.Errorf("limit %q is not a whole number", raw)
|
||||||
|
}
|
||||||
|
// Zero and negative are read as "no preference", the same as omitting
|
||||||
|
// the parameter. A page of nothing is not what anyone meant by it, and
|
||||||
|
// the request is still answerable, so answering it beats refusing it.
|
||||||
|
if limit > 0 {
|
||||||
|
parsed.limit = limit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if parsed.limit > maxSearchPageSize {
|
||||||
|
parsed.limit = maxSearchPageSize
|
||||||
|
}
|
||||||
|
|
||||||
|
return parsed, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func nonEmpty(values []string) []string {
|
||||||
|
kept := make([]string, 0, len(values))
|
||||||
|
for _, v := range values {
|
||||||
|
if v != "" {
|
||||||
|
kept = append(kept, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return kept
|
||||||
|
}
|
||||||
|
|
||||||
// SearchVideos serves the catalogue's lexical search.
|
// SearchVideos serves the catalogue's lexical search.
|
||||||
//
|
//
|
||||||
// It is served on QUERY rather than GET or POST: a search is safe and
|
// It is served on GET: a search is safe and idempotent, which POST is not, and
|
||||||
// idempotent, which POST is not, but its parameters are a structured document
|
// its parameters are small enough to say in a query string — which keeps the
|
||||||
// that does not belong in a query string. QUERY says both. It is specified in
|
// endpoint reachable from anything that speaks HTTP, describable in the
|
||||||
// draft-ietf-httpbis-safe-method-w-body, not in RFC 9110 — still a draft, so
|
// OpenAPI spec, and cacheable by the intermediaries between the reader and
|
||||||
// expect intermediaries that have never heard of it. Note it also cannot be
|
// this service.
|
||||||
// expressed in the generated OpenAPI spec — Swagger 2.0 and OpenAPI 3.x define a fixed set of
|
//
|
||||||
// per-method fields with no slot for QUERY, and swag rejects the annotation
|
// @Summary Search the catalogue
|
||||||
// outright — so this endpoint is deliberately absent from /swagger and
|
// @Description Finds catalogued videos by the words of their title and the categories they are filed under, most relevant first. Every parameter is optional — a request with none of them browses the whole catalogue, newest first. Paging is by cursor rather than by offset, so a page stays the same page while videos are being announced into the catalogue around it.
|
||||||
// documented here instead.
|
// @Tags videos
|
||||||
|
// @Produce json
|
||||||
|
// @Param title query string false "Matched lexically against video titles: the words are all required, and the last is treated as a prefix so a part-typed word still finds something." example(desert fal)
|
||||||
|
// @Param categories query []string false "Narrows the search to videos filed under any one of these names — not all of them. Repeat the parameter per category. Omitted means every category." collectionFormat(multi) example(documentary)
|
||||||
|
// @Param limit query int false "How many videos to return. Absent, zero or negative takes the default of 20; anything above the maximum of 100 is capped to it rather than refused, and the cursor still reaches the rest." default(20)
|
||||||
|
// @Param cursor query string false "Asks for the page after the one a previous search ended at. Send back the nextCursor from that search unchanged; it is opaque, and the only thing to do with it is return it."
|
||||||
|
// @Success 200 {object} api.SearchResults
|
||||||
|
// @Failure 400 {object} api.ProblemDetails
|
||||||
|
// @Failure 500 {object} api.ProblemDetails
|
||||||
|
// @Router /api/videos [get]
|
||||||
func SearchVideos(w http.ResponseWriter, r *http.Request) {
|
func SearchVideos(w http.ResponseWriter, r *http.Request) {
|
||||||
var request api.SearchRequest
|
request, err := parseSearchQuery(r.URL.Query())
|
||||||
|
if err != nil {
|
||||||
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
|
|
||||||
writeProblem(w, http.StatusBadRequest, "Malformed Search",
|
writeProblem(w, http.StatusBadRequest, "Malformed Search",
|
||||||
"The request body could not be read as JSON. Send a JSON object describing what to search for; an empty object browses the whole catalogue.")
|
"The search parameters could not be read: "+err.Error()+". Every parameter is optional; a request with none of them browses the whole catalogue.")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
limit := request.Limit
|
cacheKey := searchCacheKey(request.title, request.categories, request.limit, request.cursor)
|
||||||
if limit <= 0 {
|
|
||||||
limit = searchPageSize
|
|
||||||
}
|
|
||||||
if limit > maxSearchPageSize {
|
|
||||||
limit = maxSearchPageSize
|
|
||||||
}
|
|
||||||
|
|
||||||
// Read through the cache before touching Postgres. A ranked search is the
|
|
||||||
// expensive request this service serves — ts_rank is computed per matching
|
|
||||||
// row, so deep pages scan rather than seek — and readers ask for the same
|
|
||||||
// few things over and over, which is exactly the shape a cache pays for.
|
|
||||||
//
|
|
||||||
// Only successful pages are ever stored, so a malformed cursor still
|
|
||||||
// reaches the repository and is still rejected below.
|
|
||||||
cacheKey := searchCacheKey(request.Title, request.Categories, limit, request.Cursor)
|
|
||||||
if cached, hit := cachedSearch(r.Context(), cacheKey); hit {
|
if cached, hit := cachedSearch(r.Context(), cacheKey); hit {
|
||||||
writeJSON(w, http.StatusOK, cached)
|
writeJSON(w, http.StatusOK, cached)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
found, next, err := repositories.VideoRepo.SearchVideos(r.Context(), request.Title, request.Categories, limit, request.Cursor)
|
found, next, err := repositories.VideoRepo.SearchVideos(r.Context(), request.title, request.categories, request.limit, request.cursor)
|
||||||
if errors.Is(err, repositories.ErrInvalidCursor) {
|
if errors.Is(err, repositories.ErrInvalidCursor) {
|
||||||
writeProblem(w, http.StatusBadRequest, "Malformed Search Cursor",
|
writeProblem(w, http.StatusBadRequest, "Malformed Search Cursor",
|
||||||
"The cursor could not be read. Send back the nextCursor from a previous search unchanged, or omit it to start from the first page.")
|
"The cursor could not be read. Send back the nextCursor from a previous search unchanged, or omit it to start from the first page.")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("Something Went Wrong Searching The Catalogue: title=%q: %s", request.Title, err)
|
log.Printf("Something Went Wrong Searching The Catalogue: title=%q: %s", request.title, err)
|
||||||
writeProblem(w, http.StatusInternalServerError, "Something Went Wrong Searching The Catalogue",
|
writeProblem(w, http.StatusInternalServerError, "Something Went Wrong Searching The Catalogue",
|
||||||
"The catalogue could not be searched. This is a server-side fault and the request was not processed; retrying in a few moments may succeed.")
|
"The catalogue could not be searched. This is a server-side fault and the request was not processed; retrying in a few moments may succeed.")
|
||||||
return
|
return
|
||||||
|
|||||||
+6
-4
@@ -26,7 +26,7 @@ import (
|
|||||||
// @version 1.0
|
// @version 1.0
|
||||||
// @description JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.
|
// @description JSON API for browsing the Thamanyah catalogue. Read-side counterpart to the CMS, which is what ingests videos.
|
||||||
// @description
|
// @description
|
||||||
// @description Not listed below: `QUERY /api/videos`, the catalogue search. It takes a JSON body `{title, categories, limit, cursor}` and answers `{videos, nextCursor}` — title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search. It is absent here because OpenAPI has no slot for the QUERY method, not because it is unsupported; see handlers.SearchVideos.
|
// @description The catalogue search is `GET /api/videos`, listed below: title is matched lexically with the last word as a prefix, categories narrow to videos filed under any one of them, limit defaults to 20 and is capped at 100, and cursor is the opaque `nextCursor` of a previous search.
|
||||||
// @BasePath /
|
// @BasePath /
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
@@ -117,9 +117,11 @@ func runServer() {
|
|||||||
|
|
||||||
mux.HandleFunc("GET /health", handlers.Health)
|
mux.HandleFunc("GET /health", handlers.Health)
|
||||||
mux.HandleFunc("GET /api/videos/{id}", handlers.GetVideo)
|
mux.HandleFunc("GET /api/videos/{id}", handlers.GetVideo)
|
||||||
// QUERY, not GET: a search is safe and idempotent but carries a structured
|
// The catalogue search. GET rather than POST because a search is safe and
|
||||||
// body. See handlers.SearchVideos for why it is absent from the spec.
|
// idempotent, and its parameters — title, categories, limit, cursor — fit
|
||||||
mux.HandleFunc("QUERY /api/videos", handlers.SearchVideos)
|
// a query string. ServeMux keeps this apart from the {id} pattern above:
|
||||||
|
// the more specific one wins.
|
||||||
|
mux.HandleFunc("GET /api/videos", handlers.SearchVideos)
|
||||||
|
|
||||||
// Swagger UI and the generated spec. The UI assets are embedded in the
|
// Swagger UI and the generated spec. The UI assets are embedded in the
|
||||||
// binary by swaggo/files, so this needs no static directory on disk.
|
// binary by swaggo/files, so this needs no static directory on disk.
|
||||||
|
|||||||
+1
-1
@@ -77,7 +77,7 @@ Both steps of the upload flow, end to end:
|
|||||||
not there at all, and a video announced a second time is updated rather than
|
not there at all, and a video announced a second time is updated rather than
|
||||||
colliding with its own row
|
colliding with its own row
|
||||||
|
|
||||||
- searching that catalogue over `QUERY /api/videos`: finding a video by a word
|
- searching that catalogue over `GET /api/videos`: finding a video by a word
|
||||||
in its title and by a part-typed one, narrowing to a category and being left
|
in its title and by a part-typed one, narrowing to a category and being left
|
||||||
out of another, several categories meaning "any of", paging by cursor so the
|
out of another, several categories meaning "any of", paging by cursor so the
|
||||||
pages tile the results exactly once, refusing a cursor no search issued, and
|
pages tile the results exactly once, refusing a cursor no search issued, and
|
||||||
|
|||||||
+38
-15
@@ -6,7 +6,9 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
@@ -139,15 +141,37 @@ type catalogueVideoBody struct {
|
|||||||
Categories []string `json:"categories"`
|
Categories []string `json:"categories"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// searchRequest is the body of QUERY /api/videos on discovery: what a reader
|
// searchRequest is a search of GET /api/videos on discovery: what a reader is
|
||||||
// is looking for, plus where in the results to carry on from.
|
// looking for, plus where in the results to carry on from.
|
||||||
//
|
//
|
||||||
// Every field is optional. An empty body is the whole catalogue, newest first.
|
// Every field is optional. A search with none of them set is the whole
|
||||||
|
// catalogue, newest first.
|
||||||
type searchRequest struct {
|
type searchRequest struct {
|
||||||
Title string `json:"title,omitempty"`
|
Title string
|
||||||
Categories []string `json:"categories,omitempty"`
|
Categories []string
|
||||||
Limit int `json:"limit,omitempty"`
|
Limit int
|
||||||
Cursor string `json:"cursor,omitempty"`
|
Cursor string
|
||||||
|
}
|
||||||
|
|
||||||
|
// query renders the search as the query string discovery reads it from.
|
||||||
|
// Categories are repeated rather than joined, since that is how the endpoint
|
||||||
|
// takes several of them, and an unset field is left out entirely so that the
|
||||||
|
// service applies its own default.
|
||||||
|
func (r searchRequest) query() url.Values {
|
||||||
|
values := url.Values{}
|
||||||
|
if r.Title != "" {
|
||||||
|
values.Set("title", r.Title)
|
||||||
|
}
|
||||||
|
for _, category := range r.Categories {
|
||||||
|
values.Add("categories", category)
|
||||||
|
}
|
||||||
|
if r.Limit != 0 {
|
||||||
|
values.Set("limit", strconv.Itoa(r.Limit))
|
||||||
|
}
|
||||||
|
if r.Cursor != "" {
|
||||||
|
values.Set("cursor", r.Cursor)
|
||||||
|
}
|
||||||
|
return values
|
||||||
}
|
}
|
||||||
|
|
||||||
// searchResultsBody is what that query answers with: the page of videos, and
|
// searchResultsBody is what that query answers with: the page of videos, and
|
||||||
@@ -177,15 +201,14 @@ func (b searchResultsBody) holds(id string) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
// queryJSON sends a QUERY request — the method the catalogue search is served
|
// search sends the catalogue search: a plain GET with the search spelled out
|
||||||
// on, chosen because a search is safe and idempotent like a GET but carries a
|
// in the query string.
|
||||||
// structured body no query string would hold comfortably.
|
func (c *client) search(path string, request searchRequest) (response, error) {
|
||||||
func (c *client) queryJSON(path string, payload any) (response, error) {
|
query := request.query().Encode()
|
||||||
encoded, err := json.Marshal(payload)
|
if query != "" {
|
||||||
if err != nil {
|
path += "?" + query
|
||||||
return response{}, err
|
|
||||||
}
|
}
|
||||||
return c.send("QUERY", c.base+path, "application/json", encoded)
|
return c.get(path)
|
||||||
}
|
}
|
||||||
|
|
||||||
// postJSON marshals payload and posts it to a path on the CMS.
|
// postJSON marshals payload and posts it to a path on the CMS.
|
||||||
|
|||||||
+1
-1
@@ -973,7 +973,7 @@ func searchTheCatalogue(t gobdd.StepTest, w *world, request searchRequest) {
|
|||||||
w.discovery = newDiscoveryClient()
|
w.discovery = newDiscoveryClient()
|
||||||
}
|
}
|
||||||
|
|
||||||
result, err := w.discovery.queryJSON("/api/videos", request)
|
result, err := w.discovery.search("/api/videos", request)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("could not search the catalogue: %s", err)
|
t.Fatalf("could not search the catalogue: %s", err)
|
||||||
return
|
return
|
||||||
|
|||||||
Executable
+357
@@ -0,0 +1,357 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# upload-video.sh — drive the cms upload flow end to end from a local file.
|
||||||
|
#
|
||||||
|
# 1. GET /api/categories pick what the video is filed under
|
||||||
|
# 2. POST /api/videos/presign ask for an upload slot
|
||||||
|
# 3. PUT <presigned url> send the file straight to S3, not through cms
|
||||||
|
# 4. POST /api/videos register it, which queues the transcode
|
||||||
|
# 5. GET /api/videos/{id} poll until playbackUrl appears
|
||||||
|
#
|
||||||
|
# The playback URL is empty until the MediaConvert job reports success and
|
||||||
|
# cms's consumer records it, so step 5 is a wait, not a formality. A job that
|
||||||
|
# comes back "failed" never gets a URL — the script stops rather than polls on.
|
||||||
|
#
|
||||||
|
# Usage: scripts/upload-video.sh [options]
|
||||||
|
# Run with --help for the full list. With no options it asks for everything.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
readonly DEFAULT_BASE_URL="http://cms-alb-d478b02-648162889.us-east-1.elb.amazonaws.com"
|
||||||
|
readonly DEFAULT_POLL_INTERVAL=5
|
||||||
|
readonly DEFAULT_POLL_TIMEOUT=900
|
||||||
|
|
||||||
|
base_url="${CMS_BASE_URL:-$DEFAULT_BASE_URL}"
|
||||||
|
file_path=""
|
||||||
|
title=""
|
||||||
|
description=""
|
||||||
|
tags=""
|
||||||
|
categories_arg=""
|
||||||
|
poll_interval="$DEFAULT_POLL_INTERVAL"
|
||||||
|
poll_timeout="$DEFAULT_POLL_TIMEOUT"
|
||||||
|
skip_poll=false
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'USAGE'
|
||||||
|
Upload a video to the Thamanyah cms and wait for its playback URL.
|
||||||
|
|
||||||
|
Usage: upload-video.sh [options]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
-f, --file PATH Video file to upload (.mp4 or .mov). Prompted for if omitted.
|
||||||
|
-t, --title TITLE Video title. Prompted for if omitted.
|
||||||
|
-d, --description TEXT Video description. Optional.
|
||||||
|
--tags TAGS Free-text comma-separated tags. Optional.
|
||||||
|
-c, --categories LIST Comma-separated category names or ids (e.g. "news,podcast"
|
||||||
|
or "2,4"). Prompted for if omitted.
|
||||||
|
-u, --url URL cms base URL. Default: $CMS_BASE_URL or http://localhost:8081
|
||||||
|
-i, --interval SECONDS Seconds between status polls. Default: 5
|
||||||
|
--timeout SECONDS Give up waiting after this long. Default: 900
|
||||||
|
--no-poll Register the video and exit without waiting.
|
||||||
|
-h, --help Show this help.
|
||||||
|
|
||||||
|
Requires: curl, jq.
|
||||||
|
|
||||||
|
Exit codes: 0 ready, 1 usage/upload error, 2 transcode failed, 3 poll timed out.
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '\nerror: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
info() { printf '%s\n' "$*" >&2; }
|
||||||
|
|
||||||
|
# The poll writes over one line on a terminal and one line per check when
|
||||||
|
# redirected — an escape-littered log file helps nobody.
|
||||||
|
progress() {
|
||||||
|
if [[ -t 2 ]]; then
|
||||||
|
printf '\r\033[K %s' "$*" >&2
|
||||||
|
else
|
||||||
|
printf ' %s\n' "$*" >&2
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
clear_progress() { [[ -t 2 ]] && printf '\r\033[K' >&2 || true; }
|
||||||
|
|
||||||
|
require_tools() {
|
||||||
|
local missing=()
|
||||||
|
for tool in curl jq; do
|
||||||
|
command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
|
||||||
|
done
|
||||||
|
[[ ${#missing[@]} -eq 0 ]] || die "missing required command(s): ${missing[*]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# The API reports failures as RFC 9457 problem details; surface the human parts
|
||||||
|
# of that rather than dumping the raw body.
|
||||||
|
report_api_error() {
|
||||||
|
local context="$1" status="$2" body="$3" problem_title problem_detail
|
||||||
|
|
||||||
|
problem_title=$(jq -r '.title // empty' <<<"$body" 2>/dev/null || true)
|
||||||
|
problem_detail=$(jq -r '.detail // empty' <<<"$body" 2>/dev/null || true)
|
||||||
|
|
||||||
|
if [[ -n "$problem_title" ]]; then
|
||||||
|
printf '\nerror: %s (HTTP %s)\n %s\n' "$context" "$status" "$problem_title" >&2
|
||||||
|
[[ -n "$problem_detail" ]] && printf ' %s\n' "$problem_detail" >&2
|
||||||
|
else
|
||||||
|
printf '\nerror: %s (HTTP %s)\n %s\n' "$context" "$status" "${body:-<empty response>}" >&2
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Splits curl's "body + trailing status line" into the two globals the callers
|
||||||
|
# read, so a request needs one subshell rather than two round trips.
|
||||||
|
http_status=""
|
||||||
|
http_body=""
|
||||||
|
request() {
|
||||||
|
local response
|
||||||
|
response=$(curl --silent --show-error --location --write-out $'\n%{http_code}' "$@") ||
|
||||||
|
die "request to cms failed — is it running at $base_url ?"
|
||||||
|
http_status="${response##*$'\n'}"
|
||||||
|
http_body="${response%$'\n'*}"
|
||||||
|
}
|
||||||
|
|
||||||
|
content_type_for() {
|
||||||
|
case "${1,,}" in
|
||||||
|
*.mp4) printf 'video/mp4' ;;
|
||||||
|
*.mov) printf 'video/quicktime' ;;
|
||||||
|
*) die "unsupported file type: $1 — cms accepts only .mp4 and .mov" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
prompt_required() {
|
||||||
|
local varname="$1" prompt="$2" value=""
|
||||||
|
[[ -t 0 ]] || die "$varname not given and stdin is not a terminal — pass it as an option"
|
||||||
|
while [[ -z "$value" ]]; do
|
||||||
|
read -r -e -p "$prompt" value
|
||||||
|
value="${value#"${value%%[![:space:]]*}"}"
|
||||||
|
done
|
||||||
|
printf '%s' "$value"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
-f | --file)
|
||||||
|
file_path="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-t | --title)
|
||||||
|
title="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-d | --description)
|
||||||
|
description="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--tags)
|
||||||
|
tags="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-c | --categories)
|
||||||
|
categories_arg="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-u | --url)
|
||||||
|
base_url="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-i | --interval)
|
||||||
|
poll_interval="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--timeout)
|
||||||
|
poll_timeout="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--no-poll)
|
||||||
|
skip_poll=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h | --help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
usage >&2
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
require_tools
|
||||||
|
|
||||||
|
base_url="${base_url%/}"
|
||||||
|
[[ "$poll_interval" =~ ^[0-9]+$ && "$poll_interval" -gt 0 ]] || die "--interval must be a positive integer"
|
||||||
|
[[ "$poll_timeout" =~ ^[0-9]+$ && "$poll_timeout" -gt 0 ]] || die "--timeout must be a positive integer"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- the file ---
|
||||||
|
|
||||||
|
if [[ -z "$file_path" ]]; then
|
||||||
|
# -e gives readline's filename completion, which is the whole point of
|
||||||
|
# asking here rather than making the flag mandatory.
|
||||||
|
file_path=$(prompt_required "--file" "Video file to upload: ")
|
||||||
|
fi
|
||||||
|
file_path="${file_path/#\~/$HOME}"
|
||||||
|
|
||||||
|
[[ -f "$file_path" ]] || die "no such file: $file_path"
|
||||||
|
[[ -r "$file_path" ]] || die "file is not readable: $file_path"
|
||||||
|
[[ -s "$file_path" ]] || die "file is empty: $file_path"
|
||||||
|
|
||||||
|
file_name=$(basename -- "$file_path")
|
||||||
|
content_type=$(content_type_for "$file_name")
|
||||||
|
|
||||||
|
info "Checking cms at $base_url ..."
|
||||||
|
request "$base_url/health"
|
||||||
|
[[ "$http_status" == "200" ]] || report_api_error "cms is not healthy" "$http_status" "$http_body"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------- the categories ---
|
||||||
|
|
||||||
|
request "$base_url/api/categories"
|
||||||
|
[[ "$http_status" == "200" ]] || report_api_error "could not list categories" "$http_status" "$http_body"
|
||||||
|
categories_json="$http_body"
|
||||||
|
|
||||||
|
if [[ -z "$categories_arg" ]]; then
|
||||||
|
info ""
|
||||||
|
info "Available categories:"
|
||||||
|
jq -r '.categories[] | " \(.id)) \(.name)"' <<<"$categories_json" >&2
|
||||||
|
info ""
|
||||||
|
categories_arg=$(prompt_required "--categories" "Categories (comma-separated ids or names): ")
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Accept either spelling — an id straight from the list, or the name it goes by
|
||||||
|
# — and resolve both to the ids POST /api/videos wants. Unmatched entries come
|
||||||
|
# back in their own member rather than as a jq error, so they can be named.
|
||||||
|
resolved=$(jq -c --arg raw "$categories_arg" '
|
||||||
|
[$raw | split(",") | .[] | gsub("^\\s+|\\s+$"; "") | select(length > 0)] as $wanted
|
||||||
|
| {
|
||||||
|
ids: [ $wanted[] as $w
|
||||||
|
| $categories.categories[]
|
||||||
|
| select((.id | tostring) == $w or (.name | ascii_downcase) == ($w | ascii_downcase))
|
||||||
|
| .id ] | unique,
|
||||||
|
unknown: [ $wanted[] as $w
|
||||||
|
| select([ $categories.categories[]
|
||||||
|
| select((.id | tostring) == $w or (.name | ascii_downcase) == ($w | ascii_downcase)) ] | length == 0)
|
||||||
|
| $w ]
|
||||||
|
}
|
||||||
|
' --argjson categories "$categories_json" -n)
|
||||||
|
|
||||||
|
unknown_categories=$(jq -r '.unknown | join(", ")' <<<"$resolved")
|
||||||
|
[[ -z "$unknown_categories" ]] || die "no category is called \"$unknown_categories\" — pick from the list with GET $base_url/api/categories"
|
||||||
|
|
||||||
|
category_ids_json=$(jq -c '.ids' <<<"$resolved")
|
||||||
|
[[ "$category_ids_json" != "[]" ]] || die "at least one category is required"
|
||||||
|
|
||||||
|
category_names=$(jq -r --argjson ids "$category_ids_json" \
|
||||||
|
'[.categories[] | select(.id as $i | $ids | index($i)) | .name] | join(", ")' <<<"$categories_json")
|
||||||
|
|
||||||
|
# -------------------------------------------------------------- the metadata ---
|
||||||
|
|
||||||
|
[[ -n "$title" ]] || title=$(prompt_required "--title" "Title: ")
|
||||||
|
|
||||||
|
info ""
|
||||||
|
info "Uploading : $file_path ($content_type)"
|
||||||
|
info "Title : $title"
|
||||||
|
info "Categories : $category_names"
|
||||||
|
info ""
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- 1. presign ---
|
||||||
|
|
||||||
|
info "Requesting an upload URL ..."
|
||||||
|
presign_body=$(jq -n --arg fileName "$file_name" --arg contentType "$content_type" \
|
||||||
|
'{fileName: $fileName, contentType: $contentType}')
|
||||||
|
|
||||||
|
request -X POST "$base_url/api/videos/presign" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
--data-binary "$presign_body"
|
||||||
|
[[ "$http_status" == "200" ]] || report_api_error "could not get an upload URL" "$http_status" "$http_body"
|
||||||
|
|
||||||
|
upload_url=$(jq -r '.uploadUrl' <<<"$http_body")
|
||||||
|
storage_key=$(jq -r '.key' <<<"$http_body")
|
||||||
|
[[ -n "$upload_url" && "$upload_url" != "null" ]] || die "cms returned no upload URL"
|
||||||
|
|
||||||
|
# ----------------------------------------------------- 2. PUT the file to S3 ---
|
||||||
|
|
||||||
|
# The content type is signed into the URL, so this header has to match the one
|
||||||
|
# sent to /presign exactly or S3 rejects the PUT as a signature mismatch.
|
||||||
|
info "Uploading $(du -h -- "$file_path" | cut -f1) to storage ..."
|
||||||
|
upload_status=$(curl --silent --show-error --progress-bar \
|
||||||
|
--request PUT \
|
||||||
|
--header "Content-Type: $content_type" \
|
||||||
|
--upload-file "$file_path" \
|
||||||
|
--write-out '%{http_code}' \
|
||||||
|
--output /dev/null \
|
||||||
|
"$upload_url") || die "upload to storage failed"
|
||||||
|
|
||||||
|
[[ "$upload_status" =~ ^2 ]] || die "storage rejected the upload (HTTP $upload_status)"
|
||||||
|
info "Upload complete: $storage_key"
|
||||||
|
|
||||||
|
# ------------------------------------------- 3. register + queue transcoding ---
|
||||||
|
|
||||||
|
info "Registering the video ..."
|
||||||
|
complete_body=$(jq -n \
|
||||||
|
--arg title "$title" \
|
||||||
|
--arg description "$description" \
|
||||||
|
--arg tags "$tags" \
|
||||||
|
--arg fileName "$file_name" \
|
||||||
|
--arg key "$storage_key" \
|
||||||
|
--argjson categoryIds "$category_ids_json" \
|
||||||
|
'{title: $title, description: $description, categoryIds: $categoryIds, tags: $tags, fileName: $fileName, key: $key}')
|
||||||
|
|
||||||
|
request -X POST "$base_url/api/videos" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
--data-binary "$complete_body"
|
||||||
|
[[ "$http_status" == "201" ]] || report_api_error "could not register the video" "$http_status" "$http_body"
|
||||||
|
|
||||||
|
video_id=$(jq -r '.id' <<<"$http_body")
|
||||||
|
info "Registered as $video_id (status: $(jq -r '.status' <<<"$http_body"))"
|
||||||
|
|
||||||
|
if [[ "$skip_poll" == true ]]; then
|
||||||
|
info ""
|
||||||
|
info "Not waiting for transcoding (--no-poll). Follow it with:"
|
||||||
|
info " curl -s $base_url/api/videos/$video_id | jq"
|
||||||
|
printf '%s\n' "$video_id"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ------------------------------------------ 4. poll until the URL is there ---
|
||||||
|
|
||||||
|
info ""
|
||||||
|
info "Waiting for transcoding to finish (timeout ${poll_timeout}s, checking every ${poll_interval}s) ..."
|
||||||
|
|
||||||
|
started_at=$SECONDS
|
||||||
|
while :; do
|
||||||
|
request "$base_url/api/videos/$video_id"
|
||||||
|
[[ "$http_status" == "200" ]] || report_api_error "could not read the video" "$http_status" "$http_body"
|
||||||
|
|
||||||
|
status=$(jq -r '.status' <<<"$http_body")
|
||||||
|
playback_url=$(jq -r '.playbackUrl // empty' <<<"$http_body")
|
||||||
|
elapsed=$((SECONDS - started_at))
|
||||||
|
|
||||||
|
# The URL is what the caller is actually waiting for, and it is written
|
||||||
|
# alongside the status — so wait for the URL, not merely for "ready".
|
||||||
|
if [[ -n "$playback_url" ]]; then
|
||||||
|
clear_progress
|
||||||
|
info "Ready after ${elapsed}s."
|
||||||
|
info ""
|
||||||
|
info "Playback URL:"
|
||||||
|
printf '%s\n' "$playback_url"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$status" == "failed" ]]; then
|
||||||
|
clear_progress
|
||||||
|
printf '\nerror: transcoding failed for video %s after %ss — no playback URL will be produced.\n' "$video_id" "$elapsed" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ((elapsed >= poll_timeout)); then
|
||||||
|
clear_progress
|
||||||
|
printf '\nerror: timed out after %ss with status %q and no playback URL.\n Check again with: curl -s %s/api/videos/%s | jq\n' \
|
||||||
|
"$elapsed" "$status" "$base_url" "$video_id" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
|
||||||
|
progress "status: $status — ${elapsed}s elapsed"
|
||||||
|
sleep "$poll_interval"
|
||||||
|
done
|
||||||
Reference in New Issue
Block a user