Skip to content

Coreapi

The main application backend — talks to Postgres (its own coreapi database), SeaweedFS (asset storage, shared playtelly bucket with tusd), NATS JetStream, and authapi (via a shared PSK). Also passes MQTT connection details through to frontend clients so they can connect to VerneMQ directly. Depends on PostgreSQL, SeaweedFS, NATS JetStream, and authapi already being set up.


Overview

  • Backend service, connects directly to Postgres (own coreapi database, not via pooler)
  • Reads/writes assets in SeaweedFS — same playtelly bucket tusd uploads into
  • Publishes/subscribes via NATS JetStream
  • Validates requests from authapi using a shared pre-shared key (PSK) — the reverse side of the same COREAPI_AUTHAPI_PSK documented in the authapi doc
  • Hands MQTT connection details (host/port/path/credentials) to frontend clients, who then connect straight to VerneMQ's public route themselves — coreapi doesn't proxy MQTT traffic, just passes the config through

Architecture

Client apps (tellyid, playout, admin.telltboard, developer, spatio, biz, shaka)
   ↓ CORS-allowed origins / VITE_API_BASE_URL
coreapi (production namespace)
   ├─→ postgres-cluster-rw.database.svc.cluster.local  (coreapi database, direct)
   ├─→ seaweedfs-s3 + seaweedfs-filer  (playtelly bucket — same one tusd uses)
   ├─→ nats-cluster-client.messaging.svc.cluster.local  (JetStream)
   └─→ authapi.production.svc.cluster.local:3002  (PSK-authenticated)

Frontend clients also connect directly to:
   mqtt.castis.io:443/mqtt  (VerneMQ public route — coreapi just supplies these values)

Prerequisites

  • PostgreSQL (CNPG) already running, with a coreapi database created
  • SeaweedFS already running, with the playtelly bucket created
  • NATS JetStream already running
  • authapi already deployed, with a COREAPI_AUTHAPI_PSK value already generated there
  • VerneMQ's prod_app MQTT credentials already set (from the infrastructure doc) — coreapi just passes these through to clients, it doesn't create or manage them

Environment variables (coreapi-configmap)

None of the actual sensitive values from your example are repeated here (passwords, keys) — this documents what each one is for.

⚠️ This is a ConfigMap, but several of these values are secrets (DB_PASS, MINIO_SECRET_KEY, VITE_MQTT_PASSWORD, COREAPI_AUTHAPI_PSK) — ConfigMaps aren't encrypted at rest the way Secrets are, and they show up in plain text in kubectl describe/get -o yaml. Worth considering splitting these into an actual Secret, same as authapi's secret.yml / configmap.yaml split, rather than one ConfigMap holding everything. Not fixing this here, just flagging it as a deliberate choice to revisit.

Server

Key Example Notes
CORE_API_PORT 3000 Port the container listens on
VITE_API_BASE_URL https://api.castis.io The public URL frontend clients use to reach coreapi — implies a public APISIX route on this hostname, see Routing below

NATS

Key Notes
NATS_URL Points at NATS's client-facing Service (load-balanced across the 3-node cluster) — see the NATS doc

authapi integration

Key Notes
AUTHAPI_BASE_URL Internal cluster address for authapi — must match its actual Service name/port (3002)
COREAPI_AUTHAPI_PSK Must exactly match the same key documented in the authapi doc — this is one shared secret used by both sides, not two independent values. Generate once (openssl rand -base64 32), set identically in both services

PostgreSQL

Key Notes
DB_HOST / DB_PORT Already correctly split into two variables
DB_USER postgres — the real superuser, same pattern (and same caveat about using it directly for app traffic) as authapi and Zitadel
DB_PASS Must be the same real postgres password already configured when PostgreSQL was set up — not a separate/new password
DB_NAME coreapi — must exist before first deploy

SeaweedFS

Key Example Notes
MINIO_ENDPOINT seaweedfs-s3.seaweedfs.svc.cluster.local:8333 Internal S3 gateway address — see the SeaweedFS doc
MINIO_ACCESS_KEY / MINIO_SECRET_KEY Must match a real identity configured in SeaweedFS's 01-configmap-s3.yaml, not the placeholder admin/password
MINIO_USE_SSL "false" Correct for this internal-only SeaweedFS deployment
MINIO_BUCKET playtelly Same bucket tusd uploads into
SEAWEEDFS_FILER http://seaweedfs-filer.seaweedfs.svc.cluster.local:8888 Filer UI/API address
ASSET_BASE_URL https://stg.playout.castis.io ⚠️ This has a stg. prefix — double check this is actually the intended production asset URL and not a staging value that slipped into the production config

CORS

Key Notes
CORS_ORIGIN Comma-separated allowed origins — must exactly match real frontend domains (credentialed requests, no wildcard)
ALLOWED_REDIRECT_URIS OAuth2-style redirect target(s)

MQTT (passed through to frontend clients)

Key Example Notes
VITE_MQTT_HOST mqtt.castis.io VerneMQ's public route hostname — see the infrastructure doc
VITE_MQTT_PORT "443"
VITE_MQTT_PATH "/mqtt"
VITE_MQTT_USE_SSL "true" Matches the wss:// scheme documented for VerneMQ's public route
VITE_MQTT_USER prod_app Must match a username actually configured on VerneMQ (DOCKER_VERNEMQ_USER_prod_app)
VITE_MQTT_PASSWORD Must match that same VerneMQ user's real password — the example value is a placeholder (CHANGE_ME_PROD_PASSWORD) copied straight from VerneMQ's own placeholder

Dummy / ticketing

Key Notes
TICKETING_DB_NAME dummy — placeholder value; confirm whether this needs a real value before production traffic depends on it, or whether it's genuinely unused right now

⚠️ Must configure before applying

What Why
DB_PASS Must match the real postgres password, not a placeholder
MINIO_ACCESS_KEY / MINIO_SECRET_KEY Must match a real SeaweedFS identity
COREAPI_AUTHAPI_PSK Must be generated once and match exactly on both coreapi and authapi
VITE_MQTT_USER / VITE_MQTT_PASSWORD Must match real VerneMQ credentials
ASSET_BASE_URL Confirm this isn't an accidentally-leftover staging value
TICKETING_DB_NAME Confirm whether dummy is intentional or a placeholder that still needs a real value
CORS_ORIGIN / ALLOWED_REDIRECT_URIS Must match real frontend domains

Repository structure (Kustomize)

services/application/coreapi/
├── base/
│   ├── deployment.yml
│   ├── kustomization.yaml
│   ├── pvc.yml                        # PVC — content not yet shared, purpose unconfirmed
│   └── service.yml
└── overlays/
    └── prod/
        ├── apisix/
        │   ├── private/
        │   │   └── httproute.yaml     # Gateway API route, private APISIX env
        │   └── public/
        │       ├── 01-certificate.yaml
        │       ├── 02-apisixtls.yaml
        │       ├── 03-apisixroute.yaml
        │       └── 04-apisixupstream.yaml
        ├── kustomization.yaml
        └── configmap.yaml              # env vars — documented in full above

pvc.yml in base/ means coreapi mounts a PersistentVolumeClaim — content not shared yet, so its purpose (logs? temp processing? something not covered by SeaweedFS?) isn't documented here. Share it and I'll add it.


Routing

Same private+public pattern as authapi and Zitadel — two separate APISIX installations connected over Tailscale, not a namespace split.

Private route (apisix/private/httproute.yaml)

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: coreapi-route
  namespace: production
spec:
  parentRefs:
    - name: apisix-gateway
      namespace: apisix
  hostnames:
    - api.castis.io
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: coreapi-service
          port: 3000

Routes all traffic (PathPrefix: /) for api.castis.io on the private APISIX environment straight to the coreapi-service Service on port 3000 — confirms VITE_API_BASE_URL: https://api.castis.io from the ConfigMap is exactly this hostname.

Public route (apisix/public/0104)

01-certificate.yaml (confirmed):

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: coreapi-cert
  namespace: production
spec:
  secretName: coreapi-tls-secret
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - api.castis.io

02-apisixtls.yaml, 03-apisixroute.yaml, 04-apisixupstream.yaml weren't shared, but by this point the pattern is identical across authapi, Zitadel, and now coreapi's certificate — so with high confidence:

  • 02-apisixtls.yamlApisixTls, binding coreapi-tls-secret to APISIX
  • 03-apisixroute.yaml — an ApisixRoute matching api.castis.io, referencing a coreapi-upstream, with the same redirect (HTTP→HTTPS) and proxy-rewrite (X-Forwarded-*) plugins as authapi's
  • 04-apisixupstream.yaml — an ApisixUpstream with externalNodes pointing at the private APISIX environment's Tailscale address (the same 100.64.0.2:80 authapi's points at — that address belongs to the private environment itself, not any one service)

⚠️ Only 01-certificate.yaml and the private httproute.yaml are confirmed. 0204 are inferred from the now-3x-repeated pattern (authapi, Zitadel, coreapi's own certificate) — share the actual files to replace this with a fully verified version.


Autoscaling (hpa.yml)

  • Scales between 2 and 5 replicas on CPU or memory utilization, whichever hits first — target 70% for both (one more max replica than authapi's 2–4)
  • Scale-up is fast: no stabilization delay, adds 1 pod per 30s if still over target
  • Scale-down is deliberately slower: a 5-minute (300s) stabilization window, then removes at most 1 pod per 60s — same asymmetric behavior as authapi's, avoiding flapping

Deploy

# From services/application/coreapi/ — targets the private node/context
kubectl apply -k overlays/prod

Then the private route:

kubectl apply -f overlays/prod/apisix/private/httproute.yaml

Then, switching context to the public node:

kubectl apply -f overlays/prod/apisix/public/01-certificate.yaml
kubectl apply -f overlays/prod/apisix/public/02-apisixtls.yaml
kubectl apply -f overlays/prod/apisix/public/03-apisixroute.yaml
kubectl apply -f overlays/prod/apisix/public/04-apisixupstream.yaml

Verify:

kubectl get pods -n production -l app=coreapi
kubectl logs -n production -l app=coreapi


Still needed for this doc

  • deployment.yml / service.yml / pvc.yml contents — replica count, probes, resource limits, and what the PVC is actually for
  • 02-apisixtls.yaml, 03-apisixroute.yaml, 04-apisixupstream.yaml — to replace the inferred versions above with verified ones