feat(rls): Postgres row-level-tenant-isolation prototype + Postgres 18 bump #138

Merged

fastie81 merged 5 commits from feature/rls-tenant-isolation into main 2026-09-06 19:42:55 +12:00

Conversation 0 Commits 5 Files changed 8 +435 -16

fastie81 commented 2026-09-06 19:10:31 +12:00

Owner

Copy link

What

Replaces the abandoned database-per-club split with Postgres Row-Level Security for tenant isolation, plus a fully-automated, CI-driven Postgres 15 → 18 upgrade run through the app startup entrypoint.

RLS (single-DB, database-enforced)

• app/infrastructure/database/rls.py — opt-in bootstrap: creates a non-superuser honbu_app role, transfers ownership of the 13 club-scoped tables, enables FORCE ROW LEVEL SECURITY with tenant policies keyed on the app.club_id GUC.
• Direct tables (members, families, attendance, class_schedules, user_club_assignments, api_tokens, form_configurations, content_items, content_club_approvals) filter on club_id.
• Indirect tables (belt_levels, gradings, notes_history via members; family_guardians via families) derive club through parent FK.
• Auth decorators set per-request SET LOCAL app.club_id — transaction-scoped, pool-safe.
• Opt-in via HONBU_ENABLE_RLS=1 + runtime engine connects as honbu_app. Default (superuser) behaviour byte-for-byte unchanged.

Automated Postgres 15 → 18 upgrade (CI-driven, zero operator DB touch)

Triggered by a Forgejo Actions variable, POSTGRES_MAJOR. Routine deploys are unchanged.

POSTGRES_MAJOR	Image	Data volume	Behavior
unset (default)	postgres:15-alpine	postgres_data (stable, = current prod)	routine deploy, no change
18	postgres:18-alpine	postgres_data_18 (fresh)	upgrade path — old postgres_data preserved as rollback/safety
15	postgres:15-alpine	postgres_data_15 (fresh)	deliberate re-init

Sequence on upgrade:

1. CI deploy job injects POSTGRES_MAJOR='${{ vars.POSTGRES_MAJOR || '' }}' into the test+prod docker compose pull/up commands (read from the Forgejo Actions variable).
2. compose creates a fresh postgres_data_<major> volume + postgres:<major> image (old volume intact).
3. App entrypoint (runs before web server): takes an idempotent pre-upgrade pg_dump once per major → auto-restores the prior-major dump into the fresh DB → applies migrations → RLS bootstrap → starts server.
4. Maintenance window = exactly the upgrade; app not serving during it. Idempotent, no double-restore.

CI safety — the hard-won workflow is untouched

Only additive change to .forgejo/workflows/ci.yml: 4 POSTGRES_MAJOR injections (pull+up × test+prod). Gating, needs, if: conditions, secrets, env-file handling, SSH/scp flow all unchanged. Empty var → compose :-15 fallback → identical behavior. Verified: YAML parses, all 4 jobs intact.

Verified (real, isolated, not production)

• 21/21 RLS checks PASS on a real Postgres cluster (tenant filters, system tables readable, cross-club insert rejected).
• Auto-restore proven end-to-end: source cluster (2 clubs, 3 users) → dump → restored into fresh DB → verified data intact.
• Volume interpolation verified: unset→postgres_data, 18→postgres_data_18, 15→postgres_data_15.
• bash -n OK; existing suite 196 pass / 2 pre-existing fail (no regression).

Enable RLS in prod (separate from merge)

1. Set HONBU_ENABLE_RLS=1 on the superuser/migration boot. 2. Point runtime DATABASE_URL at honbu_app. Safe to leave off — ships disabled (legacy parity).

## What Replaces the abandoned database-per-club split with **Postgres Row-Level Security** for tenant isolation, plus a **fully-automated, CI-driven Postgres 15 → 18 upgrade** run through the app startup entrypoint. ## RLS (single-DB, database-enforced) - `app/infrastructure/database/rls.py` — opt-in bootstrap: creates a non-superuser `honbu_app` role, transfers **ownership** of the 13 club-scoped tables, enables `FORCE ROW LEVEL SECURITY` with tenant policies keyed on the `app.club_id` GUC. - **Direct tables** (members, families, attendance, class_schedules, user_club_assignments, api_tokens, form_configurations, content_items, content_club_approvals) filter on `club_id`. - **Indirect tables** (belt_levels, gradings, notes_history via members; family_guardians via families) derive club through parent FK. - Auth decorators set per-request `SET LOCAL app.club_id` — transaction-scoped, pool-safe. - Opt-in via `HONBU_ENABLE_RLS=1` + runtime engine connects as `honbu_app`. Default (superuser) behaviour byte-for-byte unchanged. ## Automated Postgres 15 → 18 upgrade (CI-driven, zero operator DB touch) **Triggered by a Forgejo Actions variable, `POSTGRES_MAJOR`.** Routine deploys are unchanged. | `POSTGRES_MAJOR` | Image | Data volume | Behavior | |---|---|---|---| | unset (default) | `postgres:15-alpine` | `postgres_data` (stable, = current prod) | routine deploy, no change | | `18` | `postgres:18-alpine` | `postgres_data_18` (fresh) | upgrade path — old `postgres_data` preserved as rollback/safety | | `15` | `postgres:15-alpine` | `postgres_data_15` (fresh) | deliberate re-init | Sequence on upgrade: 1. **CI deploy job** injects `POSTGRES_MAJOR='${{ vars.POSTGRES_MAJOR || '' }}'` into the test+prod `docker compose pull`/`up` commands (read from the Forgejo Actions variable). 2. compose creates a **fresh `postgres_data_<major>`** volume + `postgres:<major>` image (old volume intact). 3. **App entrypoint** (runs before web server): takes an idempotent pre-upgrade `pg_dump` once per major → auto-**restores** the prior-major dump into the fresh DB → applies migrations → RLS bootstrap → starts server. 4. Maintenance window = exactly the upgrade; app not serving during it. Idempotent, no double-restore. ## CI safety — the hard-won workflow is untouched Only additive change to `.forgejo/workflows/ci.yml`: 4 `POSTGRES_MAJOR` injections (pull+up × test+prod). Gating, `needs`, `if:` conditions, secrets, env-file handling, SSH/scp flow all unchanged. Empty var → compose `:-15` fallback → identical behavior. Verified: YAML parses, all 4 jobs intact. ## Verified (real, isolated, not production) - **21/21 RLS checks PASS** on a real Postgres cluster (tenant filters, system tables readable, cross-club insert rejected). - **Auto-restore proven end-to-end**: source cluster (2 clubs, 3 users) → dump → restored into fresh DB → verified data intact. - **Volume interpolation verified**: unset→`postgres_data`, 18→`postgres_data_18`, 15→`postgres_data_15`. - `bash -n` OK; existing suite 196 pass / 2 pre-existing fail (no regression). ## Enable RLS in prod (separate from merge) 1. Set `HONBU_ENABLE_RLS=1` on the superuser/migration boot. 2. Point runtime `DATABASE_URL` at `honbu_app`. Safe to leave off — ships disabled (legacy parity).

fastie81 added 1 commit 2026-09-06 19:10:32 +12:00

feat(rls): Postgres row-level tenant isolation prototype + Postgres 18 ... 9c7302663b

- Add app/infrastructure/database/rls.py: opt-in RLS bootstrap.
  Creates honbu_app role, transfers club-table ownership, enables
  FORCE ROW LEVEL SECURITY with tenant policies keyed on the
  app.club_id GUC (verified against a real Postgres: 21/21 checks).
- Auth decorators set per-request club context via SET LOCAL
  app.club_id (transaction-scoped, pool-safe). No-op unless
  HONBU_ENABLE_RLS=1, so existing behaviour is byte-for-byte intact.
- enable RLS in init_database when env set + running as superuser;
  clear context on request teardown.
- Bump Postgres 15 -> 18 in docker-compose.yml and CI (single-DB
  model retained; no data migration needed for RLS approach).

fastie81 referenced this pull request from a commit 2026-09-06 19:16:11 +12:00

feat(rls): drive RLS bootstrap + PG pre-upgrade safety dump from entrypoint

fastie81 added 1 commit 2026-09-06 19:16:12 +12:00

feat(rls): drive RLS bootstrap + PG pre-upgrade safety dump from entrypoint ... b611abb69e

entrypoint.sh already called app.db.init_database() on every boot, which
runs bootstrap_rls when HONBU_ENABLE_RLS=1 + superuser (PR #138 core change).

This adds:
- An explicit idempotent pre-upgrade safety dump (once per PG major) written
  to BACKUP_PATH/pgupgrade/ before migration, using the bundled
  postgresql-client pg_dump.  Guards the postgres:15 -> 18 image/volume swap
  so no data is lost if the operator flips versions.
- Documented that the APP container cannot perform the volume swap itself
  (postgres must start first); the dump is the restore source.
- Disable with HONBU_UPGRADE_SAFETY_DUMP=0.

fastie81 added 1 commit 2026-09-06 19:21:57 +12:00

feat(upgrade): zero-touch automated Postgres major upgrade via entrypoint ... d4c4fac3b4

- Compose: versioned postgres volume + image driven by POSTGRES_MAJOR
  (postgres_data_${POSTGRES_MAJOR}). Deploying 15->18 creates a FRESH data
  volume automatically while the prior major volume is preserved as a
  safety copy.
- Entrypoint (maintenance mode): on boot, auto-restores the newest
  prior-major pg_dump into the fresh/empty DB, then runs migrations + RLS
  bootstrap, then starts the web server LAST.  Server is down only for the
  maintenance window covering the upgrade; no operator DB touch.
- Idempotency: dump taken once per major; restore gated on DB being empty,
  so a restart never double-restores, and a populated DB never clobbered.
- .env.example documents POSTGRES_MAJOR, HONBU_UPGRADE_SAFETY_DUMP,
  HONBU_AUTO_RESTORE.

fastie81 added 1 commit 2026-09-06 19:35:48 +12:00

fix(upgrade): stable postgres_data volume + wire POSTGRES_MAJOR from Forgejo Actions var ... 4ff00ff8d1

- CRITICAL FIX: revert the versioned postgres_data_${POSTGRES_MAJOR} volume.
  It would have orphanned prod data on the first routine deploy (prod runs PG15
  on a volume literally named postgres_data). Data volume is now stable so a
  code deploy can never orphan live data.
- postgres image default back to postgres:${POSTGRES_MAJOR:-15}-alpine (matches
  current prod/test) so nothing changes until POSTGRES_MAJOR is explicitly set.
- CI deploy job: inject POSTGRES_MAJOR="${{ vars.POSTGRES_MAJOR || "" }}" into
  both test and prod docker compose pull + up commands, read from a Forgejo
  Actions variable. Empty (unset) -> compose falls back to :15 -> no behavior
  change. CI structure, gating, needs, secrets, env-file handling untouched.
- test job service bumped to postgres:18-alpine so tests exercise the target.

fastie81 added 1 commit 2026-09-06 19:37:20 +12:00

refine(upgrade): conditional postgres volume - stable when unset, versioned on explicit POSTGRES_MAJOR ... 232f72f15a

- When POSTGRES_MAJOR is UNSET (default): volume stays exactly postgres_data
  (matches current prod/test) - routine deploys never orphan live data.
- When POSTGRES_MAJOR is EXPLICITLY set (e.g. 18): volume becomes
  postgres_data_<major> (fresh), image becomes postgres:<major>. Old
  postgres_data (PG15) is preserved as rollback/safety. The app entrypoint
  auto-restores a prior-major dump into the fresh volume on first boot.
- Interpolation verified: unset->postgres_data, 18->postgres_data_18,
  15->postgres_data_15.

fastie81 merged commit a9f4f9d52b into main 2026-09-06 19:42:55 +12:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

architecture

Architecture change

  

backend

Backend (Python/Flask)

  

bug

Something is not working

  

ci

CI/CD pipeline

  

config

Configuration issue

  

database

Database related

  

docker

Docker/container related

  

documentation

Documentation

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Frontend (React/TypeScript)

  

help wanted

Need some help

  

invalid

Something is wrong

  

issue-25

Related to issue #25

  

issue-26

Related to issue #26

  

issue-42

Related to issue #42

  

issue-65

Related to issue #65

  

issue-67

Related to issue #67

  

issue-82

Related to issue #82

  

migrations

Database migrations

  

p0-critical

Critical priority - immediate fix needed

  

p1-high

High priority

  

p2-medium

Medium priority

  

p3-low

Low priority

  

performance

Performance issue

  

quality

Code quality

  

question

More information is needed

  

security

Security issue

  

state-management

Frontend state management

  

testing

Testing related

  

timezone

Timezone related

  

wontfix

This won't be fixed

No labels

architecture

backend

bug

ci

config

database

docker

documentation

duplicate

enhancement

frontend

help wanted

invalid

issue-25

issue-26

issue-42

issue-65

issue-67

issue-82

migrations

p0-critical

p1-high

p2-medium

p3-low

performance

quality

question

security

state-management

testing

timezone

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

fastie81/honbu-manager!138

No description provided.