Feature flags: design
Status: items 1 to 5 built (SWFT-2393, epic SWFT-2391); item 6 (pilot) not started. This design replaces the Flagsmith plan the epic first described.
Why
Large PRs are slow to review and hard to verify. Product can only test a change
after it merges into develop, and once there it is hard to revert. Feature
flags let unfinished work merge in small PRs while it stays off. Product turns
the flag on for a QA tenant, and turning it off replaces a revert.
Decisions
Decision |
Choice |
Why |
|---|---|---|
Build or buy |
Build. Keep the API close to OpenFeature ( |
|
Scope |
A global default, then a per-tenant override, then a per-user override. The most specific setting wins. |
Per-tenant lets product test in QA. Per-user lets one person try a feature in a shared tenant. |
Declaration |
Flags are declared in code ( |
A typo fails instead of reading as off, flag metadata goes through code review, and the expired-flag CI check needs no database. |
Storage |
|
|
Permissions |
Two internal permissions, |
Clients use the admin console and can manage their own roles; turning flags on and off is an internal operation. |
Flag lifetime |
Each flag stores an owner, its Jira ticket and a removal date. A CI check fails once a flag is past its removal date. |
Without this, old flags pile up. |
Who switches flags in prod |
Only people with an ops or admin role, and only through Django admin or the management command. Every change is logged. Dev and QA are open to the team. |
Switching a flag on in prod is effectively a release, so it needs an approval record that fits the change management policy (SWFT-1944). |
Existing |
Leave them as they are. |
Those are permanent product settings per tenant, not temporary rollout flags. |
When the lookup fails |
Keep using the last snapshot that loaded successfully. A flag counts as off only when a process has never loaded a snapshot (cold start with the database unavailable). |
A Redis or database outage must not quietly switch released features back off across the fleet. |
Performance
The runtime is uvicorn with 4 workers per pod, several pods, and django-q2
workers. Redis is reached through django_redis with a 5-second socket timeout
(swifty_django/settings.py, _make_cache). A flag check must therefore never
call Redis or the database directly: a Redis hiccup would add up to 5 seconds to
every request that checks a flag.
Each process holds every flag and override in memory. That is tens of flags and a few hundred overrides, a few KB, loaded with one query.
flag_enabled()is a dictionary lookup with no I/O.Each process runs at most one Redis
GET flags:versionevery 5 seconds, and reloads from the database only when the version changes.Total cost: a few Redis GETs per second across the fleet, and one small query per process per flag change.
How a change propagates
admin / command ──► DB row saved (source of truth) + FeatureFlagChange row
└─ transaction.on_commit ──► Redis INCR flags:version
each process, every ≤5s ──► GET flags:version ── changed? ──► reload snapshot from DB
safety net: reload from DB every 60s regardless
Flutter ──► fetch flags at login, on app resume, and when the API's
X-Flags-Version response header differs from what it has
How stale a flag can be: about 5 seconds normally, and at most 60 seconds while Redis is down.
Race conditions
A flag changes during a request. Middleware takes a snapshot of the flags into a context variable when the request starts, and the whole request uses it. A request never runs half on the old code path and half on the new one.
A flag changes during a django-q2 chain or any other long-running process. The process decides once, when it starts. It stores the flag values in the task arguments (or on the domain object, such as the prescription), and every later step reads them from there instead of calling
flag_enabled()again. This is a firm rule.Pods disagree for a few seconds. That window can’t be avoided. The old code path must handle data written by the new one, because turning a flag off is the rollback. A flag rolls back code, not data. Migrations follow expand/contract (SWFT-1948).
The UI and backend disagree. The backend always decides. A flagged endpoint returns a clear error when the flag is off, and the UI reloads its flags when it gets that error. A UI flag only hides or shows things; it doesn’t enforce anything.
The version is bumped before the database commit. Bump the version only in
transaction.on_commit, and when reloading, read the version before querying the database. A change that lands between the two is caught on the next poll. When two admins edit the same flag, the last write wins and the history table records both changes.A process starts cold. A new pod, or a django-q2 worker recycled after 100 tasks, loads the snapshot the first time it checks a flag.
Backend (item 2)
Code: swifty_django/swifty_backend_shared/feature_flags/.
Module |
What it does |
|---|---|
|
Declare flags; |
|
Pick the value: user, then tenant, then global, then code default |
|
The per-process snapshot: 5-second version poll, 60-second reload, keep the last good snapshot on failure |
|
|
|
One snapshot per request: Django middleware for the admin, and a WSGI wrapper for the API’s connexion app ( |
|
Flag pages with overrides inline and a required reason; read-only change history |
|
For dev, QA and scripts; |
flag_enabled(flag, tenant=..., user=...) uses connection.tenant when no
tenant is passed. Outside a request (django-q2), checks use the process
snapshot, so call capture_flags() once at the start of a chain.
Tests: feature_flags/tests/ (no database, in scripts/python_unit_tests.sh)
and swifty_api/tests/test_feature_flags.py (Postgres). The API test
conftest.py loads the snapshot before each database test, as a running
process already has it, so query-count assertions don’t see the load.
API (item 3)
GET /swifty/1/feature_flags(swifty_api/controllers/feature_flags_controller.py) returns{"version": n, "flags": {"name": true, ...}}: every declared flag, resolved for the signed-in user in the request’s tenant. Any signed-in user may call it; flag names are visible to clients.Every API response carries
X-Flags-Version(the request’s snapshot version, exposed through CORS). It is 0 when the version is unknown.The version only goes up. The Redis counter starts at the current time in milliseconds whenever the key is missing (first use, a flush, or Redis restarting empty), so it is always past any earlier count. Clients can therefore ignore a lower version: it comes from a process that hasn’t seen the latest change yet.
The values only show or hide UI. A flagged endpoint checks the flag itself.
Flutter (item 4)
Code: swifty_ui/lib/providers/feature_flags.dart.
FeatureFlagsNotifier.instanceholds each pharmacy’s flags, keyed by domain, and is registered as a provider inmain().FeatureFlagVersionInterceptoris on every Swifty API client (createRetryingDiowithendsExpiredSession). When a response’sX-Flags-Versiondiffers from the version loaded for that host, it loads the flags with the same credentials as that request. Sign-in, switching pharmacy and a flag change therefore need no explicit load. A version lower than (or equal to) the one loaded or loading is ignored, as is 0. A failed load keeps the old flags and waits 60 seconds before trying that version again.context.flagEnabled(Flags.x)andFeatureFlagGateread the flags for the selected pharmacy. A flag is off until loaded. Sign-out clears them.Page tests:
pumpPage(..., flags: {...}).
Process and compliance (item 5)
AGENTS.mdhas the rules: when to use a flag, declaring it in both places, PR size, “the backend decides”, “decide once” for chains, expand/contract, tests, and removal.CI (
test_flags.py, part ofscripts/python_unit_tests.sh) fails when a flag is pastremove_by, or when Flutter’sFlagsnames a flag thatflags.pydoesn’t declare. CI runs it for Django changes and for changes toswifty_ui/lib/providers/feature_flags.dart.Change management (for SWFT-1944; needs the policy owner’s decision). Proposed wording: “A feature flag and both code paths it selects are reviewed and approved in the PR that adds it. Turning a flag on or off in production is a standard change: it is made by someone holding the Change feature flags permission, through the admin, with a recorded reason. The append-only flag history records who, when, what and why. Turning a flag off to stop a problem follows the same path, with the incident as the reason.” Open question for the policy owner: does a production flip also need a second person’s approval? Today the system records the change but doesn’t require a second approver.
Plan
Design decisions (this document)
Django backend: models,
flag_enabled(), snapshot cache, request middleware, admin and management command, testsAPI: an endpoint for the active flags, the OpenAPI spec, regenerated clients, and the
X-Flags-VersionheaderFlutter: load flags, a
FeatureFlagsprovider and helper,pumpPagesupportProcess and compliance: prod change approval (SWFT-1944), PR size rule, a removal ticket per flag,
AGENTS.mdsectionPilot: one real feature behind a flag end to end