Observability

The observability / production layer (Track O) gives your app telemetry, structured logs, an error boundary, feature flags and client auth — all in typed Python, identical whether Python runs in the browser (Mode A) or on the server (Mode B). 📊

Shipped surface (Track O · O0–O4)

All five phases are in the package and importable from tempestweb.observability. Each one has a complete app under Examples: feature flags, error boundary + telemetry and JWT auth.

The adapter pattern

Every provider follows the same principle: a minimal interface that you swap without touching the app. You program against the provider; the adapter decides where it goes (console, Sentry, GrowthBook, …).

   your app  ──calls──▶  Provider (stable API)  ──delegates──▶  Adapter (backend)
                                                                console / sentry / posthog / ...

Swapping backend does not change calls

Migrating from console to sentry changes no track() call. It is the same promise as tempest-react-sdk, now in typed Python.

A provider is an object, not a singleton

There is no global init(): you build the provider with the adapter you want and keep the instance (in a module, in your State, wherever fits). In Mode A every tab has its own; in Mode B every session has its own. No hidden global state to leak between users.

O0 — Telemetry

Instruments framework and app events (service worker, push, offline replay, errors) with a pluggable provider.

from tempestweb.observability import ConsoleTelemetryAdapter, TelemetryProvider

telemetry = TelemetryProvider(ConsoleTelemetryAdapter())

telemetry.track("order_submitted", {"items": 3, "total": 99.9})
telemetry.identify("user-42", {"plan": "pro"})

The constructor takes two knobs that matter in production:

from tempestweb.observability import ConsoleTelemetryAdapter, TelemetryProvider

telemetry = TelemetryProvider(
    ConsoleTelemetryAdapter(),
    default_props={"app": "checkout", "release": "1.4.0"},
    sample_rate=0.1,
)

• default_props rides along on every event, so you stop repeating the same dict.
• sample_rate=0.1 sends 10% of events — the cut happens in the provider, before the adapter, so the backend never sees the rest.

Swapping backend means swapping the adapter: PostHogTelemetryAdapter, SentryTelemetryAdapter, or your own (the interface is TelemetryAdapter). To capture events in a test, the console adapter takes a sink:

from typing import Any

from tempestweb.observability import ConsoleTelemetryAdapter, TelemetryProvider

captured: list[Any] = []
telemetry = TelemetryProvider(ConsoleTelemetryAdapter(sink=captured.append))
telemetry.track("checkout_opened")

Do not leak PII

Keep personal data out of props and use sample_rate so you do not flood the backend. Telemetry is diagnostics, not a user database.

O1 — Logger

Structured logging with pluggable sinks and typed levels (LogLevel is Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]).

from tempestweb.observability import console_sink, create_logger

log = create_logger(sinks=[console_sink], level="INFO")

log.info("order created", order_id="o-1", total=99.9)
log.error("payment failed", order_id="o-1", reason="card_declined")

Every extra **fields rides in the LogRecord (level, message, fields), so a network sink serializes the whole record instead of parsing a string.

In Mode A the default sink is the browser console

Network sinks (shipping logs to a server) must be async/non-blocking — in Mode A a blocking sink freezes the tab.

O2 — Error boundary

Catches a render error → shows a visual fallback and fires a report, without taking the app down. The rest of the tree stays alive.

ErrorBoundary is a widget, and it takes its child as a builder (child_builder), not as an already-built widget — that is what lets it run the build inside the try.

from tempest_core import Text, Widget

from tempestweb.observability import (
    ErrorBoundary,
    ErrorInfo,
    TelemetryProvider,
    telemetry_reporter,
)


def panel(telemetry: TelemetryProvider) -> Widget:
    """Build the dashboard panel, guarded by a boundary."""
    return ErrorBoundary(
        key="dashboard",
        child_builder=lambda: build_dashboard(),
        fallback_builder=lambda info: Text(content=f"Something broke: {info.message}"),
        on_error=telemetry_reporter(telemetry),
    )

The ErrorInfo handed to the fallback and to on_error carries error, error_type, message and stack — enough to show the type on screen and ship the stack to the backend.

When the pattern is always the same, with_error_boundary builds the decorator:

from tempest_core import Text, Widget

from tempestweb.observability import ErrorInfo, with_error_boundary


@with_error_boundary(
    fallback_builder=lambda info: Text(content=f"Something broke: {info.message}"),
)
def risky_panel() -> Widget:
    """Build a panel that may raise during build."""
    return build_dashboard()

The decorator wraps a zero-argument builder and returns another callable: calling risky_panel() hands you the ready ErrorBoundary to put in the tree. Without fallback_builder, default_fallback takes over.

Render error ≠ async handler error

The boundary catches render errors (during the child's build). Async handler errors go to the event loop's handling. In both cases, report — never swallow the stack.

O3 — Feature flags

Toggles features at runtime with gradual rollout. The adapter interface is tiny (get + subscribe), so writing a new one takes ~20 lines.

from tempestweb.observability import FeatureFlagsProvider, InMemoryFeatureFlagsAdapter

flags = FeatureFlagsProvider(InMemoryFeatureFlagsAdapter({"new_checkout": True}))


def view() -> object:
    """Render checkout, gated by a feature flag."""
    if flags.is_enabled("new_checkout"):
        return new_checkout()
    return legacy_checkout()

• is_enabled(key, default=False) coerces the value to bool — a missing flag falls back to default.
• get(key, default) returns the raw value (bool, str, number) for variant flags: flags.get("checkout_variant", "control").
• on_change(listener) registers a zero-argument callback (it says something changed; you re-read the flag) and returns the unsubscribe function.

unsubscribe = flags.on_change(lambda: app.request_rebuild())

In production, swap the adapter for GrowthBookFeatureFlagsAdapter or LaunchDarklyFeatureFlagsAdapter — no is_enabled call changes.

Flags are not secrets; keep a safe default

When the flag backend is down, is_enabled falls back to the safe default — it never breaks the app. And never use flags to hide secrets: they are visible on the client.

O4 — Client auth

Auth store + route guard + JWT helpers + a refresh queue that serializes concurrent renewals (one renewal, many waiters).

from tempestweb.observability import (
    create_auth_store,
    create_refresh_queue,
    is_jwt_expired,
    route_guard,
)

auth = create_auth_store()


async def renew() -> str:
    """Fetch a fresh token from the backend.

    Returns:
        The new access token.
    """
    response = await app.native.http.request("POST", "/api/refresh")
    return str(response.json_body["token"])


refresh = create_refresh_queue(auth, renew)
guard = route_guard(auth, redirect_to="/login")


async def call_api() -> dict[str, object]:
    """Call a protected endpoint, refreshing the token once if needed.

    Returns:
        The decoded JSON response.
    """
    token = auth.token
    if token is None or is_jwt_expired(token):
        token = await refresh.refresh()
    response = await app.native.http.request(
        "GET", "/api/me", headers={"Authorization": f"Bearer {token}"}
    )
    return dict(response.json_body)

The queue is the subtle part: refresh.refresh() is single-flight. Ten concurrent callers that find the token expired trigger one renewal and all await the same result — refresh.refresh_calls counts the real renewals, and that is what you assert in a test.

The store holds the session and notifies subscribers:

from tempestweb.observability import create_auth_store

auth = create_auth_store()
auth.login(token, {"name": "Ana"})   # or set_token(token) to swap only the token
unsubscribe = auth.subscribe(lambda: app.request_rebuild())

print(auth.is_authenticated, auth.user, auth.token)
auth.logout()

decode_jwt(token) reads the claims without verifying the signature (this is the client: verification is the server's job) and is_jwt_expired(token, leeway_seconds=30) decides expiry with slack.

The token lives in different places per mode

In Mode A the token lives in the browser (storage) — treat XSS as a real risk. In Mode B it lives in the server session, better protected. The server reuses JWTUtils from tempest-fastapi-sdk, and server_decode_jwt verifies it with a secret.

S8 — Server observability (Mode B)

create_app(..., metrics=True) already answered how many sessions exist. It did not answer whether they are slow, where the time goes, or what the server did for the client that just complained — and Mode B is the mode that gets operated in production.

from tempestweb.observability import (
    PatchMetrics,
    ServerObservability,
    create_logger,
    json_log_sink,
    otel_tracer,
)
from tempestweb.server import create_app

app = create_app(
    state_factory=lambda: 0,
    view=view,
    metrics=True,
    observability=ServerObservability(
        metrics=PatchMetrics(),
        logger=create_logger(sinks=[json_log_sink], level="INFO"),
        tracer=otel_tracer(),  # optional: needs tempestweb[otel]
    ),
)

Latency and throughput

The histogram lands in GET /metrics, next to the connection counters:

tempestweb_patch_seconds_bucket{le="0.005"} 40
tempestweb_patch_seconds_sum 0.012
tempestweb_patch_seconds_count 40
tempestweb_patches_total 40

What it measures is the wait the client feels: from the event arriving to its patches reaching the transport, rebuild included. That matters because the rebuild is coalesced — it runs after the handler returns. Timing the handler would report a number that stops before the work the client is waiting for (measured: it read as rounds with zero patches).

The number agrees with the client's

Measured on a real 40-row app: the client saw 0.62 ms round-trip, the server 0.30 ms of patch time — 49% of the wait is the server, the rest is WebSocket and loopback. The server's number is always the smaller one; when it approaches the client's, the network is not your problem.

Structured logs

One JSON line per lifecycle event, with session_id as a field — which is the point: it joins to the span by the same key.

{"level": "INFO", "message": "session.open", "session_id": "s-7f60c8ba0980", "transport": "ws"}
{"duration_s": 0.027, "level": "INFO", "message": "session.close", "reason": "closed", "session_id": "s-7f60c8ba0980", "transport": "ws"}

A session that dies of an exception closes with reason set to the exception's name, not "closed".

Tracing

A span per session, per dispatch and per patch batch, behind an adapter. otel_tracer() imports opentelemetry inside the function: the default never touches the library, and an app that does not trace does not pay the import. Exporter and sampler stay with OpenTelemetry (env vars or an SDK setup the app owns) — wrapping those would be a second configuration surface, worse than the first.

The cost when it is off, measured

Default (observability=None): dispatch takes no clock and opens no span. With metrics and structured logs on, 200 clicks on a 40-row app went from 0.665 ms to 0.689 ms average — +3.6%. That is the price of knowing what is happening.

Performance: what is measured, and how the gate avoids being a flake

Three measurements in three places, because they cost very different amounts to collect.

The gate that blocks a PR

uv run python benchmarks/perf_gate.py

benchmarks/perf_gate.py runs in CI and fails the job. The hard part of a perf gate is not measuring — it is not being a flake: a shared runner varies by more than the regressions worth catching, so an absolute threshold either fires on noise (and gets disabled in the first week) or is loose enough to catch nothing. So it asserts only what survives a slow machine:

Claim	Why it survives noise
doubling the rows costs at most ~2.6x	it is a ratio between two measurements on the same machine, back to back; an O(n²) lands near 4x. Each measurement is the fastest of five rounds, not the median: interference can only add time, and on a shared runner it holds for most of a window — which is how a diff that scales 2.04x locally reported 2.85x in CI and failed main
one changed row still yields 2 patches	correctness, not time — and the cheapest way to make a diff look fast is to stop being right
calibrated cost within 2.5x of the baseline	the cost is divided by a calibration loop measured in the same process, which takes CPU speed out of the comparison — but not memory/GC: the same build cost 975.8, 1130.9 and 1206.9 units on three CI runners against a 667.7 baseline taken on a developer machine, so this limit is a coarse tripwire (it catches a doubling) and the precision lives in the scale ratios
N sessions sustain one session's total throughput	the loop is single-threaded and the rebuild is CPU-bound, so the total stays flat while the per-session share divides; a drop in the total is contention, not load

A deliberate change in cost: --update-baseline, and justify it in the PR. The baseline is versioned (benchmarks/baseline.json).

Mode B throughput

uv run python benchmarks/bench_ws_throughput.py --sessions 10 --events 100

Measures the loop a Mode B app lives in: an event arrives, a handler mutates state, the core diffs, the transport ships a batch. The transport counts instead of writing to a socket — what is under test is the Python above it, and the network only ever makes the number smaller; this is the ceiling.

What the measurement shows: the total stays roughly constant while the per-session share divides. In other words, Mode B saturates on CPU in the rebuild, inside a single event loop. Scaling means more processes (Mode B is stateful per session, so each process needs session affinity), not more threads.

Mode A cold start

npm install --no-save playwright && npx playwright install chromium
node benchmarks/bench_cold_start.mjs http://127.0.0.1:8000/

Runs in a scheduled job (perf-cold-start.yml), not on a PR: a ~6 MB Pyodide download in every PR's critical path buys a number nobody reads at that moment. It measures two values for the same page, and both matter:

• cold — no service worker, no cache: Pyodide and the core come over the network. This is a first visit.
• warm — the SW precache serves them. This is every visit after.

The clock stops when the app's first tree is on screen (the first [data-tw-key]), which is the honest definition of "the reader can use it" — waiting for load would stop before Pyodide even starts.

Measured, and the result is counter-intuitive

examples/counter in a real Chrome, against the built artifact: cold 2,394 ms with 14,593 KB transferred; warm 2,354 ms with 8,751 KB. The service worker saved 5.8 MB of network and 40 ms of time — 40% of the bytes and 1.7% of the clock.

The reading matters more than the numbers: in Mode A the dominant cost is the Pyodide boot (CPU), not the download. Optimising the network there does not move the needle; anyone who needs first paint uses Mode B or C — which the architecture docs already said, and now there is a measurement.

Recap

• Observability uses the adapter pattern: swap the backend without changing the app.
• A provider is an object you build — TelemetryProvider(adapter), FeatureFlagsProvider(adapter) — there is no global init().
• Telemetry (O0), Logger (O1), Error boundary (O2), Feature flags (O3) and Auth (O4) are all typed Python, identical in Modes A and B.
• ErrorBoundary takes builders, not built widgets; the refresh queue is single-flight.
• Safe defaults and care with PII/tokens are part of the contract.

This layer mirrors the tempest-react-sdk providers. To see it all together in a running app, start with error boundary + telemetry. 🚀