Architecture Proposal
Definitive tech stack for the OpenRally Protocol server. Rust (Axum) on Fly.io, Dragonfly for cache/pub-sub, Turso for edge reads, and Neon Postgres for primary writes. This is the best-in-class architecture — modeled after how Stripe, Cloudflare, and OTA build protocol infrastructure. The Setpoint admin app stays on its current stack — only the protocol server moves.
Definitive architecture recommendation — not a proposal, the answer.
This is not one option among many — it is the definitive recommendation. The v1 stack (Supabase/Vercel/Next.js) is architecturally incapable of running a protocol server: SSE breaks at 300s, serverless cold starts violate SLAs, and Node.js GC pauses make hold-creation latency non-deterministic. Every serious protocol and API infrastructure in production (Stripe, Cloudflare, OTA, Google Maps Platform) runs on compiled languages with always-on servers. This stack is the proven pattern.
Definitive tech stack for the OpenRally Protocol server. Rust (Axum) on Fly.io, Dragonfly for cache/pub-sub, Turso for edge reads, and Neon Postgres for primary writes. This is the best-in-class architecture — modeled after how Stripe, Cloudflare, and OTA build protocol infrastructure. The Setpoint admin app stays on its current stack — only the protocol server moves.
The v1 architecture was designed as an extension of the Setpoint admin app. A protocol server has fundamentally different requirements.
| Requirement | v1 Stack | Problem |
|---|---|---|
| Persistent connections (SSE) | Vercel serverless, 300s max | SSE breaks every 5 minutes. Reconnect overhead. |
| Deterministic latency | Node.js single-threaded, GC pauses | p99 spikes during garbage collection |
| Always-on | Serverless cold starts (500ms-2s) | Unacceptable for Google Booking Server callbacks |
| Concurrent SSE | ~50 connections (serverless limits) | Protocol needs 10,000+ per region |
| Cache layer | None defined | Every availability query hits database |
| Global distribution | Single region (Tokyo via Supabase) | 200ms+ latency for non-APAC consumers |
10-15x throughput vs Node.js. Sub-ms p99 latency. Zero GC pauses. Native async SSE via tokio. Serde for zero-copy JSON. SQLx for compile-time checked queries.
Always-on MicroVMs in 40+ regions. Anycast routing. Native long-lived connections. No cold starts. ~$0.0000573/s per shared CPU.
Serverless Postgres with branching. Strong consistency for writes (holds, bookings). Connection pooling built-in. Scale-to-zero for dev.
Embedded replicas — sub-1ms reads at edge. 26+ global locations. Perfect for availability queries. Replication lag < 100ms.
Multi-threaded Redis-compatible. 25x throughput over Redis. 60% better memory. Pub/Sub for SSE fan-out. Runs on Fly.io.
Distributed tracing, metrics, logs. Rust tracing crate → OTLP exporter.
In-Process HashMap (DashMap)
Hot paths: sport list, facility metadata
Dragonfly
Availability results, rate limits, sessions
Turso Embedded Replica
Facility/resource/slot reads, discovery
Neon Postgres
Booking writes, holds, event log
Invalidation: Write to L4 (Neon) → trigger emits event → Dragonfly Pub/Sub → invalidate L1 + L2 in all instances → Turso replicates automatically (<100ms)
Neon Write
DB trigger → or_events
NOTIFY
Postgres → Rust listener
Dragonfly
PUBLISH to channels
Axum SSE
Fan-out to 10K+ clients
10,000+
Concurrent connections/VM
Unlimited
Connection duration
<5ms
Event delivery
| Operation | v1 (Node/Vercel) | v2 (Rust/Fly.io) | Improvement |
|---|---|---|---|
| Availability query | 200-500ms | <5ms (cache hit) | 40-100x |
| Availability (miss) | 200-500ms | <50ms (Turso edge) | 4-10x |
| Hold creation | 300-800ms | <10ms | 30-80x |
| Booking confirmation | 500-1500ms | <50ms | 10-30x |
| SSE event delivery | Broken (300s limit) | <5ms | Fixed |
| Concurrent SSE | ~50 | 10,000+ per VM | 200x |
| Cold start | 500ms-2s | 0ms (always-on) | Eliminated |
| Throughput (req/s) | ~1,000 | 50,000+ | 50x |
Line-item costs based on official vendor pricing (Fly.io, Neon, Turso, Grafana Cloud). All prices in USD.
$4.02/mo
1× shared-cpu-1x / 256MB (protocol server)
Anycast IPv4
Free tier (100 CU-hrs, 0.5GB)
Free tier (500M reads, 5GB)
Skip (in-process cache only)
Free tier
$19.13/mo
1× shared-cpu-1x / 512MB (primary, Madrid)
1× shared-cpu-1x / 256MB (Dragonfly)
Anycast IPv4
Volumes (1GB, Dragonfly snapshots)
Bandwidth (~50GB/mo)
Launch plan (~50 CU-hrs, 1GB storage)
Developer plan
Free tier
$71.49/mo
2× shared-cpu-2x / 1GB (primary + replica)
2× shared-cpu-1x / 256MB (Dragonfly ×2 regions)
Anycast IPv4
Volumes (2GB)
Bandwidth (~200GB/mo)
Launch plan (~200 CU-hrs, 5GB storage)
Scaler plan
Free tier (still under limits)
$241.19/mo
4× shared-cpu-2x / 1GB (2 primary + 2 replicas)
3× shared-cpu-1x / 512MB (Dragonfly ×3 regions)
Anycast IPv4
Volumes (5GB)
Bandwidth (~1TB/mo)
Launch plan (~500 CU-hrs, 20GB storage)
Scaler plan + overage (~50GB storage)
Pro ($19 base + ~20K series)
Pricing basis: Fly.io shared-cpu-1x/256MB $2.02/mo, 512MB $3.32/mo, 2x/1GB $6.64/mo, IPv4 $2/mo, bandwidth $0.02/GB. Neon Launch $0.106/CU-hr + $0.35/GB storage. Turso Dev $4.99/mo, Scaler $24.92/mo. Grafana Free 10K series / 50GB logs, Pro $19 base.
axumHTTP framework (tower-based, async)tokioAsync runtimeserde / serde_jsonZero-copy JSON serializationsqlxCompile-time checked SQL (Postgres + SQLite)redisDragonfly client (Redis-compatible)libsqlTurso embedded replica clienttower / tower-httpMiddleware: rate limiting, auth, CORS, compressionaxum-extraSSE support (Sse, Event)schemarsGenerate JSON Schema from Rust typestracing + tracing-opentelemetryStructured logging + distributed tracingdashmapLock-free concurrent HashMap (L1 cache)uuidUUID v7 generation (time-ordered)chronoDate/time handling with timezone supportmad2x shared-cpu-1x, 512MB + Dragonfly + Turso replica
iad1x shared-cpu-1x, 256MB + Dragonfly + Turso replica
nrt1x shared-cpu-1x, 256MB + Dragonfly + Turso replica
Anycast: All regions share one IP address. Traffic routes to the nearest healthy instance automatically.
Setpoint Admin
Vercel / Next.js / Supabase
Stays on current stack
OpenRally Server
Fly.io / Rust / Neon + Turso
New protocol-grade infra
Two deployment models: Standalone (own Neon DB) or Setpoint Adapter (reads from Supabase, writes to Neon)
Slots: materialized or computed?
v1 Answer
Undecided
v2 Answer
Materialized in Turso — pre-compute into edge replicas, sub-1ms reads
SSE: Vercel or Supabase Edge?
v1 Answer
Vercel streaming (300s limit)
v2 Answer
Rust on Fly.io — unlimited connections, Dragonfly pub/sub fan-out
Facilities without schedules?
v1 Answer
Undecided
v2 Answer
Default schedule — 7am-11pm, 1hr slots, overridable per facility
Google Actions Center timing?
v1 Answer
Undecided
v2 Answer
Start now — v2 stack is Google-ready from day one
| Decision | Choice | Rationale |
|---|---|---|
| Separate protocol server | Own binary, own infra | Protocol server is infrastructure, not a web app feature. Different scaling, reliability, and latency requirements. |
| Rust over Node.js | Rust (Axum) | Deterministic latency. GC pauses unacceptable during hold creation. OTA runs Java/C++, Stripe core is Java, Cloudflare edge is Rust. |
| Fly.io over Vercel | Fly.io MicroVMs | Always-on, Anycast routing, native SSE/WebSocket, 40+ regions, cheaper at scale. |
| Neon over Supabase (writes) | Neon Postgres | Serverless Postgres with branching, connection pooling, no vendor lock-in. |
| Turso for edge reads | Turso embedded replicas | Sub-1ms availability reads at edge. 90%+ of traffic is reads. In-process — no network hop. |
| Dragonfly over Redis | Dragonfly | Same API, 25x throughput (multi-threaded), 60% better memory. Doubles as SSE pub/sub. |
| 4-layer cache | L1→L2→L3→L4 | Each layer serves a purpose. Invalidation via Dragonfly pub/sub propagates to all instances. |
| Setpoint stays on Supabase | No migration | Admin app is a CRUD interface — Supabase/Vercel/Next.js is fine. Only the protocol server needs protocol-grade infra. |