RTK Nowy Sącz (Regionalna Telewizja Kablowa), a regional cable TV and internet provider, runs its fiber-to-the-home network on GPON. Optical line terminals (OLTs) sit in the network, and an optical network unit (ONU) sits in every subscriber’s home. The operators use a web tool that lists the OLTs and ONUs, reads the live status of an ONU, manages ONU profiles and adds new units. It also shows the subscriber behind each one.
That tool has run since 2014 on Python 2 and Flask. It worked, but every query to the hardware blocked the server, and the code was at the end of its life. I rebuilt it while the old one keeps working.
From Python 2 to an async stack#
The backend is Python 3.13 with FastAPI, asyncpg and PySNMP, behind a new versioned API. The frontend is Vue 3 with PrimeVue. The old API runs next to the new one, so every part can be switched over, or back, on its own.
Making the hardware fast and safe to ask#
The OLTs answer over SNMP. They are slow and easy to overload. The old code opened a new SNMP session for every call, had no timeouts or retries, and silently cut off tables longer than one bulk read. Five half-finished SNMP implementations had piled up in the repository. I kept the one with the best architecture, fixed it, and put protections in front of the hardware:
- A short-lived cache. If a device stops answering, the last good data is served, marked as stale.
- Identical requests that arrive together share one query to the device.
- Each OLT has its own queue, with a limit on parallel operations and writes before reads.
- Each OLT has a circuit breaker. After repeated failures the device rests and the API answers at once instead of hanging.
- An empty table, a timeout, an unavailable device, a full queue and an unknown unit are different answers, and errors are never cached.
- Timeouts, retries and deadlines add up to less than the request limit, and a test checks that they still do.
Results#
All numbers are relative, measured on the real hardware or on an SNMP simulator.
- Reading an ONU’s host table without the old “trigger” writes is 1.6–3.1× faster, depending on the OLT. That is now the default.
- A read from the warm cache is about 50× faster than a cold read from the device.
- 20 identical concurrent requests make exactly 1 query to the device. 100 requests on a warm cache make none.
- Large tables now come back complete.
Testing and rollout#
An OLT simulator and an SNMP lab let the whole stack run locally, and the integration tests run against a simulated device. Tests that would touch live equipment refuse to start without an explicitly named target.
Writes to the database and SNMP writes to the network stay off until enabled. A new release goes live only after its health check passes, and one command rolls it back. I removed credentials and real network addresses from the repository and its whole history, and git hooks block them from coming back.