Backend & Data Platform
The Luma Optics digital twin is a streaming control plane over an AI training fabric. Telemetry flows from eBPF probes on whitebox switches through Kafka into a Flink RCA pipeline, which writes durable state to TimescaleDB + Postgres and pushes healing actions back to the edge via a Temporal workflow guarded by a policy lookup table.
Runtime Flow
Telemetry → RCA → Policy LUT → Edge Remediation- · OSFP-800G DDM
- · eBPF probe
- · gNMI push
- · Kafka topics
- · Protobuf frames
- · OTel spans
- · Flink windows
- · XGBoost classifier
- · Signature match
- · Redis O(1)
- · Risk-tier gate
- · Human override
- · Temporal.io
- · Rollback safe
- · Edge RPC
- · SSE stream
- · React 19
- · Grafana embeds
Multi-Vendor Telemetry · A-Side / Z-Side
4 of 6 endpoints streaming · normalized to canonical schemaTelemetry Normalization Matrix
vendor path → luma canonical field| Canonical | Unit | Juniper | Arista | NVIDIA Mellanox | Cisco | Nokia |
|---|---|---|---|---|---|---|
| link.rate_gbps | Gbps | interfaces/interface/state/negotiated-port-speed | Sysdb/interface/counter/eth/.../speed | sx_api_port_speed_get() | Cisco-IOS-XR-.../interface/data-rates/bandwidth | srl_nokia-interfaces:ethernet/port-speed |
| link.ber | log10 | components/optical-channel/state/pre-fec-ber | Smash/interfaces/optical/preFecBer | mlxlink --show-fec-histogram | Cisco-.../optics-oper/bit-error-rate | components/transceiver/channel/state/pre-fec-ber |
| optic.rx_power_dbm | dBm | components/transceiver/physical-channels/.../input-power | Sysdb/hardware/xcvr/.../rxPower | ethtool -m rx_power | Cisco-.../optics-oper/rx-power | components/transceiver/channel/state/input-power |
| optic.temp_c | °C | components/component/state/temperature/instant | Sysdb/environment/xcvr/temperature | sx_api_module_temperature_get() | Cisco-.../env-oper/temp/current | components/transceiver/state/temperature |
| port.flap_rate_per_min | flaps/min | interfaces/interface/state/counters/carrier-transitions Δ | Smash/interface/counter/linkStatusChanges Δ | port-toggle-count Δ | Cisco-.../generic-counters/link-flaps Δ | interfaces/interface/state/last-change Δ |
Technology Stack
Best-of-breed OSS glued by a thin control planeEdge / Layer 1 Agents
- Rust + eBPFsub-ms flap detection
In-switch telemetry probe
- gNMI / OpenConfig
Streaming model-driven telemetry
- SONiC + FRR
Whitebox NOS with policy hooks
- OSFP-800G DDM
Digital diag monitoring (BER, Rx, Tx bias, temp)
Ingest & Streaming
- Apache Kafka
Telemetry bus · 1.2M msg/s peak
- Apache Flink
Windowed anomaly + flap-rate aggregation
- Protobuf
Wire format for telemetry frames
- OpenTelemetry
Trace + span correlation across fabric
Storage & Model
- TimescaleDB
Hypertables for per-link telemetry (14-day hot)
- PostgreSQL 16
Topology, inventory, policies, incidents
- Neo4j
Fabric graph · shortest-path / blast-radius queries
- S3 + Parquet
Cold telemetry archive for model training
Intelligence & Control
- PyTorch + XGBoost
RCA classifier · 42 signature types
- Policy LUT (Redis)
O(1) signature → remediation action
- Temporal.io
Durable healing workflows w/ rollback
- Grafana + Prom
Operator dashboards & SLO alerts
Frontend Twin
- React 19 + TanStack Start
SSR digital twin console
- Tailwind v4
Design system · obsidian theme
- SVG + Canvas
Topology & telemetry rendering
- WebSocket / SSE
Live link-state push from Flink