- Python 63.6%
- JavaScript 27.1%
- CSS 6%
- HTML 3.3%
| smartstats | ||
| tests | ||
| tools | ||
| .gitignore | ||
| app.py | ||
| README.md | ||
| requirements.txt | ||
Uvos Smart Statistics
Logs sensor readings from a smartvos primary instance into SQLite and serves a web UI for plotting them as interactive time-series graphs.
smartvos primary ──TCP 38940──▶ collector thread ──▶ SQLite ──▶ Flask API ──▶ Chart.js UI
Run
pip install -r requirements.txt
python3 app.py
Then open http://localhost:8080. The collector starts in a background thread
inside the same process, so one command is all that is needed. It connects to
10.0.0.1:38940 by default and creates data/sensorlog.db on first start.
Configuration
All settings come from environment variables.
| Variable | Default | Meaning |
|---|---|---|
SMARTVOS_HOST |
10.0.0.1 |
smartvos primary address |
SMARTVOS_PORT |
38940 |
TCP port of the primary |
SMARTVOS_TRANSPORT |
tcp |
tcp or ws — see the note below |
SMARTVOS_WS_URL |
ws://<host>:<port+1>/ |
override the WebSocket URL |
SMARTVOS_DB |
sqlite:///data/sensorlog.db |
SQLAlchemy database URL |
SMARTVOS_RETENTION_DAYS |
0 |
drop readings older than N days (0 = keep forever) |
SMARTVOS_DEDUPE_SECONDS |
2.0 |
suppress an identical value seen this soon after the last one |
SMARTVOS_LOG_HIDDEN |
1 |
also log sensors flagged Hidden |
SMARTVOS_SOCKET_TIMEOUT |
15.0 |
per-read socket timeout in seconds |
SMARTVOS_POLL_INTERVAL |
10.0 |
snapshot interval, only used by the ws transport |
WEB_HOST / WEB_PORT |
0.0.0.0 / 8080 |
where the UI listens |
WEB_MAX_POINTS |
1500 |
target points per series before SQL downsampling |
Transport: TCP vs WebSocket
TCP is the default. Historically it was the only endpoint that pushed live
data: in SHinterface/src/mainobject.cpp the sensor store was wired up as
connect(&globalSensors, &SensorStore::sensorChangedState, tcpServer, &TcpServer::sensorEvent);
with no equivalent for webServer. The WebSocket server answered GetSensors
with a snapshot but never received change notifications — verified against a
live instance at the time: over TCP the collector received continuous
SensorUpdate pushes, over WebSocket exactly one message in 15 seconds, the
reply to its own request.
That gap has since been closed upstream: SHinterface commit e9d92de
(2026-10-06, "server: websocket: fix clients not getting messages about sensor
updates") also connects sensorChangedState to webServer, so current builds
push over WebSocket as well.
TCP remains the default because deployed primaries often predate that fix, and
the polling fallback costs nothing. SMARTVOS_TRANSPORT=ws still re-requests
the snapshot every SMARTVOS_POLL_INTERVAL seconds, which keeps older
primaries working at polling resolution; against a fixed build the pushes
arrive on their own and the extra polling is harmless (dedupe absorbs the
duplicate values).
Wire protocol
Reference: SHinterface/src/service/tcpserver.cpp, tcpclient.cpp,
service.cpp. Framing is symmetric:
MSG JSON LEN <n>\n
<n raw bytes of UTF-8 JSON>
Interleaved with JSON frames, the plain-text command MSG GETVERSION\n is
answered with MSG VERSION <version>\n.
The collector sends {"MessageType":"GetSensors","Data":[]} on connect and then
persists every SensorUpdate (and AddSensor) it receives. SensorStore only
emits on an actual value or metadata change, so the write rate tracks real
activity instead of a polling interval.
A sensor object looks like:
{ "Type": "Sensor", "SensorType": 12, "Id": -4009434731138787459,
"Field": 42.36, "Name": "Server Power", "GroupName": "Ground Floor",
"Unit": "W", "Hidden": false, "LastSeen": "Tue Oct 6 09:31:57 2026" }
Two details worth knowing:
- Identity is
(SensorType, Id), matchingSensor::operator==.Idis a signed 64-bit value — MQTT-backed sensors use hashed ids that are often negative, so the column isBigInteger, not unsigned. LastSeenis not usable. It isQDateTime::toString(), which is locale-dependent and unparseable. Readings are timestamped on arrival in UTC instead.
Data model
| Table | Columns |
|---|---|
sensors |
pk, sensor_type, ext_id, name, group_name, unit, hidden, first_seen, last_seen, last_value |
readings |
id, sensor_pk → sensors, ts (UTC), value |
sensors has a unique constraint on (sensor_type, ext_id); readings has a
composite index on (sensor_pk, ts) because every chart query is one sensor
over one time window. SQLite runs in WAL mode so the collector can write while
the web app reads.
HTTP API
| Endpoint | Purpose |
|---|---|
GET / |
the plotting UI |
GET /healthz |
liveness probe |
GET /api/sensors |
all sensors with their latest value, plus the group list |
GET /api/series?sensor=1&sensor=2&range=24h&agg=avg |
series data as {x: epoch_ms, y: value} |
GET /api/status |
collector state, server version, reading counts |
/api/series parameters:
sensor— repeatable, up to 128 idsrange—1h,6h,24h,7d,30d,90d,all; ignored ifstartis givenstart/end— epoch millis or an ISO timestampagg—avg,min,maxmax_points— cap per series, defaultWEB_MAX_POINTS
When a window holds more points than max_points, values are aggregated into
equal-width buckets in SQL, so a 90-day query over dense data stays fast.
Buckets align to the Unix epoch, which keeps the returned series sorted —
required by the chart layer, which consumes {x,y} pairs without re-parsing.
UI notes
- One chart per unit, so temperatures never share an axis with watts. Unit-less sensors are split by sensor type instead.
- Line colours are assigned by position within a unit group, not by global sensor index, so two charts can never collide and neighbours in one chart get maximally different hues. The sidebar swatch matches the plotted line. More than 16 sensors in a single unit still repeat hues.
- Doors, buttons and other binary types render as stepped lines.
- Ctrl+scroll zooms — plain scroll keeps scrolling the page — and dragging pans the x axis; per-chart "Reset zoom" returns to the current time window. Legend clicks toggle a series.
- Tooltips interpolate every series at the exact cursor time, so sensors with different reporting cadences all show a value instead of only the fastest one. Outside a series' own time range the value reads "–".
- The sidebar groups sensors by room group or by sensor type (toggle next to the Clear button).
- Auto-refresh updates charts in place: page scroll, the zoom/pan window and legend toggles survive it.
- Selection, range, aggregation, grouping and refresh interval persist in
localStorage. - Chart.js, its date adapter, the zoom plugin and hammerjs (required by the
zoom plugin for drag-pan) are vendored under
smartstats/static/vendor/, so the page needs no internet access.
Tests
python3 -m pytest tests/ -q
If collection crashes inside a plugin import, a stale entry point in the active
environment is the cause; run with PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 to skip
third-party plugin discovery.
tests/mock_server.py is a stand-in smartvos primary that speaks the real
framing, so the end-to-end tests cover snapshot ingest, live pushes, reconnect
with backoff, dedupe, hidden-sensor handling and the HTTP API without needing
hardware.
tests/ui_logic_test.js exercises the browser-side helpers (unit grouping,
value formatting, URL building) in Node with stubbed DOM:
PROJ=$PWD node tests/ui_logic_test.js
Visual check without a browser binary
tools/qt_shot.py renders the UI to PNG using Qt WebEngine (PySide6), which
works where a standalone Chromium cannot. It drives the page with injected JS,
prints what is on screen, and saves a screenshot:
pip install PySide6
python3 app.py & # serve first
QT_QPA_PLATFORM=offscreen python3 tools/qt_shot.py temps /tmp/shot.png
Scenarios: empty, all, temps, power, search. Override with
BASE_URL, SHOT_W, SHOT_H.
Known limitations
- Timestamps are recorded on arrival, not at the sensor. If the collector is disconnected, readings during the gap are lost — the primary does not backfill history.
- Only sensors are logged.
ItemUpdate,MessageandMqttDevicesframes are parsed and ignored. - The UI has no authentication; keep it on a trusted network.