Repository navigation
CI: test the real firmware against the full NSPanelManager stack - #44
Draft
jamesmulcahy wants to merge 13 commits into
Draft
jamesmulcahy wants to merge 13 commits into
jamesmulcahy wants to merge 13 commits into
Conversation
[env:qemu] is original_sonoff built with -DNSPM_QEMU and an sdkconfig overlay (qemu/sdkconfig.qemu.defaults, merged into a generated sdkconfig.qemu by qemu/gen_sdkconfig.py): OpenCores Ethernet, 4 MB flash, halt on panic. Each NSPM_QEMU block works around something Espressif's QEMU does not emulate. Hardware builds are unchanged: - WiFiManager: no Wi-Fi radio. init() sets up no Wi-Fi driver, and start_client() brings the network up over the emulated OpenCores Ethernet MAC (generic 802.3 PHY; IDF 6 keeps no other in-tree), reported through the same connected() and ip_info() state. There is no fallback AP. - Nextion: QEMU's UART has no AT_CMD_CHAR pattern detection, so the display's replies would be dropped. Pattern-detect setup is skipped: as well as being unemulated, it divides by the baud rate, which IDF 6's uart_get_baudrate() fails to read because QEMU's DPORT model does not reflect the UART clock-enable bits. Frames are split on FF FF FF in software instead, reading exactly what is buffered rather than what each UART_DATA event reports, since events are lost when their queue is full. Buffers rotate so a frame is not overwritten before the uart_handle_data loop dispatches it. - StatusUpdateManager: the SAR ADC is unimplemented, and a conversion spins forever with the ADC spinlock held, freezing core 0. Skip ADC setup and report a fixed NTC voltage. Wi-Fi is never initialised, and esp_wifi_sta_get_rssi() then dereferences missing driver state, so report a fixed RSSI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
firmware_espidf/qemu/run_tests.sh builds [env:qemu], installs a pinned Espressif QEMU, and runs a pytest suite that boots the real firmware image against: - a fake Nextion on UART2, which models the screen from the panel's commands, answers connect/get, and injects touches, slider moves, sleep and wake, - a per-test mosquitto, observed by a paho client, with a TCP proxy in front of the panel so tests can cut its connection, - a fake NSPanelManager, which handles registration, config, room and entity-page state, and decodes the panel's commands. Every wait gives up as soon as the firmware panics or QEMU exits, so a crash fails the test immediately and names the panic. There are 13 tests covering boot and registration, touch input on the home and entities pages, MQTT reconnect and last will, and the web UI. .github/workflows/qemu-tests.yml runs the suite and uploads per-test console and mosquitto logs. See firmware_espidf/qemu/README.md, including what differs from hardware. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them The firmware repo carries only protobuf-c output, so the harness needs the .proto sources from NSPanelManager, which owns the schema. Fetch docker/protobuf/*.proto from GitHub at a pinned commit (the one the vendored copies came from), cache them under qemu/build/, and compile on first use. NSPM_PROTO_REF selects another revision, and NSPM_PROTO_DIR points at a local checkout, for testing a schema change before it is pushed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion When the manager renames a panel, the panel adopts the new name as its hostname (0fde923) but kept listening for register_accept on nspanel/<old name>/command. register_request carries the new name, so the manager answers on nspanel/<new name>/command: after the next MQTT reconnect the accept went unheard and the panel re-sent register_request every 5 s, indefinitely. Move the command subscription when the hostname changes. Found by the QEMU tests; test_reregisters_after_rename_and_reconnect fails without this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Showing a page registers handlers on both the default event loop and RoomManager's loop, and esp_event takes a loop's mutex to do so, the same mutex that loop's task holds while dispatching. The first-load switch to the home page ran in a RoomManager-loop handler, while a touch on the screensaver switches pages in a default-loop handler. When the two coincided, each loop task held its own mutex and waited for the other's: a permanent deadlock that froze the UI. The QEMU tests caught it as a touch that was never handled; a FreeRTOS task dump of the frozen guest showed sys_evt and roommanager_eve each blocked on the other's loop mutex. Page switches requested from RoomManager-loop handlers now run on a page_transitions task, which holds no loop mutex and so cannot be part of the cycle. Handlers notify it with xTaskNotify, which never blocks. The duplicated first-load logic in the default-loop CONFIG_LOADED handler notifies the same task, and its flags become std::atomic now that a third task reads them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A page could end up on screen with no working buttons. Two behaviours combined: - show() hands the previous page's unshow to a separate task, so it can run after the next show() has registered its handlers. - The legacy esp_event_handler_register() de-duplicates by function pointer. Registering a handler that is still pending removal logs "handler already registered, overwriting", returns ESP_OK, and leaves it to be removed by the pending cleanup. So whenever a page's unshow and a fresh show() of the same page overlapped, in either order, that page lost its handlers. Showing the home page twice in quick succession was enough, e.g. the first-load switch plus a touch on the screensaver. InterfaceManager::begin_page() now replaces call_unshow_callback(): it gives each show() a PageHandlers that records the instances it registered, using the instance API, which never de-duplicates. Unshowing retires only that show()'s registrations, so a late unshow can no longer remove a newer show()'s handlers, and a page's own unshow() is skipped if the page has been shown again by then. Unshow stays asynchronous. It now runs on one long-lived task that is passed plain pointers instead of copying std::function objects through a FreeRTOS queue. No lock is held while calling into esp_event. EntitiesPage::_update_display() can run on the RoomManager loop, so its fallback to the home page now goes through the page_transitions task too. test_tapping_screensaver_during_first_load_does_not_deadlock covers it: 8/8 passes with this change, and fails 5 runs in 8 without it. Known limitation: EntityPage subscribes to its entity's topic in show(), before begin_page(). After very rapid entity-to-entity switches, one stale subscription can remain; EntityPage ignores its messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fake-world tests script the manager's side, so they cannot catch a protocol change in NSPanelManager. Tests marked `stack` instead point the panel at a running dev stack (NSPanelManager's docker/dev): its mosquitto and its real MQTTManager. - Broker can observe an external broker instead of starting mosquitto. - StackManager watches the real manager over MQTT and acts through the web UI's own interfaces (StackWeb): STOMP over websocket to find a panel's id and whether it is awaiting accept, then the REST accept call. So each panel joins as a new panel with a random MAC, any number can join, and each is deleted at teardown. - conftest: make_rig() factory for several panels in one test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Open the STOMP websocket with no subprotocol, as the web UI does (stomp_wrapper.js); MQTTManager's server rejects "v12.stomp". - Delete a panel at teardown even when the test failed before learning its id, by looking it up by MAC, so failed runs don't leave panels awaiting accept. - Give the room a ceiling light (ceiling_light fixture, via the same REST call as the UI's add-light dialog). The dev stack seeds no lights, and the panel's ceiling button sends nothing in a room without them. - Fix the AffectLightsOptions enum reference. All five stack tests pass against NSPanelManager feat/dev-stack with NSPManager/NSPanelManager#399, whose MQTTManager crash ("database is locked" on reload, then std::stoi("")) these tests found. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new workflow builds the firmware for QEMU and runs the stack tests against NSPanelManager's dev stack: Django, MQTTManager and mosquitto, all built from source. The stack and MQTTManager build in the background while the firmware builds. MQTTManager's Conan packages are cached, as NSPanelManager's own CI does. The dev stack is not upstream yet, so the workflow pulls it from jamesmulcahy/NSPanelManager. A comment marks where to re-point it once it and NSPManager/NSPanelManager#399 are merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jamesmulcahy
commented
Oct 16, 2026
| web.delete_entity(light_id) | ||
|
|
||
|
|
||
| def test_new_panel_is_accepted_and_configured(booted): |
Contributor
Author
There was a problem hiding this comment.
These are the tests which test the whole stack, end-to-end -- with real panel firmware, and the real manager/django stack
Contributor
Author
There was a problem hiding this comment.
Tests succeeded in CI - output is here: https://github.com/NSPManager/NSPanelManager-firmware/actions/runs/38943420039/job/117116445062?pr=44#step:11:31
ESP-IDF's OpenCores EMAC ISR is IRAM-safe, but when the RX ring overflows it logs "RX frame dropped" with ESP_EARLY_LOGW, whose tag and format string live in flash. If that happens while flash cache is disabled, for example while ButtonManager saves config.json to LittleFS, the guest panics with "Cache disabled but cached memory region accessed". A burst of MQTT traffic during a config save was enough to hit it with the 4-buffer ring. Raise it to 16. This only affects the qemu env. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The manager's first config never matches the harness's config.json, so ButtonManager saves it to LittleFS from the default loop. Under QEMU that takes ~20 s of guest time. The fixtures used to hand the panel to the test as soon as the screensaver appeared, so tests ran during the save: - MQTT traffic overflowed the Ethernet RX ring while flash cache was off, which panics the OpenCores ISR, even with 16 buffers. - The unshow task waited on the default loop, so a page shown during the save was unshown seconds late. Wait for the "Saved ... config.json" line in Rig.boot(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: - Nextion: NSPManager#46 posts RECEIVED_DATA as a size-prefixed copy of the bytes (_post_received_data) instead of a std::vector. The QEMU framing now posts each frame through _post_received_data too, so the handler gets the format it expects. That copy also makes the rotating frame buffers unnecessary, so they are gone. - EntitiesPage: NSPManager#40 passes the page to _update_display(); keep NSPManager#43's request_show_home_page() for the unknown page type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this tests
The real firmware and the real NSPanelManager, together, in CI. Nothing on either side is faked:
docker/dev).Each emulated panel joins the manager the way a real one does:
register_requestand shows up in the manager as awaiting accept.POST /rest/nspanel/<id>/accept) the web UI uses.Every assertion is on what the real manager sent or received, so a protocol change in either repository fails this job.
Tests (
firmware_espidf/qemu/tests/test_stack.py)make_rig()builds any number of panels.Each panel gets a random MAC, so it is new to the stack, and it is deleted from the stack at teardown.
Already found a real bug
On their first full run, these tests crashed MQTTManager. A reload read the database while Django was still writing, got
database is locked, and aborted onstd::stoi(""). The fix is NSPManager/NSPanelManager#399.CI (
.github/workflows/stack-tests.yml)qemu/run_tests.sh -m stack.MQTTManager's Conan packages are cached as NSPanelManager's own CI caches them. A cold cache costs about an hour, once. On failure the job uploads the test logs, the stack's container logs and the MQTTManager log.
To test against a different NSPanelManager branch or commit, run the workflow by hand and set
nspm_ref.Dependencies to track
devel. Review only these 3:d353e1dQEMU tests: run the firmware against the NSPanelManager dev stackc9af5feQEMU stack tests: fix the first run against a real dev stackef5cc75CI: run the firmware against the full NSPanelManager stackdocker/dev) is not in NSPanelManager upstream yet. Until it is, the workflow pulls it fromjamesmulcahy/NSPanelManager, branchfeat/dev-stack.feat/dev-stackis now current NSPanelManagerdevelplus the dev stack (docker/devonly), so these tests also run against the manager that sends panels the time (MQTTManager: send NSPanels the current time NSPanelManager#404).NSPM_REPO/NSPM_REFat the top ofstack-tests.yml, marked with a comment) toNSPManager/NSPanelManagerdevelonce the dev stack is merged there.Known limits
devel.🤖 Generated with Claude Code