Bluetooth runs its controller from flash, is chosen at boot, and the firmware makes room for it
M6 has the official MeshCore app drive the device over Bluetooth Low Energy. The rebuilt framework (ADR 0006) already has NimBLE, ESP-IDF's BLE host, and Arduino's BLE library on top of it. Linked as it comes, it didn't fit (M6…
M6 has the official MeshCore app drive the device over Bluetooth Low Energy. The rebuilt framework (ADR 0006) already has NimBLE, ESP-IDF's BLE host, and Arduino's BLE library on top of it. Linked as it comes, it didn't fit (M6 step 1, 2026-11-12):
- It cost about 27 KB of free heap without ever being started. On the ESP32-S3, IRAM comes out of the same internal RAM as the heap, and the controller's code goes in IRAM by default (+21.7 KB).
- Starting it crashed the device every time. The controller asks for one contiguous 30,720-byte block at start (
emi.c), and the firmware's largest free block is about 31.7 KB at best, less once anything else has run. The failed allocation ends in an assert that spins with interrupts off, until the interrupt watchdog resets the device. That is the "loop in ROM" (r_lld_core_init) first seen. Arduino's own precompiled framework does the same.
We set Bluetooth up differently:
- The controller runs from flash (
CONFIG_BT_CTRL_RUN_IN_FLASH_ONLY): its code stays out of IRAM (2.3 KB of it must stay), and its memory comes from the heap only once it starts. ESP-IDF then links a different controller library and leaves out the ROM's controller functions. Arduino's precompiled list of libraries and linker scripts doesn't follow the setting, soscripts/ble_controller_flash.pydoes it after the framework's: it linkslibbtdm_app_flash.ainstead oflibbtdm_app.a, and removesesp32s3.rom.bt_funcs.ld. Without the first the controller refuses to start ("need flash lib"); without the second it calls ROM code whose data it never set up (a null pointer inr_rwip_init). - The controller's memory is released at boot unless Bluetooth is on. Arduino asks
bleInUse()beforesetup(); false gives the controller's reserved memory back to the heap for good. The setting is read from NVS there. - NimBLE is cut to what a companion needs: one connection and two bonds, no observer role, no extended advertising, 8 buffers of each kind instead of 12 to 30, two controller activities instead of six (
platformio.ini). As it comes, BLE running took about 70 KB and stopping it left the heap fragmented; cut down, about 50 KB, given back whole. The central role stays, though a companion never connects out: NimBLE's GATT client depends on it, and without the client the phone's first indication goes unanswered and the phone leaves before pairing ends (M6 step 3). - Bluetooth and Wi-Fi together (M6, Q335, replacing Q331's either-or). BLE takes about 50 KB running, and Wi-Fi with the VPN about 40 KB, so the firmware made more room (step 3):
- Code from IRAM to flash, 17.5 KB (
.iram0.text85.8 → 68.3 KB): FreeRTOS's functions (its interrupt ones stay in IRAM), the ring buffer's, the RMT driver's interrupt handlers and encoder (the LED), the radio PHY's helpers, the I2C master's interrupt handler. Newlib's misc functions and esp_timer's would have been two more, but ESP-IDF lets them move only with flash auto-suspend, which is off: both settings are asked for and ignored (corrected 2026-11-19;scripts/check_sdkconfig.pynames them). The heap's own functions (6 KB more) stay in IRAM: moving them is safe only if no interrupt allocates while flash is busy, which can't be ruled out for Wi-Fi. - IRC's task (6 KB of stack) exists only while IRC is wanted, as the update task does.
- Even so, with Bluetooth on, a TLS connection (IRC, Gemini, an update check) finds too little memory and waits, saying so.
- Code from IRAM to flash, 17.5 KB (
- The firmware makes room for it (M6, Q334). Linked and off, Bluetooth still takes about 6 KB, enough to keep IRC on TLS from starting beside the mesh and the VPN. So:
- The update task (7 KB of stack) exists only while it has work: a check, an install, an Update File being received, an installed update waiting for its restart. The main loop listens on port 3232 and starts the task for a connection. A flag set before the task is made, and cleared by the task as it ends, says whether it is there.
- Stacks are cut to what they use, with room to spare: esp_timer 8 → 3.5 KB and the FreeRTOS timer task 3.1 → 2 KB (ESP-IDF's defaults; Arduino's are larger, and nothing of ours runs on either), Arduino's network event task 4 → 3 KB (
ARDUINO_NETWORK_EVENT_TASK_STACK_SIZE), the Debug Console's 6 → 4.5 KB.
Consequences
-
With Bluetooth on, Wi-Fi, the VPN and the mesh, and the MeshCore app connected: about 44 KB free, the lowest 30.6 KB (M6 step 3). With Bluetooth off: 96.7 KB at idle, against 64.5–65.4 KB for v0.27.0, and IRC on TLS goes no lower than 20.9 KB (3.4 KB on v0.27.0).
-
An Update File pushed over Wi-Fi with Bluetooth on runs out of memory (the device fell to 340 bytes and refused it, "update file too short"): Bluetooth off for the time of the install.
-
With Bluetooth linked and off, the mesh Sending and the VPN up, the firmware has 74.6–75.2 KB free at idle, against 64.5–65.4 KB for v0.27.0, and IRC on TLS beside them doesn't go as low as before (6.6 KB at the lowest, 3.4 KB on v0.27.0). One firmware for everyone; M6's fallback of a second "with Bluetooth" firmware (Q333) isn't needed.
-
With Bluetooth on and Wi-Fi off (step 1), 46.7 KB was left free while advertising with the mesh listening, and the largest block stayed 31.7 KB.
-
Turning Bluetooth on takes a restart, since its memory is released at boot when it is off. Stopping it gives the memory back whole, so turning it off needn't.
-
The firmware grows by about 420 KB, to 3.10 MB, 93% of its app slot (M6 step 3).
-
The functions moved to flash run a little slower, and can't run while flash is busy (an OTA write): none of them may be called from an interrupt that runs then. A crash in an interrupt during an install points here first.
-
The link fix depends on how Arduino's build lists the controller library and the ROM linker scripts. A framework update that changes either can break the build or, worse, the controller's start:
scripts/ble_controller_flash.pyprints what it changed, and an update must be tried with Bluetooth on. -
The update task's stack is now taken when it is needed: a check or an install starts only if the 7 KB can be had, and is refused ("not enough memory") otherwise. An Update File pushed while memory is short is refused too:
install()used a throwingnew, which aborted the device. -
The stack sizes are measured, not proven. A new caller on one of them (an esp_timer callback of ours, heavier Wi-Fi event handlers, a bigger console command) must check
tasksfor its free stack. -
Bluetooth's settings and the stacks' are part of the rebuilt framework's
custom_sdkconfig, so changing them costs a 4-minute framework rebuild (ADR 0006). One of them, the FreeRTOS timer task's, must also be given under its legacy name: the generatedsdkconfig.defaultsrepeats it after ours, with Arduino's value, and the last one wins.
This page is generated from docs/adr/0012-bluetooth-from-flash-chosen-at-boot.md in the repository. To change it, change that file and run site/tools/gen_dev_docs.py.