release: crumble-v4.7.2

Author: imshentastic

CHANGELOG.md

@@ -2,6 +2,22 @@
 
 ## [Unreleased]
 
+## [crumble-v4.7.2] - 2026-08-05
+
+Single firmware: `crumble-v4.7.2-tiny-bitter.bin` (Bitter built-in, fits the stock 6.25 MB slot for SD-card flashing).
+
+### Added
+- **Newer X4 units that only stayed awake on USB power now run on battery.** One X4 hardware revision does not self-latch its battery MOSFET, so it appeared dead or unresponsive the moment it was unplugged. CrumBLE now asserts that latch (GPIO13) at boot, matching the fix CrossInk shipped in 1.5.0-rc-2. This is a no-op on every X4 sold before that revision — those self-latch through a pull and never needed firmware to hold the rail — so nothing changes for existing devices.
+
+### Changed
+- **Vendored `freeink-sdk` updated to upstream `e6a8048`** (154 commits, from the 2026-07-01 snapshot shipped since 4.6.0). For existing panels this brings: the refresh-completion wait now sleeps on the BUSY interrupt edge instead of polling on a 1 ms tick (X4 joins X3, which already did this), power-up sequencing before a grayscale refresh, and an X4 RED-plane baseline fix. The X4 grayscale waveform table and the X3 LUT bank are byte-for-byte unchanged. The X3 grayscale `wb_gc` fix CrumBLE has carried since v4.6.0 is now upstream's own value, so it survives the update unmodified. The update also compiles in upstream's UC8179 and UC8279 drivers for the newer UltraChip panel controllers — see the known limitation below.
+
+### Fixed
+- **Opening a book no longer floods the log with deferred-save errors.** Two heap thresholds disagreed: the settings-save retry admitted a write whenever `maxAlloc` cleared 20 KB, but the re-check immediately before serializing rejected anything under 28 KB. In that 20–28 KB band — easily reached while a book is open — every main-loop tick ran the directory/filesystem work, failed the floor, logged an error, and tried again, hundreds of times a second. Both now use the same floor, so the save defers quietly and retries once there is genuinely room. The save behaviour itself is unchanged; only the futile retry loop is gone.
+
+### Known limitations
+- **Newer X3/X4 units with UltraChip UC8279/UC8179 panel controllers are still unsupported.** The drivers are compiled in, but nothing selects them: CrumBLE does not fingerprint the panel controller at boot. The bus probe that would do it hung both an X4 and an X3 during development, and upstream marks the UC8279 path itself as "Pending hardware validation — no UC8279 X3 unit has been on the bench yet". Shipping detection that cannot be validated, and that demonstrably broke working devices, was not a trade worth making. If your display never comes up on CrumBLE, use CrossInk 1.4.0.1 or newer.
+
 ### Fixed
 - **UI font fallback no longer taxes the reader, and reader UI keeps CJK coverage for free when the primary is CJK-capable.** All three reader activities (EPUB, TXT, XTC) release the standalone UI glyph fallback family on entry and re-arm it on exit, matching the pattern that File Transfer, OPDS, and KOReader auth already use. The fallback exists to fill glyph misses in carousel/shelf/settings labels; book text renders through the primary SD font, which already carries whatever glyphs the book itself needs. Previously the fallback stayed resident through the reading session (~10 KB scattered across the heap) and starved contiguous allocations the reader depends on (page DOM 25-40 KB, CSS rule-table grow, BT font subset). Users can now leave the "UI Font Fallback" setting on and still have full contiguous heap while reading. On top of the release-and-suppress path, if the reader's primary SD font is loaded on entry (the normal case when the user has picked an SD font family like Bitter-LXGWWenKai), the reader points the UI fallback at the primary's already-loaded `EpdFontFamily` -- zero extra heap, and CJK glyph misses in reader UI (title bar, chapter label, progress bar, drawer, in-reader menu) render through the primary. Rendered at the primary's point size, so CJK reads visibly larger than surrounding 10-12pt UI text, but far better than tofu. The alias survives font-change / reindex within the reader (ensureLoaded breaks the fallback pointer before unloading the primary and re-establishes it after the new primary loads).
 - **Progressive JPEG EPUB covers now render smoothly in generated BMP cover assets.** The cover/thumbnail BMP path already detected progressive JPEGs and forced the required 1/8 JPEGDEC decode, but it still upscaled that reduced grid with blocky sampling. `JpegToBmpConverter` now uses a progressive-only bilinear smoothing pass before dithering when those covers are enlarged for home thumbnails and sleep covers, matching the higher-quality behavior already used in the framebuffer renderer while keeping memory bounded to a small line buffer.

UC8179-PORT-PLAN.md

@@ -94,6 +94,165 @@ split) unless the facade is also updated, which is most of strategy A anyway.
    release checklist. Third frozen-dependency incident this cycle (prebake wasm,
    sim shim, SDK) and the only one with hardware consequences.
 
+## Execution record (steps 1-4 done)
+
+### Step 1 — local-patch direction: resolved
+
+The vendored SDK was an unmodified snapshot of upstream **`6ee2d06`** (2026-07-01),
+found by matching blob hashes across all 293 upstream commits: 37 of the 42 files
+in the six compiled libs matched that tree exactly. The other 5 carried content
+never seen in any upstream commit — that is the complete local-patch surface:
+
+| File | Local patch | Direction |
+|---|---|---|
+| `lut/Uc8253X3Luts.h` | X3 `lut_x3_wb_gc` lead byte `0x54` -> `0x00` (the v321 fix) | **take upstream** — HEAD is byte-identical; only our comment differed |
+| `include/FreeInkDisplay.h`, `src/FreeInkDisplay.cpp` | heap framebuffers, `releaseBuffers`/`reallocBuffers`, `syncWriteBufferFromActive`, secondary-buffer release, `cleanupGrayscale*` restore memcpy, `swapBuffers` null guard | **take upstream** — HEAD has all of them, several comment-for-comment identical (they were backports that landed upstream) |
+| `driver/Ssd1677Driver.cpp` | initial paint FULL -> HALF | **take upstream** — converged; on X4 (`halfSeqOverride=0xD7`) upstream also resolves to HALF |
+| `driver/Ssd1677Driver.h` | `skipInitialResync()` override defusing `_needsInitialFull` | **KEEP — re-applied.** Upstream made `skipInitialResync` a `PanelDriver` virtual and overrides it on X3/UC8179/UC8279 but *not* SSD1677; the base is a no-op, so without this the X4 seamless silent-restart flash returns |
+
+The `Uc8253X3Driver.{h,cpp}` hunks the plan flagged are **entirely upstream
+evolution** — the async `displayStart`/`displayFinish` split, the `factoryP1/P2`
+reference banks, and a `forcedFullSync` -> `_forceFullSyncNext` fix. Nothing of
+ours to lose there.
+
+### Step 2 — strategy A taken
+
+`freeink-sdk/` mirrored to upstream `e6a8048`, excluding `libs/book/` (4 MB
+EPUB engine — expat/miniz/libunibreak plus a test TTF — that CrumBLE never
+compiles) and the nested VCS/`__pycache__` dirs. Re-applied the one kept hunk.
+**Zero firmware call-site changes were needed**; `pio run -e tiny-bitter` built
+clean on the first attempt.
+
+Only six SDK libs are actually compiled (`lib_deps`): FreeInkDisplay,
+BatteryMonitor, InputManager, SDCardManager, BoardConfig, PowerManager —
+FreeInkUI, network and book are vendored but never built, which is why the
+"312 diff lines in FreeInkDisplay.h" scoped down to nothing.
+
+### Step 3 — detection: a gap the plan did not anticipate
+
+Copying the drivers is **not** sufficient, and neither is updating the facade.
+`selectDriver()` routes on `BoardConfig::ACTIVE.displayController`, which is set
+by `XteinkDetect::applyXteinkDisplayController()` — a lib that was **not in our
+`lib_deps` and is called from nowhere in our firmware**. Without wiring it, the
+UC8179/UC8279 drivers link but can never be selected, so a new-panel unit still
+gets a dead display. Added `XteinkDetect` to `lib_deps` and called the probe from
+`HalDisplay::begin()`, after the X3/X4 profile is chosen and before
+`einkDisplay.begin()`.
+
+Regression safety, verified by reading the probe:
+- It promotes **only** on `Uc81xxConfirmed` — two independent passes must both
+  match the UC81xx VER/FLG signature *and* agree byte-for-byte. An SSD1677 or
+  UC8253 does not answer register `0x70`, so the bus floats and both passes fail
+  -> `PrimaryAssumed` -> the profile default stands. Inconclusive also falls back.
+- Pins are read from `BoardConfig::ACTIVE` and released to `INPUT` afterwards;
+  `EpdBus::begin()` re-runs `SPI.begin()` immediately after.
+- SD shares SCLK/MOSI and is already mounted at this point, but
+  `HalSpiBus::Lock` is held across all of `HalDisplay::begin()`, so no SD
+  transaction can interleave.
+- Cost: roughly 70 ms at boot.
+
+### Step 3b — waveform regression audit: clears
+
+The plan's headline worry ("Replace grayscale LUT waveforms with stock 2-frame
+set" changing appearance on existing panels) does not apply. That commit
+(`0647bfd`), plus `6662faf` "Restore OLD plane baseline after grayscale refresh"
+and `7babfab` "Fix e-ink ghosting by tracking previous frame", touch **only
+`Uc8179Driver.cpp`** — the new-panel driver.
+
+- **X4 `lut_grayscale`: unchanged.** What actually changed in `Ssd1677Luts.h` is
+  that `lut_grayscale_revert` was deleted and its slot repurposed for a Seeed
+  Sticky table, plus an additive X4 Pro partial LUT. Neither is our X4.
+- **X3 LUT bank: unchanged**, plus additive `factory_p1/p2` reference tables that
+  are deliberately not wired up.
+- Removing the X4 revert waveform is a no-op in our flows: the reader always
+  calls `cleanupGrayscaleWithFrameBuffer()` after `displayGrayBuffer()`, and
+  `cleanupGrayscaleBuffers()` (byte-identical old vs new) already cleared
+  `_inGrayscaleMode`, so the old `grayscaleRevert()` branch was dead code.
+- `POWER_BUTTON_PIN` and the ADC ladder ranges `{3100, 2090, 750}` are unchanged;
+  the large InputManager/BoardConfig diffs are clang-format reflow plus X4 Pro
+  additions that are not compiled here.
+
+Real behavior deltas that do land on existing panels, and are therefore what
+hardware testing is for: the completion wait moved from 1 ms polling to an
+ISR edge wait on BUSY (`waitRefreshComplete`) on **both** X4 and X3;
+power-up sequencing before grayscale refresh; and the X4 RED-RAM baseline /
+boot first-paint fix. The ISR itself checks out against the repo's rules —
+`IRAM_ATTR`, `xSemaphoreGiveFromISR` (never `Take`) in the handler, a
+polling fallback when the semaphore can't be created, a bounded 20 ms
+arm-confirmation, an already-done fast path, and a 30 s timeout.
+
+The async `displayStart`/`displayFinish` split is inert for us: our app never
+calls `displayAsync`/`triggerDisplay`, so `display()` runs the blocking path,
+which on both drivers is exactly `displayStart()` + `displayFinish()`.
+
+### Step 4 — builds and size
+
+All six envs build clean, no warnings from SDK or probe code.
+
+| env | 4.7.1 | new | delta | vs 6,553,600 SD slot |
+|---|---:|---:|---:|---|
+| tiny-bitter | 5,352,608 | 5,361,888 | +9,280 | 1,191,712 under |
+| tiny | — | 5,361,744 | — | 1,191,856 under |
+| tiny-lexend | — | 5,024,864 | — | 1,528,736 under |
+| tiny-chareink | — | 5,316,976 | — | 1,236,624 under |
+| xlarge | 7,655,840 | 7,665,136 | +9,296 | over — **already over at 4.7.1** |
+| no_emoji | 7,235,984 | 7,245,280 | +9,296 | over — **already over at 4.7.1** |
+
+The whole change costs **+9.3 KB**: +7,344 for the SDK sync including both new
+drivers, +1,936 for the probe. `xlarge` / `no_emoji` exceed the 6.25 MB SD-flash
+slot but are unchanged in that respect — they were 7.66 / 7.24 MB on 4.7.1 and
+fit only the 7,864,320-byte app partition (USB flash), which is why the shipping
+SD-flashable build is a `tiny-*` variant.
+
+Simulator envs are unaffected: `[env:simulator*]` declares its own `lib_deps`
+without the freeink-sdk symlinks and sets `lib_ignore = hal`, so neither the SDK
+nor `HalDisplay.cpp` compiles there.
+
+### Step 5 — hardware gate: what actually happened
+
+The plan's premise was wrong in one important way, and hardware is what proved it.
+
+**The newer-X4 problem is not a panel-controller swap.** CrossInk's release notes
+split it: 1.4.0.1 (Jul 29) added display-driver detection for newer **X3** units,
+while 1.5.0-rc-2 (Aug 1) added "support for latest X4 battery latch on newer
+models (without this patch, newer X4's could seem unresponsive unless connected
+to USB power)". Upstream's own consumer entry point agrees —
+`selectXteinkDevice()` probes the display bus **only on a confirmed X3**; on X4 it
+just selects the profile. No shipping firmware probes an X4's panel bus.
+
+Three hardware failures, all caused by this port, all from the same root mistake
+(running things at boot that upstream does not run, or runs in a different order):
+
+1. **X4 hang, probe inside `HalDisplay::begin()`** — `applyXteinkDisplayController()`
+   was called unconditionally on every device. Verdict was *correct*
+   (`VER=FF..FF -> default controller`, SSD1677 kept) but display init never
+   completed. Bisected with a `-DCRUMBLE_DISABLE_PANEL_PROBE` build, which booted
+   clean and cleared the SDK update + both new drivers.
+2. **X4 hang, probe moved before `Storage.begin()`** — same symptom, which
+   disproved the "SD was already mounted" theory. The real problem was simply
+   that the X4 bus must not be probed at all.
+3. **X3 hang in `Storage.begin()`** — `[SD] SD card detected` never printed. Calling
+   `selectDevice(XteinkX3)` early switched `sd.powerEnable` from `PIN_UNASSIGNED`
+   (X4 default) to GPIO13, so `SDCardManager` began driving the X3 SD rail for the
+   first time ever (upstream `2da0700`, unvalidated here). Every prior CrumBLE
+   release mounts SD while `ACTIVE` is still `XTEINK_X4`; copying upstream's call
+   site into a different init order is not "1:1".
+
+**Shipped state:** no panel-controller probe on either device. X4 gets
+`selectDevice(XteinkX4)` for the battery latch only. X3's boot path is byte-for-byte
+4.7.1. UC8179/UC8279 drivers compile in but nothing selects them — documented as a
+known limitation rather than pretended-to-work.
+
+Verified on hardware: X4 boots and opens books (init waits 2/5/5 ms); X3 boots
+past SD mount. Settings log-spam fix verified at `maxAlloc=19444`, below both old
+thresholds, with zero spam.
+
+Nothing has been released.
+
+Working tree is uncommitted. The 21 new SDK paths (including
+`Uc8179Driver.{h,cpp}` and `Uc8279Driver.{h,cpp}`) are untracked — they compile
+because PlatformIO globs the source dir, but they need `git add` before a commit.
+
 ## Do not
 
 - Do not ship without the on-device X4 + X3 regression pass. A bad waveform