Extensions to the Espressif spi_nand_flash component for ESP-IDF, focused on FTL evaluation, fault simulation, and wear-leveling improvements built on top of the Dhara FTL library.
Source code lives in a fork of
idf-extra-components:github.com/omnitex/idf-extra-componentson theDIPbranch. This repo contains documentation, evaluation results, and a build setup.
The DIP branch extends the upstream spi_nand_flash component with:
By storing Logical Page Number (LPN) of each page in its out-of-band (OOB) area, mapping layer information is now durable with each page write, compared to previously only on page group sized checkpoint intervals.
A configurable fault-injection layer (nand_fault_sim) that replaces the Linux mmap emulation at link time.
Tracks which page is currently loaded in the NAND chip's internal data register. Skips redundant READ PAGE commands (~25-100 us) on back-to-back reads of the same page. Reset automatically after every program or erase. Toggle via CONFIG_NAND_PAGE_REGISTER_CACHE.
DRAM-resident cache for Dhara metadata pages (CONFIG_DHARA_META_CACHE_SLOTS, 0-8 slots). Avoids re-reading metadata that the wear-leveling layer accesses repeatedly.
Caches the Dhara radix-tree traversal path for sequential sector lookups (CONFIG_DHARA_MAP_PATH_CACHE). Eliminates most per-sector metadata reads during sequential workloads. 136 bytes static RAM.
Before programming a page, reads it back to check ECC condition. If correctable-bit count is above a configurable threshold, the page is skipped and the next available page is used -- preventing writes to already-worn pages. Three aggressiveness levels via Kconfig.
Host-test application for automated parameter sweeps:
- Configurable workloads: sequential, random, mixed, Zipf-distributed
- Sweep parameters: GC factor, write sizes, ECC noise, page relief thresholds
- JSON output with metrics (WAF, lifetime, erase distribution)
- Built-in visualization (
ftl_viz.py) for heatmaps and comparison plots - Fault-simulation integration for robustness testing
Host-test application measuring raw throughput and latency:
- Sequential/random/Zipf benchmarks
- Metadata cache hit-rate statistics
- Automated run scripts and result inventory (
perf_inventory.py,perf_viz.py)
- BDL (Block Device Layer) test suite
- Fault-injection test suite (fault simulation + FTL robustness)
- Page relief unit tests
Application / FS
|
spi_nand_flash API (backward compatible)
|
Dhara FTL (wear leveling + garbage collection)
| |
L3 path L2 meta cache
cache (DRAM slots)
|
NAND Wear-Leveling BDL
|
NAND Flash BDL
|
+------------ or ------------+
| |
SPI NAND Operations NAND Emulation
(ESP chips) (Linux host test)
|
+-----+------+
| |
mmap emul fault sim
(baseline) (inject faults)
- ESP-IDF v6.x (installation guide)
# 1. Clone this repo
git clone https://github.com/omnitex/spi-nand-flash-ftl-ext.git
cd spi-nand-flash-ftl-ext
# 2. Run setup (clones the fork, builds ftl_eval and perf_app)
./setup.sh# Or build manually:
git clone -b DIP https://github.com/omnitex/idf-extra-components.git
cd idf-extra-components/spi_nand_flash/ftl_eval
idf.py --preview set-target linux
idf.py build
cd idf-extra-components/spi_nand_flash/ftl_eval
# Run a single sweep config
make run CONFIG=gc_vs_wear_sequential_monotonic
# Visualize results
python3 ftl_viz.py --no-texThe host_test/ directory contains Catch2-based test suites that exercise the NAND driver and FTL on the Linux target (no hardware needed). Which tests are compiled depends on Kconfig options.
Tests for the standard FTL API: write/read/copy/trim/sync, GC stability, sequential sweeps, edge cases. Runs with the mmap NAND emulator.
cd idf-extra-components/spi_nand_flash/host_test
# Default build (baseline, stats enabled)
idf.py --preview set-target linux
idf.py build monitorAvailable test tags: [spi_nand_flash], [ftl], [ftl][gc], [ftl][rw], [ftl][sequential], [ftl][copy], [ftl][trim], [ftl][sync], [ftl][bounds], [ftl][patterns].
When CONFIG_NAND_FLASH_ENABLE_BDL=y, an extended test suite is compiled including the OOB LPN orphan replay tests. These verify that pages written after the last Dhara checkpoint (orphan pages) are correctly recovered on remount by replaying their OOB-stored logical page numbers — the core crash-recovery mechanism.
Two overlay configs are provided. Apply one with SDKCONFIG_DEFAULTS:
cd idf-extra-components/spi_nand_flash/host_test
# BDL only (bare — orphan replay tests, raw BDL tests, WL BDL tests)
SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.bdl" idf.py --preview set-target linux
idf.py build monitor
# BDL + all read-path optimizations (L1/L2/L3 caches + page relief)
SDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.bdl.caches" idf.py --preview set-target linux
idf.py build monitorAvailable BDL test tags: [bdl], [bdl][raw], [bdl][wl], [bdl][raw][sequential], [bdl][raw][stress], [bdl][raw][copy].
Orphan replay tests: [dhara_oob], [dhara_oob][replay], [dhara_oob][replay][boundary].
# Run only orphan replay tests (crash-recovery feature)
./build/nand_flash_host_test.elf "[dhara_oob][replay]"| Config file | What's enabled |
|---|---|
| (defaults only) | Linux target, stats — baseline |
sdkconfig.bdl |
+ BDL, orphan replay tests |
sdkconfig.bdl.caches |
+ BDL, L1/L2/L3 caches, page relief |
Individual options can also be toggled via idf.py menuconfig under Component config → SPI NAND Flash configuration. Cache statistics (spi_nand_cache_stats_t) and relief statistics (spi_nand_relief_stats_t) are printed at the end of each test run when CONFIG_NAND_ENABLE_STATS=y.
| Option | Description |
|---|---|
CONFIG_NAND_PAGE_REGISTER_CACHE |
L1 page-register cache |
CONFIG_DHARA_META_CACHE_SLOTS |
L2 metadata cache (0-8 DRAM slots) |
CONFIG_DHARA_MAP_PATH_CACHE |
L3 map-path cache (136 bytes) |
CONFIG_NAND_ENABLE_STATS |
Runtime statistics collection |
CONFIG_NAND_FLASH_PROG_PAGE_RELIEF |
Program page relief |
CONFIG_NAND_FLASH_FAULT_SIM |
Fault-injection simulator |
spi-nand-flash-ftl-ext/
setup.sh Clone fork and build apps
docs/ Deep architecture and design docs
results/ Evaluation outputs (plots, data)
The actual source code is in the idf-extra-components fork:
spi_nand_flash/
include/ Public API headers
priv_include/ Internal headers
src/ Driver implementation
nand_fault_sim.c Fault-injection simulator
dhara_glue.c Dhara FTL integration (caches, relief)
nand_impl.c NAND operations + L1 cache
ftl_eval/ FTL evaluation app + configs + viz
perf_app/ Performance benchmark app + viz
host_test/ Extended test suites
Kconfig All configuration options
This repository is licensed under Apache-2.0.
The spi_nand_flash component in the fork incorporates the Dhara library, licensed under its own MIT-style license.