Rebase/cache refactoring onto main (#20)

* [RTL] Add folded data bank plumbing

- wire DataPartSplit/folded params through cluster/group/tile
- implement skewed folded data SRAM mapping in cachepool_tile
- adjust cluster wrapper tb for the new configuration

* [SW] Add scalar and vector cache tests with 128-bit aligned accesses

- add scalar cache tests that run basic and stress patterns without crossing 128-bit parts
- add vector cache tests that use RVV loads/stores on 128-bit chunks and verify data integrity
- integrate both tests into the test CMake and keep patterns aligned to folded-cache part size

* [RTL] prioritize writes in skewed folded bank selection

Select a single part per column/bank per cycle and prioritize write parts over read parts to avoid clobbering bank signals.

* cachepool folded-path integration and sim-time protocol hardening

  - cachepool_tile: use EffectiveCoalFactor=1 in folded mode; pass to cache ctrl.
  - cachepool_cc: size Spatz response FIFO with NumSpatzOutstandingLoads; add overflow assert.
  - tcdm_cache_interco: add non-synthesis outstanding scoreboard/asserts for req/rsp matching.

* [SW] Add cache mix smoke and pressure tests for scalar and vector cache access interleaving.

* [RTL] plumb hash-way folded cache integration

Bump insitu-cache to the folded/hash-way revision, thread
UseHashWaySelect through cluster/tile, and queue Spatz memory
responses through the local response FIFO instead of bypassing
write acks.

* [RTL] Bump insitu cache dep.

* [RTL] Add Spatz<->TCDM id-indexed req/rsp scoreboard for debug
  cachepool_cc: per-port sb_q[user.req_id] slot table for out-of-order
  rsp matching; watchdog dumps stuck ids. Gated by parameter
  (default off, +define+ENABLE_SPATZ_REQ_SCOREBOARD to enable).

* [RTL] tile: propagate skew-bank grant to prevent silent read drops

The skew-bank arbiter at (col, bank_sel) picks writes over reads
without exposing the loser; a hardwired l1_data_bank_gnt=1 caused the
upstream to consume stale rdata when another way wrote the same
column.  Compute any_other_write_in_col (loop-free, depends only on
part_we) and gate gnt by it: writes always granted, reads granted iff
no OTHER way writes the same (col, bank_sel).  Excludes own way's
writes so own idle words aren't spuriously stalled.  Fixes multi-core
coherence in rlc-mimic and unlocks AllowReadDuringWrite=1 on data
banks.

* [SW] fix runtime/test bugs

- l1cache: flush+wait before xbar commit so the reconfig doesn't leave
  dirty lines bound to the old hash layout.
- mcs-lock: move cluster barrier before the non-zero-core spin loop
  (otherwise cores 1+ never barrier and core 0 deadlocks).
- load-store: print the correct buffer name (B/C, not A) in the B/C
  error messages; add c_ptr to the pointer dump.
- idotp-32b: include got/expected in Check Failed! print.

* [SW] add cache-{coverage,coverage-min,line-rw-smoke,rlc-mimic,vector-rw} tests

Register five new cache-focused tests in CMakeLists.txt:
- cache-line-rw-smoke   single-core line-granular RW smoke
- cache-rlc-mimic        RLC traffic mimic (vector load/store)
- cache-vector-rw        multi-iteration vector load-store kernel
- cache-coverage         12-phase multi-core cache stress / coverage
- cache-coverage-min     minimal phase-06 writeback-loss repro

* [VERIF] enable Spatz-SB + add tile-level memory-model VIP

- Bender.lock: bump insitu-cache to the rev with the wrapper/coalescer
  SBs and the SYNC_CTRL_CHECK_PEND fix.
- Makefile: define ENABLE_SPATZ_REQ_SCOREBOARD so the in-RTL Spatz
  req/rsp watchdog is on by default.
- cachepool_tile.sv: per-port pre-strip TCDM req tracer
  (+sb_pretrace_addr_lo/hi) and byte-granular shadow-memory model
  (+mm_enable) that $errors on DATA / TYPE / ORPHAN_RSP mismatches.
  Both passive, off by default, sim-only.

* [Fix] axi_user_width sizing and group xbar req/rsp slot routing

- config.mk: derive axi_user_width as base + 2*(idx_width(num_tiles)-1).
  Previous widths truncated bank_id MSB on the AXI loopback, routing
  cache_ctrl refill responses to the icache bypass slot.

- cachepool_group.sv: use the source tile id `t` (not target_tile) for
  the request destination slot, so the response (routed by user.tile_id
  mod NumRemotePortCore) lands on the same xbar mst port as the request.

* [SW] cache-mix-pressure: keep per-core offset 4-byte aligned

The `win` offset combined `it * 64u + cid * 7u`.  The `cid * 7u` term
is odd for cid > 0, so `wp = (base + win + j * 4U)` ended up
unaligned for any non-zero core.  Snitch raises a misaligned
load/store exception for unaligned uint32_t accesses, and this
runtime has no exception handler installed, so cores 1+ entered a
trap loop at PC 0x800005fc while cores 0/2/3 stalled at the next
sync_all.  Result: the test always timed out without printing UART.

Change the per-core stride to `cid * 28u` (= 7 * 4) so the offset
stays varied per core but is always 4-aligned, restoring the
original "varied window" intent.  Test now passes with retval=0.

* [SW] mcs-lock: drop deadlocking exit pattern

After the per-core stats printf, non-zero cores entered `while(1){}`
and were never able to reach the second `snrt_cluster_hw_barrier()`
below.  Core 0 then waited forever at that barrier for cores 1+.
Result: the kernel never reached `return 0`, _snrt_exit was never
called, and EOC was never asserted -- the sim always timed out.

Removing the if/while-loop (and the now-pointless second barrier)
lets every core return cleanly; _snrt_exit only fires set_eoc on
core 0 anyway, and the other cores halt naturally.

mcs-lock now reaches EOC retval=0 cleanly.

* [SW] fft-32b: add 1024-point / 4-core variant

The existing fft-32b_M1024_N16 test is parameterized for 16 cores --
data_1024_16.h has active_cores=16 baked in and the kernel slices
the work by active_cores.  On a 4-core config only 4/16 of the FFT
actually executes, so the output is uniformly wrong (r:1024,i:1024)
and the test self-fails with retval=1.

Add a 1024_4 variant alongside, generated via gen_data.py from a
new fft_1024_4.json config.  Both variants now coexist; the N16
variant is appropriate for 4t/16c and the N4 variant for 1t/4c.

The new 1024_4 test passes cleanly (r:0, i:0, retval=0).

* [CFG] add cachepool_2t_fpu_512 (2 tile / 8 core) bisect config

Midpoint between cachepool_fpu_512 (1t/4c — passes) and
cachepool_4t_fpu_512 (4t/16c — broken).  Used to isolate whether the
multi-tile cache failures are specific to 4 tiles or to any
configuration with NumTiles > 1.  cache-line-rw-smoke fails at 2t/8c
with the same DATA-MISMATCH signature seen at 4t/16c, confirming
the bug is in the inter-tile / group-xbar path itself, not a
4-tile-only artefact.

* [SW] add minimal-tile0-repro for multi-tile bug isolation

Reduces cache-line-rw-smoke to the smallest pattern that still
triggers the multi-tile cache bug:

  * only core 0 does work (1 store + 1 load to one cache line)
  * all other cores immediately return 0
  * no printf, no library calls
  * 16 words written + read

On cachepool_fpu_512 (1 tile) this passes cleanly.
On cachepool_2t_fpu_512 and cachepool_4t_fpu_512 the SB still flags
RESP DATA MISMATCH on cache lines touched by the startup/exit
runtime path (not by the test data).  The test's own data check
PASSES because the corrupted line is not the test's buf line, but
the underlying cache-state bug is reproduced.

Conclusion from this repro: the bug fires the moment NumTiles > 1
even on purely single-core local activity -- it is NOT a coherence
problem (no cross-tile sharing happens here) and NOT a remote-port
routing problem (no real remote traffic from cores 1+).  The
suspect surface narrows to the multi-tile-conditional rotation
math in tcdm_cache_interco/cachepool_tile (bits_to_rotate widens
from CacheBankBits to CacheBankBits+TileBits at NumTiles>1) or the
remote-port muxing inside the local cache_ctrl when those ports
are wired in even though they carry no traffic.

Use for future waveform-level debug:
  make vsim config=cachepool_2t_fpu_512 -B
  ./sim/bin/cachepool_cluster.vsim \
      software/build/CachePoolTests/test-cachepool-minimal-tile0-repro

* [RTL] cluster/group: propagate UseHashWaySelect through multi-tile path

The multi-tile (NumTiles>1) cluster instantiates cachepool_group, which then
instantiates cachepool_tile -- but cachepool_group did not forward
UseHashWaySelect, so the tile fell back to its own 1'b0 default. This silently
disabled hash-way select on every multi-tile build and triggered the
forwarding-buffer / skewed-fold data-corruption path. Add the missing
parameter wiring; default to 1'b1 to match cachepool_cluster.sv.

* [Bender] bump insitu-cache to zexin/sync-flush-fixes

* [VERIF] cc/interco: add plusarg-gated write-ack + addr-watch probes

* [VERIF] cc: demote benign EOC write-ack FIFO tail to info

* [VERIF] cc/tile: guard debug probes with ifndef TARGET_SYNTHESIS

* [Bender] bump insitu-cache lock to tcdm_wrapper comb-loop fix (2710920)

* [VERIF] cc/tile/interco: wrap long lines + verible waivers for debug probes

* [Lint] cluster: fix W110 user_i width mismatch; waive W123 false-positives

- cachepool_cluster.sv: zero-extend refill_user_t to the AxiUserWidth user_i
  port of reqrsp_to_axi + ASSERT_INIT(AxiUserWidth >= $bits(refill_user_t)).
  Behavior-preserving (was an implicit zero-extend); closes the refill-misroute
  hazard at elaboration.
- config.mk: correct stale axi_user_width comment (cache_info_t has no tile_id;
  the tile term is over-provisioned headroom, not an exact fit).
- lint.tcl: DU-scoped W123 waivers for cachepool_cache_ctrl (coalescer_resp/
  bypass_resp driven via i_bypass_xbar slv_rsp_o aggregate) + spatz_decoder.

* [TEST] multi_producer: atomic rlc_ctx updates for multi-consumer

Mark vtNext/pduWithoutPoll/byteWithoutPoll _Atomic and use atomic_fetch_add
(unique per-PDU SN + shared stats) instead of plain +=; add a release fence
before each lock unlock. Fixes the multi-consumer data race; K100 passes
eoc_clean with 0 scoreboard mismatches on 2t/4t x rp1/rp2.

* [Lint] address review: relocate tile verif, fix W123 root-cause, always_ff/rst_ni

- Move per-tile TCDM tracer + memory-model VIP out of cachepool_tile.sv into
  hardware/src/verif/cachepool_tile_tcdm_checker.sv (bind-attached); keeps the
  RTL body synthesis-clean.
- tcdm_cache_interco: put rst_ni in the Probe-D always_ff sensitivity list.
- Drop the cachepool_cache_ctrl W123 waiver: root-caused to '{}-as-lvalue on
  the bypass_xbar output ports (fixed in insitu-cache to concatenation {});
  lint confirms the cache_ctrl W123 is gone.
- Bump insitu-cache lock to 65940a3 (the concatenation fix).

* [CFG] make L1 folded/hash-way/fwd-buffer config-selectable

- Add l1d_use_folded / l1d_fold_way_group / l1d_use_hash_way / l1d_use_fwd_buf
  knobs to cachepool_512.mk + cachepool_fpu_512.mk (default = production:
  folded+hash+fwd); emit them as VLOG_DEFS macros.
- Thread UseForwardingBuffer through cluster->group->tile->cache_ctrl, and
  macro-default UseFoldedDataBanks/FoldWayGroup/UseHashWaySelect/UseForwardingBuffer
  at the wrapper from those macros.
- Fix dropped param paths: forward UseFoldedDataBanks/FoldWayGroup through the
  cluster->group instantiation, and UseHashWaySelect/UseForwardingBuffer through
  wrapper->cluster (previously stuck at module defaults).
- Bump insitu-cache lock to fbabd6a (fwd-buffer param + PartSplit=1 fixes).
Verified: folded default bit-identical pass; unfolded conventional passes.

Author: Aquaticfuller

Bender.lock

@@ -71,7 +71,7 @@ packages:
     - common_verification
     - register_interface
   insitu-cache:
-    revision: fa761ddebc946f9b46509d84945bf41ee1a9ec49
+    revision: fbabd6a06fd801c960078517ad47f7994130b944
     version: null
     source:
       Git: https://github.com/pulp-platform/Insitu-Cache.git

Bender.yml

@@ -14,7 +14,7 @@ dependencies:
   register_interface:  { git: "https://github.com/pulp-platform/register_interface.git", version: 0.3.8                }
   riscv-dbg:           { git: "https://github.com/pulp-platform/riscv-dbg.git",          version: 0.7.0                }
   tech_cells_generic:  { git: "https://github.com/pulp-platform/tech_cells_generic.git", version: 0.2.11               }
-  Insitu-Cache:        { git: "https://github.com/pulp-platform/Insitu-Cache.git",       rev:     zexin/cachepool_dev  }
+  Insitu-Cache:        { git: "https://github.com/pulp-platform/Insitu-Cache.git",       rev:     zexin/sync-flush-fixes }
   spatz:               { git: "https://github.com/pulp-platform/spatz.git",              rev:     cachepool-32b        }
   dram_rtl_sim:        { git: "https://github.com/pulp-platform/dram_rtl_sim.git",       rev:     cachepool            }
 
@@ -49,6 +49,10 @@ sources:
   - hardware/src/cachepool_cluster.sv
   # Level 4
   - hardware/tb/cachepool_cluster_wrapper.sv
+  # sim-only verification IP (bind-attached; excluded from synth/lint)
+  - target: simulation
+    files:
+    - hardware/src/verif/cachepool_tile_tcdm_checker.sv
   # testbench
   - target: cachepool_test
     files:

Makefile

@@ -251,6 +251,12 @@ VLOG_DEFS += -DL1D_TILE_SIZE=$(l1d_tile_size)
 VLOG_DEFS += -DL1D_TAG_DATA_WIDTH=$(l1d_tag_data_width)
 VLOG_DEFS += -DL1D_NUM_BANKS=$(l1d_num_banks)
 VLOG_DEFS += -DL1D_DEPTH=$(l1d_depth)
+# L1 data-bank micro-architecture knobs (1=on/0=off). Production = folded(1) +
+# hash-way(1) + fwd-buffer(1). Unfolded conventional = folded(0)+hash(0)+fwd(0).
+VLOG_DEFS += -DL1D_USE_FOLDED=$(l1d_use_folded)
+VLOG_DEFS += -DL1D_FOLD_WAY_GROUP=$(l1d_fold_way_group)
+VLOG_DEFS += -DL1D_USE_HASH_WAY=$(l1d_use_hash_way)
+VLOG_DEFS += -DL1D_USE_FWD_BUF=$(l1d_use_fwd_buf)
 
 # CachePool CC / core cluster
 VLOG_DEFS += -DSPATZ_FPU_EN=$(spatz_fpu_en)
@@ -277,6 +283,8 @@ VLOG_DEFS += -DPERIPH_START_ADDR=$(periph_start_addr)
 VLOG_DEFS += -DBOOT_ADDR=$(boot_addr)
 VLOG_DEFS += -DUART_ADDR=$(uart_addr)
 
+VLOG_DEFS += -DENABLE_SPATZ_REQ_SCOREBOARD
+
 ENABLE_CACHEPOOL_TESTS ?= 1
 
 # Bender targets

config/cachepool_2t_fpu_512.mk

@@ -0,0 +1,101 @@
+# Copyright 2026 ETH Zurich and University of Bologna.
+# Licensed under the Apache License, Version 2.0, see LICENSE for details.
+# SPDX-License-Identifier: Apache-2.0
+
+# 2-tile / 8-core FPU variant of cachepool_fpu_512.
+# Bisect point between 1t/4c (works) and 4t/16c (broken) to isolate
+# whether the bug is in the inter-tile (group-level) xbar at all, or
+# only at 4t.
+
+#########################
+##  CachePool Cluster  ##
+#########################
+
+# Number of tiles
+num_tiles ?= 2
+
+# Number of cores
+num_cores ?= 8
+
+# Core datawidth
+data_width ?= 32
+
+# Core addrwidth
+addr_width ?= 32
+
+num_remote_ports_per_tile ?= 1
+
+
+######################
+##  CachePool Tile  ##
+######################
+
+# Number of cores per CachePool tile
+num_cores_per_tile ?= 4
+
+# Refill interconnection data width
+refill_data_width ?= 128
+
+##### L1 Data Cache #####
+
+# L1 data cacheline width (in Bit)
+l1d_cacheline_width ?= 512
+
+# L1 data cache size (in KiB)
+l1d_size ?= 256
+
+# L1 data cache banking factor (how many banks per core?)
+l1d_bank_factor ?= 1
+
+# L1 coalecsing window
+l1d_coal_window ?= 2
+
+# L1 data cache number of ways per
+l1d_num_way ?= 4
+
+# L1 data cache size per tile (KiB)
+l1d_tile_size ?= 256
+
+# L1 data cache tag width (TODO: should be calcualted)
+l1d_tag_data_width ?= 92
+
+####################
+##  CachePool CC  ##
+####################
+# Spatz fpu support?
+spatz_fpu_en ?= 1
+
+# Spatz number of FPU
+spatz_num_fpu ?= 4
+
+# Spatz number of IPU
+spatz_num_ipu ?= 4
+
+# Spatz max outstanding transactions
+spatz_max_trans ?= 32
+
+# Snitch/FPU max outstanding transactions
+snitch_max_trans ?= 16
+
+
+#####################
+##  L2 Main Memory ##
+#####################
+# L2 number of channels
+l2_channel ?= 4
+
+# L2 bank width (DRAM width, change with care)
+l2_bank_width ?= 512
+
+# L2 interleaving factor (in order of bank_width)
+l2_interleave ?= 16
+
+
+##################
+##  Peripherals ##
+##################
+# Hardware stack size (in Byte)
+stack_hw_size ?= 1024
+
+# Stack size (total, including share and private, 32'h800)
+stack_tot_size ?= 2048

config/cachepool_512.mk

@@ -9,10 +9,10 @@
 #########################
 
 # Number of tiles
-num_tiles ?= 4
+num_tiles ?= 1
 
 # Number of cores
-num_cores ?= 16
+num_cores ?= 4
 
 # Core datawidth
 data_width ?= 32
@@ -54,6 +54,18 @@ l1d_tile_size ?= 256
 # L1 data cache tag width (TODO: should be calcualted)
 l1d_tag_data_width ?= 92
 
+# L1 data-bank micro-architecture (1=on, 0=off).
+# Production cache  = folded(1) + hash-way(1) + fwd-buffer(1).
+# Unfolded "conventional" cache = folded(0) + hash-way(0) + fwd-buffer(0).
+# Constraints (enforced by RTL elaboration asserts):
+#   - folded (l1d_use_folded=1) REQUIRES l1d_use_hash_way=1
+#   - fwd-buffer (l1d_use_fwd_buf=1) REQUIRES l1d_use_hash_way=1
+# l1d_fold_way_group=0 => auto (min(4, ways)).
+l1d_use_folded     ?= 1
+l1d_fold_way_group ?= 0
+l1d_use_hash_way   ?= 1
+l1d_use_fwd_buf    ?= 1
+
 ####################
 ##  CachePool CC  ##
 ####################

config/cachepool_fpu_512.mk

@@ -56,6 +56,18 @@ l1d_tile_size ?= 256
 # L1 data cache tag width (TODO: should be calcualted)
 l1d_tag_data_width ?= 92
 
+# L1 data-bank micro-architecture (1=on, 0=off).
+# Production cache  = folded(1) + hash-way(1) + fwd-buffer(1).
+# Unfolded "conventional" cache = folded(0) + hash-way(0) + fwd-buffer(0).
+# Constraints (enforced by RTL elaboration asserts):
+#   - folded (l1d_use_folded=1) REQUIRES l1d_use_hash_way=1
+#   - fwd-buffer (l1d_use_fwd_buf=1) REQUIRES l1d_use_hash_way=1
+# l1d_fold_way_group=0 => auto (min(4, ways)).
+l1d_use_folded     ?= 1
+l1d_fold_way_group ?= 0
+l1d_use_hash_way   ?= 1
+l1d_use_fwd_buf    ?= 1
+
 ####################
 ##  CachePool CC  ##
 ####################

config/config.mk

@@ -105,14 +105,57 @@ snitch_max_trans ?= 16
 ##  AXI configuration  ##
 #########################
 
+# axi_user_width sets the AXI 'user' field that carries refill_user_t over the
+# cache->L2 AXI.  Hard requirement: axi_user_width >= $bits(refill_user_t).
+# This floor is now enforced in RTL by ASSERT_INIT(CheckAxiUserFitsRefillUser)
+# in cachepool_cluster.sv, and the cluster zero-extends refill_user_t onto the
+# (wider) AXI user port at the reqrsp_to_axi call site.
+#
+#   refill_user_t = bank_id(BankIDWidth) + tile_id(TileIDWidth)
+#                 + cache_info_t + burst_req_t
+#   cache_info_t  = {for_write_pend, depth, way}   -- NOTE: NO tile_id inside.
+#
+# So refill_user_t carries TileIDWidth exactly ONCE.  The values below are
+# deliberately conservative headroom, not an exact fit: the per-tile adjustment
+# adds 2*(idx_width(NumTiles)-1) bits, and cachepool_pkg.sv:163 then adds a
+# further clog2(NumTiles) (SpatzAxiUserWidth = AXI_USER_WIDTH + clog2(NumTiles)),
+# so the tile term ends up over-provisioned.  This is harmless: no RTL reads any
+# AXI user bit above $bits(refill_user_t)-1.
+# (An earlier version of this comment claimed cache_info_t held a second
+#  TileIDWidth copy and therefore doubled idx_width(NumTiles) -- it does not;
+#  the 2x factor is just slack, kept as-is for headroom.)
+#
+# Do NOT let axi_user_width drop below $bits(refill_user_t): the MSB of bank_id
+# (or higher tile_id) would be truncated on the AXI loopback and refill
+# responses would route back to the wrong slv port (e.g. bank_id=4 aliases to
+# bank_id=0, sending cb=3's refill response to the icache bypass slot, making
+# cb=3 hang).  The ASSERT_INIT above catches this at elaboration.
+
+ifeq ($(num_tiles),1)
+  axi_user_tile_adj := 0
+else ifeq ($(num_tiles),2)
+  axi_user_tile_adj := 0
+else ifeq ($(num_tiles),4)
+  axi_user_tile_adj := 2
+else ifeq ($(num_tiles),8)
+  axi_user_tile_adj := 4
+else ifeq ($(num_tiles),16)
+  axi_user_tile_adj := 6
+else
+  $(error num_tiles=$(num_tiles) not handled by axi_user_width formula; add a case in config.mk)
+endif
+
+# Base widths for NumTiles=1 (= reference values, verified working).
 ifeq ($(l1d_cacheline_width),512)
-  axi_user_width := 17
+  axi_user_base := 18
 else ifeq ($(l1d_cacheline_width),256)
-  axi_user_width := 18
+  axi_user_base := 19
 else ifeq ($(l1d_cacheline_width),128)
-  axi_user_width := 21
+  axi_user_base := 22
 endif
 
+axi_user_width := $(shell echo $$(( $(axi_user_base) + $(axi_user_tile_adj) )))
+
 #####################
 ##  L2 Main Memory ##
 #####################