Potential top-down skip routing bug in non-biological unroll mode

[leibniz21c]

Potential top-down skip routing bug in the non-biological unroll

Description

There may be an indexing issue in the V4-to-V1 top-down skip connection when bio_unroll=False.

The README describes the intended long-range skip connections as:

• Bottom-up skip: V1 → V4
• Top-down skip: V4 → V1

During layer construction, the top-down skip module is created only for V1:

skip_connections_td=self.skip_connections and (idx == 2)

This is consistent with self.areas[2] == "V1".

However, the non-biological top-down sweep iterates through the areas in reverse order:

for idx, area in enumerate(reversed(self.areas[1:-1])):

The resulting mapping is:

idx	area
0	LOC
1	V4
2	V3
3	V2
4	V1
5	LGN

The top-down skip input is currently routed using:

td_skip_input=td_activations[5] if idx + 1 == 2 else None

In this reversed loop, (idx + 1) == 2 is true for idx == 1, which corresponds to V4 rather than V1.

Consequently:

1. td_activations[5], corresponding to V4, is passed to the V4 layer as td_skip_input.
2. V4 does not contain the top-down skip convolution; its skip modules are NoOpModules.
3. V1 contains the intended V4-to-V1 skip convolution but receives td_skip_input=None.
4. The intended V4-to-V1 top-down skip therefore appears to be inactive in the bio_unroll=False path.

The same condition is used in both the initial top-down sweep and the recurrent top-down sweep:

• blt_vs_model/blt_vs.py, initial top-down sweep
• blt_vs_model/blt_vs.py, recurrent top-down sweep
• The duplicated model under blt_vs_model/training_code/models/BLT_VS.py contains the same conditions.

Why this appears specific to bio_unroll=False

The bio_unroll=True path iterates in forward area order:

for idx, area in enumerate(self.areas[1:-1]):

In that loop, (idx + 1) == 2 correctly corresponds to V1. The issue therefore appears specific to the reversed indexing used by the non-biological unroll.

Possible fix

Using the area name directly would avoid ambiguity between the absolute area index and the index within the reversed list:

td_skip_input=td_activations[5] if area == "V1" else None

This would need to be changed in both top-down sweeps and in both copies of the model definition.

Potential checkpoint implication

The released ImageNet model uses bio_unroll=False. If that checkpoint was trained with this exact implementation, the V4-to-V1 skip parameters may not have received gradients during training.

Fixing the routing could therefore change the behavior of existing checkpoints, because previously unused skip parameters may still contain their initialization values. A corrected model may need to be retrained, or backward-compatible checkpoint behavior may need to be considered.

Questions

Could you please confirm:

1. Whether the current routing is intentional?
2. Whether the released ImageNet checkpoint was trained using this implementation?
3. Whether you would welcome a small pull request with the routing fix and a regression test?

Version inspected

• Repository: KietzmannLab/BLT-VS
• Branch: main
• Commit: 032a4d9434e8f33f2d4d1523e7012db856d66e19

[novelmartis]

Thank you for bringing this to our attention. You are right - in the current implementation (and release v1.0.0), in the non-bio-unroll condition, the top-down V4 -> V1 connection is inactive due to a bug in the code. I'll fix the bug as soon as I get a moment. The network should still be useful for most purposes as the lateral connections (which typically do the heavy-lifting in RNNs) are functioning properly.