fast-weights

Official repository for the paper "A Modern Self-Referential Weight Matrix That Learns to Modify Itself" (ICML 2022 & NeurIPS 2021 Deep RL Workshop) and "Accelerating Neural Self-Improvement via Bootstrapping" (ICLR 2023 Workshop)

Implementation of the paper [Using Fast Weights to Attend to the Recent Past](https://arxiv.org/abs/1610.06258)

Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers" (NeurIPS 2021)

Implementation of Fast Weight Attention

Official repository for the paper "Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules" (NeurIPS 2022)

PyTorch Language Modeling Toolkit for Fast Weight Programmers

Official repository for the paper "Automating Continual Learning"

Weights that never freeze. A continual-learning architecture, and the pre-registered experiment that falsified its central claim. https://arxiv.org/abs/2607.20792

Official repository for the paper "Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules" (ICLR 2023)

Usage kit for the Fractale models — LMs whose long-term memory is a self-written fast-weight thought bank

PyTorch implementation of DCT fast weight RNNs

Official repository for the paper "Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions" (EMNLP 2023)

Your language model forgets everything. Sillage gives it a 4 MB memory that persists across sessions, no gradients, no fine-tuning, no growing index. Beats an unbounded kNN-LM at 1/13 the storage (perplexity 31 → 17). CPU-only, fully reproducible, four preprints with Zenodo DOIs.

PyTorch Implementation of the paper [Using Fast Weights to Attend to the Recent Past]

A lightweight, Hebbian-based AI engine running natively on WebGPU. Zero-latency, privacy-focused.

Inference-time drift experiment demonstrating reduced repetition collapse in frozen LLMs.

Distributed Engrammics — gradient-free transfer of a persistent fast-weight associative state between model instances (DeltaNet, RWKV-7). Paper + reproducible code.

Naturally fractal, self-modifying language model research with persistent in-weights memory.

Measuring what a frozen LLM can cite after context is written into a fixed-size mid-layer state.

Build fast weight attention for episodic memory in Transformers with causal and chunked sequence support