Visual transformer and next token prediction from scratch in Pytorch

This repo contains an implementation of a Transformer for next token prediction and Vision Transformer (ViT) for digit classification, all built from scratch using PyTorch.

Files

• next_word_prediction.ipynb: Transformer for next token prediction.
• transformer.py: Core Transformer implementation (attention, embeddings, etc.).
• vit.ipynb: Vision Transformer for digit classification.