README.md

This repository contains notebooks and materials for teaching Deep Learning. Below is the Table of Contents (TOC) categorized by topics.

Table of Contents

1. Foundations

• Vectors and Linear Algebra
• Systems of Linear Equations
• Tensors

2. Machine Learning Basics

• Supervised Learning
• Loss Functions

3. Neural Networks

• First Neural Network
• Activation Functions (ReLU)
• Shallow vs. Deep Neural Networks
• Backpropagation
• Parameter Initialization (He Initialization)

4. Optimization Techniques

• Gradient Descent Variants (SGD, Momentum, Nesterov Momentum, Adam)
• Learning Rate Schedulers

5. Model Performance

• Overfitting and Bias-Variance Tradeoff
• Regularization Techniques (Dropout, Data Augmentation)

6. Convolutional Neural Networks (CNNs)

• Basics of Convolutions (1D Convolutions, Stride, Dilation)
• CNN Architectures and Projects
• Transfer Learning

7. Natural Language Processing (NLP)

• Tokenization and Embedding
• Transformers Overview (including BERT and Vision Transformers)
• Large Language Model (LLM)
• Sentiment Analysis with Transformers

8. Residual Networks

• Overview and Analysis
• ResNet Architecture

9. Autoencoders

• Variants (Denoising Autoencoders, Variational Autoencoders)
• Applications (Anomaly Detection)

10. Generative Models

• GANs (Generative Adversarial Networks)
• Pix2Pix and CycleGAN

11. Normalizing Flows

• Introduction and Advanced Topics

12. Diffusion Models

• Theory and Implementation
• Guided Diffusion

13. Graph Neural Networks (GNNs)

• Motivation and Common Tasks
• GCNs (Graph Convolutional Networks)
• Attention Mechanisms and Edge Embedding

14. Reinforcement Learning

• Foundations and Mathematical Background
• Value Functions and Optimality
• Deep Q-Networks (DQN)