GitHub - Ayushmaan06/Deepseek: DeepSeek Implementation from Scratch

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
Implementation		Implementation
00_DeepSeek_Complete_Implementation_Overview.md		00_DeepSeek_Complete_Implementation_Overview.md
01_LLM.md		01_LLM.md
02_attention.md		02_attention.md
03_self_attention.md		03_self_attention.md
04_casual_attention.md		04_casual_attention.md
05_MultiHeadAttention.ipynb		05_MultiHeadAttention.ipynb
06_MultiHeadAttention_Implementation.ipynb		06_MultiHeadAttention_Implementation.ipynb
07_KeyValueCache.md		07_KeyValueCache.md
08_MultiQueryAttention.md		08_MultiQueryAttention.md
09_GroupedQueryAttention.md		09_GroupedQueryAttention.md
10_MultiheadLatentAttention.md		10_MultiheadLatentAttention.md
11_MultiHeadLatentAtention.ipynb		11_MultiHeadLatentAtention.ipynb
12_Rotart_Positional_Encoding.md		12_Rotart_Positional_Encoding.md
13_SinusoidalPositionalEncodings.md		13_SinusoidalPositionalEncodings.md
14_Rotatory_Positional_Encoding.md		14_Rotatory_Positional_Encoding.md
15_DeepSeek_MHLA_RoPE_Inference.md		15_DeepSeek_MHLA_RoPE_Inference.md
16_MoE.md		16_MoE.md
17_MoE_Balancing.md		17_MoE_Balancing.md
18_MoE_DeepSeek.md		18_MoE_DeepSeek.md
19_MoE.ipynb		19_MoE.ipynb
20_MultiTokenPrediction.md		20_MultiTokenPrediction.md
21_MTP.ipynb		21_MTP.ipynb
22_DeepseekQuantization.md		22_DeepseekQuantization.md
2412.19437v2.pdf		2412.19437v2.pdf
Amazon_ML_Summer_School_Project_Description.md		Amazon_ML_Summer_School_Project_Description.md
Transformers.md		Transformers.md
image.png		image.png
input.txt		input.txt