Skip to content

Commit 5c7e97e

Browse files
tianyumyumtianyumyum
committed
2.2
1 parent a5bf622 commit 5c7e97e

8 files changed

+3345
-3
lines changed

‎.DS_Store

0 Bytes
Binary file not shown.

‎.ipynb_checkpoints/1.2_矩阵和导数-checkpoint.ipynb

+334
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,334 @@
1+
{
2+
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "cd9f9a3b",
7+
"metadata": {},
8+
"source": [
9+
" ![image.png](Images/1.2矩阵与导数图片.png)"
10+
]
11+
},
12+
{
13+
"cell_type": "code",
14+
"execution_count": null,
15+
"id": "774e19ed",
16+
"metadata": {},
17+
"outputs": [],
18+
"source": [
19+
"自动求导\n",
20+
"符号求导\n",
21+
"数值求导\n",
22+
"计算图：\n",
23+
"将代码分解成操作子，将计算表示成一个无环图"
24+
]
25+
},
26+
{
27+
"cell_type": "code",
28+
"execution_count": 2,
29+
"id": "4971f6b5",
30+
"metadata": {},
31+
"outputs": [
32+
{
33+
"data": {
34+
"text/plain": [
35+
"tensor([0., 1., 2., 3.])"
36+
]
37+
},
38+
"execution_count": 2,
39+
"metadata": {},
40+
"output_type": "execute_result"
41+
}
42+
],
43+
"source": [
44+
"import torch\n",
45+
"x=torch.arange(4.0)\n",
46+
"x"
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": 3,
52+
"id": "f72fc4ce",
53+
"metadata": {},
54+
"outputs": [
55+
{
56+
"data": {
57+
"text/plain": [
58+
"tensor([0., 1., 2., 3.], requires_grad=True)"
59+
]
60+
},
61+
"execution_count": 3,
62+
"metadata": {},
63+
"output_type": "execute_result"
64+
}
65+
],
66+
"source": [
67+
"x.requires_grad_(True)"
68+
]
69+
},
70+
{
71+
"cell_type": "markdown",
72+
"id": "adbe5cb4",
73+
"metadata": {},
74+
"source": [
75+
"现在让我们计算y"
76+
]
77+
},
78+
{
79+
"cell_type": "code",
80+
"execution_count": 4,
81+
"id": "03accfc8",
82+
"metadata": {},
83+
"outputs": [
84+
{
85+
"data": {
86+
"text/plain": [
87+
"tensor(28., grad_fn=<MulBackward0>)"
88+
]
89+
},
90+
"execution_count": 4,
91+
"metadata": {},
92+
"output_type": "execute_result"
93+
}
94+
],
95+
"source": [
96+
"y=2*torch.dot(x,x)\n",
97+
"y"
98+
]
99+
},
100+
{
101+
"cell_type": "markdown",
102+
"id": "c93e90fc",
103+
"metadata": {},
104+
"source": [
105+
"通过调用反向传播函数来自动计算y关于x每个分量的梯度"
106+
]
107+
},
108+
{
109+
"cell_type": "code",
110+
"execution_count": 6,
111+
"id": "ddf77250",
112+
"metadata": {},
113+
"outputs": [
114+
{
115+
"data": {
116+
"text/plain": [
117+
"tensor([ 0., 4., 8., 12.])"
118+
]
119+
},
120+
"execution_count": 6,
121+
"metadata": {},
122+
"output_type": "execute_result"
123+
}
124+
],
125+
"source": [
126+
"y.backward()\n",
127+
"x.grad"
128+
]
129+
},
130+
{
131+
"cell_type": "code",
132+
"execution_count": 7,
133+
"id": "83c3201d",
134+
"metadata": {},
135+
"outputs": [
136+
{
137+
"data": {
138+
"text/plain": [
139+
"tensor([True, True, True, True])"
140+
]
141+
},
142+
"execution_count": 7,
143+
"metadata": {},
144+
"output_type": "execute_result"
145+
}
146+
],
147+
"source": [
148+
"x.grad==4*x"
149+
]
150+
},
151+
{
152+
"cell_type": "markdown",
153+
"id": "fcd7608d",
154+
"metadata": {},
155+
"source": [
156+
"现在我们计算x的另一个函数"
157+
]
158+
},
159+
{
160+
"cell_type": "code",
161+
"execution_count": 17,
162+
"id": "01d6ef51",
163+
"metadata": {},
164+
"outputs": [
165+
{
166+
"name": "stdout",
167+
"output_type": "stream",
168+
"text": [
169+
"tensor(6., grad_fn=<SumBackward0>)\n",
170+
"tensor([1., 1., 1., 1.])\n"
171+
]
172+
}
173+
],
174+
"source": [
175+
"#在默认情况下，Pytorch会累积梯度，我们需要清除之前的值\n",
176+
"x.grad.zero_()#pytorch中下划线表示重写内容\n",
177+
"y=x.sum()\n",
178+
"print(y)\n",
179+
"y.sum().backward()\n",
180+
"print(x.grad)"
181+
]
182+
},
183+
{
184+
"cell_type": "markdown",
185+
"id": "16f40283",
186+
"metadata": {},
187+
"source": [
188+
"深度学习中，我们的目的不是计算微分矩阵，而是批量中每个样本单独计算的偏导数之和。"
189+
]
190+
},
191+
{
192+
"cell_type": "code",
193+
"execution_count": 18,
194+
"id": "e123c634",
195+
"metadata": {},
196+
"outputs": [
197+
{
198+
"data": {
199+
"text/plain": [
200+
"tensor([0., 2., 4., 6.])"
201+
]
202+
},
203+
"execution_count": 18,
204+
"metadata": {},
205+
"output_type": "execute_result"
206+
}
207+
],
208+
"source": [
209+
"x.grad.zero_()\n",
210+
"y=x*x\n",
211+
"#等价于y.backward(torch.ones(len(x)))\n",
212+
"y.sum().backward()\n",
213+
"x.grad"
214+
]
215+
},
216+
{
217+
"cell_type": "markdown",
218+
"id": "c26a32f5",
219+
"metadata": {},
220+
"source": [
221+
"将某些计算移动到记录的计算图之外"
222+
]
223+
},
224+
{
225+
"cell_type": "code",
226+
"execution_count": 19,
227+
"id": "9c46d0f5",
228+
"metadata": {},
229+
"outputs": [
230+
{
231+
"data": {
232+
"text/plain": [
233+
"tensor([True, True, True, True])"
234+
]
235+
},
236+
"execution_count": 19,
237+
"metadata": {},
238+
"output_type": "execute_result"
239+
}
240+
],
241+
"source": [
242+
"x.grad.zero_()\n",
243+
"y=x*x\n",
244+
"u=y.detach()#把y当作一个常数\n",
245+
"z=u*x\n",
246+
"\n",
247+
"z.sum().backward()\n",
248+
"x.grad==u"
249+
]
250+
},
251+
{
252+
"cell_type": "code",
253+
"execution_count": 20,
254+
"id": "e5453970",
255+
"metadata": {},
256+
"outputs": [
257+
{
258+
"data": {
259+
"text/plain": [
260+
"tensor([True, True, True, True])"
261+
]
262+
},
263+
"execution_count": 20,
264+
"metadata": {},
265+
"output_type": "execute_result"
266+
}
267+
],
268+
"source": [
269+
"x.grad.zero_()\n",
270+
"y.sum().backward()\n",
271+
"x.grad==2*x"
272+
]
273+
},
274+
{
275+
"cell_type": "markdown",
276+
"id": "925bf83e",
277+
"metadata": {},
278+
"source": [
279+
"构建函数的计算图，通过Python控制流"
280+
]
281+
},
282+
{
283+
"cell_type": "code",
284+
"execution_count": 22,
285+
"id": "3fdf0d53",
286+
"metadata": {},
287+
"outputs": [],
288+
"source": [
289+
"def f(a):\n",
290+
" b=a*2\n",
291+
" while b.norm()<1000:\n",
292+
" b=b*2\n",
293+
" if b.sum()>0:\n",
294+
" c=b\n",
295+
" else:\n",
296+
" c=100*b\n",
297+
" return c\n",
298+
"\n",
299+
"a=torch.randn(size=(),requires_grad=True)#标量，随机数\n",
300+
"d=f(a)\n",
301+
"d.backward()"
302+
]
303+
},
304+
{
305+
"cell_type": "code",
306+
"execution_count": null,
307+
"id": "fe0d7f7d",
308+
"metadata": {},
309+
"outputs": [],
310+
"source": []
311+
}
312+
],
313+
"metadata": {
314+
"kernelspec": {
315+
"display_name": "py38",
316+
"language": "python",
317+
"name": "py38"
318+
},
319+
"language_info": {
320+
"codemirror_mode": {
321+
"name": "ipython",
322+
"version": 3
323+
},
324+
"file_extension": ".py",
325+
"mimetype": "text/x-python",
326+
"name": "python",
327+
"nbconvert_exporter": "python",
328+
"pygments_lexer": "ipython3",
329+
"version": "3.8.13"
330+
}
331+
},
332+
"nbformat": 4,
333+
"nbformat_minor": 5
334+
}

0 commit comments

Comments
 (0)