You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: RELEASENOTES.md
+6
Original file line number
Diff line number
Diff line change
@@ -4,10 +4,16 @@ Releases, starting with 9/2/2021, are listed with the most recent release at the
4
4
5
5
# NuGet Version 0.103.1
6
6
7
+
__Breaking Changes__:
8
+
#1376`torch.Tensor.backward`'s function signature has been updated to match PyTorch's implementation. Previously, passing `create_graph` or `retain_graph` by position would work like PyTorch's `torch.Tensor.backward`, but not if passing by name (`create_graph`'s value was swapped with `retain_graph`). This has been corrected; however, this means any code that passes `create_graph` or `retain_graph` by name needs to be updated to reflect the intended functionality.<br/>
9
+
7
10
__Bug Fixes__:
8
11
9
12
#1383`torch.linalg.vector_norm`: Make `ord`-argument optional, as specified in docs<br/>
10
13
#1385 PackedSequence now participates in the DisposeScope system at the same level as Tensor objects.<br/>
14
+
#1387 Attaching tensor to a DisposeScope no longer makes Statistics.DetachedFromScopeCount go negative.<br/>
15
+
#1390 DisposeScopeManager.Statistics now includes DisposedOutsideScopeCount and AttachedToScopeCount. ThreadTotalLiveCount is now exact instead of approximate. ToString gives a useful debug string, and documentation is added for how to troubleshoot memory leaks. Also DisposeScopeManager.Statistics.TensorStatistics and DisposeScopeManager.Statistics.PackedSequenceStatistics provide separate metrics for these objects.<br/>
Copy file name to clipboardexpand all lines: docfx/articles/memory.md
+11-9
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,8 @@ In both cases, you may want to experiment with using a smaller batch size -- tem
10
10
11
11
Note DiffSharp (which uses TorchSharp) relies on techniques 1.
12
12
13
+
Also refer to [Memory Leak Troubleshooting](memory leak troubleshooting.md) for help on fixing any leaks.
14
+
13
15
> Most of the examples included will use technique #1, doing frequent explicit calls to GC.Collect() in the training code -- if not after each batch in the training loop, at least after each epoch.
14
16
15
17
## Technique 1. Automatic disposal via Garbage Collection
@@ -44,7 +46,7 @@ __Note__: Even with this approach, it is a good idea to place a call to `GC.Coll
44
46
45
47
It is important to understand that all TorchSharp "tensors" (type Tensor) are actually "tensor aliases", referring to a C++ tensor. When a C++ tensor is created and returned to .NET as a tensor alias, and the reference count on the C++ tensor is incremented. When you call `Dispose()` on the TorchSharp tensor alias (that is, type Tensor), it is decremented. If the tensor alias is finalized instead, the decrement happens implicitly.
46
48
47
-
To enable this technique, all operations that return one or more TorchSharp `Tensor`s should return "fresh" Tensor aliases (though that doesn't always mean freshly copied C++ tensors). This is true even for in-place, destructive operations like `add_()`, which overwrites the underlying native tensor with data, but still returns a fresh tensor alias to that same tensor.
49
+
To enable this technique, all operations that return one or more TorchSharp `Tensor`s should return "fresh" Tensor aliases (though that doesn't always mean freshly copied C++ tensors). This is true even for in-place, destructive operations like `add_()`, which overwrites the underlying native tensor with data, but still returns a fresh tensor alias to that same tensor.
48
50
49
51
Thus, when you write methods and functions that take and produce type Tensor, for example in the `forward()` method of a model, you should always make sure to return a fresh alias. Most of the time, this happens automatically, because the last action of your code will normally be to call another tensor function, which itself will be returning a fresh alias, but there are cases when it's not, especially when returning input tensors or tensors stored in some lookaside table.
50
52
@@ -55,7 +57,7 @@ Tensor flatten(Tensor input) {
55
57
if (input.shape.Length==1)
56
58
returninput.alias();
57
59
else
58
-
returninput.reshape(input.numel());
60
+
returninput.reshape(input.numel());
59
61
}
60
62
```
61
63
@@ -100,10 +102,10 @@ let myTensorFunction0(input: Tensor) =
100
102
input.alias()
101
103
102
104
let myTensorFunction1() =
103
-
if today then
104
-
table[4].alias()
105
+
if today then
106
+
table[4].alias()
105
107
else
106
-
table[5].alias()
108
+
table[5].alias()
107
109
108
110
let myTensorFunction2(input: Tensor) =
109
111
input.add(tensor(1))
@@ -124,9 +126,9 @@ let myTensorFunction5(go: bool, input: Tensor) =
124
126
tmp2.add(tensor(1))
125
127
else
126
128
input.alias()
127
-
129
+
128
130
let myTensorFunction5(go: bool, input: Tensor) =
129
-
if go then
131
+
if go then
130
132
use tmp1 = input.add_(tensor(1)) // NOTE: even for in-place mutations
131
133
use tmp2 = input.add_(tensor(1)) // NOTE: even for in-place mutations
132
134
tmp2.add(tensor(1))
@@ -173,13 +175,13 @@ use d = torch.NewDisposeScope()
If you need to dispose some tensors before the scope is disposed, you can use `DisposeEverything()`, or `DisposeEverythingBut(...)` if you want to exclude a few tensors from disposal. These can be useful when tensor lifetimes aren't cleanly nested in dynamic scopes.
178
+
If you need to dispose some tensors before the scope is disposed, you can use `DisposeEverything()`, or `DisposeEverythingBut(...)` if you want to exclude a few tensors from disposal. These can be useful when tensor lifetimes aren't cleanly nested in dynamic scopes.
177
179
178
180
__NOTE: It is absolutely essential for the proper functioning of dynamic dispose scopes that the scope is created with a 'using' statemen (C#) or 'use' expression (F#).__
179
181
180
182
It's important to note that these scopes are dynamic -- if any functions are called, the tensors inside them are also registered and disposed, unless there's a nested scope within those functions.
181
183
182
-
It is advisable to place a dispose scope around your training and test code, and in any library code that can be called from contexts that do not have dispose scopes.
184
+
It is advisable to place a dispose scope around your training and test code, and in any library code that can be called from contexts that do not have dispose scopes.
183
185
184
186
That said, you should use dispose scope very carefully: having _too few_ scope raises the pressure on native memory, which is particularly bad for GPUs. Having too _many_ scopes, managing too few temporaries, will add runtime overhead to computations. For example, it may be better to put a scope outside an inner loop that contains multiple computations than to place it inside the loop. There is no single best answer.
0 commit comments