You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<h2id="Interface-caching:-why-and-how">Interface caching: why and how<aclass="anchor-link" href="#Interface-caching:-why-and-how">¶</a></h2><ul>
11808
-
<li><p><code>Pipelines</code> (also called <code>workflows</code>) specify processing by an execution graph. This is useful because it opens the door to dependency checking and enable</p>
11798
+
<li><p><code>Pipelines</code> (also called <code>workflows</code>) specify processing by an execution graph. This is useful because it opens the door to dependency checking and enables</p>
11809
11799
<ul>
11810
11800
<li>to minimize recomputations, </li>
11811
11801
<li>to have the execution engine transparently deal with intermediate file manipulations.</li>
11812
11802
</ul>
11813
-
<p>They however do not blend in well with arbitrary Python code, as they must rely on their own execution engine.</p>
11803
+
<p>They, however, do not blend in well with arbitrary Python code, as they must rely on their own execution engine.</p>
11814
11804
</li>
11815
11805
</ul>
11816
11806
<ul>
@@ -11852,7 +11842,7 @@ <h2 id="A-big-picture-view:-using-the-Memory-object">A big picture view: using t
<p>Note that the caching directory is a subdirectory called <code>nipype_mem</code> of the given <code>base_dir</code>. This is done to avoid polluting the base director.</p>
11855
-
<p>In the corresponding execution context, nipype interfaces can be turned into callables that can be used as functions using the <ahref="http://nipype.readthedocs.io/en/latest/api/generated/nipype.caching.memory.html#nipype.caching.memory.Memory.cache"><code>Memory.cache</code></a> method. For instance if we want to run the fslMerge command on a set of files:</p>
11845
+
<p>In the corresponding execution context, nipype interfaces can be turned into callables that can be used as functions using the <ahref="http://nipype.readthedocs.io/en/latest/api/generated/nipype.caching.memory.html#nipype.caching.memory.Memory.cache"><code>Memory.cache</code></a> method. For instance, if we want to run the fslMerge command on a set of files:</p>
11856
11846
11857
11847
</div>
11858
11848
</div>
@@ -11876,7 +11866,7 @@ <h2 id="A-big-picture-view:-using-the-Memory-object">A big picture view: using t
<p>Note that the <ahref="http://nipype.readthedocs.io/en/latest/api/generated/nipype.caching.memory.html#nipype.caching.memory.Memory.cache"><code>Memory.cache</code></a> method takes interfaces <strong>classes</strong>, and not instances.</p>
11879
-
<p>The resulting <code>fsl_merge</code> object can be applied as a function to parameters, that will form the inputs of the <code>merge</code> fsl commands. Those inputs are given as keyword arguments, bearing the same name as the name in the inputs specs of the interface. In IPython, you can also get the argument list by using the <code>fsl_merge?</code>synthax to inspect the docs:</p>
11869
+
<p>The resulting <code>fsl_merge</code> object can be applied as a function to parameters, that will form the inputs of the <code>merge</code> fsl commands. Those inputs are given as keyword arguments, bearing the same name as the name in the inputs specs of the interface. In IPython, you can also get the argument list by using the <code>fsl_merge?</code>syntax to inspect the docs:</p>
<h2id="Usage-patterns:-working-efficiently-with-caching">Usage patterns: working efficiently with caching<aclass="anchor-link" href="#Usage-patterns:-working-efficiently-with-caching">¶</a></h2><p>The goal of the <code>caching</code> module is to enable writing plain Python code rather than workflows. Use it: instead of data grabber nodes, use for instance the <code>glob</code> module. To vary parameters, use <code>for</code> loops. To make reusable code, write Python functions.</p>
12078
12068
<p>One good rule of thumb to respect is to avoid the usage of explicit filenames apart from the outermost inputs and outputs of your processing. The reason being that the caching mechanism of <code>nipy.caching</code> takes care of generating the unique hashes, ensuring that, when you vary parameters, files are not overridden by the output of different computations.</p>
12079
12069
<divclass="alert alert-info">
12080
-
**Debuging**:
12070
+
**Debugging**:
12081
12071
If you need to inspect the running environment of the nodes, it may be useful to know where they were executed. With `nipype.caching`, you do not control this location as it is encoded by hashes.
12082
12072
To find out where an operation has been persisted, simply look in it's output variable:
<h1id="Dataset-structure">Dataset structure<aclass="anchor-link" href="#Dataset-structure">¶</a></h1><p>To be able to import data, you first need to be aware about the structure of your dataset. The structure of the dataset for this tutorial is according to BIDS, and looks as follows:</p>
11809
+
<h1id="Dataset-structure">Dataset structure<aclass="anchor-link" href="#Dataset-structure">¶</a></h1><p>To be able to import data, you first need to be aware of the structure of your dataset. The structure of the dataset for this tutorial is according to BIDS, and looks as follows:</p>
string template to match. So in the previous case <code>DataGrabber</code> is looking for
11981
11971
path matches of the form <code>/data/ds000114/s*/ses-test/func/*fingerfootlips*.nii.gz</code>.</p>
11982
11972
<divclass="alert alert-info">
11983
-
**Note**: When used with wildcards (e.g., `s*` and `*fingerfootlips*` above) `DataGrabber` does not return data in sorted order. In order to force it to return data in sorted order, one needs to set the input `sorted = True`. However, when explicitly specifying an order as we will see below, `sorted` should be set to `False`.
11984
-
</div><p>More useful cases arise when the template can be filled by other inputs. In the
11973
+
**Note**: When used with wildcards (e.g., `s*` and `*fingerfootlips*` above) `DataGrabber` does not return data in sorted order. In order to force it to return data in a sorted order, one needs to set the input `sorted = True`. However, when explicitly specifying an order as we will see below, `sorted` should be set to `False`.
11974
+
</div><p>More use cases arise when the template can be filled by other inputs. In the
11985
11975
example below, we define an input field for <code>DataGrabber</code> called <code>subject_id</code>. This is
11986
11976
then used to set the template (see <code>%d</code> in the template).</p>
11987
11977
@@ -12200,7 +12190,7 @@ <h2 id="A-more-realistic-use-case">A more realistic use-case<a class="anchor-lin
<p>Above, two more fields are introduced: <code>field_template</code> and <code>template_args</code>. These fields are both dictionaries whose keys correspond to the <code>outfields</code> keyword. The <code>field_template</code> reflects the search path for each output field, while the <code>template_args</code> reflect the inputs that satisfy the template. The inputs can either be one of the named inputs specified by the <code>infields</code> keyword arg or it can be raw strings or integers corresponding to the template. For the <code>func</code> output, the <strong>%s</strong> in the <code>field_template</code> is satisfied by <code>subject_id</code> and the <strong>%d</strong> is field in by the list of numbers.</p>
12193
+
<p>Above, two more fields are introduced: <code>field_template</code> and <code>template_args</code>. These fields are both dictionaries whose keys correspond to the <code>outfields</code> keyword. The <code>field_template</code> reflects the search path for each output field, while the <code>template_args</code> reflect the inputs that satisfy the template. The inputs can either be one of the named inputs specified by the <code>infields</code> keyword arg or it can be raw strings or integers corresponding to the template. For the <code>func</code> output, the <strong>%s</strong> in the <code>field_template</code> is satisfied by <code>subject_id</code> and the <strong>%d</strong> is filled in by the list of numbers.</p>
<p><code>SelectFiles</code> uses the {}-based string formating syntax to plug values into string templates and collect the data. These templates can also be combined with glob wild cards. The field names in the formatting template (i.e. the terms in braces) will become inputs fields on the interface, and the keys in the templates dictionary will form the output fields.</p>
12503
+
<p><code>SelectFiles</code> uses the {}-based string formatting syntax to plug values into string templates and collect the data. These templates can also be combined with glob wild cards. The field names in the formatting template (i.e. the terms in braces) will become inputs fields on the interface, and the keys in the templates dictionary will form the output fields.</p>
12514
12504
<p>Let's focus again on the data we want to import:</p>
<h2id="FreeSurferSource">FreeSurferSource<aclass="anchor-link" href="#FreeSurferSource">¶</a></h2><p><code>FreeSurferSource</code> is a specific case of a file grabber that felicitates the data import of outputs from the FreeSurfer recon-all algorithm. This of course requires that you've already run <code>recon-all</code> on your subject.</p>
12819
+
<h2id="FreeSurferSource">FreeSurferSource<aclass="anchor-link" href="#FreeSurferSource">¶</a></h2><p><code>FreeSurferSource</code> is a specific case of a file grabber that felicitates the data import of outputs from the FreeSurfer recon-all algorithm. This, of course, requires that you've already run <code>recon-all</code> on your subject.</p>
0 commit comments