You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/dsl1.md
+47-34Lines changed: 47 additions & 34 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,24 +1,12 @@
1
1
(dsl1-page)=
2
2
3
-
# Migrating from DSL 1
3
+
# Migrating from DSL1
4
4
5
-
In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed, and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 in order to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2, and how to migrate to DSL2.
6
-
7
-
In Nextflow versions prior to `22.03.0-edge`, you must enable DSL2 explicitly in order to use it. You can either set the feature flag in your pipeline script:
8
-
9
-
```nextflow
10
-
nextflow.enable.dsl=2
11
-
```
12
-
13
-
Or set the environment variable where you launch Nextflow:
14
-
15
-
```bash
16
-
export NXF_DEFAULT_DSL=2
17
-
```
5
+
In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2 and how to migrate to DSL2.
18
6
19
7
## Processes and workflows
20
8
21
-
In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. Here is the {ref}`your-first-script` example written in DSL1:
9
+
In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. You can see a basic Nextflow script written in DSL1 here:
22
10
23
11
```nextflow
24
12
nextflow.enable.dsl=1
@@ -53,44 +41,69 @@ result.view { it.trim() }
53
41
54
42
To migrate this code to DSL2, you need to move all of your channel logic throughout the script into a `workflow` definition. Additionally, you must call each process explicitly, passing any input channels as arguments (instead of `from ...`) and receiving any output channels as return values (instead of `into ...`).
55
43
56
-
Refer to the {ref}`workflow-page` page to learn how to define a workflow. The DSL2 version of the above script is duplicated here for your convenience:
44
+
See {ref}`workflow-page` page to learn how to define a workflow.
57
45
58
-
```{literalinclude} snippets/your-first-script.nf
59
-
:language: nextflow
60
-
```
46
+
You can see the DSL1 Nextflow script from above written in DSL2 here:
61
47
62
-
## Channel forking
48
+
```nextflow
49
+
params.str = 'Hello world!'
50
+
51
+
process splitLetters {
52
+
output:
53
+
path 'chunk_*'
54
+
55
+
script:
56
+
"""
57
+
printf '${params.str}' | split -b 6 - chunk_
58
+
"""
59
+
}
63
60
64
-
In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator.
61
+
process convertToUpper {
62
+
input:
63
+
path x
65
64
66
-
In DSL2, channels are automatically forked when connecting two or more consumers.
65
+
output:
66
+
stdout
67
67
68
-
For example, this would not work in DSL1 but is not a problem in DSL2:
In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator. In DSL2, channels are automatically forked when connecting two or more consumers. For example:
69
82
70
83
```nextflow
71
84
Channel
72
-
.from('Hello','Hola','Ciao')
73
-
.set{ cheers }
85
+
.of('Hello','Hola','Ciao')
86
+
.set{ cheers }
74
87
75
88
cheers
76
-
.map{ it.toUpperCase() }
89
+
.map { v -> v.toUpperCase() }
77
90
.view()
78
91
79
92
cheers
80
-
.map{ it.reverse() }
93
+
.map { v -> v.reverse() }
81
94
.view()
82
95
```
83
96
84
-
Similarly, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.
97
+
Similarly, in DSL2, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.
85
98
86
99
## Modules
87
100
88
-
In DSL1, the entire Nextflow pipeline must be defined in a single file (e.g. `main.nf`). This restriction becomes quite cumbersome as a pipeline becomes larger, and it hinders the sharing and reuse of pipeline components.
101
+
In DSL1, the entire Nextflow pipeline must be defined in a single file. For example, `main.nf`. This restriction becomes cumbersome as a pipeline grows and hinders the sharing and reuse of pipeline components.
89
102
90
-
DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2 by any means, modules can help you organize a large pipeline into multiple smaller files, and take advantage of modules created by others. Check out the {ref}`module-page` to get started.
103
+
DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2, modules can help you organize a large pipeline into multiple smaller files and take advantage of modules created by others. See {ref}`module-page` to learn more about modules.
91
104
92
105
:::{note}
93
-
DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be split into modules to avoid this limit.
106
+
DSL2 scripts cannot exceed 64 KB in size. Split large DSL1 scripts into modules to avoid this limit.
94
107
:::
95
108
96
109
## Deprecations
@@ -103,14 +116,13 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli
103
116
104
117
- The `mode flatten` option for process outputs is no longer available. Use the {ref}`operator-flatten` operator on the corresponding output channel instead.
105
118
106
-
- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier.
107
-
108
-
For example:
119
+
- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier. For example:
109
120
110
121
```nextflow
111
122
process foo {
112
123
input:
113
124
tuple X, 'some-file.sam'
125
+
114
126
output:
115
127
tuple X, 'some-file.bam'
116
128
@@ -127,6 +139,7 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli
Copy file name to clipboardExpand all lines: docs/install.md
+14-46Lines changed: 14 additions & 46 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,11 +2,17 @@
2
2
3
3
# Installation
4
4
5
+
Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). This page describes how to install Nextflow.
6
+
7
+
:::{note}
8
+
New versions of Nextflow are released regularly. See {ref}`updating-nextflow-page` for more information about Nextflow release cadence, how to update Nextflow, and how select your version of Nextflow.
9
+
:::
10
+
5
11
(install-requirements)=
6
12
7
13
## Requirements
8
14
9
-
Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). It requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. You can see which version you have using the following command:
15
+
Nextflow requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. To see which version of Java you have, run the following command:
10
16
11
17
```{code-block} bash
12
18
:class: copyable
@@ -17,7 +23,7 @@ java -version
17
23
Support for Java versions prior to 17 was dropped.
18
24
:::
19
25
20
-
If you don't have a compatible version of Java installed in your computer, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest LTS version of Temurin. See [this website](https://whichjdk.com/) for more information.
26
+
If you don't have a compatible version of Java installed, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest Long-Term-Support (LTS) version of Temurin. See [Which version of JDK should I use?](https://whichjdk.com/) for more information about different versions of Java.
21
27
22
28
To install Java with SDKMAN:
23
29
@@ -50,15 +56,15 @@ To install Java with SDKMAN:
50
56
51
57
Nextflow is distributed as a self-installing package, in order to make the installation process as simple as possible:
52
58
53
-
1. Install Nextflow:
59
+
To install Nextflow:
60
+
61
+
1. Download Nextflow:
54
62
55
63
```{code-block} bash
56
64
:class: copyable
57
65
curl -s https://get.nextflow.io | bash
58
66
```
59
67
60
-
This will create the `nextflow` executable in the current directory.
61
-
62
68
:::{tip}
63
69
Set `export CAPSULE_LOG=none` to make the installation logs less verbose.
64
70
:::
@@ -93,27 +99,12 @@ Nextflow is distributed as a self-installing package, in order to make the insta
93
99
nextflow info
94
100
```
95
101
96
-
## Updates
97
-
98
-
With Nextflow installed in your environment, you can update to the latest version using the following command:
99
-
100
-
```{code-block} bash
101
-
:class: copyable
102
-
nextflow self-update
103
-
```
104
-
105
-
You can also temporarily switch to a specific version of Nextflow with the `NXF_VER` environment variable. For example:
106
-
107
-
```{code-block} bash
108
-
:class: copyable
109
-
NXF_VER=23.10.0 nextflow info
110
-
```
111
-
112
102
## Seqera Platform
113
103
114
104
You can launch workflows directly from [Seqera Platform](https://seqera.io/platform/) without installing Nextflow locally.
115
105
116
106
Launching from Seqera Platform provides you with:
107
+
117
108
- User-friendly launch interfaces.
118
109
- Automated cloud infrastructure creation.
119
110
- Organizational user management.
@@ -122,34 +113,11 @@ Launching from Seqera Platform provides you with:
122
113
Seqera Cloud Basic is free for small teams. Researchers at qualifying academic institutions can apply for free access to Seqera Cloud Pro.
123
114
See the [Seqera Platform documentation](https://docs.seqera.io/platform) for set-up information and tutorials to get started.
124
115
125
-
## Stable and edge releases
126
-
127
-
A *stable* version of Nextflow is released every six months, in the 4th and 10th month of each year.
128
-
129
-
Additionally, an *edge* version is released on a monthly basis. The edge releases can be used to access the latest updates and experimental features.
130
-
131
-
To use the latest edge release, set `NXF_EDGE=1` when updating:
132
-
133
-
```{code-block} bash
134
-
:class: copyable
135
-
NXF_EDGE=1 nextflow self-update
136
-
```
137
-
138
-
You can also use `NXF_VER` to temporarily switch to any edge release. For example:
139
-
140
-
```{code-block} bash
141
-
:class: copyable
142
-
NXF_VER=24.06.0-edge nextflow info
143
-
```
144
-
145
116
## Standalone distribution
146
117
147
-
The Nextflow standalone distribution (i.e. the `dist` distribution) consists of self-contained `nextflow` executable file
148
-
that includes all the application dependencies for core functionalities, and it can run without downloading third parties
149
-
libraries. This distribution is mainly useful for offline environments.
118
+
The Nextflow standalone distribution (i.e. the `dist` release) is a self-contained `nextflow` executable that can run without needing to download core dependencies at runtime. This distribution is useful for offline environments, as well as building and testing Nextflow locally.
150
119
151
-
Note however the support for cloud services e.g. AWS, Seqera Platform, Wave, etc. still require the download
152
-
of the corresponding Nextflow plugins.
120
+
The standalone distribution will still download core and third-party plugins as needed at runtime.
Copy file name to clipboardExpand all lines: docs/sharing.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -115,6 +115,8 @@ For example, shebang definitions `#!/usr/bin/python` and `#!/usr/local/bin/pytho
115
115
```
116
116
:::
117
117
118
+
(lib-directory)=
119
+
118
120
#### The `lib` directory
119
121
120
122
The `lib` directory can be used to add utility code or external libraries without cluttering the pipeline scripts. The `lib` directory in the Nextflow project root is added to the classpath by default. Classes defined in the `lib` directory will be available in pipeline scripts. Functions defined outside of classes will not be available in pipeline scripts.
This page describes Nextflow release cadence, how to self-update Nextflow, and how select your version of Nextflow.
6
+
7
+
## Releases
8
+
9
+
A stable version of Nextflow is released in the 4th and 10th month of each year. A edge version of Nextflow is released on a monthly basis. The edge version can be used to access the latest updates and experimental features.
10
+
11
+
You can find an exhaustive list of releases and updates in the [Nextflow changelog](https://github.com/nextflow-io/nextflow/blob/master/changelog.txt).
12
+
13
+
## Self-update
14
+
15
+
To update to the latest stable release of Nextflow, run the `self-update` command:
16
+
17
+
```{code-block} bash
18
+
:class: copyable
19
+
nextflow self-update
20
+
```
21
+
22
+
To use the latest edge release, set `NXF_EDGE=1` when you self-update Nextflow:
23
+
24
+
```{code-block} bash
25
+
:class: copyable
26
+
NXF_EDGE=1 nextflow self-update
27
+
```
28
+
29
+
:::{warning}
30
+
Nextflow will update its executable during the self-update process. The update can fail if the Nextflow executable is in a directory with restricted permissions.
31
+
:::
32
+
33
+
## Version selection
34
+
35
+
The `NXF_VER` environment variable can be used to define which version of Nextflow to use. To switch to a specific version of Nextflow for a single run, set the `NXF_VER` environment variable in your execution command. For example:
36
+
37
+
```{code-block} bash
38
+
:class: copyable
39
+
NXF_VER=23.10.0 nextflow info
40
+
```
41
+
42
+
To set a specific version of Nextflow for a terminal session, export the `NXF_VER` environment variable. For example:
43
+
44
+
```{code-block} bash
45
+
:class: copyable
46
+
export NXF_VER=23.10.0
47
+
```
48
+
49
+
To set a specific version of Nextflow for your user profile, add the above `NXF_VER` export command to your shell configuration file, such as `~/.bashrc` or `~/.zshrc`, and restart your session.
50
+
51
+
:::{tip}
52
+
You can use `NXF_VER` to switch to an edge release. For example:
53
+
54
+
```{code-block} bash
55
+
:class: copyable
56
+
NXF_VER=24.06.0-edge nextflow info
57
+
```
58
+
:::
59
+
60
+
:::{warning}
61
+
Nextflow will update its executable during the self-update process. The update can fail if the Nextflow executable is in a directory with restricted permissions.
0 commit comments