Skip to content

Commit 70080c1

Browse files
Create updates section (#5687)
Signed-off-by: Christopher Hakkaart <[email protected]> Signed-off-by: Ben Sherman <[email protected]> Co-authored-by: Ben Sherman <[email protected]>
1 parent e79b9ef commit 70080c1

File tree

9 files changed

+693
-622
lines changed

9 files changed

+693
-622
lines changed

docs/developer/plugins.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
(plugins-dev-page)=
12

23
# Plugins
34

docs/dsl1.md

Lines changed: 47 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,12 @@
11
(dsl1-page)=
22

3-
# Migrating from DSL 1
3+
# Migrating from DSL1
44

5-
In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed, and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 in order to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2, and how to migrate to DSL2.
6-
7-
In Nextflow versions prior to `22.03.0-edge`, you must enable DSL2 explicitly in order to use it. You can either set the feature flag in your pipeline script:
8-
9-
```nextflow
10-
nextflow.enable.dsl=2
11-
```
12-
13-
Or set the environment variable where you launch Nextflow:
14-
15-
```bash
16-
export NXF_DEFAULT_DSL=2
17-
```
5+
In Nextflow version `22.03.0-edge`, DSL2 became the default DSL version. In version `22.12.0-edge`, DSL1 support was removed and the Nextflow documentation was updated to use DSL2 by default. Users who are still using DSL1 should migrate their pipelines to DSL2 to use the latest versions of Nextflow. This page describes the differences between DSL1 and DSL2 and how to migrate to DSL2.
186

197
## Processes and workflows
208

21-
In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. Here is the {ref}`your-first-script` example written in DSL1:
9+
In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels using `from` and `into`. You can see a basic Nextflow script written in DSL1 here:
2210

2311
```nextflow
2412
nextflow.enable.dsl=1
@@ -53,44 +41,69 @@ result.view { it.trim() }
5341

5442
To migrate this code to DSL2, you need to move all of your channel logic throughout the script into a `workflow` definition. Additionally, you must call each process explicitly, passing any input channels as arguments (instead of `from ...`) and receiving any output channels as return values (instead of `into ...`).
5543

56-
Refer to the {ref}`workflow-page` page to learn how to define a workflow. The DSL2 version of the above script is duplicated here for your convenience:
44+
See {ref}`workflow-page` page to learn how to define a workflow.
5745

58-
```{literalinclude} snippets/your-first-script.nf
59-
:language: nextflow
60-
```
46+
You can see the DSL1 Nextflow script from above written in DSL2 here:
6147

62-
## Channel forking
48+
```nextflow
49+
params.str = 'Hello world!'
50+
51+
process splitLetters {
52+
output:
53+
path 'chunk_*'
54+
55+
script:
56+
"""
57+
printf '${params.str}' | split -b 6 - chunk_
58+
"""
59+
}
6360
64-
In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator.
61+
process convertToUpper {
62+
input:
63+
path x
6564
66-
In DSL2, channels are automatically forked when connecting two or more consumers.
65+
output:
66+
stdout
6767
68-
For example, this would not work in DSL1 but is not a problem in DSL2:
68+
script:
69+
"""
70+
cat $x | tr '[a-z]' '[A-Z]'
71+
"""
72+
}
73+
74+
workflow {
75+
splitLetters | flatten | convertToUpper | view { v -> v.trim() }
76+
}
77+
```
78+
79+
## Channel forking
80+
81+
In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must be forked using the `into` operator. In DSL2, channels are automatically forked when connecting two or more consumers. For example:
6982

7083
```nextflow
7184
Channel
72-
.from('Hello','Hola','Ciao')
73-
.set{ cheers }
85+
.of('Hello','Hola','Ciao')
86+
.set { cheers }
7487
7588
cheers
76-
.map{ it.toUpperCase() }
89+
.map { v -> v.toUpperCase() }
7790
.view()
7891
7992
cheers
80-
.map{ it.reverse() }
93+
.map { v -> v.reverse() }
8194
.view()
8295
```
8396

84-
Similarly, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.
97+
Similarly, in DSL2, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts much easier to read and write.
8598

8699
## Modules
87100

88-
In DSL1, the entire Nextflow pipeline must be defined in a single file (e.g. `main.nf`). This restriction becomes quite cumbersome as a pipeline becomes larger, and it hinders the sharing and reuse of pipeline components.
101+
In DSL1, the entire Nextflow pipeline must be defined in a single file. For example, `main.nf`. This restriction becomes cumbersome as a pipeline grows and hinders the sharing and reuse of pipeline components.
89102

90-
DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2 by any means, modules can help you organize a large pipeline into multiple smaller files, and take advantage of modules created by others. Check out the {ref}`module-page` to get started.
103+
DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are they mandatory in DSL2, modules can help you organize a large pipeline into multiple smaller files and take advantage of modules created by others. See {ref}`module-page` to learn more about modules.
91104

92105
:::{note}
93-
DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be split into modules to avoid this limit.
106+
DSL2 scripts cannot exceed 64 KB in size. Split large DSL1 scripts into modules to avoid this limit.
94107
:::
95108

96109
## Deprecations
@@ -103,14 +116,13 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli
103116

104117
- The `mode flatten` option for process outputs is no longer available. Use the {ref}`operator-flatten` operator on the corresponding output channel instead.
105118

106-
- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier.
107-
108-
For example:
119+
- Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit `val` or `path` qualifier. For example:
109120

110121
```nextflow
111122
process foo {
112123
input:
113124
tuple X, 'some-file.sam'
125+
114126
output:
115127
tuple X, 'some-file.bam'
116128
@@ -127,6 +139,7 @@ DSL2 scripts cannot exceed 64 KB in size. Large DSL1 scripts may need to be spli
127139
process foo {
128140
input:
129141
tuple val(X), path('some-file.sam')
142+
130143
output:
131144
tuple val(X), path('some-file.bam')
132145

docs/index.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,6 @@ notifications
8080
secrets
8181
sharing
8282
vscode
83-
dsl1
8483
```
8584

8685
```{toctree}
@@ -124,6 +123,16 @@ reference/channel
124123
reference/operator
125124
```
126125

126+
```{toctree}
127+
:hidden:
128+
:caption: Updates
129+
:maxdepth: 1
130+
131+
updating-nextflow
132+
updating-syntax
133+
dsl1
134+
```
135+
127136
```{toctree}
128137
:hidden:
129138
:caption: Contributing

docs/install.md

Lines changed: 14 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,17 @@
22

33
# Installation
44

5+
Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). This page describes how to install Nextflow.
6+
7+
:::{note}
8+
New versions of Nextflow are released regularly. See {ref}`updating-nextflow-page` for more information about Nextflow release cadence, how to update Nextflow, and how select your version of Nextflow.
9+
:::
10+
511
(install-requirements)=
612

713
## Requirements
814

9-
Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). It requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. You can see which version you have using the following command:
15+
Nextflow requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. To see which version of Java you have, run the following command:
1016

1117
```{code-block} bash
1218
:class: copyable
@@ -17,7 +23,7 @@ java -version
1723
Support for Java versions prior to 17 was dropped.
1824
:::
1925

20-
If you don't have a compatible version of Java installed in your computer, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest LTS version of Temurin. See [this website](https://whichjdk.com/) for more information.
26+
If you don't have a compatible version of Java installed, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest Long-Term-Support (LTS) version of Temurin. See [Which version of JDK should I use?](https://whichjdk.com/) for more information about different versions of Java.
2127

2228
To install Java with SDKMAN:
2329

@@ -50,15 +56,15 @@ To install Java with SDKMAN:
5056
5157
Nextflow is distributed as a self-installing package, in order to make the installation process as simple as possible:
5258
53-
1. Install Nextflow:
59+
To install Nextflow:
60+
61+
1. Download Nextflow:
5462
5563
```{code-block} bash
5664
:class: copyable
5765
curl -s https://get.nextflow.io | bash
5866
```
5967
60-
This will create the `nextflow` executable in the current directory.
61-
6268
:::{tip}
6369
Set `export CAPSULE_LOG=none` to make the installation logs less verbose.
6470
:::
@@ -93,27 +99,12 @@ Nextflow is distributed as a self-installing package, in order to make the insta
9399
nextflow info
94100
```
95101
96-
## Updates
97-
98-
With Nextflow installed in your environment, you can update to the latest version using the following command:
99-
100-
```{code-block} bash
101-
:class: copyable
102-
nextflow self-update
103-
```
104-
105-
You can also temporarily switch to a specific version of Nextflow with the `NXF_VER` environment variable. For example:
106-
107-
```{code-block} bash
108-
:class: copyable
109-
NXF_VER=23.10.0 nextflow info
110-
```
111-
112102
## Seqera Platform
113103
114104
You can launch workflows directly from [Seqera Platform](https://seqera.io/platform/) without installing Nextflow locally.
115105
116106
Launching from Seqera Platform provides you with:
107+
117108
- User-friendly launch interfaces.
118109
- Automated cloud infrastructure creation.
119110
- Organizational user management.
@@ -122,34 +113,11 @@ Launching from Seqera Platform provides you with:
122113
Seqera Cloud Basic is free for small teams. Researchers at qualifying academic institutions can apply for free access to Seqera Cloud Pro.
123114
See the [Seqera Platform documentation](https://docs.seqera.io/platform) for set-up information and tutorials to get started.
124115
125-
## Stable and edge releases
126-
127-
A *stable* version of Nextflow is released every six months, in the 4th and 10th month of each year.
128-
129-
Additionally, an *edge* version is released on a monthly basis. The edge releases can be used to access the latest updates and experimental features.
130-
131-
To use the latest edge release, set `NXF_EDGE=1` when updating:
132-
133-
```{code-block} bash
134-
:class: copyable
135-
NXF_EDGE=1 nextflow self-update
136-
```
137-
138-
You can also use `NXF_VER` to temporarily switch to any edge release. For example:
139-
140-
```{code-block} bash
141-
:class: copyable
142-
NXF_VER=24.06.0-edge nextflow info
143-
```
144-
145116
## Standalone distribution
146117
147-
The Nextflow standalone distribution (i.e. the `dist` distribution) consists of self-contained `nextflow` executable file
148-
that includes all the application dependencies for core functionalities, and it can run without downloading third parties
149-
libraries. This distribution is mainly useful for offline environments.
118+
The Nextflow standalone distribution (i.e. the `dist` release) is a self-contained `nextflow` executable that can run without needing to download core dependencies at runtime. This distribution is useful for offline environments, as well as building and testing Nextflow locally.
150119
151-
Note however the support for cloud services e.g. AWS, Seqera Platform, Wave, etc. still require the download
152-
of the corresponding Nextflow plugins.
120+
The standalone distribution will still download core and third-party plugins as needed at runtime.
153121
154122
To use the standalone distribution:
155123

docs/reference/syntax.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@ A Nextflow script may contain the following top-level declarations:
2727

2828
- Shebang
2929
- Feature flags
30-
- Includes
31-
- Parameter definitions
30+
- Include declarations
31+
- Parameter declarations
3232
- Workflow definitions
3333
- Process definitions
3434
- Function definitions

docs/sharing.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@ For example, shebang definitions `#!/usr/bin/python` and `#!/usr/local/bin/pytho
115115
```
116116
:::
117117

118+
(lib-directory)=
119+
118120
#### The `lib` directory
119121

120122
The `lib` directory can be used to add utility code or external libraries without cluttering the pipeline scripts. The `lib` directory in the Nextflow project root is added to the classpath by default. Classes defined in the `lib` directory will be available in pipeline scripts. Functions defined outside of classes will not be available in pipeline scripts.

docs/updating-nextflow.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
(updating-nextflow-page)=
2+
3+
# Updating Nextflow
4+
5+
This page describes Nextflow release cadence, how to self-update Nextflow, and how select your version of Nextflow.
6+
7+
## Releases
8+
9+
A stable version of Nextflow is released in the 4th and 10th month of each year. A edge version of Nextflow is released on a monthly basis. The edge version can be used to access the latest updates and experimental features.
10+
11+
You can find an exhaustive list of releases and updates in the [Nextflow changelog](https://github.com/nextflow-io/nextflow/blob/master/changelog.txt).
12+
13+
## Self-update
14+
15+
To update to the latest stable release of Nextflow, run the `self-update` command:
16+
17+
```{code-block} bash
18+
:class: copyable
19+
nextflow self-update
20+
```
21+
22+
To use the latest edge release, set `NXF_EDGE=1` when you self-update Nextflow:
23+
24+
```{code-block} bash
25+
:class: copyable
26+
NXF_EDGE=1 nextflow self-update
27+
```
28+
29+
:::{warning}
30+
Nextflow will update its executable during the self-update process. The update can fail if the Nextflow executable is in a directory with restricted permissions.
31+
:::
32+
33+
## Version selection
34+
35+
The `NXF_VER` environment variable can be used to define which version of Nextflow to use. To switch to a specific version of Nextflow for a single run, set the `NXF_VER` environment variable in your execution command. For example:
36+
37+
```{code-block} bash
38+
:class: copyable
39+
NXF_VER=23.10.0 nextflow info
40+
```
41+
42+
To set a specific version of Nextflow for a terminal session, export the `NXF_VER` environment variable. For example:
43+
44+
```{code-block} bash
45+
:class: copyable
46+
export NXF_VER=23.10.0
47+
```
48+
49+
To set a specific version of Nextflow for your user profile, add the above `NXF_VER` export command to your shell configuration file, such as `~/.bashrc` or `~/.zshrc`, and restart your session.
50+
51+
:::{tip}
52+
You can use `NXF_VER` to switch to an edge release. For example:
53+
54+
```{code-block} bash
55+
:class: copyable
56+
NXF_VER=24.06.0-edge nextflow info
57+
```
58+
:::
59+
60+
:::{warning}
61+
Nextflow will update its executable during the self-update process. The update can fail if the Nextflow executable is in a directory with restricted permissions.
62+
:::

0 commit comments

Comments
 (0)