You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: docs/source/basics/cpu_only_mode.md
+18-18
Original file line number
Diff line number
Diff line change
@@ -16,13 +16,13 @@ limitations under the License.
16
16
-->
17
17
18
18
# Morpheus CPU-Only Mode
19
-
By default Morpheus is designed to take advantage of the GPU for accelerated processing. However, there are cases where it may be necessary to run Morpheus on a system without access to a GPU. To address this need, Morpheus provides a CPUonly execution mode. Many stages within Morpheus require a GPU to run while others can operate in both GPU and CPU execution mode. Attempting to add a GPUonly stage to a pipeline that is configured to operate in CPUonly mode will result in an error.
19
+
By default, Morpheus is designed to take advantage of the GPU for accelerated processing. However, there are cases where it may be necessary to run Morpheus on a system without access to a GPU. To address this need, Morpheus provides a CPU-only execution mode. Many stages within Morpheus require a GPU to run, while others can operate in both GPU and CPU execution mode. Attempting to add a GPU-only stage to a pipeline that is configured to operate in CPU-only mode will result in an error.
20
20
21
21
## Execution Modes
22
-
By default Morpheus will run in GPU execution mode. Users have the choice of specifying the execution mode with either the Python API or from the command line.
22
+
By default, Morpheus will run in GPU execution mode. Users have the choice of specifying the execution mode with either the Python API or from the command line.
23
23
24
24
### Python API
25
-
Execution modes are defined in the `morpheus.config.ExecutionMode` enumeration, which is then specified in the `execution_mode` attribute of the `morpheus.config.Config` object. The following example demonstrates how to set the execution mode of a pipeline to CPUonly:
25
+
Execution modes are defined in the `morpheus.config.ExecutionMode` enumeration, which is then specified in the `execution_mode` attribute of the `morpheus.config.Config` object. The following example demonstrates how to set the execution mode of a pipeline to CPU-only:
26
26
27
27
```python
28
28
from morpheus.config import Config
@@ -47,12 +47,12 @@ morpheus run --use_cpu_only pipeline-other --help
47
47
```
48
48
49
49
#### Example
50
-
The following is a simple command line example of a pipeline that can execute in CPUonly mode. To begin ensure that you have fetched the examples dataset by running the following command from the root of the Morpheus repository:
50
+
The following is a simple command line example of a pipeline that can execute in CPU-only mode. To begin, ensure that you have fetched the examples dataset by running the following command from the root of the Morpheus repository:
51
51
```bash
52
52
./scripts/fetch_data.py fetch examples
53
53
```
54
54
55
-
Then to run the pipeline run the following command:
55
+
Then, run the following command to run the pipeline:
56
56
```bash
57
57
morpheus --log_level=INFO \
58
58
run --use_cpu_only pipeline-other \
@@ -64,18 +64,18 @@ morpheus --log_level=INFO \
64
64
```
65
65
66
66
## Designing Stages for CPU Execution
67
-
It is up to the author of each stage to decide which execution modes are supported. Options are: CPU, GPU or both. As mentioned previously the default execution mode is GPU, authors of stages which require a GPU do not need to make any changes to their stage definitions.
67
+
It is up to the author of each stage to decide which execution modes are supported. Options are: CPU, GPU, or both. As mentioned previously, the default execution mode is GPU; authors of stages which require a GPU do not need to make any changes to their stage definitions.
68
68
69
69
### DataFrames and Tensors
70
-
With the selection of the execution mode implies selection of DataFrame and tensor types. In GPU mode Morpheus will use [cuDF](https://docs.rapids.ai/api/cudf/stable/) DataFrames and tensors are represented as [CuPy](https://cupy.dev/)`ndarray` objects. In CPU mode Morpheus will use [pandas](https://pandas.pydata.org/) DataFrames and [NumPy](https://numpy.org/)`ndarray` objects.
70
+
The selection of the execution mode implies selection of DataFrame and tensor types. In GPU mode, Morpheus will use [cuDF](https://docs.rapids.ai/api/cudf/stable/) DataFrames and tensors are represented as [CuPy](https://cupy.dev/)`ndarray` objects. In CPU mode, Morpheus will use [pandas](https://pandas.pydata.org/) DataFrames and [NumPy](https://numpy.org/)`ndarray` objects.
### Stages defined with `@stage` and `@source`decorators
78
-
Both the `@stage` and `@source` decorators have an optional `execution_modes` parameter that accepts a tuple of `morpheus.config.ExecutionMode` values which is used to specify the supported executions mode of the stage.
77
+
### Stages Defined with `@stage` and `@source`Decorators
78
+
Both the `@stage` and `@source` decorators have an optional `execution_modes` parameter that accepts a tuple of `morpheus.config.ExecutionMode` values, which is used to specify the supported execution mode of the stage.
79
79
80
80
#### CPU-only Source & Stage Examples
81
81
```python
@@ -115,10 +115,10 @@ if __name__ == "__main__":
115
115
main()
116
116
```
117
117
118
-
#### CPU & GPU Source & Stage Examples
119
-
Supporting both CPU and GPU execution modes requires writing code that can handle both types of DataFrames and `ndarray` objects. In many cases code designed to work with pandas will work with cuDF, and code designed to work with NumPy will work with CuPy without requiring any changes to the code. In some cases however, the API may differ slightly and there is a need to know the payload type, care must be taken not to directly import `cudf` or any other package requiring a GPU when running in CPU mode on a system without a GPU. Morpheus provides some helper methods to assist with this in the {py:mod}`~morpheus.utils.type_utils` module, such as {py:func}`~morpheus.utils.type_utils.is_cudf_type`, {py:func}`~morpheus.utils.type_utils.get_df_class`, and {py:func}`~morpheus.utils.type_utils.get_array_pkg`.
118
+
#### CPU and GPU Source and Stage Examples
119
+
Supporting both CPU and GPU execution modes requires writing code that can handle both types of DataFrames and `ndarray` objects. In many cases, code designed to work with pandas will work with cuDF, and code designed to work with NumPy will work with CuPy, without requiring any changes to the code. However, in some cases, the API may differ slightly and there is a need to know the payload type. Care must be taken not to directly import `cudf` or any other package requiring a GPU when running in CPU mode on a system without a GPU. Morpheus provides some helper methods to assist with this in the {py:mod}`~morpheus.utils.type_utils` module, such as {py:func}`~morpheus.utils.type_utils.is_cudf_type`, {py:func}`~morpheus.utils.type_utils.get_df_class`, and {py:func}`~morpheus.utils.type_utils.get_array_pkg`.
120
120
121
-
With a few simple modifications the previous example now supports both CPU and GPU execution modes. The `get_df_class` function is used to determine the DataFrame type to use, and we added a command line flag to switch between the two execution modes.
121
+
With a few simple modifications, the previous example now supports both CPU and GPU execution modes. The `get_df_class` function is used to determine the DataFrame type to use, and we added a command line flag to switch between the two execution modes.
122
122
123
123
```python
124
124
import logging
@@ -177,15 +177,15 @@ if __name__ == "__main__":
177
177
main()
178
178
```
179
179
180
-
### Source & Stages Classes
181
-
Similar to the `@source` and `@stage` decorators, classbased sources and stages can also be defined to advertise which execution modes they support. The base class for all source and stage classes `StageBase` defines a `supported_execution_modes` method for this purpose which can be overridden in a derived class. The method in the base class is defined as:
180
+
### Source and Stage Classes
181
+
Similar to the `@source` and `@stage` decorators, class-based sources and stages can also be defined to advertise which execution modes they support. The base class for all source and stage classes,`StageBase`, defines a `supported_execution_modes` method for this purpose, which can be overridden in a derived class. The method in the base class is defined as:
Stage authors are free to inspect constructor arguments of the stage to determine which execution modes are supported. However for many stages the supported execution modes do not change based upon the constructor arguments. In these cases the {py:class}`~morpheus.pipeline.execution_mode_mixins.GpuAndCpuMixin` and {py:class}`~morpheus.pipeline.execution_mode_mixins.CpuOnlyMixin` mixins can be used to simplify the implementation.
188
+
Stage authors are free to inspect constructor arguments of the stage to determine which execution modes are supported. However, for many stages the supported execution modes do not change based upon the constructor arguments. In these cases the {py:class}`~morpheus.pipeline.execution_mode_mixins.GpuAndCpuMixin` and {py:class}`~morpheus.pipeline.execution_mode_mixins.CpuOnlyMixin` mixins can be used to simplify the implementation.
189
189
190
190
Example class definition:
191
191
```python
@@ -201,13 +201,13 @@ class PassThruStage(PassThruTypeMixin, GpuAndCpuMixin, SinglePortStage):
201
201
```
202
202
203
203
#### GpuAndCpuMixin
204
-
In the previous decorators example we discussed utilizing various helper methods available in the {py:mod}`~morpheus.utils.type_utils` module to assist in writing code which is able to operate in both CPU and GPU execution modes. To simplify this further the `GpuAndCpuMixin` mixin adds these helper methods to the class. At time of writing they are:
204
+
In the previous decorators example, we discussed utilizing various helper methods available in the {py:mod}`~morpheus.utils.type_utils` module to assist in writing code that is able to operate in both CPU and GPU execution modes. To simplify this further, the `GpuAndCpuMixin` mixin adds these helper methods to the class. At the time of this writing, they are:
205
205
206
206
-`df_type_str` - Returns either `"cudf"` or `"pandas"`.
207
207
-`get_df_pkg` - Returns either the `cudf` or `pandas` module.
208
208
-`get_df_class` - Returns either the `cudf.DataFrame` or `pandas.DataFrame` class.
209
209
210
-
### Stages with C++ implementations
211
-
C++ stages have the ability to interact with cuDF DataFrames via the [libcudf](https://docs.rapids.ai/api/libcudf/stable/) library, however no such C++ library exists for pandas DataFrames. As a result, any stages which contain both a Python and a C++ implementation, the Python implementation will be used in CPU mode, and the C++ implementation will be used in GPU mode. For these stages, the Python implementation is then free to assume DataFrames are of type `pandas.DataFrame` and tensors are of type `numpy.ndarray`.
210
+
### Stages with C++ Implementations
211
+
C++ stages have the ability to interact with cuDF DataFrames via the [libcudf](https://docs.rapids.ai/api/libcudf/stable/) library; however, no such C++ library exists for pandas DataFrames. As a result, any stages which contain both a Python and a C++ implementation, the Python implementation will be used in CPU mode, and the C++ implementation will be used in GPU mode. For these stages, the Python implementation is then free to assume DataFrames are of type `pandas.DataFrame` and tensors are of type `numpy.ndarray`.
212
212
213
213
A stage which contains only a C++ implementation will not be able to run in CPU mode.
0 commit comments