Skip to content

Commit 0a74b03

Browse files
Draft commit
1 parent 89e1c3c commit 0a74b03

1 file changed

Lines changed: 27 additions & 0 deletions

File tree

R/Writing R Code that Runs in Parallel.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,33 @@ Inherently (or embarrassingly) parallel tasks are those that can be easily divid
4545
1. Summarising data by groups (e.g., calculating the mean or sum for each group) can be done independently for each group.
4646
2. Running the same computation with different sets of parameters; each parameter set can be processed independently.
4747

48+
## Multithreading
49+
50+
### What is multithreading?
51+
52+
Multithreading is a method of executing code in parallel, rather than sequentially, making better use of computer resources, and potentially reducing the amount of time it takes to execute the code. The tasks carried out using multithreading do not necessarily need to be related to one another and do not need to wait for each to complete.
53+
54+
### How does multithreading relate to R?
55+
56+
As stated previously, R is a single-threaded language by default, meaning it processes tasks sequentially. It is not possible to implement multithreading in R without calling on additional R packages, such as the [`{data.table}`](https://rdatatable.gitlab.io/data.table/) package in which many common operations will execute using multiple CPU threads.
57+
58+
### Multithreading inside a Kubernetes container in Azure
59+
60+
Your R session on Posit Workbench runs inside a Kubernetes container. That container is allocated a certain amount of CPU resource. This resource is provided by the Azure Kubernetes Service (AKS), where 1 CPU corresponds to 1 vCPU (virtual CPU). 1 vCPU is the equivalent of a single hyper-thread on a physical CPU core.
61+
62+
If you attempt to run multithreaded code in a session with just 1 CPU in Posit Workbench, the multiple threads will be executed by taking turns on the single thread. This will give the illusion of multithreading, but all that is happening is that each thread is using a slice of CPU time, running sequentially.
63+
64+
In order to run multithreaded code in Posit Workbench, you must open a session with more than 1 CPU. To demonstrate this, below are the results of running a computationally expensive operation on a large `{data.table}` of 600 million rows in Posit Workbench sessions with 1, 2, 4 and 8 CPUs, and using 1, 2, 4 and 8 threads:
65+
66+
| Threads | 1 vCPU | 2 vCPUs | 4 vCPUs | 8 vCPUs |
67+
|---------|--------|---------|---------|---------|
68+
| 1 | 72.285 | 72.612 | 67.869 | 67.244 |
69+
| 2 | 75.373 | 48.828 | 44.957 | 45.019 |
70+
| 4 | 87.979 | 52.276 | 34.109 | 34.641 |
71+
| 8 | 100.207| 60.684 | 37.004 | 29.259 |
72+
73+
The results are the execution times in seconds for each combination of number of CPUs and threads. As you can see, running code with multiple threads on 1 vCPU takes longer than running the same code in a single thread of execution. A reduction in execution time is only seen if the number of CPUs is increased, and the most optimal combination is where the number of CPUs matches the number of threads of execution.
74+
4875
## The `{multidplyr}` R package
4976

5077
The `{multidplyr}` R package is a backend for `{dplyr}` that facilitates parallel processing by partitioning data frames across multiple cores. The package is part of the [Tidyverse](https://www.tidyverse.org/).

0 commit comments

Comments
 (0)