You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: R/Writing R Code that Runs in Parallel.md
+27Lines changed: 27 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -45,6 +45,33 @@ Inherently (or embarrassingly) parallel tasks are those that can be easily divid
45
45
1. Summarising data by groups (e.g., calculating the mean or sum for each group) can be done independently for each group.
46
46
2. Running the same computation with different sets of parameters; each parameter set can be processed independently.
47
47
48
+
## Multithreading
49
+
50
+
### What is multithreading?
51
+
52
+
Multithreading is a method of executing code in parallel, rather than sequentially, making better use of computer resources, and potentially reducing the amount of time it takes to execute the code. The tasks carried out using multithreading do not necessarily need to be related to one another and do not need to wait for each to complete.
53
+
54
+
### How does multithreading relate to R?
55
+
56
+
As stated previously, R is a single-threaded language by default, meaning it processes tasks sequentially. It is not possible to implement multithreading in R without calling on additional R packages, such as the [`{data.table}`](https://rdatatable.gitlab.io/data.table/) package in which many common operations will execute using multiple CPU threads.
57
+
58
+
### Multithreading inside a Kubernetes container in Azure
59
+
60
+
Your R session on Posit Workbench runs inside a Kubernetes container. That container is allocated a certain amount of CPU resource. This resource is provided by the Azure Kubernetes Service (AKS), where 1 CPU corresponds to 1 vCPU (virtual CPU). 1 vCPU is the equivalent of a single hyper-thread on a physical CPU core.
61
+
62
+
If you attempt to run multithreaded code in a session with just 1 CPU in Posit Workbench, the multiple threads will be executed by taking turns on the single thread. This will give the illusion of multithreading, but all that is happening is that each thread is using a slice of CPU time, running sequentially.
63
+
64
+
In order to run multithreaded code in Posit Workbench, you must open a session with more than 1 CPU. To demonstrate this, below are the results of running a computationally expensive operation on a large `{data.table}` of 600 million rows in Posit Workbench sessions with 1, 2, 4 and 8 CPUs, and using 1, 2, 4 and 8 threads:
The results are the execution times in seconds for each combination of number of CPUs and threads. As you can see, running code with multiple threads on 1 vCPU takes longer than running the same code in a single thread of execution. A reduction in execution time is only seen if the number of CPUs is increased, and the most optimal combination is where the number of CPUs matches the number of threads of execution.
74
+
48
75
## The `{multidplyr}` R package
49
76
50
77
The `{multidplyr}` R package is a backend for `{dplyr}` that facilitates parallel processing by partitioning data frames across multiple cores. The package is part of the [Tidyverse](https://www.tidyverse.org/).
0 commit comments