You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: hadoop-hdfs-project/hadoop-hdfs-rbf/src/site/markdown/HDFSRouterFederation.md
+33-1
Original file line number
Diff line number
Diff line change
@@ -557,4 +557,36 @@ Enable Router to rename across namespaces. Currently it is implemented based on
557
557
| dfs.federation.router.federation.rename.bandwidth || Specify bandwidth per map in MB.|
558
558
| dfs.federation.router.federation.rename.delay | 1000 | Specify the delayed duration(millie seconds) when the job needs to retry.|
559
559
| dfs.federation.router.federation.rename.diff | 0 | Specify the threshold of the diff entries that used in incremental copy stage.|
560
-
| dfs.federation.router.federation.rename.trash | trash | This options has 3 values: trash (move the source path to trash), delete (delete the source path directly) and skip (skip both trash and deletion).|
560
+
| dfs.federation.router.federation.rename.trash | trash | This options has 3 values: trash (move the source path to trash), delete (delete the source path directly) and skip (skip both trash and deletion).|
561
+
562
+
Asynchronous Router RPC
563
+
-------
564
+
Asynchronous router RPC is designed to address the performance bottlenecks of synchronous router RPC in scenarios with high concurrency and multiple named services.
565
+
By introducing an asynchronous processing mechanism, it optimizes the request-handling process, enhances the system's concurrency capacity and resource utilization efficiency.
566
+
567
+
See the Apache JIRA ticket [HDFS-17531](https://issues.apache.org/jira/browse/HDFS-17531) for more information on this feature.
568
+
569
+
### Asynchronous router RPC threads
570
+
571
+
-**Handler**: Retrieves ```RpcCall``` from the ```CallQueue``` for preliminary processing. In case of exceptions (such as the non-existence of the mount point),
572
+
it directly places the response into the response queue. Otherwise, it forwards the ```RpcCall``` to the **Async-Handler**.
573
+
-**Async-Handler**: Puts the ```RpcCall``` into the ```connection.calls``` of the connection thread and returns immediately without blocking and waiting.
574
+
-**Async-Responder**: Is responsible for processing the response received by the connection thread. If the ```RpcCall``` needs to be retried (such as the downstream service returns a ```StandbyException```),
575
+
it re-adds the ```RpcCall``` to the ```connection.calls```; otherwise, it puts the response into the ```ResponseQueue```.
576
+
-**Responder**: Retrieves the response from the ```ResponseQueue``` and returns it to the client.
577
+
578
+
### Advantages of the Asynchronous Router Rpc
579
+
580
+
-**High Processing Performance**: Benefiting from the asynchronous RPC (Remote Procedure Call) processing mechanism, the asynchronous router is capable of handling a large number of requests simultaneously.
581
+
This not only significantly enhances the system's concurrent processing capacity but also optimizes the overall throughput. This mechanism enables the system to respond rapidly to high - traffic requests and maintain efficient operation even under high - load conditions.
582
+
-**High Resource Utilization**: The asynchronous design effectively reduces thread blocking and frequent thread switching. As a result, it minimizes the resource waste associated with threads, thereby improving the overall efficiency of the system and reducing CPU idle time.
583
+
-**Isolation**: Different name-services employ distinct asynchronous processor thread pools. This architecture achieves isolation among name-services. If a particular name-service experiences a performance degradation,
584
+
it will not impact the processing capabilities of other name-services, ensuring the stability and reliability of the entire system.
585
+
586
+
### Asynchronous Router Rpc configuration
587
+
| Property | Default | Description|
588
+
|:---- |:---- |:---- |
589
+
| dfs.federation.router.async.rpc.enable | false | If true, router will process the ```RpcCall``` asynchronously. |
590
+
| dfs.federation.router.async.rpc.ns.handler.count || The number of async-handlers per nameservice, separated by commas, internally separated by colons. The identifier of nameservice is in dfs.nameservices configuration entry. Such as: ns1:count1,ns2:count2,ns3:count3. |
591
+
| dfs.federation.router.async.rpc.handler.count | 10 | For those nameservices not in dfs.federation.router.async.rpc.ns.handler.count configuration entry, use this value as the async-handler counts. |
592
+
| dfs.federation.router.async.rpc.responder.count | 10 | The thread counts of async-responders. |
0 commit comments