diff --git a/up-l2/dispatchers/README.adoc b/up-l2/dispatchers/README.adoc index 73d10f0..4327119 100644 --- a/up-l2/dispatchers/README.adoc +++ b/up-l2/dispatchers/README.adoc @@ -30,11 +30,11 @@ SPDX-License-Identifier: Apache-2.0 ---- -Like IP packets, uProtocol messages (CE) have a source attribute (originator of the message) and sink attribute (where should the message be sent to). These attributes are used to route the CEs from one uE to the next if the destination is not the receiving uE. +Like IP packets, uProtocol messages (UMessage) have a source attribute (originator of the message) and sink attribute (where should the message be sent to). These attributes are used to route the CEs from one uE to the next if the destination is not the receiving uE. NOTE: Dispatcher and router shall be used For the remainder of this document we will use the term dispatcher and router interchangeably. -The header contains information for routing as well as metadata (of the data). One of the core principles of uProtocol is that the data portion of CE *MUST* be untouched by message routers, this is very similar to how most Internet standards work today. Only the source who generated the CE and the sink who will consume the CE needs to understand/analyze the payload of the CE. +The header contains information for routing as well as metadata (of the data). One of the core principles of uProtocol is that the data portion of UMessage *MUST* be untouched by message routers, this is very similar to how most Internet standards work today. Only the source who generated the UMessage and the sink who will consume the UMessage needs to understand/analyze the payload of the UMessage. To be able to forward/dispatch/route CEs through the network, we must define specific purpose built uEs to perform these tasks (ex. Ethernet switches, IP routers, etc...). Platform uEs that are responsible for event dispatching and implementing the communication layer are described in the sections below. We will elaborate on these specific uEs in the Platform uEs section below. @@ -47,7 +47,7 @@ To be able to forward/dispatch/route CEs through the network, we must define spe |Message bus that dispatches CEs between uEs over a common transport. It provides multicast and forwarding functionality (works like a network switch) |*uStreamer* -|Provides Device-2-Device CE routing either using the same or different transport protocols , i.e. when events need to move form one transport to the next it flows through the streamer (can be equated to an IP router) +|Provides Device-2-Device UMessage routing either using the same or different transport protocols , i.e. when events need to move form one transport to the next it flows through the streamer (can be equated to an IP router) |*Cloud Gateway* |A uE that sits at the edge of the cloud to connect non-cloud devices (ex. vehicles, phones, etc...) to the cloud @@ -64,16 +64,16 @@ In this section we will elaborate on the requirements of the platform Dispatcher NOTE: These communication layer requirements are still for point-2-point uE communication to and from a dispatcher -* *MUST* support At-least-once delivery policy, this means that the dispatcher will make every attempt to dispatch the CE to the intended Receiver +* *MUST* support At-least-once delivery policy, this means that the dispatcher will make every attempt to dispatch the UMessage to the intended Receiver ** *MUST* queue CEs not successfully acknowledged (transport level at-least-once delivery confirmation described above) - ** *MUST* attempt to retry transmission of the CE. Retry policy is specific to the dispatcher implementation - ** Dispatcher *MUST NOT* discard CEs unless either CE has expired (CE.ttl), or the egress queue is full. CEs that cannot be delivered are sent to a Dead Letter Office Topic + ** *MUST* attempt to retry transmission of the UMessage. Retry policy is specific to the dispatcher implementation + ** Dispatcher *MUST NOT* discard CEs unless either UMessage has expired (UMessage.ttl), or the egress queue is full. CEs that cannot be delivered are sent to a Dead Letter Office Topic -* *MAY* support additional CE delivery policies in general or per topic in the future -* *SHOULD* dispatch in order that it received the CE +* *MAY* support additional UMessage delivery policies in general or per topic in the future +* *SHOULD* dispatch in order that it received the UMessage * *MAY* batch CEs when delivering to the Receiver * CEs that cannot be delivered *MUST* be sent to the Dead Letter topic (DLT) - ** DLT *MUST* include at least the CE header, SHOULD contain the full CE + ** DLT *MUST* include at least the UMessage header, SHOULD contain the full UMessage ** DLT *MUST* include the reason for the failed delivery attempt using error codes defined in google.rpc.Code ** uEs MUST be able to subscribe to the DLT to be notified of message deliver failures @@ -93,10 +93,10 @@ When a dispatcher is unable to dispatch an event for a given reason (queue full, |*google.rpc.Code* |*Reason* |`*UNAVAILABLE*` -|The req.v1 has expired due to the downstream uE was unavailable (ex. uDevice was disconnected). uE that issued the req.v1 MAY retry with back-off +|The Request message has expired due to the downstream uE was unavailable (ex. uDevice was disconnected). uE that issued the request *MAY* retry with back-off |`*DEADLINE_EXCEEDED*` -|CE has timed out per the ttl attribute specifications defined in req.v1 event +|UMessage has timed out per the ttl attribute specifications defined in req.v1 event |`*PERMISSION_DENIED*` |source is not permitted to access sink @@ -108,7 +108,7 @@ When a dispatcher is unable to dispatch an event for a given reason (queue full, |The dispatcher ran out of resources (buffer full) |`*INVALID_ARGUMENT*` -|Invalid CE header attributes not covered above (ex. any mal-formatted attributes) +|Invalid UMessage header attributes not covered above (ex. any mal-formatted attributes) |`*UNKNOWN*` |An unknown (but not critical) error has occurred @@ -117,8 +117,3 @@ When a dispatcher is unable to dispatch an event for a given reason (queue full, |There is a serious error has occurred not described by error codes mentioned above !=== -<> figure below illustrates the sequence of messages for RPC flows and the role dispatchers play in error handling. - -.RPC Error Flow -[#rpc-error-flow] -image::rpc_flow.png[RPC Error Handling] \ No newline at end of file diff --git a/up-l2/dispatchers/legend.png b/up-l2/dispatchers/legend.png new file mode 100644 index 0000000..f3dc7ef Binary files /dev/null and b/up-l2/dispatchers/legend.png differ diff --git a/up-l2/dispatchers/legend.puml b/up-l2/dispatchers/legend.puml new file mode 100644 index 0000000..2ba10f6 --- /dev/null +++ b/up-l2/dispatchers/legend.puml @@ -0,0 +1,14 @@ +@startuml +'https://plantuml.com/sequence-diagram + +title Legend + +participant foo +participant bar + + +foo <-->bar: RpcClient & RpcServer APIs +foo <-[#green]->bar: UTransport & UListener APIs (uT) +foo <-[#blue]> bar: Communication Middleware + +@enduml \ No newline at end of file diff --git a/up-l2/dispatchers/rpc_flow.png b/up-l2/dispatchers/rpc_flow.png index fb78ee7..fbfcaa0 100644 Binary files a/up-l2/dispatchers/rpc_flow.png and b/up-l2/dispatchers/rpc_flow.png differ diff --git a/up-l2/dispatchers/rpc_flow.puml b/up-l2/dispatchers/rpc_flow.puml index 95aec0a..cbc8d31 100644 --- a/up-l2/dispatchers/rpc_flow.puml +++ b/up-l2/dispatchers/rpc_flow.puml @@ -1,59 +1,108 @@ @startuml 'https://plantuml.com/sequence-diagram + autonumber -actor uApp #red -collections dispatchers as UB -entity uService as US #blue +title Direct RPC Flow (not through Gateway/DPR) + +box uP-Foo (UAuthority="Device1") #white + actor uApp #red + boundary uPClientFoo as upc1 #black + boundary uPClientFoo as upc2 #black + entity uStreamer as uS1 #green + boundary uPClientBar as upc3 #black +end box + +box uP-Bar (UAuthority="Device2") #white + boundary uPClientBar as upc4 #black + entity uService #blue +end box -uApp --\ UB: req.v1 + +== Setup uTransport Listeners == +uS1 -[#green]>upc2: registerListener(\n\tUUri,\n\tUListener) note right - **req.v1:** - "id" : "cf8b1bcd-30bd-43be-a8d3-ad1cde652e10" - "source": "up://Device1/uApp/1/rpc.response" - "sink": "up://Device1/uService/1/rpc.Method - "specversion": "1.0" - "type": "req.v1" - "dataschema": //url of Method schema// - "data": //Protobuf serialized Request Message// + UUri = "//Device2/*" end note +upc2 -[#green]->uS1: UStatus -alt Delivery Error - UB -> UB: Create res.v1 - note right - **res.v1:** - "id" : "f8c48bcf-2b54-4d64-83df-ad1cde652e10" - "source": "up://Device1/dispatcher/1/rpc.Method - "sink": "up://Device1/uApp/1/rpc.response" - "specversion": "1.0" - "type": "res.v1" - "request_id": "cf8b1bcd-30bd-43be-a8d3-ad1cde652e10" - "commstatus": /* google.rpc.Code for the error */ - end note -else Delivery Success - UB --\ US: req.v1 - US -> US: Process req.v1 - note right - **res.v1:** - "id" : "f8c48bcf-2b54-4d64-83df-ad1cde652e10" - "source": "up://Device1/dispatcher/1/rpc.Method - "sink": "up://Device1/uApp/1/rpc.response" - "specversion": "1.0" - "type": "res.v1" - "request_id": "cf8b1bcd-30bd-43be-a8d3-ad1cde652e10" - "dataschema": //url of Method Response Message// - "data": // Protobuf serialized Response Message// - end note - US --/ UB: res.v1 -end alt -UB --/ uApp: res.v1 -uApp -> uApp: Process res.v1 +uS1 -[#green]>upc3: registerListener(\n\tUUri,\n\tUListener) note right - **Process res.v1:** - if (commstatus) - // Process L2 Error// - else - //handle returned L3 message// + UUri = "//Device1/*" +end note +upc3 -[#green]->uS1: UStatus +uService -> upc4: registerRpcListener(\n\tUUri,\n\tURpcListener) +note left + UUri = "/uService/rpc.myMethod" end note +upc4 --> uService: UStatus + +== Request == +uApp -> upc1 : invokeMethod(\n\tUUri, \n\tUPayload, \n\tCallOptions) +note right + **UUri:** + authority:{name: "Device2", ip: "192.168.1.100"} + entity: {name: "uService", version_major: 1, id: 99} + resource: {name: "rpc", instance: "myMethod", id: 1} + +end note +upc1 -> upc1: Build request +note right + **Request UMessage:** + source: { + \tauthority:{name: "Device1", ip: "192.168.1.101"} + \tentity: {name: "uApp", version_major: 1, id: 300} + \tresource: {name: "rpc", instance: "response", id: 0} + } + payload: { /* Request Payload */ } + attributes: { + \tid: /* UUID */, + \ttype: UMESSAGE_TYPE_REQUEST + \tpriority: UPRIORITY_CS4 + \tttl: 10000 + \tsink: { + \t\tauthority:{name: "Device2", ip: "192.168.1.100"} + \t\tentity: {name: "uService", version_major: 1, id: 99} + \t\tresource: {name: "rpc", instance: "myMethod", id: 1} + \t} + } +end note +upc1 -[#green]> upc1:send(request) +upc1 <-[#blue]> upc2: Foo +upc1 --> uApp: Future + +upc2 -[#green]> uS1: onReceive(request) +uS1 -> uS1: Queuing & routing +uS1 -[#green]> upc3:send(request) +upc3 <-[#blue]> upc4: Bar + +upc4 -[#green]> upc4: onReceive(request) +upc4 ->uService: onReceive(request) + + +== Response == +uService -> uService: Process req,\nbuild res +note left + **Response UMessage:** + source: //Device2/uService/rpc.myMethod"" } + ""payload: { /* Request Payload */ } + ""attributes: { + \tid: /* UUID */, + \ttype: UMESSAGE_TYPE_RESPONSE + \tpriority: UPRIORITY_CS4 + \tttl: 10000 + \tsink: //Device1/uApp/1/rpc.response + \treqid: /* Request attributes.id */ + } +end note + +uService -[#green]> upc4:send(response) +upc4 <-[#blue]> upc3: Bar +upc3 -[#green]> uS1: onReceive(response) +uS1 -[#green]>upc2:send(response) +upc2 <-[#blue]> upc1: Foo +upc2 -[#green]->uS1: UStatus +upc1 --\uApp: Future\n\t.complete() + @enduml \ No newline at end of file diff --git a/up-l2/dispatchers/streamer.adoc b/up-l2/dispatchers/streamer.adoc new file mode 100644 index 0000000..8b9e1d7 --- /dev/null +++ b/up-l2/dispatchers/streamer.adoc @@ -0,0 +1,72 @@ += uStreamer +:toc: +:sectnums: + +The key words "*MUST*", "*MUST NOT*", "*REQUIRED*", "*SHALL*", "*SHALL NOT*", "*SHOULD*", "*SHOULD NOT*", "*RECOMMENDED*", "*MAY*", and "*OPTIONAL*" in this document are to be interpreted as described in https://www.rfc-editor.org/info/bcp14[IETF BCP14 (RFC2119 & RFC8174)] + +---- +Copyright (c) 2023 General Motors GTO LLC + +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. + +SPDX-FileType: DOCUMENTATION +SPDX-FileCopyrightText: 2023 General Motors GTO LLC +SPDX-License-Identifier: Apache-2.0 +---- + +== Overview + +The streamer is a uProtocol dispatcher responsible to route messages to/from the local link:../../up-l1/README.adoc[uTransport] to one or more remote uTransports. Local uTransport refers to the transport implementation that the streamer is running on (the transport for the device that the streamer is deployed to). Remote UTransports are any non-local transports that messages need to be forwarded to. + +NOTE: The term _local transport_ could also be commonly referred as _downstream transport_ vs _remote transport_ referred as _upstream transport_, we use the term local vs remote as they are also commonly used for our addressing scheme in uProtocol. + +* locally addressed messages *MUST NOT* be routed to remote transports + +Streamer use the link:../../up-l1/README.adoc[uTransport APIs] to configure the flow of messages between the local and remote transports. + + +== Routing Policies +Streamer dispatches messages by declaring simple routing policies to either route or forward messages to/from a given transport to another using the UTransport API. + +The Transport boundaries match up with UUri link:../../basics/uri.adoc[UAuthority] to make the declaration of the policies simpler. + +NOTE: UDevices that have connected through the same transport do not require defining routing policies as the underlining transport takes care of the dispatching of messages. + +=== Routing Examples + +Below is an example of a simple routing configuration to connect a single local transport with a remote transport. for the purpose of this example, the local device will have UAuthority `uLocal` and the remote device will be have the UAuthority `URemote`. + +``` +yes one sec +``` + + +.Legend +image::legend.png[#legend] + +=== Building uStreamers using uTransport APIs +<> diagram below illustrates how the uTransport APIs are used to build a uStreamer. The uTransport APIs are used to send/receive UMessages over the underlining communication middleware. The uStreamer is responsible for routing the UMessages to the appropriate uE based on the UMessage header attributes. + +.Streamer Flows using uTransport +image:streamer_flow.png[#streamer_flow] + +=== RPC Flows +<> diagram below illustrates how the uPClient RpcClient & RpcServer interfaces are then connected to the uTransport such that a transport can send/receive the UMessages over the underlining communication middleware +.Rpc Flows +image:rpc_flow.png[#rpc_flow] + diff --git a/up-l2/dispatchers/streamer.drawio.svg b/up-l2/dispatchers/streamer.drawio.svg new file mode 100644 index 0000000..73a2990 --- /dev/null +++ b/up-l2/dispatchers/streamer.drawio.svg @@ -0,0 +1,49 @@ + + + + + + + +
+
+
+ uStreamer +
+
+
+ + + uStreamer + + + + + + + + + +
+
+
+ local uTransport +
+
+
+ + + local uTransp... + + + + + + + + + Text is not SVG - cannot display + + + + \ No newline at end of file diff --git a/up-l2/dispatchers/streamer_flow.png b/up-l2/dispatchers/streamer_flow.png new file mode 100644 index 0000000..4b3f8df Binary files /dev/null and b/up-l2/dispatchers/streamer_flow.png differ diff --git a/up-l2/dispatchers/streamer_flow.puml b/up-l2/dispatchers/streamer_flow.puml new file mode 100644 index 0000000..ae70a63 --- /dev/null +++ b/up-l2/dispatchers/streamer_flow.puml @@ -0,0 +1,86 @@ +@startuml +'https://plantuml.com/sequence-diagram + +skinparam BoxPadding 40 + +autonumber +title RPC Flow Through a Gateway + +box uP-Foo (UAuthority="Device1") #white + entity uStreamer as uS1 #green + boundary uPClientBaz as upc3 #black +end box + +box uP-Baz (Gateway) + boundary uPClientBaz as upc4 #black + entity DPR #green +end box + +box uP-Bar (UAuthority="Device2") #white + boundary uPClientBaz as upc5 #black + entity uStreamer as uS2 #green + boundary uPClientBar as upc6 #black +end box + +== Setup uTransport Listeners == +uS1 -[#green]>upc3: registerListener(\nUUri, UListener) +note right + UUri = "//Device2/*" +end note +upc3 -[#green]->uS1: UStatus + + +DPR -[#green]>upc4: registerListener(\nUUri, UListener) +note right + UUri = "//*" +end note +upc4 -[#green]->DPR: UStatus + + +uS2 -[#green]>upc5: registerListener(\nUUri, UListener) +note right + UUri = "//Device1*" +end note +upc5 -[#green]->uS2: UStatus +uS2 -[#green]>upc6: registerListener(\nUUri, UListener) +note right + UUri = "//Device2/*" +end note +upc6 -[#green]->uS2: UStatus + +== MSG1 tx (Device1 to Device 2) == + -[#blue]> upc3: Foo Protocol +upc3 -[#green]>uS1: OnReceive(msg1) +uS1 -[#green]> upc3: send(msg1) +upc3 <-[#blue]> upc4: Baz Protocol +upc3 -[#green]-> uS1: UStatus + +upc4 -[#green]> DPR: onReceive(msg1) +DPR ->DPR: Process &\n Route Request + +DPR -[#green]> upc4: send(msg1) +upc4 <-[#blue]> upc5: Baz Protocol +upc4 -[#green]-> DPR: UStatus + +upc5 -[#green]> uS2: onReceive(msg1) +uS2 -[#green]> upc6: send(msg1) +upc6 -[#blue]> : Bar Protocol + +== MSG2 rx (Device 2 to Device1) == +upc6 <[#blue]-: Bar Protocol +upc6 -[#green]> uS2: onReceive(msg2) +uS2 -[#green]>upc5: send(msg2) +upc5 <-[#blue]> upc4: Bax Protocol +upc5 -[#green]->uS2: UStatus + + +upc4 -[#green]> DPR: onReceive(msg2) +DPR ->DPR: Process &\n Route Response +upc5 <-[#blue]> upc4: Bax Protocol +DPR -[#green]>upc4: send(msg2) +upc4 <-[#blue]> upc3: Bax Protocol +upc4 -[#green]->DPR: UStatus + +upc3 -[#green]> uS1: onReceive(msg2) +<[#blue]- upc3: Foo Protocol +@enduml \ No newline at end of file