Publish with RTCRtpTransceivers. (#498)

* Add a property to ConferenceClient to get PeerConnection.

* Publish a stream with RTPTransceivers.

* Allow publish with transceivers.

* Add examples to design doc.

* Update docs/design/owt_with_webrtc_apis.md

Co-authored-by: Philip Giuliani <[email protected]>

Author: jianjunz

docs/design/owt_with_webrtc_apis.md

@@ -1,19 +1,71 @@
 # OWT SDK with WebRTC APIs
 ## Introduction
 OWT(Open WebRTC Toolkit) Client SDKs provide convenient APIs to create, publish, and subscribe streams. Most of these APIs are wrappers of WebRTC APIs with signaling support. It helps WebRTC beginners easily involve WebRTC technology into their applications without too much knowledge of WebRTC evolution and browser differences. As WebRTC 1.0 is moving to PR, which means it is quite stable, we are planning to expose more WebRTC APIs to developers to enable advanced and custom usages with OWT.
-## WebRTC APIs
-### RTCRtpTransceiver, RTCRtpSender, RTCRtpReceiver
-#### Potential Usages
+## Potential Usages
 - Replace a track in the middle of a call.
 - Set custom encoding parameters, perhaps for simulcast.
 - Set preferred codecs.
 - Disable or enable RTX / RED / FEC.
-#### API Changes
+## API Changes
 - A new member `rtpTransceivers` will be added to `TransportSettings`. It returns an array `RTCRtpReceiver`s for RTP transport.
 - A new member `transport` will be added to `Publication` and `Subscription`. Developers could get `RTPTransceiver`s from its `rtpTransceivers` property.
-- A new method `addTransceiver(DOMString trackKind, sequence<RTCRtpEncodingParameters> sendEncodings)` will be added to `ConferenceClient`. It invokes `RTCPeerConnection.addTransceiver(trackKind, {direction:inactive, sendEncodings:sendEncodings})`, returns an `RTCRtpTransceiver`. Please note that direction is `inactive` until a `publish` with return transceiver is called.
-- The second parameter of `ConferenceClient.publish` accepts an `RTCRtpTransceiver` created by `RTCPeerConnection.addTransceiver`. When this method is called, certain `RTCRtpTransceiver`'s direction is changed to `sendonly`, and its sender's `setStreams` is called with the first parameter's `mediaStream`.
-#### Server Requirements
+- A new member `peerConnection` will be added to `ConferenceClient` for developers to get the `PeerConnection`.
+- The second parameter of `ConferenceClient.publish` accepts an `RTCRtpTransceiver` created by `RTCPeerConnection.addTransceiver`. `RTCRtpTransceiver`s' direction must be `sendonly`, and its sender's `setStreams` is called with the first parameter's `mediaStream`.
+## Server Requirements
 - `addTransceiver` and new `publish` needs renegotiation support.
-#### Remarks
-- Every transceiver could be associated with at most one `Publication` or one `Subscription`.
+## Remarks
+- Every transceiver could be associated with at most one `Publication` or one `Subscription`.
+## Examples
+### Replace a video track
+```
+const transceivers = publication.transport.rtpTransceivers;
+for (const transceiver of transceivers) {
+  if (transceiver.sender.track?.kind === 'video'){
+    transceiver.sender.replaceTrack(newTrack);
+  }
+}
+```
+
+### Publish a stream with codec preferences
+```
+const codecCapabilities = RTCRtpSender.getCapabilities('video').codecs;
+// Reorder codecCapabilities or remove some items.
+const transceivers = [];
+for (const track of mediaStream.getTracks()) {
+  const transceiver =
+      conference.peerConnection.addTransceiver(track, {
+        direction: 'sendonly',
+        streams: [stream],
+      });
+  if (track.kind==='video'){
+      transceiver.setCodecPreferences(codecCapabilities);
+  }
+  transceivers.push(transceiver);
+}
+const publication = await conference.publish(localStream, transceivers);
+```
+
+
+### Publish with a codec supports SVC.
+This example uses some WebRTC APIs may not be supported by all browsers.
+```
+const transceiver =
+    conference.peerConnection.addTransceiver(track, {
+      direction: 'sendonly',
+      streams: [stream],
+      sendEncodings: [
+        {
+          rid: 'q',
+          scaleResolutionDownBy: 4.0,
+          scalabilityMode: 'L1T3'
+        },
+        {
+          rid: 'h',
+          scaleResolutionDownBy: 2.0,
+          scalabilityMode: 'L1T3'
+        },
+        {rid: 'f', scalabilityMode: 'L1T3'}
+      ],
+    });
+const publication = await conference.publish(localStream, [transceiver]));
+```

docs/mdfiles/changelog.md

@@ -4,6 +4,8 @@ Change Log
 * When subscribe a stream in conference mode, the subscribed MediaStream or BidirectionalStream is associated with a `Owt.Conference.Subscription` instead of a `Owt.Base.RemoteStream`. The `stream` property of a RemoteStream in conference mode is always undefined, while a new property `stream` is added to `Subscription`. It allows a RemoteStream to be subscribed multiple times, as well as subscribing audio and video tracks from different streams.
 * Add a new property `transport` to `Publication` for getting `TransportSettings`.
 * Add a new property `rtpTransceivers` to `TransportSettings` and `TransportConstraints`.
+* Add a new property `peerConnection` to `ConferenceClient`.
+* The second argument of `ConferenceClient.publish` could be a list of `RTCRtpTransceiver`s.
 # 5.0
 * Add WebTransport support for conference mode, see [this design doc](../../design/webtransport.md) for detailed information.
 * All publications and subscriptions for the same conference use the same `PeerConnection`.

src/.eslintrc.json

@@ -4,7 +4,7 @@
 {
   "extends": ["eslint:recommended", "google", "webrtc"],
   "parserOptions": {
-    "ecmaVersion": 2018,
+    "ecmaVersion": 2020,
     "sourceType": "module"
   },
   "rules": {

src/samples/conference/public/scripts/index.js

@@ -29,6 +29,9 @@
 'use strict';
 var conference;
 var publicationGlobal;
+// Change to your sample server's URL if it's not deployed on the same machine
+// as this page.
+const serverUrlBase = undefined;
 const runSocketIOSample = function() {
 
     let localStream;
@@ -124,7 +127,7 @@ const runSocketIOSample = function() {
         console.log('A new stream is added ', event.stream.id);
         isSelf = isSelf?isSelf:event.stream.id != publicationGlobal.id;
         subscribeForward && isSelf && subscribeAndRenderVideo(event.stream);
-        mixStream(myRoom, event.stream.id, 'common');
+        mixStream(myRoom, event.stream.id, 'common', serverUrlBase);
         event.stream.addEventListener('ended', () => {
             console.log(event.stream.id + ' is ended.');
         });
@@ -144,7 +147,7 @@ const runSocketIOSample = function() {
                 myId = resp.self.id;
                 myRoom = resp.id;
                 if(mediaUrl){
-                     startStreamingIn(myRoom, mediaUrl);
+                     startStreamingIn(myRoom, mediaUrl, serverUrlBase);
                 }
                 if (isPublish !== 'false') {
                     // audioConstraintsForMic
@@ -174,9 +177,22 @@ const runSocketIOSample = function() {
                             mediaStream, new Owt.Base.StreamSourceInfo(
                                 'mic', 'camera'));
                         $('.local video').get(0).srcObject = stream;
+                        // Publish with RTCRtpTransceivers.
+                        // const transceivers = [];
+                        // for (const track of mediaStream.getTracks()) {
+                        //   transceivers.push(
+                        //       conference.peerConnection.addTransceiver(track, {
+                        //         direction: 'sendonly',
+                        //         streams: [stream],
+                        //       }));
+                        // }
+                        // publication =
+                        //     await conference.publish(localStream, transceivers);
+
+                        // Publish with options.
                         conference.publish(localStream, publishOption).then(publication => {
                             publicationGlobal = publication;
-                            mixStream(myRoom, publication.id, 'common')
+                            mixStream(myRoom, publication.id, 'common', serverUrlBase)
                             publication.addEventListener('error', (err) => {
                                 console.log('Publication error: ' + err.error.message);
                             });
@@ -216,7 +232,7 @@ const runSocketIOSample = function() {
                     $p.appendTo($('body'));
                 }
             });
-        });
+        }, serverUrlBase);
     };
 };
 window.onbeforeunload = function(event){