Test Ideas

[Sean-Der]

Echo (Round trip audio/video packets)

Signaling/Media

• Renegotation
• Simulcast
• Multiple media sections
• Offerer controls PayloadTypes
• Answerer properly responds to enabled/disabled codecs

ICE

• pwd/ufrag validity is assert
• Trickle
• Restart
• STUN
• TURN (UDP, TCP, DTLS and TLS)
• mDNS

DTLS

• assert that fingerprint is checked

DataChannels

• Ordered and Unordered
• Lossy and Lossless
• Assert IDs are properly chosen

RTP/RTCP

• Sender/Receiver Reports
• Responds to NACK

PeerConnection APIs

• ICEConnectionState, PeerConnectionState

[Sean-Der]

@sipsorcery @shinyoshiaki @steely-glint @jlaine what do you think? Some of these come from bugs/mistakes in Pion that really hurt, and I don't want to see others have to deal with.

[sipsorcery]

+1 from me.

There would need to be some sort of config to indicate which tests to attempt against each library. From my point of view I don't know if/when the sipsorcery library will even catch up to the list of features mentioned above let alone the full list of RFC's libwebrtc supports and of which more seem to regularly appear.

[Sean-Der]

100% agree

I can take a shot at ‘Docker and run’ next weekend. All yours if you want it earlier :)

Would you be interested in submitting to IETF hackathon? Don’t know if I can do live, but would get eyes on it!

[sipsorcery]

Yeah I'll take a look at an automated docker set up.

Would you be interested in submitting to IETF hackathon?

I'm guessing the benefit is exposure of a project to a wider audience?

[sipsorcery]

A GitHub Actions Workflow is now up and running to build docker images and run echo tests.

@Sean-Der I've added instructions for the Dockerfile. If you get a chance to test them out it would be nice to know if they are sufficient or not (they could well be missing steps I overlooked). Once you've got a Dockerfile add "pion" to the list of libraries in the workflow and the image should be automatically built.

[jlaine]

Do you think we could enable runs of the workflow for pull requests too? For instance, for my PR #19 which adds an aiortc client: I'd like to also include it as a client but don't want to discover I broke everything!

Obviously for PRs we don't want to have a change to the results commited :)

[sipsorcery]

Do you think we could enable runs of the workflow for pull requests too?

Sure done.

[jlaine]

Do you think we could enable runs of the workflow for pull requests too?

Sure done.

Hm, maybe this wasn't such a great idea after all. Currently the PR builds are failing, probably due to (legitimate) restrictions on access to the secrets.

[Sean-Der]

@sipsorcery sorry been busy :/

Submitting to the IETF hackathon might also get more acceptance of non-libwebrtc implementations! That is my big end goal. I would love to see a world where people build WebRTC projects with their favorite language.

Instead of saying 'Oh I use $x language for WebRTC support' it would be like building a HTTP/REST service. You use the language you are comfortable in :)