Low frame rate and high frames drop rate when using Sunshine vs GFE

[Adelyann]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the nightly release?

• This issue is present in the nightly release

Describe the Bug

I have set up both GFE and Sunshine (using different ports).

When I stream my desktop with GFE I got these stats :

When I stream my desktop with Sunshine in the exact same condition I got these stats :

Plus I get huge frame drop (and slow connection message) when reading PDF with SumatraPDF or using some programs

Expected Behavior

I should see similar stats and not experiencing frame drop with Sunshine.

Additional Context

No response

Host Operating System

Docker

Operating System Version

Windows 11

Architecture

32 bit

Sunshine commit or version

0.21

Package

Windows - installer

GPU Type

Nvidia

GPU Model

ASUS RTX 3090

GPU Driver/Mesa Version

551.61

Capture Method (Linux Only)

No response

Config

encoder = nvenc

Apps

No response

Relevant log output

No relevant or specifics entries in logs

[cgutman]

The variable frame rate is expected from Sunshine. Unlike GFE, Sunshine only encodes frames when there is new content being displayed, hence you will see a frame rate that matches the rate of content updates on screen (with a floor of about 10 FPS). Pull up a 60 FPS video/game or something like https://testufo.com and you'll see it jump up to 60 just like GFE.

As for the frame drops (sounds like packet loss to me) over the network connection, we'd probably need client logs to have a chance to understand what's going on there.

[Adelyann]

Thanks. It looks like that sometimes during stream Sunshine does not detect HEVC capabilities and most of the rest of the time it does... Maybe that's the reason ?

Error: NvEnc: encoding format is not supported by the gpu

---

[2024:02:29:08:07:33]: Info: Active GPU has HAGS enabled
[2024:02:29:08:07:33]: Info: Using realtime GPU priority
[2024:02:29:08:07:33]: Info: Desktop resolution [3840x2160]
[2024:02:29:08:07:33]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:29:08:07:33]: Info: Display refresh rate [59.997Hz]
[2024:02:29:08:07:33]: Info: Requested frame rate [60fps]
[2024:02:29:08:07:33]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 8
Red Primary : [0.632812,0.339844]
Green Primary : [0.310547,0.632812]
Blue Primary : [0.158203,0.0605469]
White Point : [0.313477,0.329102]
Min Luminance : 0.5 nits
Max Luminance : 270 nits
Max Full Luminance : 270 nits
[2024:02:29:08:07:33]: Info: SDR color coding [Rec. 601]
[2024:02:29:08:07:33]: Info: Color depth: 8-bit
[2024:02:29:08:07:33]: Info: Color range: [JPEG]
[2024:02:29:08:07:33]: Error: NvEnc: encoding format is not supported by the gpu
[2024:02:29:08:07:33]: Info:
Device Description : NVIDIA GeForce RTX 3090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002204
Device Video Mem : 24326 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 32724 MiB
Feature Level : 0x0000B100
Capture size : 3840x2160
Offset : 0x0
Virtual Desktop : 3840x2160
[2024:02:29:08:07:33]: Info: Active GPU has HAGS enabled
[2024:02:29:08:07:33]: Info: Using realtime GPU priority
[2024:02:29:08:07:33]: Info: Desktop resolution [3840x2160]
[2024:02:29:08:07:33]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:29:08:07:33]: Info: Display refresh rate [59.997Hz]
[2024:02:29:08:07:33]: Info: Requested frame rate [60fps]
[2024:02:29:08:07:33]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 8
Red Primary : [0.632812,0.339844]
Green Primary : [0.310547,0.632812]
Blue Primary : [0.158203,0.0605469]
White Point : [0.313477,0.329102]
Min Luminance : 0.5 nits
Max Luminance : 270 nits
Max Full Luminance : 270 nits
[2024:02:29:08:07:33]: Info: SDR color coding [Rec. 709]
[2024:02:29:08:07:33]: Info: Color depth: 10-bit
[2024:02:29:08:07:33]: Info: Color range: [JPEG]
[2024:02:29:08:07:33]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:02:29:08:07:33]: Info:
[2024:02:29:08:07:33]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:02:29:08:07:33]: Info:
[2024:02:29:08:07:33]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024:02:29:08:07:33]: Info: Found HEVC encoder: hevc_nvenc [nvenc]
[2024:02:29:08:07:33]: Info: Executing [Desktop]
[2024:02:29:08:07:33]: Info: nvprefs: No need to modify application profile settings
[2024:02:29:08:07:33]: Info: nvprefs: No need to modify global profile settings
[2024:02:29:08:07:34]: Info: CLIENT CONNECTED
[2024:02:29:08:07:34]: Info:
Device Description : NVIDIA GeForce RTX 3090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002204
Device Video Mem : 24326 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 32724 MiB
Feature Level : 0x0000B100
Capture size : 3840x2160
Offset : 0x0
Virtual Desktop : 3840x2160

[RebelliousX]

@cgutman

The variable frame rate is expected from Sunshine. Unlike GFE, Sunshine only encodes frames when there is new content being displayed, hence you will see a frame rate that matches the rate of content updates on screen (with a floor of about 10 FPS). Pull up a 60 FPS video/game or something like https://testufo.com/ and you'll see it jump up to 60 just like GFE.

As for the frame drops (sounds like packet loss to me) over the network connection, we'd probably need client logs to have a chance to understand what's going on there.

Interesting, could you please take a look at this problem here. It seems what you mentioned and what this guy experiencing (in the link below) is the cause for the random stuttering. And if there is a forced activity like constantly moving the mouse for example will make the stutter go away. I think it might be better to have an option to allow encoding of frames even if they seem identical, duplicated frames dropping might cause weird stutter. I too kind of experience random stutter even with very low latency < 5-8ms. Probably the time Sunshine needs to switch from low frame rate to 60, at that moment we get random micro hitching, seemingly dropping possible duplicated frames thus disrupting the fluidity of frame transitions.

https://www.reddit.com/r/MoonlightStreaming/comments/15y68yz/stutteringhitching_issues_with_sunshinemoonlight/

He posted few videos, but this one is extreme, the mouse pointer not showing in the video, but sound stutters too and no visual changes on screen. Sound stutter goes away when moving the mouse.

https://imgur.com/a/gN24hBF

For me, I tried to run Godot Game Engine on my TV, and I get bad stutter and low frame rate, sprites movements are bumpy when there is not much changes on screen, fps in 30-55s, forcing the project's game window of the game engine to go full screen fixes the issue and I get solid 60fps and smooth.

[cgutman]

I think it might be better to have an option to allow encoding of frames even if they seem identical, duplicated frames dropping might cause weird stutter.

It's not us actually dropping the frames - we don't even see them. The DXGI Desktop Duplication API doesn't report new frames until the DWM actually composites a new image (which won't happen if no visible surfaces change). It would be possible in theory to have the encoder thread wait for a fixed time and encode if a new frame doesn't arrive in say 16.6ms, but this has clock drift problems between the display refresh rate and encode loop and runs into issues with the 1ms minimum timer granularity on Windows.

For me, I tried to run Godot Game Engine on my TV, and I get bad stutter and low frame rate, sprites movements are bumpy when there is not much changes on screen, fps in 30-55s, forcing the project's game window of the game engine to go full screen fixes the issue and I get solid 60fps and smooth.

I think this is probably more likely related to the Vulkan/OpenGL presentation path. Nvidia's driver provides an option to force Vulkan and OpenGL to present using DXGI, which allows us to capture it at full rate. I'm not aware of a similar option for AMD or Intel GPUs though. When you fullscreened it, the surface was probably promoted to the DirectFlip path which increased capture performance too.

There's a new display capture method coming soon as part of #2149 which may provide significant improvements on both fronts. We can retest these issues with WCG capture once that lands in nightly.

[RebelliousX]

I think this is probably more likely related to the Vulkan/OpenGL presentation path. Nvidia's driver provides an option to force Vulkan and OpenGL to present using DXGI, which allows us to capture it at full rate. I'm not aware of a similar option for AMD or Intel GPUs though. When you fullscreened it, the surface was probably promoted to the DirectFlip path which increased capture performance too.

Well, it is still kind of bad at full screen or maximizing the game window, but to lesser extent. And Godot uses the high-performant GPU not the iGPU. I still see the 2D sprites very jumpy and sometimes they go back and forth, lol. The Nvidia control panel already set to use DXGI for Vulkan/OpenGL.

There's a new display capture method coming soon as part of #2149 which may provide significant improvements on both fronts. We can retest these issues with WCG capture once that lands in nightly.

Good to know,, I can't wait to test it. 👍

[Adelyann]

Update : It is getting worse with 0.22
Now even the mouse seem to be "laggy" for instance when opening a new tab in my browser or still when reading a PDF...

[LizardByte-bot]

It seems this issue hasn't had any activity in the past 90 days. If it's still something you'd like addressed, please let us know by leaving a comment. Otherwise, to help keep our backlog tidy, we'll be closing this issue in 10 days. Thanks!

[LizardByte-bot]

This issue was closed because it has been stalled for 10 days with no activity.