AddMousePosEvent() with both absolute and relative mouse position

[tksuoran]

Version/Branch of Dear ImGui:

Version 1.92.7 WIP (19262), Branch: docking

Back-ends:

erhe

Compiler, OS:

Windows 11, MSVC 18.2.1

Full config/build information:

Dear ImGui 1.92.7 WIP (19262)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=202002
define: IMGUI_DISABLE_OBSOLETE_FUNCTIONS
define: _WIN32
define: _WIN64
define: _MSC_VER=1950
define: _MSVC_LANG=202002
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
IM_ASSERT: runs expression: OK. expand size: OK
--------------------------------
io.BackendPlatformName: imgui_impl_erhe
io.BackendRendererName: erhe
io.ConfigFlags: 0x00000081
 NavEnableKeyboard
 DockingEnable
io.ConfigViewportsNoDecoration
io.ConfigViewportsNoDefaultParent
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigWindowsMoveFromTitleBarOnly
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x0000001A
 HasMouseCursors
 RendererHasVtxOffset
 RendererHasTextures
--------------------------------
io.Fonts: 6 fonts, Flags: 0x00000000, TexSize: 512,256
io.Fonts->FontLoaderName: FreeType
io.DisplaySize: 1920.00,1080.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 2.00,2.00
style.WindowBorderSize: 1.00
style.FramePadding: 2.00,2.00
style.FrameRounding: 3.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 2.00,2.00
style.ItemInnerSpacing: 2.00,2.00

Details:

My Issue/Question:

I have augmented mouse input events in ImGui so that backend can at will - when requested by app - change the mouse behavior to "relative" only mode:

• When user moves the mouse
  • Absolute position x, y remains the same as it was when relative mode was entered
  • Relative mouse dx, dy is reported normally
  • Mouse motion is not constrained to screen boundaries

Motivation / Use Cases

Requesting stationary mouse cursor with relative-only mouse motion events unlocks ability to interact with ImGui widgets that need only relative mouse input, without screen boundaries limiting the drag input.

My actual use case:

• Navigation gizmos like in https://docs.blender.org/manual/en/latest/editors/3dview/navigate/introduction.html

Possible use cases I have not yet implemented:

• Drag int, float etc. sliders that could be dragged even if mouse cursor would otherwise stop at screen borders.

Changes in my backend

• I have implemented changes in my custom ImGui backend, which internally uses SDL3, SDL_WarpMouseInWindow() and SDL_SetWindowRelativeMouseMode().
• The old mouse motion event only had absolute x and y.
• The new mouse motion event keeps absolute x and y, and adds relative dx and dy.
• The relative motion dx and dy is always reported truthfully
• In relative only mode, the absolute position is "frozen", and the backend uses SDL API to warp the mouse to keep in the window

My backend has had this functionality for quite some time, due to interaction that did not involve ImGui widgets. When I wanted to use the relative mode with ImGui widgets, I figured out I need a way to tell ImGui about relative mouse motion while keeping mouse absolute position stationary.

Changes in ImGui

• Added relative dx, dy to ImGuiInputEventMousePos
• Added AddMousePosEventWithRelative() which backend can use to feed mouse events that have x, y, dx, and dy.
• In UpdateMouseInputs() mouse delta is cleared to zero, not computed from absolute position in previous and current frame (*)
• In UpdateInputEvents() mouse delta is cumulated from mouse events (*)
• (*): I made an attempt to also support backends which do not report relative mouse events, but I have not tested these paths - and there may be simpler/better ways.

The changes can be seen in tksuoran/erhe@bc7230e

Changes / usage from application code

• Currently, my application code uses backend specific API void Context_window::set_cursor_relative_hold(bool relative_hold_enabled) to choose if mouse absolute position should be frozen or not
• No other modifications needed to the application. When in relative mode, io.MouseDelta values can be consumed and work as expected, while absolute position remains stationary as/by design.

Low level details on my backend

• Input events are processed in https://github.com/tksuoran/erhe/blob/main/src/erhe/window/erhe_window/sdl_window.cpp
• Context_window::poll_events() pulls mouse events from SDL; SDL_Event already comes with both absolute x, y and relative dx, dy
  • Before processing events, if relative mode is active, mouse is warped to the frozen absolute position with SDL_WarpMouseInWindow()
• Context_window::handle_mouse_move() builds input event with either the real or frozen absolute position. This is the mouse motion event that I eventually route to ImGui::AddMousePosEventWithRelative() (but my app also other non-ImGui input event consumers).
• Context_window::set_cursor_relative_hold() enables and disables relative mode. The mode switch is implemented with SDL_SetWindowRelativeMouseMode().

Please advice if it would make sense to develop this change further and aim to be included in ImGui - or should I just keep this as custom modification that I keep cherry-picking to my ImGui. Thank you!

[ocornut]

Hello,

It seems it would be dangerous if we tried to separate them, because lots of code assume subtracting previous and current frame position would lead to the same value as relative.

I think one way to frame it is that when you are in mouse relative hold mode, you could maintain/reconstruct a global position that matches the relative motion you want and submit it to io.AddMousePosEvent(). Presumably if you are using a custom backend there should be no issue with that?

[tksuoran]

By lots of code, do you mean ImGui internals, or code using ImGui?
Injecting fake absolute positions would break positions of mouse inputs and I think that would be more broken that just no relative motion which I currently get.

[tksuoran]

Clarification: My mouse motion event always is single event with all x, y, dx and dy, there are no separate events for absolute and relative motion. IIRC believe I copied this pattern from OpenXR, where found it worked well for me.

[ocornut]

By lots of code, do you mean ImGui internals, or code using ImGui?

Probably both, but I assume you are controlling the later.

I’m afraid I don’t fully understand your problem because well, I can’t really see the problem.
Could you perhaps setup a patch based on a SDL3 example that would replicate your situation ?

[tksuoran]

This works for me: tksuoran/erhe@bc7230e
Feel free to close this issue or leave open if you think either brings any value.

[ocornut]

Again, I don’t quite understand your problem aka why you’d need that, so I can’t evaluate if it is generally desirable.

Can you try to reword your first post with eg more pseudo code or commentary?

Thank you.

[tksuoran]

I've reworded my first post, I hope that helps a bit.