Fix/drag and drop on linux

[mikekrylov]

What does the pull request do?

Added realisation of XDnD protocol for X11 on Linux.

What is the current behavior?

There are no Drag and Drop between Avalonia window and other apps on Linux systems with X11 and Wayland-X11 mode.

What is the updated/expected behavior with this PR?

Avalonia app can receive dropped uri and text data and initiate drag for those datas on Linux (if under X11 server).
Avalonia app can receive/transmit other X11's mime types, if app's devs support them.

How was the solution implemented (if it's not obvious)?

To X11Window added helper instance of X11DropTarget, which process XDnD messages and request data according specifications. X11DropTarget works as mid-level between X11 and InputRoot system.
To X11Platform added X11DragSource (similar as OLE implementation). On drag X11DragSource creates instance of DragSourceWindow, an invisible window, which process pointer movements and sends XDnD messages to windows under the cursor.

Checklist

• Added unit tests (if possible)?
• Added XML documentation to any related classes?
• Consider submitting a PR to https://github.com/AvaloniaUI/avalonia-docs with user documentation

Breaking changes

Obsoletions / Deprecations

Fixed issues

Fixes #6085

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0057686-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[cla-avalonia]

• All contributors have signed the CLA.

[mikekrylov]

@cla-avalonia agree

[MrJul]

Thank you for your contribution! :)

I haven't reviewed the code yet, but we're currently in the process of refactoring the clipboard and drag and drop architecture. Consequently, your PR will be on hold until that work is completed and will very likely need modifications to accommodate the new types.

Edit: clipboard refactoring PR is #19347

[mikekrylov]

@MrJul Ok, but there a no much usage of Avalonia's inner DragNDrop system, mostly implementations of lots and lots of X11 requests

[kekekeks]

IDataObject can be queried for data at any stage ( DragEnter, DragOver, DragDrop, DragLeave) for any of the supported types, apps are allowed to inspect dragged contents to decide if they support the dragged data.

This behavior is actually described by XDND spec too:

If it needs to look at the data itself, it calls XConvertSelection() for XdndSelection, the data type that it is interested in, and the given time stamp. (7) It can do this more than once, if necessary.

If it can accept the drop, it should hilight its border to notify the user. If it retrieved the data, it should cache it so it does not need to be retrieved again when the actual drop occurs.

The PR only loads data for only one random data type after getting the final drop event.

So the implementation is incorrect.

[mikekrylov]

@kekekeks Changed it.
Now it requests all data at DragEnter. I've read, that some X11 drag source implementations supports only one request for data, so "standart" files and text types will be asked first.
Because IDataObject allows user to ask data by type in any moment later, we need to cache all pairs type-data inside the IDataObject.

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0057690-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[kekekeks]

Now it requests all data at DragEnter.

This is technically incorrect too. The dragged data could be a 8K image that is presented in multiple compressed and uncompressed formats. Transferring all of that via INCR's window of 16KB will take hours.

[kekekeks]

e. g. dragging image from firefox produces the following formats:

text/uri-list
_NETSCAPE_URL
text/x-moz-url-data
text/x-moz-url-desc
application/x-moz-custom-clipdata
text/_moz_htmlcontext
text/_moz_htmlinfo
text/html
text/plain
text/plain;charset=utf-8
application/x-moz-nativeimage
image/png
image/jpeg
image/jpg
image/gif
application/x-moz-file-promise
XdndDirectSave0
application/x-moz-file-promise-url
application/x-moz-file-promise-dest-filename
application/x-qt-image

[mikekrylov]

@kekekeks Changed it.
Now we preload on enter only Files and Text types, if user asks via Get other data format, we ask it from sender and cache in data object. On drop we load all the formats, asked by user via Contains, or, if there were no any data asked/loaded - first available type.
And for .net types serialization we use BinaryFormatter, like in OLE version - I have doubts about it, but "alternative" JsonSerializator are absent in the netstandart 2.0 and it's limits can cause inconsistent behaviour between Win and Linux versions.

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0058022-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[JZZQuant]

I have tried with mp3/wav and gpz guitar profile its working for me dont see any issues .

[kekekeks]

Now we preload on enter only Files and Text types, if user asks via Get other data format, we ask it from sender and cache in data object. On drop we load all the formats, asked by user via Contains, or, if there were no any data asked/loaded - first available type.

Haven't read the diff, but what you described here is not a correct implementation of IDataObject contract.

Have you tried the described usage scenario with inspecting the actual data of dragged image inside of DragMove event?

If this is not addressed, we can probably merge the PR for the sake of the most common file list drag-n-drop scenario, but we'll have to disable support for other data formats.

[kekekeks]

From XDND spec:

When the source and target windows are part of the same application, sending X Client Messages is a waste of time and bandwidth, especially if the program and server are on different machines. Implementations should therefore detect the special cases of "source = target" and "source and target in same application" and handle them via function calls.

We should switch to in-process implementation when DnD occurs between app's own windows.

[drasticactions]

No new code should be introduced that uses BinaryFormatter. It has been deprecated from .NET for years as a security hazard and is being migrated out of the framework altogether, hence the need for this flag. In net9.0+, it also requires a package install for it to work at all, and is heavily discouraged from use.

There are some legacy places within Avalonia where it is used; those should be removed for v12 or ASAP. No new code using this should be added. This needs to be replaced.

[mikekrylov]

Ok, but what should I use as alternative? Microsoft recommends JsonSerializator, but it absent in netstandart 2.0.
Or skip this functionality in netstandart 2.0?

[drasticactions]

@kekekeks what do you think?

netcore includes System.Text.Json. Framework doesn't, and obviously not in netstandard, but that could be brought in as a dependency. Is there other serialization frameworks within Avalonia that could be used here?

I'm open to pulling in System.Text.Json since it's already in the netcore, it would need to be referenced for legacy frameworks.

[MrJul]

I don't think we should do any kind of automatic magic serialization.

With #19347 we're handling conversions for known formats, but everything else should be a string or byte[]/Memory<byte>. It's up to the user to properly serialize/deserialize their payload with their preferred format.

The only exception is on Windows which always had the BinaryFormatter in place (for compatibility with WPF/Winforms), but it will be removed for v12.

[mikekrylov]

removed BinaryFormatter

[mikekrylov]

@kekekeks Sorry, I'm not sure i'm understood what is wrong with IDataObject contract.
User can request data on move event via Get it will be requested from sender. It could took time, but it is X11's trait.
As I see, only difference with OLE version, is that on Win user can store IDataObject, and ask data lately, long after drop event, but on X11 we must get datas from sender before it ends conversation.

Could you describe more, which functionality should I add?

[mikekrylov]

Ok, looks like after #19347 some of my problems, such as BinaryFormatter, will be gone)

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0058964-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[mikekrylov]

@drasticactions , @kekekeks
Removed BinnaryFormater, added Drag'n'Drop between windows in one app.

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0059053-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[MrJul]

@mikekrylov Now that #19347 is merged, could you please update this PR when you have time?

You should now implement IAsyncDataTransfer and IAsyncDataTransferItem, preferably by inheriting from PlatformAsyncDataTransfer and PlatformAsyncDataTransfer.

[mikekrylov]

@MrJul Thanks!
Already in the process.

Should only async one? Not need to implementing sync version?

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0059373-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[mikekrylov]

@MrJul @kekekeks @drasticactions
Ta-dam!
New, shiny version of X11 dnd with IDataTransfer is ready and waiting for review.

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0059397-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[avaloniaui-bot]

You can test this PR using the following package version. 12.0.999-cibuild0059694-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]