Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: Key input overhaul #19616

Merged
merged 5 commits into from
Mar 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
305 changes: 301 additions & 4 deletions core/src/events.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,10 @@ use swf::ClipEventFlag;
#[derive(Debug, Clone)]
pub enum PlayerEvent {
KeyDown {
key_code: KeyCode,
key_char: Option<char>,
key: KeyDescriptor,
},
KeyUp {
key_code: KeyCode,
key_char: Option<char>,
key: KeyDescriptor,
},
MouseMove {
x: f64,
Expand Down Expand Up @@ -830,3 +828,302 @@ impl FromStr for GamepadButton {
})
}
}

#[derive(Debug, Clone, Copy)]
pub struct KeyDescriptor {
pub physical_key: PhysicalKey,
pub logical_key: LogicalKey,
pub key_location: KeyLocation,
}
Comment on lines +832 to +837
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestions on how to jam the contents of this neat struct (especially LogicalKey) all the way across this pipeline? 😵‍💫

https://github.com/ruffle-rs/ruffle-android/blob/main/app/src/main/res/layout/keyboard.xml#L121-L127
https://github.com/ruffle-rs/ruffle-android/blob/main/app/src/main/java/rs/ruffle/PlayerActivity.kt#L179-L188
https://github.com/ruffle-rs/ruffle-android/blob/main/app/src/main/java/rs/ruffle/PlayerActivity.kt#L97-L98
https://github.com/ruffle-rs/ruffle-android/blob/main/src/lib.rs#L487-L523
https://github.com/ruffle-rs/ruffle-android/blob/main/src/lib.rs#L400-L421

Or maybe I should leave all the XML/Kotlin parts operating on "softkeyboard scancodes" of a made-up layout, and translate it all in the Rust part? But that creates an uncomfy tie (or gap, depending on how you look at it) between how and where the layout is defined (including the button text), and what it "means" for the Ruffle player...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO when it comes to a simple, custom keyboard, you should use an approach similar to one used in tests, where each key has an identifier and is mapped into PhysicalKey/LogicalKey/KeyLocation. For more advanced keyboards I would expect all 3 fields to come from the virtual keyboard.

Maybe we can explore the idea of using the system keyboard on Android?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can explore the idea of using the system keyboard on Android?

I don't think we can get separate down/up events from that, nor can it handle pressing multiple keys at the same time...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For pure text entry, sure, it would be great, but not for playing games, and such.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(The same mechanisms could/should trigger hiding our handmade keyboard and bringing up the system one as currently used for the same purpose on web IMO.)

Copy link
Member

@torokati44 torokati44 Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bur yeah, making each key have a proper identifier in its tag attribute, and carrying that string all the way into Rust, then dispatching on it there (turning it into a KeyIdentifier) sounds doable. But passing strings through the JNI FFI boundary is just so much more inconvenient than simple scalar values IIRC... 😩 Sure it's not impossible, just less nice...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can always map it in Java and pass numerical values down. Not sure if that would require code duplication or not

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but that would just require more messing around with encoding different (sometimes optional) info about all the keys into numbers and then those into Rust enumerants... This way it would be in one place at least.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it would still require separating the methods to pass in the key events coming from our handmade virtual keyboard (with the string identifiers), and those coming from an actual physical keyboard connected say via Bluetooth (or USB or adb/scrcpy or whatever), in the form of an Android KeyEvent. Oh well, this isn't a big deal either I guess...


/// Physical keys are keys that exist physically on a keyboard (or other devices).
///
/// For instance:
/// * pressing <kbd>E</kbd> while using ANSI keyboard and Colemak keyboard layout
/// will produce physical key [`PhysicalKey::KeyE`] and logical key `F`,
/// * pressing left backslash on a UK keyboard will produce physical key
/// [`PhysicalKey::IntlBackslash`] and logical key `\`.
///
/// See <https://w3c.github.io/uievents-code/#code-value-tables>.
#[derive(Debug, Clone, Copy)]
pub enum PhysicalKey {
Unknown = 0,

// Alphanumeric Section
Backquote,
Digit0,
Digit1,
Digit2,
Digit4,
Digit3,
Digit5,
Digit6,
Digit7,
Digit8,
Digit9,
Minus,
Equal,
IntlYen,
KeyA,
KeyB,
KeyC,
KeyD,
KeyE,
KeyF,
KeyG,
KeyH,
KeyI,
KeyJ,
KeyK,
KeyL,
KeyM,
KeyN,
KeyO,
KeyP,
KeyQ,
KeyR,
KeyS,
KeyT,
KeyU,
KeyV,
KeyW,
KeyX,
KeyY,
KeyZ,
BracketLeft,
BracketRight,
Backslash,
Semicolon,
Quote,
IntlBackslash,
Comma,
Period,
Slash,
IntlRo,
Backspace,
Tab,
CapsLock,
Enter,
ShiftLeft,
ShiftRight,
ControlLeft,
SuperLeft,
AltLeft,
Space,
AltRight,
SuperRight,
ContextMenu,
ControlRight,

// Control Pad Section
Insert,
Delete,
Home,
End,
PageUp,
PageDown,

// Arrow Pad Section
ArrowUp,
ArrowLeft,
ArrowDown,
ArrowRight,

// Numpad Section
NumLock,
NumpadDivide,
NumpadMultiply,
NumpadSubtract,
Numpad7,
Numpad8,
Numpad9,
Numpad4,
Numpad5,
Numpad6,
Numpad1,
Numpad2,
Numpad3,
Numpad0,
NumpadAdd,
NumpadComma,
NumpadEnter,
NumpadDecimal,

// Function Section
Escape,
F1,
F2,
F3,
F4,
F5,
F6,
F7,
F8,
F9,
F10,
F11,
F12,
F13,
F14,
F15,
F16,
F17,
F18,
F19,
F20,
F21,
F22,
F23,
F24,
F25,
F26,
F27,
F28,
F29,
F30,
F31,
F32,
F33,
F34,
F35,
Fn,
FnLock,
PrintScreen,
ScrollLock,
Pause,
}

/// Logical key represents the semantics behind a pressed physical key
/// taking into account any system keyboard layouts and mappings.
///
/// Most keys will be mapped into a [`LogicalKey::Character`], but some keys that do not produce
/// any characters (such as `F1...Fn` keys, `Home`, `Ctrl`, etc.) will be mapped into
/// [`LogicalKey::Named`], which is similar to a physical key, but:
/// * it does not take into account duplicate keys, e.g. there's only one Shift,
/// * it respects the keyboard layout, so that e.g. pressing <kbd>CapsLock</kbd>
/// with Colemak can produce `Backspace`.
///
/// See <https://w3c.github.io/uievents-key/>.
#[derive(Debug, Clone, Copy)]
pub enum LogicalKey {
Unknown,
Character(char),
Named(NamedKey),
}

impl LogicalKey {
pub fn character(self) -> Option<char> {
match self {
LogicalKey::Unknown => None,
LogicalKey::Character(ch) => Some(ch),
LogicalKey::Named(NamedKey::Backspace) => Some('\u{0008}'),
LogicalKey::Named(NamedKey::Tab) => Some('\u{0009}'),
LogicalKey::Named(NamedKey::Enter) => Some('\u{000D}'),
LogicalKey::Named(NamedKey::Escape) => Some('\u{001B}'),
LogicalKey::Named(NamedKey::Delete) => Some('\u{007F}'),
LogicalKey::Named(_) => None,
}
}
}

#[derive(Debug, Clone, Copy)]
pub enum NamedKey {
// Modifier Keys
Alt,
AltGraph,
CapsLock,
Control,
Fn,
FnLock,
Super,
NumLock,
ScrollLock,
Shift,
Symbol,
SymbolLock,

// Whitespace Keys
Enter,
Tab,

// Navigation Keys
ArrowDown,
ArrowLeft,
ArrowRight,
ArrowUp,
End,
Home,
PageDown,
PageUp,

// Editing Keys
Backspace,
Clear,
Copy,
CrSel,
Cut,
Delete,
EraseEof,
ExSel,
Insert,
Paste,
Redo,
Undo,

// UI Keys
ContextMenu,
Escape,
Pause,
Play,
Select,
ZoomIn,
ZoomOut,

// Device Keys
PrintScreen,

// General-Purpose Function Keys
F1,
F2,
F3,
F4,
F5,
F6,
F7,
F8,
F9,
F10,
F11,
F12,
F13,
F14,
F15,
F16,
F17,
F18,
F19,
F20,
F21,
F22,
F23,
F24,
F25,
F26,
F27,
F28,
F29,
F30,
F31,
F32,
F33,
F34,
F35,
}

#[derive(Debug, Clone, Copy)]
pub enum KeyLocation {
Standard = 0,
Left = 1,
Right = 2,
Numpad = 3,
}
Loading