[Meta][Autocomplete Improvement] Major tracker for autocomplete improvement issues and tasks

[ananzh]

Infrastructure and foundation

Task 1: Monaco Editor Upgrade: From 0.17.0 to 0.52.0

Current Limitations & Future Benefits

The current Monaco Editor version 0.17.0 has several limitations that impact the development experience and functionality in OpenSearch Dashboards:

• Limited Suggestion Control: The current version offers minimal control over code suggestions, with only basic options like enabling/disabling word-based suggestions.
• Outdated UI Components: The suggestion widget and other UI elements use older styling approaches that are difficult to customize.
• Inflexible API: The completion provider API lacks context-awareness and granular control options.
• Outdated Icon System: Uses an image-based sprite system instead of modern font-based icons.

Upgrading to version 0.52.0 will bring significant improvements:

• Enhanced Inline Completions: Granular word-based suggestions with options for current document, matching documents, or all documents.
• Improved Suggestion Widget: Preview functionality, better handling of deprecated items, and forward stability for consistent suggestions.
• Modern API: Provider prioritization with yieldsToGroupIds, better context-awareness, and more control over suggestion behavior.
• Modern UI Components: Codicon font-based icon system and improved theming capabilities.

Technical Challenges

The upgrade process involves several significant technical challenges:

1. Breaking API Changes: The Monaco Editor API has undergone substantial changes between versions.
2. Modern JavaScript Features: Newer Monaco versions use modern JavaScript syntax (optional chaining, static blocks, nullish coalescing) that may require additional transpilation.
3. Module System Changes: Differences in how modules are imported and exported between versions.
4. Worker Implementation Changes: Updates to how web workers are configured and used.
5. UI Component Structure Changes: The DOM structure of components like the suggestion widget has been redesigned.

Sub Tasks

• Research on monaco-editor
• Upgrade to 0.30.1
• PR: Bump monaco-editor from 0.17.0 to 0.30.1 #9497
• Bump monaco-editor from 0.30.1 to 0.52.0 #9573

Task 2: Refactor Automplete Providers

• Refactor Automplete Providers #9584