Skip to content

[v2] Lazily initialize built-in plugins to reduce command execution time#10363

Open
aemous wants to merge 28 commits into
v2from
lazy-plugin-initialization
Open

[v2] Lazily initialize built-in plugins to reduce command execution time#10363
aemous wants to merge 28 commits into
v2from
lazy-plugin-initialization

Conversation

@aemous

@aemous aemous commented May 29, 2026

Copy link
Copy Markdown
Contributor

Issue #, if available:

Notes:

Description of changes:

  • Merge lazy-plugin-initialization feature branch to v2: Switch from eagerly importing and initializing all built-in plugins eagerly to lazy-loading them when relevant events are emitted, command(s) are invoked, or help docs are generated. These changes have been pre-approved.

Description of tests:

  • Successfully ran pre-production build workflow (see Build Internal GitHub Action).

Benchmarking data:

A prototype for this solution on an m7i.xlarge EC2 instance with 2,000 iterations shows an average 112.1 ms (22.02%) reduction of execution time for the aws --version command (p < 0.0001), and an average 108 ms (18%) reduction of execution time for the aws sts get-caller-identity command (p < 0.0001).

We expect similar numbers for most modeled commands. Commands that use more plugins at runtime will generally have higher execution time, since more plugins would be loaded.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

aemous and others added 25 commits May 11, 2026 16:30
[v2] Implement LazyCommand for deferring modules until their commands are invoked, and make PrefixTrie public
[v2] Implement event handlers registry for built-in plugins
@aemous aemous requested a review from a team May 29, 2026 20:11
@aemous aemous requested a review from a team as a code owner May 29, 2026 20:12
@aemous aemous added v2 performance This PR or issue is related to performance. labels May 29, 2026
@aemous aemous changed the title [NOT READY] [v2] Lazily initialize built-in plugins to reduce command execution time [v2] Lazily initialize built-in plugins to reduce command execution time Jun 1, 2026
@aemous aemous force-pushed the lazy-plugin-initialization branch from bb63556 to 34dfb7e Compare June 5, 2026 18:55

@kdaily kdaily left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of small clarifications, otherwise looks good. There's still a merge conflict that needs to be resolved as well.

'xml.etree',
'pipes',
'colorama',
'awscli.handlers',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this still need to be here, since it was removed? PR build was successful, does a PyInstaller-based installation raise any errors or warnings at install or runtime?

Comment thread awscli/lazy_emitter.py
Comment thread awscli/lazy_emitter.py
# Mark all building-command-table.main entries as initialized so
# _ensure_initialized never imports them. The ops list fully
# accounts for all plugins registered against this event.
for entry in PLUGIN_REGISTRY.get('building-command-table.main', []):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we always want to use the global PLUGIN_REGISTRY, since this class takes a parameter plugin_registry?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance This PR or issue is related to performance. v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants