Skip to content

perf: sparse Prop tracking and @value types — up to 6.7× faster diffs#6296

Open
FeodorFitsner wants to merge 15 commits intomainfrom
object-patch-perf-2
Open

perf: sparse Prop tracking and @value types — up to 6.7× faster diffs#6296
FeodorFitsner wants to merge 15 commits intomainfrom
object-patch-perf-2

Conversation

@FeodorFitsner
Copy link
Contributor

@FeodorFitsner FeodorFitsner commented Mar 11, 2026

Close #6098
Fix #6270
Fix #6117

Problem

Every page.update() call serializes the control tree by computing a diff between
the current and previous state. The diff machinery had two hot-path bottlenecks:

1. Non-frozen diff — O(all fields) per control
The "what changed?" pass called dataclasses.fields(ctrl) and compared every field
one-by-one on every update. A control with 20 fields caused 20 comparisons even when
only 1 field changed. On a list of 300 controls that's 6,000 comparisons to produce
~600 ops.

2. Frozen diff — O(all fields) per control
The frozen (declarative / build()) diff path also iterated dataclasses.fields(dst)
for every control, comparing all scalars/objects regardless of which ones actually
differed. In the sunflower benchmark this produced O(n²) behaviour: 500 controls ×
20+ fields on every update.

3. Value type fields not tracked at all
Nested value objects (ctrl.style.color, ctrl.alignment.x) were re-serialized in
full on every update because there was no record of which sub-fields had changed.


Solution

Sparse Prop descriptors

A new Prop descriptor replaces plain dataclass field storage for public properties.
On every __set__ it writes to two per-instance dicts:

  • _values — current value (replaces __dict__ slot)
  • _dirty — fields mutated since the last diff (cleared after each serialization pass)

_dirty is cleared after construction so only post-construction mutations are visible.

@value decorator for value types

A new @value decorator (analogous to @control) applies @dataclass + Prop
descriptors to the ~150 data-only types (Alignment, BorderRadius, BoxDecoration,
Theme*, etc.). These types now also carry _values/_dirty, so nested mutations
are tracked precisely.

A Value abstract base class is registered, enabling isinstance(obj, Value) checks
in the diff engine to recurse into nested value objects.

Two-loop frozen diff

The single for f in dataclasses.fields(dst) loop was replaced with two focused loops:

# 1. Structural fields (controls, lists, dicts) — always compared
for fname in structural:
    ...

# 2. Only Prop fields set in src or dst — proportional to actual changes
for fname in {**src._values, **dst._values}:
    ...

The {**src._values, **dst._values} union is O(changed fields), not O(all fields),
and preserves definition order.


API changes

Before After
All data types were plain @dataclass All data types use @value

@value and Value are exported from the top-level flet namespace as
ft.value / ft.Value.

Event classes (*Event, inherits ControlEvent) remain plain @dataclass
they are immutable, short-lived, and need no tracking.


Benchmarks (Python 3.14.2, Apple Silicon)

Scenario 1 — frozen, all-keyed list (shuffled)

size before after speedup
150 1.662 ms 1.265 ms 1.3×
300 4.281 ms 3.360 ms 1.3×
500 9.279 ms 7.809 ms 1.2×

Scenario 2 — frozen, mixed keyed/unkeyed list (50% keyed)

size before after speedup
150 1.475 ms 0.726 ms 2.0×
300 2.852 ms 1.503 ms 1.9×
500 4.657 ms 2.452 ms 1.9×

Scenario 3 — frozen, 20-field controls, only 2 fields changed

controls before after speedup
100 1.718 ms 0.538 ms 3.2×
300 5.625 ms 1.666 ms 3.4×

Scenario 4 — non-frozen, list rebuilt each update (keyed)

size before after speedup
100 1.624 ms 0.551 ms 2.9×
300 7.613 ms 1.675 ms 4.5×

Scenario 5 — non-frozen, sparse field changes (2 of 20 changed)

controls before after speedup
100 0.751 ms 0.510 ms 1.5×
300 2.208 ms 1.501 ms 1.5×

Scenario 6 — sunflower (500 seeds, +10 per update)

fill before after speedup
250 29.248 ms 4.373 ms 6.7×
400 28.924 ms 4.364 ms 6.6×

The sunflower case was the most severe regression — previously O(n²) due to scanning
all fields of every control during frozen diff; now reduced to O(changed fields only).
The non-frozen rebuilt-list case improves up to 4.5× from better key-based
matching. Controls with many fields but few changes (the typical real-world update)
see 3–3.4× improvement from the two-loop frozen diff skipping unchanged _values.


Files changed

91 files, +1274 / −533 lines across flet and all flet-* extension packages.

Summary by Sourcery

Optimize control diffing and serialization by introducing sparse property tracking for controls and value types and updating the diff/patch pipeline to operate on changed fields only.

New Features:

  • Add a Prop-based sparse property tracking system for controls and value types, storing only non-default values and tracking mutated fields.
  • Introduce a @value decorator and Value base type for data-only value objects, and export them from the top-level flet namespace.
  • Add a standalone object_patch benchmark script covering multiple frozen and non-frozen update scenarios.

Enhancements:

  • Rework dataclass diffing to use dirty tracking for in-place updates and a two-loop frozen diff that compares only structural and non-default Prop fields.
  • Revise list reconciliation to a hybrid keyed algorithm that better preserves identity for dataclasses and explicit keys while using positional diffs for primitive lists.
  • Optimize protocol serialization to emit only non-default Prop values and structural fields via _values/_structural_fields metadata instead of scanning all dataclass fields.
  • Update patch_dataclass to write Prop fields directly into _values, avoiding unnecessary dirty-tracking and notifications for Dart-originated patches.
  • Refine control key access and various logging statements for lower overhead and more consistent formatting.
  • Adjust internal helpers (e.g., classproperty typing) and page service cleanup logging for clarity and correctness.

Documentation:

  • Document the new @value decorator in the Python docs and add it to the mkdocs navigation.

Tests:

  • Extend and update object diff tests to cover the new dirty-tracking semantics and order-insensitive patch comparisons.
  • Add and update integration tests across controls and extension packages to reflect @value usage and minor API/text changes.

Chores:

  • Convert numerous data-only dataclasses across core controls and extension packages (themes, geometry, animation, gradients, charts, maps, camera, ads, video, geolocator, secure storage, etc.) to use @value instead of plain @DataClass.
  • Tighten exports and import organization in testing utilities and various modules, and add a local typo configuration for Python packages.

Introduce sparse property tracking via a Prop descriptor, TrackedValue marker and @Tracked decorator; install Prop descriptors on BaseControl and via _apply_control, and add _values/_dirty fields (cleared after init) to record only non-default properties. Optimize object-patch machinery: fast-path handling in _control_setattr for Prop-managed fields, faster get_control_key reading from _values, and major DiffBuilder improvements (dataclass fast-path using _values/_prop_defaults, hybrid keyed list reconciliation, reduced per-field descriptor overhead). Apply @Tracked to ButtonStyle, TextStyle and StrutStyle. Add a standalone bench_object_patch.py benchmark to measure performance scenarios. These changes reduce call overhead and make diffs for sparse/non-frozen/frozen updates significantly faster and more memory-efficient.
Replace f-string based logging calls with %-style deferred formatting across the Python package to avoid eager string interpolation and to follow logging best practices. This standardizes logging usage and can improve performance when log levels suppress messages. Modified files: sdk/python/packages/flet/src/flet/app.py, flet/controls/base_control.py, flet/controls/page.py, flet/messaging/flet_socket_server.py, flet/messaging/pyodide_connection.py, flet/messaging/session.py, flet/pubsub/pubsub_hub.py.
Rename internal tracked API to value: TrackedValue -> Value and @Tracked -> @value. Updated imports and exports, docstrings, and references across modules (base_control, __init__, buttons, text_style, object_patch, messaging/protocol, utils/object_model) to use the new names and wording. Note: this is a breaking API rename for consumers — replace usages of @Tracked and TrackedValue with @value and Value respectively; __all__ now exports Value and value.
Replace the single dataclasses.fields iteration in DiffBuilder with two focused loops: one over structural fields and one over the union of active _values keys. This avoids per-field Prop.__get__ overhead, skips defaulted prop fields efficiently, and handles event fields via _event_fields. Also normalize prop_defaults to an empty dict. Add cmp_ops_unordered helper to tests/common.py and update tests to use it where patch operation order is commutative.
Convert many plain dataclass value types to flet's sparse-tracked value types by replacing @DataClass with @ft.value/@value across multiple SDK Python packages (charts, camera, ads, audio-recorder, color-pickers, geolocator, map, secure-storage, video, code-editor, and core flet controls). Move sparse property tracking (Prop, Value, _install_props, value decorator) out of base_control into a new value_types module and re-export it to avoid circular imports. Also remove unused imports (e.g. urllib.error), tidy imports/usages in integration tests, and adjust imports (remove unneeded dataclass imports, add value import) to align with the new value-based implementation.
Add a new documentation page for the @value type at sdk/python/packages/flet/docs/types/value.md (uses class_all_options("flet.value")). Update sdk/python/packages/flet/mkdocs.yml to include the new types/value.md entry in the Types navigation.
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 11, 2026

Deploying flet-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4b0eb36
Status: ✅  Deploy successful!
Preview URL: https://1ca169af.flet-docs.pages.dev
Branch Preview URL: https://object-patch-perf-2.flet-docs.pages.dev

View logs

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 11, 2026

Deploying flet-examples with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4b0eb36
Status: ✅  Deploy successful!
Preview URL: https://869c0f14.flet-examples.pages.dev
Branch Preview URL: https://object-patch-perf-2.flet-examples.pages.dev

View logs

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces sparse property tracking (Prop) and a new @value decorator to significantly speed up control diffing/serialization by tracking only mutated fields (_dirty) and storing only non-default values (_values), then updates the diff engine and many value/config types across core and extension packages to use these mechanisms.

Changes:

  • Add Prop/_values/_dirty tracking for @control and new @value data-only types, plus Value marker and top-level exports.
  • Optimize frozen + non-frozen diffing (two-loop frozen diff, improved keyed list reconciliation, updated non-frozen in-place diff behavior).
  • Update MsgPack serialization, patching logic, tests/docs, and convert many dataclass config/types in core + extensions to @value / ft.value.

Reviewed changes

Copilot reviewed 93 out of 93 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
sdk/python/packages/flet/tests/test_patch_dataclass.py Update patching test to validate _dirty tracking/clearing semantics.
sdk/python/packages/flet/tests/test_object_diff_memory_churn.py Import ordering cleanup.
sdk/python/packages/flet/tests/test_object_diff_in_place.py Update test expectations from __changes to _dirty.
sdk/python/packages/flet/tests/test_object_diff_frozen.py Switch to unordered op comparisons + _dirty assertions.
sdk/python/packages/flet/tests/test_datetime.py Remove unused flet import.
sdk/python/packages/flet/tests/common.py Add cmp_ops_unordered() helper for order-insensitive patch assertions.
sdk/python/packages/flet/tests/bench_object_patch.py Add standalone benchmark for ObjectPatch scenarios.
sdk/python/packages/flet/src/flet/utils/object_model.py Make patch_dataclass() write Prop fields directly into _values (bypassing Prop.__set__).
sdk/python/packages/flet/src/flet/utils/classproperty.py Simplify typing imports and update type annotations.
sdk/python/packages/flet/src/flet/testing/init.py Consolidate imports and reorder __all__.
sdk/python/packages/flet/src/flet/pubsub/pubsub_hub.py Convert debug logging to parameterized style.
sdk/python/packages/flet/src/flet/messaging/session.py Convert logging to parameterized style in hot paths and error handling.
sdk/python/packages/flet/src/flet/messaging/pyodide_connection.py Convert transport logging to parameterized style.
sdk/python/packages/flet/src/flet/messaging/protocol.py Add _values/_structural_fields-based serialization fast path and root-default handling.
sdk/python/packages/flet/src/flet/messaging/flet_socket_server.py Convert server logging to parameterized style.
sdk/python/packages/flet/src/flet/controls/value_types.py New Prop + @value implementation and shared helpers for sparse tracking.
sdk/python/packages/flet/src/flet/controls/types.py Convert multiple foundational data types to @value.
sdk/python/packages/flet/src/flet/controls/transform.py Convert transform value types to @value.
sdk/python/packages/flet/src/flet/controls/theme.py Convert theme-related value types to @value.
sdk/python/packages/flet/src/flet/controls/text_style.py Convert text style value types to @value.
sdk/python/packages/flet/src/flet/controls/painting.py Convert painting-related value types to @value.
sdk/python/packages/flet/src/flet/controls/page.py Parameterize logging in service registry updates.
sdk/python/packages/flet/src/flet/controls/padding.py Convert Padding to @value.
sdk/python/packages/flet/src/flet/controls/object_patch.py Major diff-engine refactor: _dirty tracking, hybrid keyed list diff, two-loop frozen diff, key fast-path.
sdk/python/packages/flet/src/flet/controls/material/tooltip.py Convert tooltip value type to @value.
sdk/python/packages/flet/src/flet/controls/material/textfield.py Convert InputFilter to @value and adjust imports.
sdk/python/packages/flet/src/flet/controls/material/tabs.py Convert UnderlineTabIndicator to @value.
sdk/python/packages/flet/src/flet/controls/material/menu_bar.py Convert MenuStyle to @value.
sdk/python/packages/flet/src/flet/controls/material/auto_complete.py Convert AutoCompleteSuggestion to @value.
sdk/python/packages/flet/src/flet/controls/margin.py Convert Margin to @value.
sdk/python/packages/flet/src/flet/controls/keys.py Avoid circular import by lazily applying @value to key classes.
sdk/python/packages/flet/src/flet/controls/gradients.py Convert gradient value types to @value.
sdk/python/packages/flet/src/flet/controls/geometry.py Convert Size/Rect to @value.
sdk/python/packages/flet/src/flet/controls/duration.py Convert Duration to @value.
sdk/python/packages/flet/src/flet/controls/device_info.py Convert device info dataclasses to @value and fix a doc example string.
sdk/python/packages/flet/src/flet/controls/core/markdown.py Convert markdown style/theme helper types to @value.
sdk/python/packages/flet/src/flet/controls/core/canvas/path.py Convert nested path element dataclasses to @value.
sdk/python/packages/flet/src/flet/controls/buttons.py Convert button/style-related value types to @value.
sdk/python/packages/flet/src/flet/controls/box.py Convert box decoration/filter/shadow/etc. to @value.
sdk/python/packages/flet/src/flet/controls/border_radius.py Convert BorderRadius to @value.
sdk/python/packages/flet/src/flet/controls/border.py Convert border value types to @value.
sdk/python/packages/flet/src/flet/controls/blur.py Convert Blur to @value.
sdk/python/packages/flet/src/flet/controls/base_control.py Re-export Prop/Value/value, install props on controls, and add _values/_dirty fields to BaseControl.
sdk/python/packages/flet/src/flet/controls/animation.py Convert animation value types to @value.
sdk/python/packages/flet/src/flet/controls/alignment.py Convert Alignment to @value.
sdk/python/packages/flet/src/flet/app.py Parameterize logging for app/web-server startup paths.
sdk/python/packages/flet/src/flet/init.py Export Value/value at top-level flet namespace.
sdk/python/packages/flet/mkdocs.yml Add docs nav entry for @value.
sdk/python/packages/flet/integration_tests/examples/material/test_slider.py Import ordering cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_reorderable_list_view.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_range_slider.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_radio.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_progress_ring.py Import ordering cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_progress_bar.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_popup_menu_button.py Remove unused walrus binding and whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_navigation_rail.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_navigation_drawer.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_navigation_bar.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_menu_item_button.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/material/test_menu_bar.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/cupertino/test_cupertino_timer_picker.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/examples/core/test_row.py Import ordering cleanup.
sdk/python/packages/flet/integration_tests/examples/core/test_placeholder.py Whitespace cleanup.
sdk/python/packages/flet/integration_tests/controls/material/test_progress_bar.py Remove unused import and whitespace cleanup.
sdk/python/packages/flet/docs/types/value.md New docs page for @value.
sdk/python/packages/flet/docs/extras/macros/pypi_index.py Remove unused urllib.error import.
sdk/python/packages/flet-video/src/flet_video/types.py Convert flet-video config dataclasses to @ft.value.
sdk/python/packages/flet-secure-storage/src/flet_secure_storage/types.py Convert secure-storage option dataclasses to @ft.value.
sdk/python/packages/flet-map/src/flet_map/types.py Convert map value/config dataclasses to @ft.value (keep dataclass for events).
sdk/python/packages/flet-geolocator/src/flet_geolocator/types.py Convert geolocator value/config dataclasses to @ft.value.
sdk/python/packages/flet-color-pickers/src/flet_color_pickers/color_picker.py Convert HSV color dataclass to @ft.value.
sdk/python/packages/flet-code-editor/src/flet_code_editor/types.py Convert GutterStyle to @ft.value and adjust language enum member.
sdk/python/packages/flet-charts/src/flet_charts/types.py Convert chart config/value dataclasses to @ft.value.
sdk/python/packages/flet-charts/src/flet_charts/scatter_chart_spot.py Convert tooltip type to @ft.value and fix “whether” typos.
sdk/python/packages/flet-charts/src/flet_charts/scatter_chart.py Convert tooltip type to @ft.value.
sdk/python/packages/flet-charts/src/flet_charts/line_chart_data_point.py Convert tooltip type to @ft.value and fix “edge” typo.
sdk/python/packages/flet-charts/src/flet_charts/line_chart.py Convert event spot + tooltip types to @ft.value.
sdk/python/packages/flet-charts/src/flet_charts/candlestick_chart_spot.py Convert tooltip type to @ft.value.
sdk/python/packages/flet-charts/src/flet_charts/candlestick_chart.py Convert tooltip type to @ft.value and adjust example punctuation.
sdk/python/packages/flet-charts/src/flet_charts/bar_chart_rod.py Convert tooltip type to @ft.value.
sdk/python/packages/flet-charts/src/flet_charts/bar_chart.py Convert tooltip type to @ft.value.
sdk/python/packages/flet-charts/integration_tests/examples/test_scatter_chart.py Import ordering/whitespace cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_radar_chart.py Import ordering cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_plotly_chart.py Import ordering cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_pie_chart.py Remove unused imports.
sdk/python/packages/flet-charts/integration_tests/examples/test_matplotlib_chart.py Import ordering cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_line_chart.py Import ordering cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_candlestick_chart.py Import ordering cleanup.
sdk/python/packages/flet-charts/integration_tests/examples/test_bar_chart.py Import ordering cleanup.
sdk/python/packages/flet-camera/src/flet_camera/types.py Convert camera value types to @ft.value.
sdk/python/packages/flet-audio-recorder/src/flet_audio_recorder/types.py Convert recorder config/value types to @ft.value.
sdk/python/packages/flet-ads/src/flet_ads/types.py Convert ads request/style dataclasses to @ft.value.
sdk/python/_typos.toml Add project typo dictionary extension.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

…ization

Fix a typo in CodeLanguage (ROUTEROS). Update control/value change tracking to skip no-op assignments, record the original value on first write, and drop tracking when a field is reverted to its original value to produce net-zero changes. Make Prop storage sparse by removing entries that are reset to the default, avoiding sending defaults. Extend the msgpack fast-path to treat Value instances like control types for more efficient encoding. These changes reduce unnecessary notifications and payloads during updates.
@FeodorFitsner FeodorFitsner marked this pull request as ready for review March 11, 2026 20:02
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reviewed this pull request using the Sourcery rules engine

Rename CodeLanguage enum member from ROUTERS to ROUTEROS and update its value to "routeros" to correctly represent RouterOS. Also add ROUTEROS to sdk/python/_typos.toml so the term is recognized/whitelisted.
Review finding: @value claimed to register decorated classes as ft.Value, but no registration happened. That left isinstance(obj, ft.Value) false and made the msgpack fast-path gate in messaging/protocol.py unreachable for value objects.

Fix by marking each @value-decorated class as a Value subtype and add regression tests for ft.Value recognition and sparse encoding of a @value instance.
Treat an empty Map the same as a non-Map when creating a WidgetStateProperty. Previously an empty Map would bypass wrapping and could break default value handling; the check now includes j.isEmpty so empty maps are converted to {"default": j} to ensure consistent default lookups.
Remove the `controls/types` entry from the matrix in .github/workflows/macos-integration-tests.yml so that the controls/types suite is no longer executed as part of the macOS integration test jobs. This reduces the CI matrix size and excludes that specific suite from the workflow.
When end_angle is omitted, use 2*math.pi (full circle) instead of 0 to produce proper full-angle gradients. Added dart:math import in drawing.dart and gradient.dart to access math.pi. This avoids zero-length angle ranges for parsed gradients.
Replace the image_for_docs.png golden asset for the macOS submenu_button example in the Python integration tests. This updates the test/docs screenshot to reflect recent UI or styling changes.
Ensure Prop.__get__ returns the declared default for class-level access (matching dataclass field behavior) instead of the Prop descriptor itself. Add _get_raw_descriptor to fetch a class attribute from the MRO without invoking __get__, and use it in _install_props to check for and retrieve existing Prop descriptors. This prevents accidental descriptor invocation during installation and preserves intended default values (e.g. TextTheme.display_small returning None).
Replace theme-based fallbacks with Colors.black for default side colors in container and menu parsing to provide a consistent black fallback (packages/flet/lib/src/controls/container.dart and packages/flet/lib/src/utils/menu.dart). Also update the macOS submenu_button image asset (sdk/python/.../image_for_docs.png).
@FeodorFitsner FeodorFitsner added this to the 0.83.0 milestone Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: GridView with large number of items items causes significant slowdown on 0.80+ bug: problem listview lags Improve Flet Performance

2 participants