Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize balance-related queries with a cache #2383

Open
wants to merge 94 commits into
base: master
Choose a base branch
from
Open

Optimize balance-related queries with a cache #2383

wants to merge 94 commits into from

Commits on Oct 14, 2024

  1. Add basic balances functionality

    @rafal-ch
    rafal-ch committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    4a70f52 View commit details
    Browse the repository at this point in the history

  2. Add support for querying all balances for user

    @rafal-ch
    rafal-ch committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    7d472fb View commit details
    Browse the repository at this point in the history

  3. Adding balances_indexation_progress to DB metadata

    @rafal-ch
    rafal-ch committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    4296fd4 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. Attempt at migrating the metadata to store the indexation progress

    @rafal-ch
    rafal-ch committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    77d0f16 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Merge remote-tracking branch 'upstream/master' into 1965_balances

    @rafal-ch
    rafal-ch committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    525e6f9 View commit details
    Browse the repository at this point in the history

  2. Hack the replace_forced() and commit_changes_forced in

    @rafal-ch
    rafal-ch committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    8eba5d7 View commit details
    Browse the repository at this point in the history

  3. Introduce ForcedCommitDatabase

    @rafal-ch
    rafal-ch committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    c083f45 View commit details
    Browse the repository at this point in the history

  4. Update dependencies

    @rafal-ch
    rafal-ch committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    6b76c37 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. DB metadata can track multiple indexation progresses

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    f246cd0 View commit details
    Browse the repository at this point in the history

  2. into_genesis() attemt

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7472db7 View commit details
    Browse the repository at this point in the history

  3. Add some TODOs with ideas for the future

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    b4d2e0f View commit details
    Browse the repository at this point in the history

  4. Use double_key! macro to define the balances key

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    d38fdb3 View commit details
    Browse the repository at this point in the history

  5. Add basic_storage_tests! for Balances

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    292acab View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'upstream/master' into 1965_balances

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    651bcc8 View commit details
    Browse the repository at this point in the history

  7. Balances DB stores separate information for coins and messages

    @rafal-ch
    rafal-ch committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    8342c8f View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Fix the recursive call

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    9a9f120 View commit details
    Browse the repository at this point in the history

  2. Init indexation progresses with 0 upon metadata migration

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    6aa9325 View commit details
    Browse the repository at this point in the history

  3. Remove debug prints

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    b153db4 View commit details
    Browse the repository at this point in the history

  4. Store incoming balance in the new Balances DB

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    512a8a3 View commit details
    Browse the repository at this point in the history

  5. Read balance from the new Balances database

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    adf9e2a View commit details
    Browse the repository at this point in the history

  6. Update coin balance, don't overwrite

    @rafal-ch
    rafal-ch committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    344ca90 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Use more detailed IndexationStatus, not just block height

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    f73dbed View commit details
    Browse the repository at this point in the history

  2. Add processing of MessageImported

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    80da09b View commit details
    Browse the repository at this point in the history

  3. Simplify processing of coins and message amounts

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    ad5216d View commit details
    Browse the repository at this point in the history

  4. Extract increase_balance()

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    c784e44 View commit details
    Browse the repository at this point in the history

  5. Store coin and message balances separately

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    5762665 View commit details
    Browse the repository at this point in the history

  6. Clean up column naming

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    5595985 View commit details
    Browse the repository at this point in the history

  7. Add test for coin balances

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    38dd8d6 View commit details
    Browse the repository at this point in the history

  8. Support both coins and messages in the new balance system

    @rafal-ch
    rafal-ch committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    530bcaf View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Merge remote-tracking branch 'upstream/master' into 1965_balances

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5604f30 View commit details
    Browse the repository at this point in the history

  2. update comment

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    562c087 View commit details
    Browse the repository at this point in the history

  3. Remove the database migration hack

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    82f7a16 View commit details
    Browse the repository at this point in the history

  4. Remove code related to handling base asset id

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    7d0b014 View commit details
    Browse the repository at this point in the history

  5. Clean-up of the balance update code

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    1fe3bba View commit details
    Browse the repository at this point in the history

  6. Remove unused tests

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fe60c98 View commit details
    Browse the repository at this point in the history

  7. Handle overflow in balance query

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    adef78e View commit details
    Browse the repository at this point in the history

  8. Clippy and formatting

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    9409d52 View commit details
    Browse the repository at this point in the history

  9. Update db tests after reverting changes to metadata

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    13b6db9 View commit details
    Browse the repository at this point in the history

  10. Add comment

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    55f9025 View commit details
    Browse the repository at this point in the history

  11. Add support for balances() (plural) query

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    d215844 View commit details
    Browse the repository at this point in the history

  12. Handle error properly instead of using expect()

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    53131a3 View commit details
    Browse the repository at this point in the history

  13. Add support for direction in balances query

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    1e24aaa View commit details
    Browse the repository at this point in the history

  14. Remove safety check from the balance query

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    eeea199 View commit details
    Browse the repository at this point in the history

  15. Fix balance logging

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    e77f33b View commit details
    Browse the repository at this point in the history

  16. Include message balance in balances() query

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    c9b7e29 View commit details
    Browse the repository at this point in the history

  17. Revert "Fix balance logging" 

    This reverts commit e77f33b.
    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    076cb42 View commit details
    Browse the repository at this point in the history

  18. Revert a couple of unintentional changes

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    f5e2a6d View commit details
    Browse the repository at this point in the history

  19. Rename database Balances to CoinBalances

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    d1c6fcc View commit details
    Browse the repository at this point in the history

  20. Update comments

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    8cc0280 View commit details
    Browse the repository at this point in the history

  21. Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

    @rafal-ch
    rafal-ch committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    ca32195 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Fix formatting

    @rafal-ch
    rafal-ch committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    f082291 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

    @rafal-ch
    rafal-ch committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    f598395 View commit details
    Browse the repository at this point in the history

  2. Add comment about potential discount on the balances query cost

    @rafal-ch
    rafal-ch committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    8ce4550 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. Update DB metadata with balances info

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    4e84ac3 View commit details
    Browse the repository at this point in the history

  2. Support both 'new' and 'old' way of querying balances

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    9577f30 View commit details
    Browse the repository at this point in the history

  3. Support both 'new' and 'old' way of querying balance

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    a16ef36 View commit details
    Browse the repository at this point in the history

  4. Do not touch the onchain DB metadata

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    80fb852 View commit details
    Browse the repository at this point in the history

  5. Write balances cache information do off-chain db on genesis

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    c94a484 View commit details
    Browse the repository at this point in the history

  6. Use balances cache in GraphQL api

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    ddf0571 View commit details
    Browse the repository at this point in the history

  7. Clean-up error handling

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    9078b05 View commit details
    Browse the repository at this point in the history

  8. Satisfy clippy and fmt

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    3b6b8c6 View commit details
    Browse the repository at this point in the history

  9. generic update function

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    5162936 View commit details
    Browse the repository at this point in the history

  10. Extract processing of balance updates

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    f736ecb View commit details
    Browse the repository at this point in the history

  11. Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

    @rafal-ch
    rafal-ch committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    d880e0a View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Use instead of for balances key

    @rafal-ch
    rafal-ch committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    6927f03 View commit details
    Browse the repository at this point in the history

  2. Revert the supporsed-to-be temporary change that disabled warnings

    @rafal-ch
    rafal-ch committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    66e26d9 View commit details
    Browse the repository at this point in the history

  3. Remove unused lifetime from HasIndexation trait

    @rafal-ch
    rafal-ch committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    a87698a View commit details
    Browse the repository at this point in the history

  4. Simplify trait bounds for HasIndexation trait

    @rafal-ch
    rafal-ch committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    ddadd80 View commit details
    Browse the repository at this point in the history

  5. Further simplify trait bounds for HasIndexation trait

    @rafal-ch
    rafal-ch committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    ccadf41 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2024

  1. Split and rename the HasIndexation trait

    @rafal-ch
    rafal-ch committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    91e8abc View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. WIP - bail when unable to update balance due to overflow

    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    ab7808f View commit details
    Browse the repository at this point in the history

  2. Prefer core::fmt::Display over std::fmt::Display

    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    90fa259 View commit details
    Browse the repository at this point in the history

  3. Use named const instead of plain true in genesis importer for clarity

    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    0b39319 View commit details
    Browse the repository at this point in the history

  4. Remove reduntant write to DatabaseMetadata::V2

    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    4d6e17a View commit details
    Browse the repository at this point in the history

  5. Ensure all indexations are enabled at genesis

    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    8f3e817 View commit details
    Browse the repository at this point in the history

  6. Fix produce_block__raises_gas_price after balance overflow check wa… 

    …s added
    @rafal-ch
    rafal-ch committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    895d9da View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. Fix more integration tests to work with the balance overflow checks

    @rafal-ch
    rafal-ch committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    de11071 View commit details
    Browse the repository at this point in the history

  2. Fix typo

    @rafal-ch
    rafal-ch committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    71226d4 View commit details
    Browse the repository at this point in the history

  3. Fix BLOB integration tests to work with the balance overflow checks

    @rafal-ch
    rafal-ch committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    474e207 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2024

  1. Revert "Fix BLOB integration tests to work with the balance overflow … 

    …checks"

This reverts commit 474e207.
    @rafal-ch
    rafal-ch committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    ca6057d View commit details
    Browse the repository at this point in the history

  2. Revert "Fix more integration tests to work with the balance overflow … 

    …checks"

This reverts commit de11071.
    @rafal-ch
    rafal-ch committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    4d43038 View commit details
    Browse the repository at this point in the history

  3. Revert "Fix produce_block__raises_gas_price after balance overflow … 

    …check was added"

This reverts commit 895d9da.
    @rafal-ch
    rafal-ch committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    e3d898d View commit details
    Browse the repository at this point in the history

  4. Do not bail when balances cannot be updated, log the error instead

    @rafal-ch
    rafal-ch committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    9848f64 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Total balance of assets is now represented as u128

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    dae0af5 View commit details
    Browse the repository at this point in the history

  2. Infer types in process_balances_update()

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    facbd2e View commit details
    Browse the repository at this point in the history

  3. Simplify trait bounds in update_balances()

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    9b8e491 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    9ea9c02 View commit details
    Browse the repository at this point in the history

  5. Fix formatting

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    54087fc View commit details
    Browse the repository at this point in the history

  6. Satisfy Clippy

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    20081fa View commit details
    Browse the repository at this point in the history

  7. Update tests

    @rafal-ch
    rafal-ch committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    8b9f4e8 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. Asset balance queries now return U128 instead of U64.

    @rafal-ch
    rafal-ch committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    e819e23 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2024

  1. Log errors when balance cannot be calculated in the legacy calculatio… 

    …n flow (without indexation)
    @rafal-ch
    rafal-ch committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    de16099 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. Mention an balance overflow follow-up issue in the comments

    @rafal-ch
    rafal-ch committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    4f307e3 View commit details
    Browse the repository at this point in the history

  2. Remove the TODO comment

    @rafal-ch
    rafal-ch committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    66d5948 View commit details
    Browse the repository at this point in the history