Skip to content

[294] Optimise PostgresMetadataFetcher#283

Open
suddendust wants to merge 13 commits into
hypertrace:mainfrom
suddendust:optimiseMetaFetcher
Open

[294] Optimise PostgresMetadataFetcher#283
suddendust wants to merge 13 commits into
hypertrace:mainfrom
suddendust:optimiseMetaFetcher

Conversation

@suddendust
Copy link
Copy Markdown
Contributor

@suddendust suddendust commented Mar 26, 2026

Description

I observed that querying information_schema for table metadata was much slower than querying pg_catalog for the same. See the following logs:

With information_schema:

2026-03-26 06:45:10.459 [pool-5-thread-3] INFO  o.h.c.d.p.PostgresLazyilyLoadedSchemaRegistry - {entityId=7a44307e-b93b-37a9-b53d-75fc07c713c3, entityType=API, requestId=b, shadowEventId=6bae0afe-6774-4cf5-b314-359ff396bb27, tenantId=3e761879-c77b-4d8f-a075-62ff28e8fa8c, writeType=CREATE_OR_REPLACE} Successfully loaded schema for table: entities_api_shadow, columns: 133, durationMs: 39

With pg_catalog:

2026-03-26 06:45:10.459 [pool-5-thread-3] INFO  o.h.c.d.p.PostgresLazyilyLoadedSchemaRegistry - {entityId=7a44307e-b93b-37a9-b53d-75fc07c713c3, entityType=API, requestId=b, shadowEventId=6bae0afe-6774-4cf5-b314-359ff396bb27, tenantId=3e761879-c77b-4d8f-a075-62ff28e8fa8c, writeType=CREATE_OR_REPLACE} Successfully loaded schema for table: entities_api_shadow, columns: 133, durationMs: 39

This is ~ 26x faster.

Testing

Tested both version in an sandbox env.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

Documentation

https://dba.stackexchange.com/questions/302587/what-is-faster-pg-catalog-or-information-schema

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.96%. Comparing base (f25bebd) to head (ebbb1c3).

❗ There is a different number of reports uploaded between BASE (f25bebd) and HEAD (ebbb1c3). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (f25bebd) HEAD (ebbb1c3)
integration 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               main     #283       +/-   ##
=============================================
- Coverage     81.63%   55.96%   -25.68%     
+ Complexity     1549     1548        -1     
=============================================
  Files           242      242               
  Lines          7450     7442        -8     
  Branches        720      719        -1     
=============================================
- Hits           6082     4165     -1917     
- Misses          916     2855     +1939     
+ Partials        452      422       -30     
Flag Coverage Δ
integration ?
unit 55.96% <100.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@suddendust suddendust changed the title Optimise PostgresMetadataFetcher [294] Optimise PostgresMetadataFetcher Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant