Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Apply correct metric configs in GenericAppenderFactory #12366

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

XBaith
Copy link

@XBaith XBaith commented Feb 21, 2025

The DataWriter created using GenericAppenderFactory in the application does not configure metric writing behavior according to the table's properties.

Based on previous modifications, such as FlinkAppenderFactory, this class should also invoke the correct API to configure the relevant properties

@XBaith
Copy link
Author

XBaith commented Feb 21, 2025

The specific cause of the issue can be referenced in apache/amoro#3273

@XBaith XBaith force-pushed the fix-generic-appender-metric-configs branch 2 times, most recently from 52ed81c to c9dfece Compare February 21, 2025 12:54
@XBaith XBaith force-pushed the fix-generic-appender-metric-configs branch from c9dfece to d0ac6f4 Compare February 21, 2025 16:12
@XBaith XBaith force-pushed the fix-generic-appender-metric-configs branch from d0ac6f4 to 00f3e02 Compare February 21, 2025 16:21
Comment on lines 121 to 127
MetricsConfig metricsConfig;
if (table == null) {
metricsConfig = MetricsConfig.fromProperties(config);
} else {
metricsConfig = MetricsConfig.forTable(table);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit questionable for me.
Do we really want to disregard the config properties specifically set by the set/setAll method if the table has provided?
We should decide on the expected behavior and make sure that we don't get wrong configuration. Maybe by throwing an exception in the set methods when the table is set?

@pvary
Copy link
Contributor

pvary commented Feb 21, 2025

Please add some tests to ensure that the metrics are honored if the table is set

@XBaith XBaith requested a review from pvary February 21, 2025 17:35
@XBaith XBaith force-pushed the fix-generic-appender-metric-configs branch from 5eeeabd to 27263c1 Compare February 21, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants