Merge Polaris-Admin to Polaris-Server #3340

flyrain · 2025-12-30T22:08:24Z

Please check the motivation here, https://lists.apache.org/thread/fs7j81kv2kn0wsh05412yk6pp7w7r5nz

Checklist

🛡️ Don't disclose security issues! (contact [email protected])
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)

dimas-b · 2026-01-06T14:12:58Z

@flyrain : CI fails? 🤔

runtime/server/src/main/java/org/apache/polaris/PolarisApplication.java

runtime/defaults/src/main/resources/application.properties

adutra · 2026-01-06T18:46:14Z

@flyrain the CI failure is an interesting one. After some investigation, it seems related to the colored output that we have by default when running server tests:

polaris/runtime/defaults/src/main/resources/application-test.properties

Line 28 in c4fc848

quarkus.console.color=true

Quarkus CLI tests seem unable to capture colored output properly.

This can be fixed in two ways: by removing the colored output completely for all tests (but that's frustrating) – or by implementing getConfigProfile() in RelationalJdbcAdminProfile and using another profile than test:

public class RelationalJdbcAdminProfile extends RelationalJdbcProfile {
  @Override
  public Map<String, String> getConfigOverrides() {...}

  @Override
  public List<TestResourceEntry> testResources() {...}

  // ADD THIS
  @Override  public String getConfigProfile() { return "cli"; }
}

The above will make the test run under the cli profile and thus solves the issue. (We might end up needing a cli-test profile one day...)

runtime/defaults/src/main/resources/application.properties

adutra · 2026-01-06T18:53:40Z

runtime/defaults/src/main/resources/application.properties

+%cli.quarkus.log.category."io.quarkus".level=ERROR
+%cli.quarkus.log.category."io.quarkus.agroal.deployment".level=ERROR
+%cli.quarkus.log.category."org.hibernate.validator".level=ERROR
+%cli.quarkus.log.category."org.apache.polaris.service.config.ServiceProducers".level=ERROR


Rather than silencing the whole class, I'd suggest modifying the maybeBootstrap method as follows:

public void maybeBootstrap( @Observes Startup event, MetaStoreManagerFactory factory, PersistenceConfiguration config, RealmContextConfiguration realmContextConfiguration) { if (ConfigUtils.isProfileActive("cli")) { return; // never bootstrap realms in CLI profile } ...

I'd also suggest moving this method to a separate class, as it is becoming quite big and complex.

SGTM. Adopted the change.

For moving it to a separate class, I'd suggest to tackle it in a followup PR given the current PR is big already.

dimas-b

Thanks for you effort, @flyrain !

I played with a local build of this PR and it seems generally usable 👍

However, I believe the PR still needs some attention in the areas of production checks and CDI in general.

Adding some more specific comments below.

.github/workflows/gradle.yml

.github/workflows/release-3-build-and-publish-artifacts.yml

...rc/main/java/org/apache/polaris/persistence/relational/jdbc/RelationalJdbcConfiguration.java

.../org/apache/polaris/persistence/relational/jdbc/RelationalJdbcProductionReadinessChecks.java

dimas-b · 2026-01-06T20:10:25Z

runtime/server/src/main/java/org/apache/polaris/admintool/PolarisAdminTool.java

  public Integer call() {
-    PrintWriter out = spec.commandLine().getOut();
-    spec.commandLine().usage(out);
+    // When invoked without a subcommand, show usage


How is it possible to invoke the Admin CLI without a subcommand now? 🤔

Both work per my test. I have removed the comment to avoid confusion.

yufei@Yufeis-MacBook-Pro-2 polaris % java -jar runtime/server/build/quarkus-app/quarkus-run.jar -h Usage: polaris-admin-tool.jar [-hV] [COMMAND] Polaris administration & maintenance tool -h, --help Show this help message and exit. -V, --version Print version information and exit. Commands: help Display help information about the specified command. bootstrap Bootstraps realms and root principal credentials. purge Purge realms and all associated entities. yufei@Yufeis-MacBook-Pro-2 polaris % java -jar runtime/server/build/quarkus-app/quarkus-run.jar purge -h Usage: polaris-admin-tool.jar purge [-hV] -r=<realm> [-r=<realm>]... Purge realms and all associated entities. -h, --help Show this help message and exit. -r, --realm=<realm> The name of a realm to purge. -V, --version Print version information and exit.

dimas-b · 2026-01-06T20:12:16Z

runtime/server/build.gradle.kts

  implementation(project(":polaris-runtime-service"))

+  // Dependencies from merged polaris-admin module
+  implementation(project(":polaris-core"))


Is this necessary? I suppose runtime/server included polaris-core before 🤔

Changed it to compileOnly, otherwise the compilation will fail.

Oh, I did not mean to imply that the dep. was wrong... I was just wondering 🙂

If core is require for complication, we should probably keep it as an implementation dep.

dimas-b · 2026-01-06T20:13:02Z

runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java

      Instance<ProductionReadinessCheck> checks,
      ReadinessConfiguration config) {
+    // Skip production readiness checks in CLI mode - they're only relevant for server deployments
+    if ("cli".equals(System.getProperty("quarkus.profile"))) {


Let's try to find a more natural way to perform readiness checks under the CLI env.

Sure, I'm open for suggestions.

It's safer to use the following idiom, as more than one profile can be specified with quarkus.profile:

if (ConfigUtils.isProfileActive("cli")) {

Fixed in the new commit. Thanks for the suggestion.

dimas-b · 2026-01-06T20:13:46Z

.asf.yaml

          - "Quarkus Tests"
          - "Quarkus Integration Tests"
-          - "Quarkus Admin Tests"
+          - "Quarkus Server Tests"


I believe we simply need to remove Quarkus Admin Tests here, not rename

Removed, however, the CI is still asking for it as a pending task. Maybe we should remove it first in a separate PR?

Filed #3370 to unblock this PR.

runtime/server/src/main/java/org/apache/polaris/admintool/config/AdminToolProducers.java

flyrain · 2026-01-07T00:54:05Z

public class RelationalJdbcAdminProfile extends RelationalJdbcProfile {
  @Override
  public Map<String, String> getConfigOverrides() {...}

  @Override
  public List<TestResourceEntry> testResources() {...}

  // ADD THIS
  @Override  public String getConfigProfile() { return "cli"; }
}

The above will make the test run under the cli profile and thus solves the issue. (We might end up needing a cli-test profile one day...)

Awesome! Thanks a lot for the suggestion, @adutra! All tests passed now!

dimas-b

from my POV this PR is almost ready to merge :) I hope a few new minor comments, below could still be squeezed in.

dimas-b · 2026-01-07T21:56:40Z

runtime/defaults/src/main/resources/application-cli.properties

+quarkus.banner.enabled=false
+
+# Logging configuration - suppress verbose output for CLI
+quarkus.log.level=WARN


Should we suppress CONSOLE log instead but keep default levels informative in case a user enabled FILE logging?

I'm open for that, but the WARN level logging was carried from the application.properties in the removed admin module,

polaris/runtime/admin/src/main/resources/application.properties

Line 67 in 476301b

quarkus.log.level=WARN

. Should we change it now?

ah, sorry, I missed that. Current code is fine for now 👍

dimas-b · 2026-01-07T21:57:54Z

runtime/distribution/bin/admin

 # Get the directory
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-cd "$SCRIPT_DIR/../admin"
+cd "$SCRIPT_DIR/../server"


for follow-up: with a single binary package, we may want to rename / restructure the binary dist for clarity.

Definitely, there are a few places needed to be changed, including doc. I will address them in followups.

runtime/distribution/bin/admin

dimas-b · 2026-01-07T22:02:34Z

runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java

      Instance<ProductionReadinessCheck> checks,
      ReadinessConfiguration config) {
+    // Skip production readiness checks in CLI mode - they're only relevant for server deployments
+    if (ConfigUtils.isProfileActive("cli")) {


This is probably ok in the interest of progress, but in general I hope we could leverage profile-specific config like polaris.production.readiness.checks.enabled=true (or false for CLI). WDYT?

That's probably a good idea, esp. if we want to group a set of methods under the same configure key, instead of spreed if (ConfigUtils.isProfileActive("cli") in multiple places. We could improve in followups.

I believe using dedicated config properties for this is a small change :) but I'm also ok with addressing this in a follow-up PR.

dimas-b · 2026-01-07T22:03:34Z

runtime/service/src/main/java/org/apache/polaris/service/config/ServiceProducers.java

      MetaStoreManagerFactory factory,
      PersistenceConfiguration config,
      RealmContextConfiguration realmContextConfiguration) {
+    if (ConfigUtils.isProfileActive("cli")) {


Same here: I'd prefer to use explicit config flags (set differently in each profile) vs. checking the name of the profile, please.

See #3340 (comment)

dimas-b · 2026-01-08T16:27:05Z

runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java

      Instance<ProductionReadinessCheck> checks,
      ReadinessConfiguration config) {
+    // Skip production readiness checks in CLI mode - they're only relevant for server deployments
+    if (ConfigUtils.isProfileActive("cli")) {


I believe using dedicated config properties for this is a small change :) but I'm also ok with addressing this in a follow-up PR.

# Conflicts: # runtime/admin/src/main/docker/Dockerfile.jvm

…ollowup PRs.

adutra

This is looking really good! I only have a few minor comments.

adutra · 2026-01-08T22:07:19Z

runtime/distribution/bin/admin

-# Launch Quarkus
-exec "$JAVA_CMD" ${POLARIS_JAVA_OPTS:-} -jar quarkus-run.jar "$@"
+# Show the usage when no argument is provided
+if [ "$#" -eq 0 ]; then


adutra · 2026-01-08T22:14:42Z

runtime/server/build.gradle.kts

+
+  // These are already provided transitively by polaris-runtime-service
+  // but we need them at compile time for CLI code
+  compileOnly(project(":polaris-core"))


Agree with @dimas-b these should be implementation for most of them. I would reorder them as below:

implementation(project(":polaris-core")) implementation(project(":polaris-version")) implementation(project(":polaris-api-management-service")) implementation(project(":polaris-api-iceberg-service")) implementation(project(":polaris-runtime-service")) compileOnly("com.fasterxml.jackson.core:jackson-annotations") implementation("io.quarkus:quarkus-picocli")

adutra · 2026-01-08T22:16:48Z

runtime/server/src/main/java/org/apache/polaris/PolarisApplication.java

+   * <ol>
+   *   <li>(400) System properties: -D flags at JVM startup
+   *   <li>(300) Environment variables
+   *   <li>(260) External config/application-cli.properties: Profile-specific external config


As I mentioned on the ML, I'm unsure whether we should mention the possibility of adding this profile-aware external config file, as it has very little added value.

+1 Let's advise users to customize via the profile-neutral application.properties (and/or corresponding -D and env. vars).

If a user runs both the server and the CLI with the same Polaris binary of the same location, a shared application.properties applies to both use cases. In that setup, having a separate application-cli.properties still seems useful to capture CLI specific configuration that should not affect the server behavior. That separation feels reasonable and keeps the shared defaults clean while allowing targeted overrides for the CLI. For example, CLI specific configs like the following will be overwritten by application.properties

quarkus.banner.enabled=false quarkus.log.level=WARN

WDYT?

AFAIK, profile-specific config works in conjunction with the profile-neutral config. In other words, application.properties is always loaded. This might be too much low-level details for end users to think about.

Even if all the code is bundled in the same Quarkus application, Server and Admin could still have distinct shell entry points and distinct application.properties files (each in its own directory). That would be my preference for the tar-based distribution.

Docker images can bundle anything that Polaris developers are ok with. In docker, users will override config via external mechanisms like env. variables. If a config map is used to define application.properties it will be specific to a particular runtime scenario anyway (Server or Admin, but not both).

profile-specific config works in conjunction with the profile-neutral config

That's true, and I can confirm that's how it work today.

This might be too much low-level details for end users to think about.

This isn't a user-face doc. This is a comment in the source code. I think it should be fine to be here to provide complete view of how it works.

Even if all the code is bundled in the same Quarkus application, Server and Admin could still have distinct shell entry points and distinct application.properties files (each in its own directory). That would be my preference for the tar-based distribution.
Docker images can bundle anything that Polaris developers are ok with. In docker, users will override config via external mechanisms like env. variables. If a config map is used to define application.properties it will be specific to a particular runtime scenario anyway (Server or Admin, but not both).

I understand that these particular use cases may not require profile specific config files today. That said, this does not mean the scenarios I mentioned will never occur. Providing clear and explicit messaging still has value in that context. Even if a user never actively uses External config/application-cli.properties as a profile specific external config, having a clear understanding of how it works can be important when diagnosing or debugging unexpected behavior.

dimas-b · 2026-01-09T16:34:06Z

runtime/server/src/main/java/org/apache/polaris/admintool/BootstrapCommand.java

Sorry for the last minute comment here, but I wonder if it might make sense to keep the runtime/admin module as a home for Admin code, but not build a Quarkus app from it. The runtime/server module will then import runtime/admin and we end up with one Quarkus app both for the Server and Admin Tool... WDYT?

My rationale for keeping the runtime/admin module is just better code isolation at compile time. The runtime/server module is mostly an assembly descriptor, ATM it does not have any feature code. It might be preferable to keep it this way to allow simpler reuse of feature code from their dedicated modules.

+1, that's actually a nice idea!

I see the appeal of this approach, but I worry it may dismiss some of the key benefits of the current refactor(we will need a few common modules to enable a separate admin module), and make the dependency structure less clearer. Please note that, the CLI will need ServiceProvider to work now.
It also makes testing a bit awkward. Tests that are specific to the CLI behavior would have to depend on the default and service module just to pick up application-cli and application properties , which feels indirect and less explicit. For those reasons, I think the current direction still has advantages in terms of modularity, clarity, and test ergonomics.

I agree on "test ergonomics" as an advantage in the current diff of this PR, but I do not think "modularity" is better - that's exactly what I meant to highlight in my comment above :)

IMHO, better modularity is preferable to "test ergonomics".

Testing the admin tool could follow the same approach as testing OPA (which is not monolithic with the server, but a pure puglin)... WDYT?

We will need a few common modules to enable a separate admin module. I believe this is one of areas we used to struggle with.

In terms of modularity, splitting functionalities across multiple modules does not automatically improve modularity. In many cases it does the opposite. It makes the dependency graph harder to reason about, and forces consumers and tests to pull in modules that exist only to host a thin slice of logic. I think good modularity is about cohesion and clear ownership. A module should group functionality that changes together and represents a meaningful boundary. And I'm not a fan of how we layout OPA modules.

In practice, over splitting often trades real modularity for superficial structure, while introducing costs in test ergonomics, dependency clarity, and mental overhead. Keeping related behavior together, even if it is small, is often the more modular choice. I have mentioned that we have to keep introducing new modules like runtime/common, runtime/test-common, runtime/distribution, they are the side effect of over splitting, which can be avoided in the beginning.

github-project-automation bot added this to Basic Kanban Board Dec 30, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Dec 30, 2025

jbonofre self-requested a review January 3, 2026 06:33

flyrain force-pushed the merge-admin branch 2 times, most recently from 542d619 to 70ad92f Compare January 6, 2026 07:30

adutra reviewed Jan 6, 2026

View reviewed changes

runtime/server/src/main/java/org/apache/polaris/PolarisApplication.java Show resolved Hide resolved

runtime/defaults/src/main/resources/application.properties Outdated Show resolved Hide resolved

adutra reviewed Jan 6, 2026

View reviewed changes

dimas-b reviewed Jan 6, 2026

View reviewed changes

flyrain marked this pull request as ready for review January 7, 2026 00:54

flyrain mentioned this pull request Jan 7, 2026

Remove Admin tests from required_status_checks #3370

Merged

6 tasks

dimas-b reviewed Jan 7, 2026

View reviewed changes

dimas-b previously approved these changes Jan 8, 2026

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jan 8, 2026

flyrain added 12 commits January 8, 2026 12:06

Merge Polaris-Admin to Polaris-Server

28bf714

# Conflicts: # runtime/admin/src/main/docker/Dockerfile.jvm

Fix test cases

9dd1f84

Resolve comments

73ef650

Restore module runtime/common

f398172

Remove release related change in the current PR, will be handled in f…

3267a87

…ollowup PRs.

Fixed the test failure

1cf6cd3

Separating cli profile

967c0ad

Resolve comments

005ac89

Resolve comments

441d158

Resolve comments

0e0182c

Resolve the comments

5feefbf

Resolve the comments

73bd962

flyrain dismissed dimas-b’s stale review via 73bd962 January 8, 2026 20:07

flyrain force-pushed the merge-admin branch from ae3644d to 73bd962 Compare January 8, 2026 20:07

Resolve the comments

4c853f4

adutra reviewed Jan 8, 2026

View reviewed changes

Resolve the comments

ef48217

adutra approved these changes Jan 9, 2026

View reviewed changes

dimas-b reviewed Jan 9, 2026

View reviewed changes

Merge Polaris-Admin to Polaris-Server #3340

Are you sure you want to change the base?

Merge Polaris-Admin to Polaris-Server #3340

Uh oh!

Conversation

flyrain commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

dimas-b commented Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

adutra commented Jan 6, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flyrain commented Jan 7, 2026

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

flyrain commented Dec 30, 2025 •

edited

Loading

flyrain Jan 7, 2026 •

edited

Loading

flyrain Jan 8, 2026 •

edited

Loading

dimas-b Jan 9, 2026 •

edited

Loading

dimas-b Jan 9, 2026 •

edited

Loading

flyrain Jan 9, 2026 •

edited

Loading