all_android_tools_deploy.jar doesn't contain AutoValue's annotation processor

[snazhmudinov]

Description of the bug:

The final all_android_tools_deploy.jar doesn't contain AutoValue's annotation processor class.

Which category does this issue belong to?

android tools

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I wasn't able to reproduce it in a sample project, but we see it in our project.

Basically, the error says:

ERROR: non zero exit: worker spawn failed for Javac
warning: [options] source value 8 is obsolete and will be removed in a future release
warning: [options] target value 8 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
error: Annotation processor 'com.google.auto.value.processor.AutoValueProcessor' not found
ERROR: non zero exit: worker spawn failed for Javac

Which operating system are you running Bazel on?

macOS

What is the output of bazel info release?

7.4.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

It might be related to: #23709

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[snazhmudinov]

@fmeum FYI.

[fmeum]

Did this work with 7.3.0? Are you depending on AutoValue explicitly? It's possible that something provided it by accident.

@ted-xie

[snazhmudinov]

Did this work with 7.3.0? Are you depending on AutoValue explicitly? It's possible that something provided it by accident.

@ted-xie

Yes, we had no issue with 7.3.x.

[snazhmudinov]

Here's a bit more detailed log

ERROR: /private/var/tmp/_bazel_sherozn/7c8e0e0343032503c540f4d2799c7b13/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/BUILD:41:12: Building external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar (5 source files) and running annotation processors (AutoValueProcessor) [for tool] failed: (Exit 1): java failed: error executing Javac command (from target @@bazel_tools//src/tools/android/java/com/google/devtools/build/android/dexer:DexFileSplitter)
  (cd /private/var/tmp/_bazel_sherozn/7c8e0e0343032503c540f4d2799c7b13/execroot/client_android && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
    PATH=/bin:/usr/bin:/usr/local/bin \
  external/remotejdk21_macos_aarch64/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.resources=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED' '--add-opens=java.base/java.nio=ALL-UNNAMED' '--add-opens=java.base/java.lang=ALL-UNNAMED' '-Dsun.io.useCanonCaches=false' -XX:-CompactStrings -Xlog:disable '-Xlog:all=warning:stderr:uptime,level,tags' -XX:-MaxFDLimit -jar external/remote_java_tools/java_tools/JavaBuilder_deploy.jar @bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar-0.params @bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar-1.params)

[fmeum]

@bazel-io fork 7.4.1

[fmeum]

A reproducer would be very helpful. What happens if you build bazel build @bazel_tools//src/tools/android/java/com/google/devtools/build/android/dexer:DexFileSplitter?

[snazhmudinov]

Building DexFileSplitter didn't produce any issue...
fwiw, this fixes the issue for me. although, the size of the android_tools.jar gets bigger.

[fmeum]

I'm wondering why all_android_tools_deploy.jar should bundle the annotation processor. It does contain some classes that use @AutoValue, but the generation has already happened when the jar has been produced. Are you using AutoValue in your build?

[iancha1992]

@bazel-io fork 7.4.1

Is this a regression? 7.4.1 is will be out today. @fmeum

[fmeum]

It's a change that can break builds that used to pass, but I'm not yet sure that it's really Bazel's compared to just the dangers of unspecified deps. It can probably be fixed without a new Bazel release, but that requires confirmation from the Android team.

[snazhmudinov]

I'm wondering why all_android_tools_deploy.jar should bundle the annotation processor. It does contain some classes that use @AutoValue, but the generation has already happened when the jar has been produced. Are you using AutoValue in your build?

Yes, we do use the annotation itself and the processor (these libs are listed in our maven dependencies).

[meisterT]

cc @ahumesky

[ahumesky]

USE_BAZEL_VERSION=7.4.1 bazelisk build @bazel_tools//src/tools/android/java/com/google/devtools/build/android/dexer:DexFileSplitter succeeds for me as well

And building an android app with 7.4.1 also works for me and indeed builds the DexFileSplitter target in the course of the build:

SUBCOMMAND: # @@bazel_tools//src/tools/android/java/com/google/devtools/build/android/dexer:DexFileSplitter [action 'Building external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar (5 source files) and running annotation processors (AutoValueProcessor) [for tool]', configuration: b8277ad354826c8bd423c87ef7cc3b2cc0719cef15afe75eeafbb90f35dc5f85, execution platform: @@internal_platforms_do_not_use//host:host, mnemonic: Javac]
(cd /usr/local/google/home/ahumesky/.cache/bazel/_bazel_ahumesky/887b3b4bbbf34aae0ce3b5a63454f9d6/execroot/__main__ && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
    PATH=/usr/local/google/home/ahumesky/.cache/bazelisk/downloads/sha256/c97f02133adce63f0c28678ac1f21d65fa8255c80429b588aeeba8a1fac6202b/bin:/usr/local/google/bin:/usr/local/google/home/ahumesky/.local/bin:/usr/local/google/home/ahumesky/bin:/usr/lib/google-golang/bin:/usr/local/buildtools/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  external/remotejdk21_linux/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.resources=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED' '--add-opens=java.base/java.nio=ALL-UNNAMED' '--add-opens=java.base/java.lang=ALL-UNNAMED' '-Dsun.io.useCanonCaches=false' -XX:-CompactStrings -Xlog:disable '-Xlog:all=warning:stderr:uptime,level,tags' -jar external/remote_java_tools/java_tools/JavaBuilder_deploy.jar @bazel-out/k8-opt-exec-ST-a828a81199fe/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar-0.params @bazel-out/k8-opt-exec-ST-a828a81199fe/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/DexFileSplitter.jar-1.params)
# Configuration: b8277ad354826c8bd423c87ef7cc3b2cc0719cef15afe75eeafbb90f35dc5f85
# Execution platform: @@internal_platforms_do_not_use//host:host

Is DexFileSplitter being used in some other context? A minimal repro would be ideal. I know that Spotify maintains a fork of Bazel to implement missing features in the native Android rules, so I understand that might be tricky.