Skip to content

Make google-cloud-aiplatform[evaluation] an optional extra in providers-google#69399

Open
gingeekrishna wants to merge 1 commit into
apache:mainfrom
gingeekrishna:fix/69323-google-aiplatform-evaluation-optional
Open

Make google-cloud-aiplatform[evaluation] an optional extra in providers-google#69399
gingeekrishna wants to merge 1 commit into
apache:mainfrom
gingeekrishna:fix/69323-google-aiplatform-evaluation-optional

Conversation

@gingeekrishna

Copy link
Copy Markdown

Motivation

Closes #69323

apache-airflow-providers-google declares google-cloud-aiplatform[evaluation] as an unconditional base dependency. The [evaluation] extra transitively installs:

  • litellm → huggingface-hub → tokenizers (large ML stack)
  • scikit-learn
  • ruamel-yaml

This bloats the install for every user of the provider, even those who only need GCSHook, BigQueryHook, or other features unrelated to Vertex AI evaluation. It also surfaces a constant stream of litellm proxy-server CVEs in pip-audit for all provider users.

Changes

providers/google/pyproject.toml

  • Change base dep from google-cloud-aiplatform[evaluation]>=1.155.0 to google-cloud-aiplatform>=1.155.0
  • Add new optional extra vertex-ai-evaluation that provides google-cloud-aiplatform[evaluation]>=1.155.0 for users who need the Rapid Evaluation API

generative_model.py

  • Guard from vertexai.preview.evaluation import EvalResult, EvalTask with try/except ImportError
  • get_eval_task() and run_evaluation() raise an informative ImportError when the extra is not installed, pointing users to pip install 'apache-airflow-providers-google[vertex-ai-evaluation]'

Upgrade path

Users who rely on Vertex AI evaluation operators (RunEvaluationOperator etc.) add the explicit extra:

pip install 'apache-airflow-providers-google[vertex-ai-evaluation]'

All other users get a smaller, CVE-quieter install with no behaviour change.

Copilot AI review requested due to automatic review settings July 5, 2026 04:45
@gingeekrishna gingeekrishna requested a review from shahar1 as a code owner July 5, 2026 04:45
@boring-cyborg boring-cyborg Bot added area:providers provider:google Google (including GCP) related issues provider:teradata labels Jul 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@jroachgolf84 jroachgolf84 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seem to be changes for two issues in this PR. Can you please move these into two, separate PRs?

@gingeekrishna gingeekrishna force-pushed the fix/69323-google-aiplatform-evaluation-optional branch from f23a227 to b1640db Compare July 5, 2026 14:07
@gingeekrishna

Copy link
Copy Markdown
Author

Thank you for the review! The passphrase-quoting fix for Windows has been moved to a separate PR: #69398

This PR now contains only the google-cloud-aiplatform[evaluation] optional-extra change (fixes #69323). I've rebased the branch to remove the Teradata commit.

…itellm/scikit-learn bloat

The [evaluation] extra of google-cloud-aiplatform transitively installs
litellm, scikit-learn, ruamel-yaml and a full HF/tokenizers chain that
is irrelevant to users of GCSHook, BigQueryHook, or most other Google
provider features. It also surfaces a steady stream of litellm proxy CVEs
in pip-audit for every provider user, even when litellm is never used.

Changes:
- pyproject.toml: drop [evaluation] from the base dep; add a new
  "vertex-ai-evaluation" optional extra that provides it for users
  who need the Rapid Evaluation API.
- generative_model.py: guard the vertexai.preview.evaluation import with
  try/except; raise an informative ImportError from get_eval_task() and
  run_evaluation() when the extra is not installed, pointing users to
  pip install apache-airflow-providers-google[vertex-ai-evaluation].

Fixes: apache#69323
@gingeekrishna gingeekrishna force-pushed the fix/69323-google-aiplatform-evaluation-optional branch from b1640db to 5d0d1d1 Compare July 5, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues provider:teradata

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Google provider base-depends on google-cloud-aiplatform[evaluation], force-installing litellm/scikit-learn for every user

3 participants