feat: prepare client-side delegation and telemetry for pandas-gbq - #17704
feat: prepare client-side delegation and telemetry for pandas-gbq#17704shuoweil wants to merge 12 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces query delegation support to the pandas-gbq package when retrieving dataframes via to_dataframe. It adds version helper utilities to check if delegation is supported, issues a deprecation warning if the core client fallback is used, and delegates the dataframe creation to pandas_gbq when available. The review feedback suggests making the pandas-gbq import and version checks more robust by catching general exceptions rather than just ImportError, using getattr to safely retrieve the package version, and ensuring that pyarrow is skipped in tests when not installed to prevent test failures.
| if self._installed_version is None: | ||
| try: | ||
| import pandas_gbq # type: ignore | ||
|
|
||
| self._installed_version = packaging.version.parse( | ||
| getattr(pandas_gbq, "__version__", "0.0.0") | ||
| ) | ||
| except ImportError: | ||
| self._installed_version = packaging.version.parse("0.0.0") |
There was a problem hiding this comment.
Catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues (e.g., compiled extension load failures, AttributeError, or TypeError from broken dependencies). Additionally, packaging.version.parse can raise InvalidVersion if the version string is malformed. Catching Exception ensures a robust fallback to version 0.0.0.
| if self._installed_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._installed_version = packaging.version.parse( | |
| getattr(pandas_gbq, "__version__", "0.0.0") | |
| ) | |
| except ImportError: | |
| self._installed_version = packaging.version.parse("0.0.0") | |
| if self._installed_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._installed_version = packaging.version.parse( | |
| getattr(pandas_gbq, "__version__", "0.0.0") | |
| ) | |
| except Exception: | |
| self._installed_version = packaging.version.parse("0.0.0") |
There was a problem hiding this comment.
I do not want to hide errors for users.
| if self._delegation_api_version is None: | ||
| try: | ||
| import pandas_gbq # type: ignore | ||
|
|
||
| self._delegation_api_version = getattr( | ||
| pandas_gbq, "_internal_delegation_api_version", 0 | ||
| ) | ||
| except ImportError: | ||
| self._delegation_api_version = 0 |
There was a problem hiding this comment.
Similar to installed_version, catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues. Catching Exception ensures a robust fallback to 0.
| if self._delegation_api_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._delegation_api_version = getattr( | |
| pandas_gbq, "_internal_delegation_api_version", 0 | |
| ) | |
| except ImportError: | |
| self._delegation_api_version = 0 | |
| if self._delegation_api_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._delegation_api_version = getattr( | |
| pandas_gbq, "_internal_delegation_api_version", 0 | |
| ) | |
| except Exception: | |
| self._delegation_api_version = 0 |
There was a problem hiding this comment.
I do not want to hide errors for users.
|
Converting to draft until presubmits are green |
…nection.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…_helpers.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…_helpers.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
5965e97 to
5d80f9a
Compare
Injects
pandas-gbq/<version>into the clientuser-agentwhen query result conversion to DataFrame / GeoDataFrame is executed and supported bypandas-gbq.This PR enables backend metrics and telemetry to distinguish between client-side conversion and delegated conversions without altering existing DataFrame output or behavior.
Key Changes
pandas-gbq/<version>toClientInfo.user_agentinRowIterator.to_dataframe()andto_geodataframe().Fixes #<540939659> 🦕