Agent Example - personalized-shopping unable to run python convert_product_file_format.py

[JCMazza]

When I get to the point of doing python convert_product_file_format.py I get the following error

(myenv) C:\Users\jchri\Documents\adk-samples\python\agents\personalized-shopping\personalized_shopping\shared_libraries\search_engine>python convert_product_file_format.py
Traceback (most recent call last):
File "C:\Users\jchri\Documents\adk-samples\python\agents\personalized-shopping\personalized_shopping\shared_libraries\search_engine\convert_product_file_format.py", line 21, in
from web_agent_site.engine.engine import load_products
File "..\web_agent_site_init_.py", line 15, in
from .envs.web_agent_text_env import WebAgentTextEnv
File "..\web_agent_site\envs\web_agent_text_env.py", line 20, in
from bs4 import BeautifulSoup
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\bs4_init_.py", line 64, in
from .builder import (
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\bs4\builder_init_.py", line 24, in
from bs4.element import (
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\bs4\element.py", line 9, in
from bs4.css import CSS
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\bs4\css.py", line 27, in
from bs4._typing import _NamespaceMapping
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\bs4_typing.py", line 63, in
_IncomingMarkup: TypeAlias = Union[str, bytes, IO[str], IO[bytes]]
File "C:\Users\jchri\AppData\Local\Programs\Python\Python310\lib\typing.py", line 287, in inner
return cached(*args, **kwds)
File "C:\Users\jchri\AppData\Local\Programs\Python\Python310\lib\typing.py", line 367, in getitem
return self._getitem(self, parameters)
File "C:\Users\jchri\AppData\Local\Programs\Python\Python310\lib\typing.py", line 482, in Union
return _UnionGenericAlias(self, parameters)
File "C:\Users\jchri\AppData\Local\Programs\Python\Python310\lib\typing.py", line 874, in init
self.parameters = _collect_type_vars(params)
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\typing_extensions.py", line 3158, in _collect_type_vars
if _should_collect_from_parameters(t):
File "C:\Users\jchri\Documents\adk-samples\myenv\lib\site-packages\typing_extensions.py", line 174, in _should_collect_from_parameters
t, (typing._GenericAlias, _types.GenericAlias, _types.UnionType)
AttributeError: module 'types' has no attribute 'UnionType'

[happyhuman]

I tried to reproduce this issue and followed the installation instruction here. However some steps in the instruction are no longer working, mainly the steps to download the following files:

# Download items_shuffle_1000 (4.5MB)
gdown https://drive.google.com/uc?id=1EgHdxQ_YxqIQlvvq5iKlCrkEKR6-j0Ib;

# Download items_ins_v2_1000 (147KB)
gdown https://drive.google.com/uc?id=1IduG0xl544V_A_jv3tHXC0kyFi7PnyBu;

# Download items_shuffle (5.1GB)
gdown https://drive.google.com/uc?id=1A2whVgOO0euk5O13n2iYDM0bQRkkRduB;

# Download items_ins_v2 (178MB)
gdown https://drive.google.com/uc?id=1s2j6NgHljiZzQNL3veZaAiyW_qDEgBNi;

[happyhuman]

My environment:
OS: Mac Os Sequoia 15.6
Python: 3.12.4

One of the gdown commands does work:

# Download items_human_ins (4.9MB)
gdown https://drive.google.com/uc?id=14Kb5SPBk_jfdLZ_CDBNitW98QLDlKR5O

So, I tried the next set of commands and got another error:

(issue191) ➜  search_engine git:(fix/issiue191) python convert_product_file_format.py

Gym has been unmaintained since 2022 and does not support NumPy 2.0 amongst other critical functionality.
Please upgrade to Gymnasium, the maintained drop-in replacement of Gym, or contact the authors of your software and request that they upgrade.
See the migration guide at https://gymnasium.farama.org/introduction/migration_guide/ for additional information.
WARNING: Using incubator modules: jdk.incubator.vector
Traceback (most recent call last):
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2292, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2322, in _get_module
    raise e
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2320, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahins/.pyenv/versions/3.12.4/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/modeling_utils.py", line 75, in <module>
    from .loss.loss_utils import LOSS_MAPPING
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/loss/loss_utils.py", line 21, in <module>
    from .loss_d_fine import DFineForObjectDetectionLoss
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/loss/loss_d_fine.py", line 21, in <module>
    from .loss_for_object_detection import (
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/loss/loss_for_object_detection.py", line 32, in <module>
    from transformers.image_transforms import center_to_corners_format
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/image_transforms.py", line 22, in <module>
    from .image_utils import (
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/image_utils.py", line 59, in <module>
    from torchvision.transforms import InterpolationMode
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/torchvision/__init__.py", line 10, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils  # usort:skip
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/torchvision/datasets/__init__.py", line 1, in <module>
    from ._optical_flow import FlyingChairs, FlyingThings3D, HD1K, KittiFlow, Sintel
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/torchvision/datasets/_optical_flow.py", line 13, in <module>
    from .utils import _read_pfm, verify_str_arg
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/torchvision/datasets/utils.py", line 4, in <module>
    import lzma
  File "/Users/shahins/.pyenv/versions/3.12.4/lib/python3.12/lzma.py", line 27, in <module>
    from _lzma import *
ModuleNotFoundError: No module named '_lzma'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/shahins/projects/adk/adk-samples/python/agents/personalized-shopping/personalized_shopping/shared_libraries/search_engine/convert_product_file_format.py", line 21, in <module>
    from web_agent_site.engine.engine import load_products
  File "/Users/shahins/projects/adk/adk-samples/python/agents/personalized-shopping/personalized_shopping/shared_libraries/search_engine/../web_agent_site/__init__.py", line 15, in <module>
    from .envs.web_agent_text_env import WebAgentTextEnv
  File "/Users/shahins/projects/adk/adk-samples/python/agents/personalized-shopping/personalized_shopping/shared_libraries/search_engine/../web_agent_site/envs/web_agent_text_env.py", line 27, in <module>
    from ..engine.engine import (
  File "/Users/shahins/projects/adk/adk-samples/python/agents/personalized-shopping/personalized_shopping/shared_libraries/search_engine/../web_agent_site/engine/engine.py", line 26, in <module>
    from pyserini.search.lucene import LuceneSearcher
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/pyserini/search/lucene/__init__.py", line 27, in <module>
    from ._impact_searcher import LuceneImpactSearcher, SlimSearcher
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/pyserini/search/lucene/_impact_searcher.py", line 32, in <module>
    from pyserini.encode import QueryEncoder, CachedDataQueryEncoder, SlimQueryEncoder, SpladeQueryEncoder, \
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/pyserini/encode/__init__.py", line 21, in <module>
    from ._aggretriever import AggretrieverDocumentEncoder, AggretrieverQueryEncoder
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/pyserini/encode/_aggretriever.py", line 27, in <module>
    from transformers import AutoModelForMaskedLM, AutoTokenizer, PreTrainedModel
  File "/Users/shahins/.pyenv/versions/issue191/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2295, in __getattr__
    raise ModuleNotFoundError(
ModuleNotFoundError: Could not import module 'PreTrainedModel'. Are this object's requirements defined correctly?

So, I seems this sample code needs some attention to make sure all the steps are working. Assigning it to Yiwen Song for further investigation.