bug: Nemo Guardrails failing with Vertex AI

[kanzyai-emirarditi]

Did you check docs and existing issues?

• I have read all the NeMo-Guardrails docs
• I have updated the package to the latest version before submitting this issue
• (optional) I have used the develop branch
• I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

3.12

Operating system/version

MacOS 15.3

NeMo-Guardrails version (if you must use a specific version and not the latest

latest

Describe the bug

When I try to use a model from vertex ai, I get the following error message:

AttributeError: property 'max_tokens' of 'VertexAI' object has no setter

Here are the package versions I am using:
`
Package Version Editable project location

---

aiofiles 24.1.0
aiohappyeyeballs 2.4.4
aiohttp 3.10.11
aiolimiter 1.2.1
aiosignal 1.3.2
aiosqlite 0.20.0
alembic 1.14.1
alt-profanity-check 1.6.1
annotated-types 0.7.0
annoy 1.17.3
anthropic 0.37.1
anyio 4.8.0
appdirs 1.4.4
appnope 0.1.4
asttokens 3.0.0
asyncer 0.0.8
asyncio 3.4.3
asyncpg 0.30.0
attrs 25.1.0
azure-core 1.32.0
backoff 2.2.1
blis 1.2.0
boto3 1.36.14
botocore 1.36.14
cachetools 5.5.1
catalogue 2.0.10
certifi 2025.1.31
cffi 1.17.1
cfgv 3.4.0
charset-normalizer 3.4.1
click 8.1.8
cloudpathlib 0.20.0
cloudpickle 3.1.1
cohere 5.13.11
coloredlogs 15.0.1
colorlog 6.9.0
comm 0.2.2
confection 0.1.5
cryptography 44.0.0
cymem 2.0.11
dataclasses-json 0.6.7
datasets 3.2.0
debugpy 1.8.12
decorator 5.1.1
deepeval 1.6.2
defusedxml 0.7.1
Deprecated 1.2.18
dill 0.3.8
diskcache 5.6.3
distlib 0.3.9
distro 1.9.0
dnspython 2.7.0
docstring_parser 0.16
docx2txt 0.8
dspy 2.5.42
ecdsa 0.19.0
email_validator 2.2.0
en_core_web_lg 3.8.0
execnet 2.1.1
executing 2.2.0
fastapi 0.111.1
fastapi-cli 0.0.7
fastavro 1.10.0
fastembed 0.5.1
filelock 3.17.0
filetype 1.2.0
flatbuffers 25.2.10
frozenlist 1.5.0
fsspec 2024.9.0
google-ai-generativelanguage 0.6.15
google-api-core 2.24.1
google-api-python-client 2.160.0
google-auth 2.38.0
google-auth-httplib2 0.2.0
google-cloud-aiplatform 1.80.0
google-cloud-bigquery 3.29.0
google-cloud-core 2.4.1
google-cloud-resource-manager 1.14.0
google-cloud-storage 2.19.0
google-crc32c 1.6.0
google-generativeai 0.8.4
google-resumable-media 2.7.2
googleapis-common-protos 1.66.0
grpc-google-iam-v1 0.14.0
grpcio 1.63.2
grpcio-status 1.62.3
h11 0.14.0
httpcore 1.0.7
httplib2 0.22.0
httptools 0.6.4
httpx 0.28.1
httpx-sse 0.4.0
huggingface-hub 0.28.1
humanfriendly 10.0
identify 2.6.6
idna 3.10
importlib-metadata 7.0.0
iniconfig 2.0.0
ipykernel 6.29.5
ipython 8.32.0
jedi 0.19.2
Jinja2 3.1.5
jiter 0.8.2
jmespath 1.0.1
joblib 1.4.2
json_repair 0.35.0
jsonpatch 1.33
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
jupyter_client 8.6.3
jupyter_core 5.7.2
kanzy-agent 0.1.0 /Users/emirarditi/PycharmProjects/kanzy-agent
langchain 0.3.17
langchain-anthropic 0.2.4
langchain-aws 0.2.12
langchain-community 0.3.16
langchain-core 0.3.33
langchain-google-genai 2.0.9
langchain-google-vertexai 2.0.13
langchain-milvus 0.1.8
langchain-openai 0.2.14
langchain-pinecone 0.2.2
langchain-tests 0.3.10
langchain-text-splitters 0.3.5
langchain-voyageai 0.1.3
langcodes 3.5.0
langgraph 0.2.69
langgraph-checkpoint 2.0.10
langgraph-checkpoint-sqlite 2.0.3
langgraph-sdk 0.1.51
langsmith 0.3.6
language_data 1.3.0
lark 1.1.9
litellm 1.51.0
loguru 0.7.3
magicattr 0.1.6
Mako 1.3.9
marisa-trie 1.2.1
markdown-it-py 3.0.0
MarkupSafe 3.0.2
marshmallow 3.26.1
matplotlib-inline 0.1.7
mdurl 0.1.2
milvus-lite 2.4.11
mmh3 4.1.0
mpmath 1.3.0
msgpack 1.1.0
multidict 6.1.0
multiprocess 0.70.16
murmurhash 1.0.12
mypy-extensions 1.0.0
nemoguardrails 0.11.0
nest-asyncio 1.6.0
nodeenv 1.9.1
numpy 1.26.4
onnxruntime 1.20.1
openai 1.61.1
opentelemetry-api 1.24.0
opentelemetry-exporter-otlp-proto-common 1.24.0
opentelemetry-exporter-otlp-proto-grpc 1.24.0
opentelemetry-proto 1.24.0
opentelemetry-sdk 1.24.0
opentelemetry-semantic-conventions 0.45b0
optuna 4.2.0
orjson 3.10.15
packaging 24.2
pandas 2.2.3
parso 0.8.4
pexpect 4.9.0
phonenumbers 8.13.54
pillow 10.4.0
pinecone 5.4.2
pinecone-plugin-inference 3.1.0
pinecone-plugin-interface 0.0.7
pip 24.3.1
platformdirs 4.3.6
pluggy 1.5.0
portalocker 3.1.1
pre-commit 3.8.0
preshed 3.0.9
presidio_analyzer 2.2.357
presidio_anonymizer 2.2.357
prompt_toolkit 3.0.50
propcache 0.2.1
proto-plus 1.26.0
protobuf 4.25.6
psutil 6.1.1
psycopg2 2.9.10
ptyprocess 0.7.0
pure_eval 0.2.3
py_rust_stemmers 0.1.3
pyarrow 19.0.0
pyasn1 0.6.1
pyasn1_modules 0.4.1
pycparser 2.22
pycryptodome 3.21.0
pydantic 2.11.0a1
pydantic_core 2.28.0
pydantic-settings 2.7.1
Pygments 2.19.1
pymilvus 2.5.4
pyparsing 3.2.1
pytest 8.3.4
pytest-asyncio 0.25.3
pytest-order 1.3.0
pytest-repeat 0.9.3
pytest-socket 0.7.0
pytest-xdist 3.6.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-jose 3.3.0
python-multipart 0.0.20
pytz 2025.1
PyYAML 6.0.2
pyzmq 26.2.1
ragas 0.2.13
redis 5.2.1
referencing 0.36.2
regex 2024.11.6
requests 2.32.3
requests-file 2.1.0
requests-toolbelt 1.0.0
rich 13.9.4
rich-toolkit 0.13.2
rpds-py 0.22.3
rsa 4.9
s3transfer 0.11.2
scikit-learn 1.6.1
scipy 1.15.1
sentry-sdk 2.20.0
setuptools 75.8.0
shapely 2.0.7
shellingham 1.5.4
simpleeval 1.0.3
six 1.17.0
smart-open 7.1.0
sniffio 1.3.1
spacy 3.8.4
spacy-legacy 3.0.12
spacy-loggers 1.0.5
SQLAlchemy 2.0.37
sqlglot 25.34.1
srsly 2.5.1
stack-data 0.6.3
starlette 0.37.2
sympy 1.13.3
syrupy 4.8.1
tabulate 0.9.0
tavily-python 0.5.0
tenacity 8.4.2
thinc 8.3.4
threadpoolctl 3.5.0
tiktoken 0.8.0
tldextract 5.1.3
tokenizers 0.21.0
tornado 6.4.2
tqdm 4.67.1
traitlets 5.14.3
typer 0.15.1
types-requests 2.32.0.20241016
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2025.1
ujson 5.10.0
uritemplate 4.1.1
urllib3 2.3.0
uvicorn 0.29.0
uvloop 0.21.0
virtualenv 20.29.1
voyageai 0.2.4
wasabi 1.1.3
watchdog 6.0.0
watchfiles 1.0.4
wcwidth 0.2.13
weasel 0.4.1
websockets 14.2
wrapt 1.17.2
xxhash 3.5.0
yarl 1.18.3
zipp 3.21.0
zstandard 0.23.0

`

Steps To Reproduce

Set config to:
models:

• type: main
  engine: vertexai
  model: gemini-2.0-flash-001

Expected Behavior

For it to work and produce rail output

Actual Behavior

The following error:

AttributeError: property 'max_tokens' of 'VertexAI' object has no setter

[trebedea]

max_tokens is a read-only property in langchain-google-vertexai 2.0.13, not sure why there is no setter. See discussion linked above. The solution should be to also add a setter in langchain-google-vertexai on my side - it does not make sense to add a patch.

This is a nice bug that appeared with the new released langchain-google-vertexai 2.0.13: they introduced max_tokens as a property, as this was missing until version 2.0.13. However, it's only a read-only property, so we cannot modify it here:

NeMo-Guardrails/nemoguardrails/llm/params.py

Line 44 in 1d9f278

setattr(self.llm, param, value)

@Pouyanpi , should we patch this on our side? A simple try-catch on AttributeError would fix this for any Langchain LLM provider,

[Pouyanpi]

Thank you, @trebedea. I noticed that vertexai originally used max_output_tokens aliased as max_tokens without defining an explicit max_tokens property. This meant that in older versions, passing max_tokens worked without raising errors, even though it wasn’t clearly exposed (we were passing max_tokens silently).

Having a property named max_tokens can be confusing. I think langchain-google-vertexai should properly support a setter as you mentioned. We can possibly resolve this by patching both the getter and setter (if the property didn't exist fro previous versions). I think, using a try-catch on AttributeError would simply ignore the issue( it would fail to set the attribute without any notification, leaving developers unaware that the intended update wasn’t applied).

I can see two options as you suggested:

1. Patch vertexai similarly to what we did for ChatNVIDIA, by implementing both a setter and getter.
2. Open an issue on langchain-google-vertexai, acknowledging that we cannot support versions prior to 2.0.14.

This lack of unified parameters across different providers is indeed problematic.

let me know if you have something else in mind.

    ...
    @property
    def max_tokens(self) -> Optional[int]:
        return self.max_output_tokens

    @max_tokens.setter
    def max_tokens(self, value: Optional[int]) -> None:
        self.max_output_tokens = value