Evaluate your AI Agent using Vertex AI Gen AI Evaluation service Notebook

[olex-snk]

This pull request introduces an example of the notebook with guides on how to evaluate an ADK (Agent Development Kit) agent using Vertex AI Gen AI Evaluation for agent evaluation.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:17Z
----------------------------------------------------------------

PS: Clear all the output cells, to keep the notebook clean

olex-snk commented on 2025-07-14T09:01:57Z
----------------------------------------------------------------

Cleaned

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:19Z
----------------------------------------------------------------

Format NOTE to be bold and don't make it a header

olex-snk commented on 2025-07-14T09:07:37Z
----------------------------------------------------------------

Reformatted

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:20Z
----------------------------------------------------------------

Can you simplify the explanation to be more understandable? Here's an option, "Normally, asyncio doesn't let you start a new "event loop" (which is like the engine that runs asynchronous code) if one is already active. Jupyter Notebooks already have an event loop running in the background. So, when you try to run your asyncio code using functions like run_until_complete(), asyncio sees that a loop is already active and throws an error saying "This event loop is already running."

To fix this, you can use a library called nest_asyncio. It essentially "patches" asyncio to allow you to run new event loops even when one is already active. This lets your asyncio code work seamlessly within Jupyter Notebooks without getting that error."

Also please add links as hyperlinks instead of adding them directly.

olex-snk commented on 2025-07-14T12:40:51Z
----------------------------------------------------------------

Fixed explanation, added hyperlinks

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:20Z
----------------------------------------------------------------

Update markdown to be more descriptive. "We'll define plotting helper functions to visualize the results from evaluation as bar, radar...etc."

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:21Z
----------------------------------------------------------------

Update formate of NOTE to be bold and not a header.

olex-snk commented on 2025-07-14T09:05:58Z
----------------------------------------------------------------

Reformatted

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:21Z
----------------------------------------------------------------

Remove this cell if you aren't going to use it.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:22Z
----------------------------------------------------------------

Remove the note regarding BYOD here since you'll cover that in the notebook later.

olex-snk commented on 2025-07-14T09:09:24Z
----------------------------------------------------------------

Removed

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:22Z
----------------------------------------------------------------

Line #1.    EXPERIMENT_NAME = "evaluate-adk-agent-v1"  # @param {type:"string"}

Remove the comments regarding parameter type since we are not using colab, this isn't required. #param {type:"string"}

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

sanjanalreddy commented on 2025-07-14T00:40:23Z
----------------------------------------------------------------

Line #4.    !gsutil mb -l {LOCATION} {BUCKET_URI}

This throws an exception if the bucket already exists. Add a conditional statement to check if the bucket exists and only create the bucket if the bucket doesn't exist.

olex-snk commented on 2025-07-14T13:28:41Z
----------------------------------------------------------------

Added conditional statement to check if the bucket exists

[olex-snk]

Cleaned

---

View entire conversation on ReviewNB

[olex-snk]

Reformatted

---

View entire conversation on ReviewNB

[olex-snk]

Reformatted

---

View entire conversation on ReviewNB

[olex-snk]

Removed

---

View entire conversation on ReviewNB

[olex-snk]

Fixed explanation, added hyperlinks

---

View entire conversation on ReviewNB

[olex-snk]

Added conditional statement to check if the bucket exists

---

View entire conversation on ReviewNB

[sanjanalreddy]

Update NOTE to be bold and not header

Reply via ReviewNB

[sanjanalreddy]

Line #2.    BUCKET_NAME = "put_your_ai_agent_evaluation_adk_bucket"

This line will throw an error if a hard-coded bucket name is used since bucket names have to be unique for everyone. My recommendation is to use the PROJECT ID as the bucket name like this instead:

PROJECT = !gcloud config list --format 'value(core.project)'
PROJECT = PROJECT[0]
BUCKET_NAME = PROJECT
BUCKET_URI = f"gs://{BUCKET_NAME}"

Reply via ReviewNB

[takumiohym]

Please use the uuid library instead of creating a this custom method for simplicity.

import uuid

uuid = uuid.uuid4()

Reply via ReviewNB

[takumiohym]

@olex-snk Here's my suggestion regarding nest_asyncio. #634
Let me know your thoughts. fyi @sanjanalreddy