feat: Added file tutorial_plangen_on_naturalplan_examples.ipynb #1804
Conversation
Added the colab file which consist of tutorial on example run of PlanGEN frameworks on two tasks from NaturalPlan: (i) Calendar Scheduling, and (ii) Trip Planning.
Added file tutorial_plangen_on_naturalplan_examples.ipynb
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello @Mihir3009, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
This pull request adds a new Colab notebook, tutorial_plangen_on_naturalplan_examples.ipynb, which provides a tutorial on how to run the PlanGEN framework on two tasks from the NaturalPlan dataset: Calendar Scheduling and Trip Planning. The notebook includes example runs and demonstrations of the framework's capabilities on these tasks.
Highlights
- New Tutorial Notebook: A new Colab notebook
tutorial_plangen_on_naturalplan_examples.ipynbhas been added to demonstrate the PlanGEN framework. - NaturalPlan Examples: The tutorial focuses on two specific tasks from the NaturalPlan dataset: Calendar Scheduling and Trip Planning.
- PlanGEN Framework: The notebook provides example runs and demonstrates the capabilities of the PlanGEN framework on the selected tasks.
Changelog
- tutorial_plangen_on_naturalplan_examples.ipynb
- Added a new Colab notebook demonstrating PlanGEN framework usage.
- Includes examples for Calendar Scheduling and Trip Planning tasks from NaturalPlan.
- Provides a tutorial on running PlanGEN with these examples.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Trivia time!
What is the file extension typically used for Jupyter Notebook files?
Click here for the answer
The file extension typically used for Jupyter Notebook files is `.ipynb`.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The pull request adds a Colab notebook demonstrating the use of the PlanGEN framework on two tasks from NaturalPlan: Calendar Scheduling and Trip Planning. The notebook provides a tutorial on how to run the examples. Overall, the addition of this tutorial is a valuable contribution to the project, providing users with a practical guide to using the PlanGEN framework. Since the file is a notebook, there aren't many code-related suggestions to make, but I've provided some feedback on the content and structure of the notebook.
Merge Readiness
The pull request introduces a valuable tutorial notebook. Given the nature of the content, there are no critical or high severity issues. The notebook appears ready to be merged, offering a practical guide to using the PlanGEN framework. However, it's recommended to review the notebook's content and structure for clarity and accuracy before merging. I am unable to directly approve the pull request, and users should have others review and approve this code before merging.
holtskinner
changed the title
| "cell_type": "markdown", | ||
| "source": [ | ||
| "### Restart current runtime\n", | ||
| "\n", | ||
| "> To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which will restart the current kernel." | ||
| ], | ||
| "metadata": { | ||
| "id": "WHhkAhyXAqqr" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Restart kernel after installs so that your environment can access the new packages\n", | ||
| "import IPython\n", | ||
| "\n", | ||
| "app = IPython.Application.instance()\n", | ||
| "app.kernel.do_shutdown(True)" | ||
| ], | ||
| "metadata": { | ||
| "colab": { | ||
| "base_uri": "https://localhost:8080/" | ||
| }, | ||
| "id": "7teNvbzBAqA7", | ||
| "outputId": "d7134c74-e014-4dbe-8ba7-3fc20376b3ec" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [ | ||
| { | ||
| "output_type": "execute_result", | ||
| "data": { | ||
| "text/plain": [ | ||
| "{'status': 'ok', 'restart': True}" | ||
| ] | ||
| }, | ||
| "metadata": {}, | ||
| "execution_count": 2 |
There was a problem hiding this comment.
This restart shouldn't be needed.
| "output_type": "stream", | ||
| "name": "stdout", | ||
| "text": [ | ||
| "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/101.8 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.8/101.8 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", | ||
| "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m93.9/93.9 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", | ||
| "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.3/7.3 MB\u001b[0m \u001b[31m62.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", | ||
| "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m153.7/153.7 kB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", | ||
| "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.4/45.4 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", | ||
| "\u001b[?25h" | ||
| ] |
| "### Install Vertex AI SDK\n", | ||
| "\n", | ||
| "> Install the latest version of the Vertex AI SDK for Python as well as extra dependencies related to PlanGEN:" |
There was a problem hiding this comment.
Can you change this Notebook to use the Google Gen AI SDK instead of the Vertex AI SDK? And use Gemini 2.0 Flash for all examples?
There was a problem hiding this comment.
Yes, I can offer guidance on how to modify the notebook to use the Google Gen AI SDK and Gemini 2.0 Flash. Here's a general outline:
- Install the Google Gen AI SDK: Replace the Vertex AI SDK installation command with the one for the Google Gen AI SDK. This typically involves
pip install google-generativeai. - Import the necessary modules: Update the import statements to reflect the new SDK. For example, instead of
from vertexai.generative_models import GenerativeModel, you would useimport google.generativeai as genai. - Initialize the Gemini 2.0 Flash model: Modify the code to initialize the Gemini 2.0 Flash model using the Gen AI SDK. This might involve specifying the model name directly, such as
model = genai.GenerativeModel('gemini-2.0-flash'). - Update the code examples: Adapt the code examples to use the new SDK's syntax for generating text, handling input, and processing output. Ensure that the examples leverage the specific capabilities of Gemini 2.0 Flash.
I can provide more specific code snippets if you share the relevant parts of the notebook where these changes need to be applied.
| "### Install Vertex AI SDK\n", | |
| "\n", | |
| "> Install the latest version of the Vertex AI SDK for Python as well as extra dependencies related to PlanGEN:" | |
| "> Install the latest version of the Google Gen AI SDK for Python as well as extra dependencies related to PlanGEN:" |
| "source": [ | ||
| "### Set Google Cloud project information and initialize Vertex AI SDK\n", | ||
| "\n", | ||
| "> To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://pantheon.corp.google.com/apis/enableflow?apiid=aiplatform.googleapis.com&project=research-01-268019).\n", |
There was a problem hiding this comment.
Change this link back to the one used in the template.
| "> To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://pantheon.corp.google.com/apis/enableflow?apiid=aiplatform.googleapis.com&project=research-01-268019).\n", | |
| "> To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", |
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "PROJECT_ID = \"research-01-268019\" # @param {type:\"string\"}\n", |
There was a problem hiding this comment.
Replace with Placeholder
| " incomplete = False\n", | ||
| " final_answer = \"\"\n", | ||
| " return child.par_str, final_answer\n", | ||
| " iteration += 1\n", | ||
| "\n", | ||
| " # If we reach here, return the best child based on reward\n", | ||
| " if iteration >= max_iterations:\n", | ||
| " best_child = max(current_node.children, key=lambda x: x.reward)\n", | ||
| " return best_child.par_str, \"Solution reached iteration limit\"\n", | ||
| "\n", | ||
| " current_node = self.select_node(current_node)\n", | ||
| " self.expand_node(current_node)" |
There was a problem hiding this comment.
Can any of the code in this notebook be simplified or use external libraries? It's easy to get lost in all of the implementations.
You can also try pulling some of these classes out into external .py files and import them.
| " \"- Best of N: [Explain why this algorithm is or isn’t suitable]\\n\"\n", | ||
| " \"- Rebase: [Explain why this algorithm is or isn’t suitable]\\n\"\n", |
There was a problem hiding this comment.
Don't use "smart quotes".
| " initial_plan = call_gemini(problem_description, temperature=0.0, model_name=model_name)\n", | ||
| " return initial_plan\n", | ||
| "\n", | ||
| "class OrchastrationUnit:\n", |
There was a problem hiding this comment.
| "class OrchastrationUnit:\n", | |
| "class OrchestrationUnit:\n", |
Fix this spelling throughout.
| "data_calendar = [{'num_people': '3',\n", | ||
| " 'num_days': '1',\n", | ||
| " 'duration': '1',\n", | ||
| " 'prompt_5shot': \"You are an expert at scheduling meetings. You are given a few constraints on the existing schedule of each participant, the meeting duration, and possibly some preferences on the meeting time. Note there exists a solution that works with existing schedule of every participant. Here are a few example tasks and solutions:\\n\\nTASK: You need to schedule a meeting for Roger, Karen and Dorothy for half an hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nRoger's calendar is wide open the entire day.\\nKaren has meetings on Monday during 10:00 to 10:30, 11:30 to 12:00, 12:30 to 13:00, 14:00 to 15:00, 15:30 to 16:00; \\nDorothy is busy on Monday during 9:00 to 10:00, 10:30 to 11:00, 11:30 to 12:00, 12:30 to 13:00, 14:00 to 15:30, 16:00 to 17:00; \\n\\nYou would like to schedule the meeting at their earlist availability.\\nFind a time that works for everyone's schedule and constraints. \\nSOLUTION: Here is the proposed time: Monday, 11:00 - 11:30 \\n\\nTASK: You need to schedule a meeting for Douglas, Lawrence and Isabella for half an hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nDouglas has meetings on Monday during 12:00 to 12:30, 15:00 to 15:30; \\nLawrence has meetings on Monday during 10:30 to 12:00, 13:00 to 13:30, 14:00 to 14:30, 15:30 to 16:00; \\nIsabella is busy on Monday during 9:00 to 12:30, 13:30 to 17:00; \\n\\nFind a time that works for everyone's schedule and constraints. \\nSOLUTION: Here is the proposed time: Monday, 12:30 - 13:00 \\n\\nTASK: You need to schedule a meeting for Joshua, Denise and Jeremy for one hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nJoshua is busy on Monday during 10:00 to 10:30, 12:00 to 12:30, 14:00 to 14:30, 15:00 to 15:30; \\nDenise's calendar is wide open the entire day.\\nJeremy has meetings on Monday during 9:30 to 10:30, 12:00 to 13:00, 13:30 to 14:00, 14:30 to 15:00, 15:30 to 16:00, 16:30 to 17:00; \\n\\nFind a time that works for everyone's schedule and constraints. \\nSOLUTION: Here is the proposed time: Monday, 10:30 - 11:30 \\n\\nTASK: You need to schedule a meeting for Alan, Elizabeth and Denise for half an hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nAlan has blocked their calendar on Monday during 13:00 to 13:30, 14:00 to 14:30; \\nElizabeth is busy on Monday during 9:00 to 9:30, 11:00 to 11:30; \\nDenise has blocked their calendar on Monday during 9:00 to 10:00, 10:30 to 11:30, 12:30 to 13:00, 13:30 to 14:00, 14:30 to 16:00, 16:30 to 17:00; \\n\\nYou would like to schedule the meeting at their earlist availability.\\nFind a time that works for everyone's schedule and constraints. \\nSOLUTION: Here is the proposed time: Monday, 10:00 - 10:30 \\n\\nTASK: You need to schedule a meeting for Mason, Bruce and Christopher for half an hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nMason has meetings on Monday during 9:30 to 10:00, 11:00 to 11:30, 14:30 to 15:00, 16:30 to 17:00; \\nBruce is free the entire day.\\nChristopher is busy on Monday during 9:30 to 10:30, 11:30 to 12:30, 15:00 to 17:00; \\n\\nMason would rather not meet on Monday before 12:30. Find a time that works for everyone's schedule and constraints. \\nSOLUTION: Here is the proposed time: Monday, 12:30 - 13:00 \\n\\nTASK: You need to schedule a meeting for Michelle, Steven and Jerry for one hour between the work hours of 9:00 to 17:00 on Monday. \\n\\nHere are the existing schedules for everyone during the day: \\nMichelle has meetings on Monday during 11:00 to 12:00; \\nSteven has blocked their calendar on Monday during 9:00 to 9:30, 11:30 to 12:00, 13:30 to 14:00, 15:30 to 16:00; \\nJerry has blocked their calendar on Monday during 9:00 to 9:30, 10:00 to 11:00, 11:30 to 12:30, 13:00 to 14:30, 15:30 to 16:00, 16:30 to 17:00; \\n\\nFind a time that works for everyone's schedule and constraints. \\nSOLUTION: \",\n", |
There was a problem hiding this comment.
Can you load these large data prompts into separate files or use Google Cloud Storage? We have a GCS bucket for large files which can be imported.
| "import pandas as pd\n", | ||
| "from IPython.display import display, HTML\n", | ||
| "\n", | ||
| "def create_analysis_table(final_predictions):\n", | ||
| " \"\"\"\n", | ||
| " Generates a Pandas DataFrame for analyzing model predictions with text wrapping.\n", | ||
| "\n", | ||
| " Args:\n", | ||
| " final_predictions: A list of dictionaries, where each dictionary represents\n", | ||
| " a data point with 'prompt_5shot', 'golden_plan', and\n", | ||
| " 'model_prediction' keys.\n", | ||
| "\n", | ||
| " Returns:\n", | ||
| " A Pandas DataFrame containing the analysis table.\n", | ||
| " \"\"\"\n", | ||
| " analysis_data = []\n", | ||
| " for data_point in final_predictions:\n", | ||
| " analysis_data.append({\n", | ||
| " 'Problem Statement': data_point['prompt_0shot'],\n", | ||
| " 'Golden Plan': data_point.get('golden_plan', 'N/A'), # Handle missing golden_plan gracefully\n", | ||
| " 'Model Prediction': data_point['model_prediction']\n", | ||
| " })\n", | ||
| "\n", | ||
| " df = pd.DataFrame(analysis_data)\n", | ||
| "\n", | ||
| " # Apply text wrapping to all columns\n", | ||
| " return df.style.set_properties(**{'text-align': 'left'}).set_table_styles([dict(selector='th', props=[('text-align', 'left')])])\n", | ||
| "\n", | ||
| "\n", | ||
| "# Example usage (assuming final_predictions is defined as in the provided code):\n", | ||
| "analysis_table = create_analysis_table(final_predictions)\n", | ||
| "display(analysis_table)\n" |
There was a problem hiding this comment.
This code looks like it was AI generated (based on the comments) please clean it up
| @@ -0,0 +1,2325 @@ | |||
| { | |||
There was a problem hiding this comment.
Be sure to fix spelling errors and add false positives to https://github.com/GoogleCloudPlatform/generative-ai/blob/main/.github/actions/spelling/allow.txt
| "\n", | ||
| "| | |\n", | ||
| "|-|-|\n", | ||
| "|Author(s) | Mihir Parmar, Hamid Palangi |" |
There was a problem hiding this comment.
Add links to your GitHub accounts here.
example:
| Author(s) |
| --- |
| [Eric Dong](https://github.com/gericdong) |
| [Holt Skinner](https://github.com/holtskinner) |
Content
Added the colab file which consist of tutorial on example run of PlanGEN frameworks on two tasks from NaturalPlan: (i) Calendar Scheduling, and (ii) Trip Planning.
Description
Thank you for opening a Pull Request!
Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
CONTRIBUTINGGuide.CODEOWNERSfor the file(s).nox -s formatfrom the repository root to format).Fixes #<issue_number_goes_here> 🦕