Skip to content

Commit

Permalink
docs: Generative ai docs (#12720)
Browse files Browse the repository at this point in the history 
* docs(generative-ai): update README.md

Updated the page to provide overview of the feature folders.

* feat(generative-ai): add an example template folder
  • Loading branch information
@msampathkumar
msampathkumar authored Oct 24, 2024
1 parent ffbe8ed commit a49e9ac
Show file tree
Hide file tree
Showing 7 changed files with 368 additions and 84 deletions.
259 changes: 175 additions & 84 deletions generative_ai/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,84 +1,175 @@
# Generative AI on Google Cloud

* Product Page: https://cloud.google.com/ai/generative-ai?hl=en
* Code samples: https://cloud.google.com/docs/samples?text=Generative%20AI

## Samples Style Guide

If you are interested in code sample contributions, see [Contributing Guide](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/CONTRIBUTING.md).

Only for Generative AI Python samples, below style guide instructions take precedence over [Samples Style Guide](https://googlecloudplatform.github.io/samples-style-guide/).

### 1. Use Scripting format

Wrap the imports, sample code and the region tags to be with in one function definition.
This is to keep the region tag section code to be in script format and also allowing you to write regular testcases.

> This change is motivated by the desire to provide code samples in a copy-paste-run
format that is helpful for popular data science community tools like
Google Colab, Jupyter Notebooks, and IPython shell.

Here is an example code.

```python
def create_hello_world_file(filename):
# <region tag: starts here>
import os

# TODO(developer): Update and uncomment below code
# filename = `/tmp/test.txt`

if os.path.isfile(filename):
print(f'Overriding content in file(name: {filename})!')

with open(filename) as fp:
fp.write('Hello world!')
# <region tag: ends here>
```

In Google Cloud documentation, this code sample will be shown as below

```python
import os

# TODO(developer): Update and uncomment below code
# filename = `/tmp/test.txt`

if os.path.isfile(filename):
print(f'Overriding content in file(name: {filename})!')

with open(filename) as fp:
fp.write('Hello world!')
```

**Note:** In the above sample, `imports` are include and `TODO's` are provided to variable
that need to update by users.

### 2. Avoid Hidden Variables

Suggestion to avoid hidden variables in code samples

* Use global variables, to defined common variables like PROJECT_ID, LOCATION.

* Keep the function definitions simple, with less or no arguments.
* Ex: Use `def generate_text() -> str:` instead of `def generate_text(temperature=..image_path=..video_path=..) -> str:`

* Use descriptive variables names and if need, use long variable name.
* Ex: Use `text_input` instead of `user_input` or `input_var` or `myvar` ...

* Resist the temptation to tell more
* Ex: Don't define unused optional arguments
* Ex: Use `Read more @ http://..` than explaining `video_config looks like {"foo": "bar",...}`

**Note**: Not all the samples are the same! "Avoid Hidden variables" is not same as "Dont/No Hidden Variables".


## Conclusion

To summarize, it's crucial to maintain the simplicity and brevity of your code
examples.
> The ideal sample is one that appears self-evident and immediately
comprehensible.



# Generative AI Samples on Google Cloud

Welcome to the Python samples folder for Generative AI on Vertex AI! In this folder, you can find the Python samples
used in [Google Cloud Generative AI documentation](https://cloud.google.com/ai/generative-ai?hl=en).

If you are looking for colab notebook, then this [link](https://github.com/GoogleCloudPlatform/generative-ai/tree/main).

## Getting Started

To try and run these Code samples, we have following recommend using Google Cloud IDE or Google Colab.

Note: A Google Cloud Project is a pre-requisite.

### Feature folders

Browse the folders below to find the Generative AI capabilities you're interested in.

<table>
<tr>
<td><strong>Python Samples Folder</strong>
</td>
<td><strong>Google Cloud Product</strong>
</td>
<td><strong>Short Description (With the help of Gemini 1.5)</strong>
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/context_caching">Context Caching</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview">https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview</a>
</td>
<td>Code samples demonstrating how to use context caching with Vertex AI's generative models. This allows for more consistent and relevant responses across multiple interactions by storing previous conversation history.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/controlled_generation">Controlled Generation</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output</a>
</td>
<td>Examples of how to control the output of generative models, such as specifying length, format, or sentiment.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/count_token">Count Token</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/list-token">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/list-token</a>
</td>
<td>Code demonstrating how to count tokens in text, which is crucial for managing costs and understanding model limitations.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/embeddings">Embeddings</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings">https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings</a>
</td>
<td>Code showing how to generate and use embeddings from text or images. Embeddings can be used for tasks like semantic search, clustering, and classification.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/extensions">Extensions</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/extensions/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/extensions/overview</a>
</td>
<td>Demonstrations of how to use extensions with generative models, enabling them to access and process real-time information, use tools, and interact with external systems.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/function_calling">Function Calling</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling</a>
</td>
<td>Examples of how to use function calling to enable generative models to execute specific actions or retrieve information from external APIs.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/grounding">Grounding</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview</a>
</td>
<td>Code illustrating how to ground generative models with specific knowledge bases or data sources to improve the accuracy and relevance of their responses.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/image_generation">Image Generation</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview</a>
</td>
<td>Samples showcasing how to generate images from text prompts using models like Imagen.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/model_garden">Model Garden</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models">https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models</a>
</td>
<td>Resources related to exploring and utilizing pre-trained models available in Vertex AI's Model Garden.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/model_tuning">Model Tuning</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models">https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models</a>
</td>
<td>Code and guides for fine-tuning pre-trained generative models on specific datasets or for specific tasks.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/rag">RAG</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/rag-api">https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/rag-api</a>
</td>
<td>Information and resources about Retrieval Augmented Generation (RAG), which combines information retrieval with generative models.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/reasoning_engine">Reasoning Engine</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/reasoning-engine">https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/reasoning-engine</a>
</td>
<td>Details about the Reasoning Engine, which enables more complex reasoning and logical deduction in generative models.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/safety">Safety</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes</a>
</td>
<td>Examples of how to configure safety attributes and filters to mitigate risks and ensure responsible use of generative models.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/system_instructions">System Instructions</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions?hl=en">https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions?hl=en</a>
</td>
<td>Code demonstrating how to provide system instructions to guide the behavior and responses of generative models.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/text_generation">Text Generation</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-chat-prompts-gemini">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-chat-prompts-gemini</a>
</td>
<td>Samples of how to generate text using Gemini models, including chat-based interactions and creative writing.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/audio">Understand Audio</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/audio-understanding">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/audio-understanding</a>
</td>
<td>Examples of how to use generative models for audio understanding tasks, such as transcription and audio classification.
</td>
</tr>
<tr>
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/video">Understand Video</a>
</td>
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding</a>
</td>
<td>Samples showcasing how to use generative models for video understanding tasks, such as video summarization and content analysis.
</td>
</tr>
</table>

## Contributing

Contributions welcome! See the [Contributing Guide](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/CONTRIBUTING.md).

## Getting help

Please use the [issues page](https://github.com/GoogleCloudPlatform/python-docs-samples/issues) to provide suggestions, feedback or submit a bug report.

## Disclaimer

This repository itself is not an officially supported Google product. The code in this repository is for demonstrative purposes only.
66 changes: 66 additions & 0 deletions generative_ai/template_folder/advanced_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# # Copyright 2024 Google LLC
# #
# # Licensed under the Apache License, Version 2.0 (the "License");
# # you may not use this file except in compliance with the License.
# # You may obtain a copy of the License at
# #
# # https://www.apache.org/licenses/LICENSE-2.0
# #
# # Unless required by applicable law or agreed to in writing, software
# # distributed under the License is distributed on an "AS IS" BASIS,
# # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# # See the License for the specific language governing permissions and
# # limitations under the License.
# import os
#
# from vertexai.generative_models import GenerationResponse
#
# PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")
#
#
# def advanced_example() -> GenerationResponse:
# # [START generativeaionvertexai_gemini_token_count_multimodal]
# import vertexai
# from vertexai.generative_models import GenerativeModel, Part
#
# # TODO(developer): Update and un-comment below line
# # PROJECT_ID = "your-project-id"
# vertexai.init(project=PROJECT_ID, location="us-central1")
#
# model = GenerativeModel("gemini-1.5-flash-002")
#
# contents = [
# Part.from_uri(
# "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
# mime_type="video/mp4",
# ),
# "Provide a description of the video.",
# ]
#
# # tokens count for user prompt
# response = model.count_tokens(contents)
# print(f"Prompt Token Count: {response.total_tokens}")
# print(f"Prompt Character Count: {response.total_billable_characters}")
# # Example response:
# # Prompt Token Count: 16822
# # Prompt Character Count: 30
#
# # Send text to Gemini
# response = model.generate_content(contents)
# usage_metadata = response.usage_metadata
#
# # tokens count for model response
# print(f"Prompt Token Count: {usage_metadata.prompt_token_count}")
# print(f"Candidates Token Count: {usage_metadata.candidates_token_count}")
# print(f"Total Token Count: {usage_metadata.total_token_count}")
# # Example response:
# # Prompt Token Count: 16822
# # Candidates Token Count: 71
# # Total Token Count: 16893
#
# # [END generativeaionvertexai_gemini_token_count_multimodal]
# return response
#
#
# if __name__ == "__main__":
# advanced_example()
42 changes: 42 additions & 0 deletions generative_ai/template_folder/noxfile_config.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Default TEST_CONFIG_OVERRIDE for python repos.

# You can copy this file into your directory, then it will be imported from
# the noxfile.py.

# The source of truth:
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/noxfile_config.py

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.7", "3.8", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": True,
# An envvar key for determining the project id to use. Change it
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
# build specific Cloud project. You can also use your own string
# to use your own Cloud project.
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
# If you need to use a specific version of pip,
# change pip_version_override to the string representation
# of the version number, for example, "20.2.4"
"pip_version_override": None,
# A dictionary you want to inject into your test. Don't put any
# secrets here. These values will override predefined values.
"envs": {},
}
4 changes: 4 additions & 0 deletions generative_ai/template_folder/requirements-test.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
backoff==2.2.1
google-api-core==2.19.0
pytest==8.2.0
pytest-asyncio==0.23.6
14 changes: 14 additions & 0 deletions generative_ai/template_folder/requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
pandas==1.3.5; python_version == '3.7'
pandas==2.0.3; python_version == '3.8'
pandas==2.1.4; python_version > '3.8'
pillow==10.3.0; python_version < '3.8'
pillow==10.3.0; python_version >= '3.8'
google-cloud-aiplatform[all]==1.69.0
sentencepiece==0.2.0
google-auth==2.29.0
anthropic[vertex]==0.28.0
langchain-core==0.2.11
langchain-google-vertexai==1.0.6
numpy<2
openai==1.30.5
immutabledict==4.2.0
Loading

0 comments on commit a49e9ac

Please sign in to comment.