Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 52 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Welcome! Content Understanding is a solution that analyzes and comprehends various media content—including **documents, images, audio, and video**—and transforms it into structured, organized, and searchable data.

Content Understanding is now a Generally Available (GA) service with the release of the `2025-11-01` API version.
Content Understanding is now a Generally Available (GA) service with the release of the `2025-11-01` API version.

- The samples in this repository default to the latest GA API version: `2025-11-01`.
- We will provide more samples for new functionalities in the GA API versions soon. For details on the updates in the current GA release, see the [Content Understanding What's New Document page](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/whats-new).
- As of November 2025, the `2025-11-01` API version is now available in a broader range of [regions](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/language-region-support).
- We will provide more samples for new functionalities in the GA API versions soon. For details on the updates in the current GA release, please see the [Content Understanding What's New Document page](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/whats-new).
- As of November 2025, the `2025-11-01` API version is available in a broader range of [regions](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/language-region-support).
- To access sample code for version `2025-05-01-preview`, please check out the corresponding Git tag `2025-05-01-preview` or download it directly from the [release page](https://github.com/Azure-Samples/azure-ai-content-understanding-python/releases/tag/2025-05-01-preview).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Formatting]

    • change: Removed an extra trailing space at the end of the first line.
    • rationale: To clean up unnecessary whitespace and ensure consistent formatting.
    • impact: Improves the document’s appearance and prevents potential formatting issues.
  • categories: [Grammar, Clarity]

    • change: Added "please" to the sentence linking to the What's New document page.
    • rationale: To make the sentence more polite and reader-friendly.
    • impact: Enhances readability and provides a more courteous tone.
  • categories: [Grammar, Clarity]

    • change: Changed "is now available" to "is available" regarding the API version's regional availability.
    • rationale: Because the phrase "as of November 2025" already implies recent availability, "now" is redundant.
    • impact: Improves the clarity and precision of the statement.

- To access sample code for version `2024-12-01-preview`, please check out the corresponding Git tag `2024-12-01-preview` or download it directly from the [release page](https://github.com/Azure-Samples/azure-ai-content-understanding-python/releases/tag/2024-12-01-preview).

👉 If you are looking for **.NET samples**, check out [this repo](https://github.com/Azure-Samples/azure-ai-content-understanding-dotnet/).
👉 If you are looking for **.NET samples**, please check out [this repo](https://github.com/Azure-Samples/azure-ai-content-understanding-dotnet/).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity]
    • change: Added the word "please" before "check out" in the sentence directing users to the .NET samples repo.
    • rationale: Including "please" makes the sentence more polite and inviting, improving the tone of the documentation.
    • impact: Enhances the readability and user-friendliness of the documentation, encouraging users to follow the recommendation.

## Getting Started

Expand All @@ -22,18 +22,18 @@ Run this repository virtually by using GitHub Codespaces, which opens a web-base

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://github.com/codespaces/new?skip_quickstart=true&machine=basicLinux32gb&repo=899687170&ref=main&geo=UsEast&devcontainer_path=.devcontainer%2Fdevcontainer.json)

After clicking the link above, follow these steps to set up the Codespace:
After clicking the link above, please follow these steps to set up the Codespace:

1. Create a new Codespace by selecting the `main` branch, your preferred Codespace region, and the 2-core machine type, as shown in the screenshot below.
![Create CodeSpace](/docs/create-codespace/1-Create%20Codespace.png)
![Create Codespace](/docs/create-codespace/1-Create%20Codespace.png)
2. Once the Codespace is ready, open the terminal and follow the instructions in the **Configure Azure AI service resource** section to set up a valid Content Understanding resource.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar]

    • change: Added the word "please" to the sentence "After clicking the link above, follow these steps to set up the Codespace."
    • rationale: Including "please" makes the instruction more polite and reader-friendly.
    • impact: Enhances the tone and engagement of the documentation without changing its meaning.
  • categories: [Formatting, Consistency]

    • change: Corrected the image alt text from "Create CodeSpace" to "Create Codespace" and added two trailing spaces after the image markdown.
    • rationale: The alt text was inconsistent in capitalization; adding trailing spaces ensures proper line breaks in markdown rendering.
    • impact: Improves visual formatting and consistency across documentation, ensuring images display correctly and alt text accurately describes the content.


### Local Environment

1. Ensure the following tools are installed:
1. Please ensure the following tools are installed:

* [Azure Developer CLI (azd)](https://aka.ms/install-azd)
* [Python 3.11+](https://www.python.org/downloads/)
* [Azure Developer CLI (azd)](https://aka.ms/install-azd)
* [Python 3.11+](https://www.python.org/downloads/)
* [Git LFS](https://git-lfs.com/)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity]

    • change: Added the word "Please" to the instruction "Ensure the following tools are installed" making it "Please ensure the following tools are installed:"
    • rationale: This softens the directive, making it more polite and reader-friendly.
    • impact: Improves tone and readability, providing a more welcoming instruction.
  • categories: [Formatting]

    • change: Added two trailing spaces after the first two list items in the tools list to enable proper Markdown line breaks.
    • rationale: In Markdown, two trailing spaces create a line break without starting a new paragraph; this helps maintain proper spacing and readability.
    • impact: Enhances the visual presentation of the list, ensuring consistent formatting and spacing in rendered documentation.


2. Create a new directory named `azure-ai-content-understanding-python` and clone this template into it using the `azd` CLI:
Expand All @@ -49,7 +49,7 @@ After clicking the link above, follow these steps to set up the Codespace:
cd azure-ai-content-understanding-python
```

- **Important:** If you use `git clone`, you must install Git LFS and run `git lfs pull` to download sample files in the `data` directory:
- **Important:** If you use `git clone`, please install Git LFS and run `git lfs pull` to download sample files in the `data` directory:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Clarity, Tone]
    • change: Replaced "you must install Git LFS" with "please install Git LFS"
    • rationale: The change softens the instruction, making it more polite and user-friendly.
    • impact: Improves the tone of the documentation, making it more approachable without altering the meaning.

```bash
git lfs install
Expand All @@ -58,7 +58,7 @@ After clicking the link above, follow these steps to set up the Codespace:

3. Set Up Dev Container Environment

- Install the following tools that support development containers:
- Please install the following tools that support development containers:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Clarity, Grammar]
    • change: Changed "Install the following tools that support development containers:" to "Please install the following tools that support development containers:"
    • rationale: Adding "Please" makes the instruction more polite and reader-friendly, improving the tone.
    • impact: Enhances the clarity and approachability of the documentation, making it more engaging for the reader.

- **Visual Studio Code**
Download and install [Visual Studio Code](https://code.visualstudio.com/).
Expand All @@ -69,70 +69,73 @@ After clicking the link above, follow these steps to set up the Codespace:

- **Docker**
Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) for Windows, macOS, or Linux. Docker manages and runs the container environment.
- Start Docker and ensure it is running in the background.
- Please start Docker and ensure it is running in the background.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Clarity, Consistency]
    • change: Changed "Start Docker and ensure it is running in the background." to "Please start Docker and ensure it is running in the background."
    • rationale: Added "Please" to make the instruction more polite and consistent with common documentation tone.
    • impact: Improves the readability and tone of the instruction, making it sound more user-friendly and approachable.

- Open the project and start the Dev Container:

- Open the project folder in VS Code.
- Open the project folder in VS Code.
- Press `F1` or `Ctrl+Shift+P`, then type and select:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Formatting]
    • change: Added two trailing spaces at the end of the sentence "Open the project folder in VS Code."
    • rationale: Two trailing spaces are used in Markdown to indicate a line break, ensuring proper rendering of the intended line spacing.
    • impact: Improves the readability and visual structure of the documentation when rendered in Markdown viewers.

```
Dev Containers: Reopen in Container
```
Alternatively, click the green icon in the lower-left corner of VS Code and select **Reopen in Container**.
```
Alternatively, click the green icon in the lower-left corner of VS Code and select **Reopen in Container**.
- VS Code will detect the `.devcontainer` folder, build the development container, and install the necessary dependencies.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Formatting]
    • change: Added trailing spaces at the end of markdown lines to enforce line breaks and removed inconsistent backticks from the fenced code block.
    • rationale: Trailing spaces in markdown ensure proper line breaks, improving readability. Fixing the fenced code block delimiters maintains correct markdown syntax.
    • impact: Enhances the visual structure and readability of the rendered documentation by ensuring proper line breaks and correctly formatted code blocks.

- ![How to set dev container environment](./docs/dev-container-setup.gif "Dev Container Setup")

## Configure Azure AI Service Resource

### Step 1: Create Azure AI Foundry Resource

First, create an Azure AI Foundry resource that will host both the Content Understanding service and the required model deployments.
First, please create an Azure AI Foundry resource that will host both the Content Understanding service and the required model deployments.

1. Follow the steps in the [Azure Content Understanding documentation](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/) to create an Azure AI Foundry resource.

2. Obtain your Foundry resource's endpoint URL from Azure Portal:

1. Follow the steps in the [Azure Content Understanding documentation](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/) to create an Azure AI Foundry resource
2. Get your Foundry resource's endpoint URL from Azure Portal:
- Go to [Azure Portal](https://portal.azure.com/)
- Navigate to your Azure AI Foundry resource
- Go to **Resource Management** > **Keys and Endpoint**
- Go to [Azure Portal](https://portal.azure.com/)
- Navigate to your Azure AI Foundry resource
- Go to **Resource Management** > **Keys and Endpoint**
- Copy the **Endpoint** URL (typically `https://<your-resource-name>.services.ai.azure.com/`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity, Formatting]

    • change: Added "please" to the introductory sentence and separated instructions into a numbered list with clear steps.
    • rationale: Including "please" makes the tone more polite; breaking the instructions into numbered steps improves readability and user guidance.
    • impact: Enhances user experience by making the instructions clearer and easier to follow.
  • categories: [Formatting]

    • change: Replaced the original bulleted sub-steps with indented lines that include two trailing spaces for proper line breaks in Markdown.
    • rationale: Ensures consistent Markdown formatting to properly render line breaks and maintain structure.
    • impact: Improves the visual presentation of the instructions, making them easier to read and understand.
  • categories: [Clarity]

    • change: Changed "Get your Foundry resource's endpoint URL" to "Obtain your Foundry resource's endpoint URL" in the heading line.
    • rationale: "Obtain" sounds more formal and instructional, aligning with professional documentation tone.
    • impact: Delivers a more precise and authoritative instruction to the user.


**⚠️ Important: Grant Required Permissions**

After creating your Azure AI Foundry resource, you must grant yourself the **Cognitive Services User** role to enable API calls for setting default GPT deployments:
After creating your Azure AI Foundry resource, please grant yourself the **Cognitive Services User** role to enable API calls for setting default GPT deployments:

1. Go to [Azure Portal](https://portal.azure.com/)
2. Navigate to your Azure AI Foundry resource
3. Go to **Access Control (IAM)** in the left menu
4. Click **Add** > **Add role assignment**
5. Select the **Cognitive Services User** role
6. Assign it to yourself (or the user/service principal that will run the samples)
1. Go to [Azure Portal](https://portal.azure.com/)
2. Navigate to your Azure AI Foundry resource
3. Go to **Access Control (IAM)** in the left menu
4. Click **Add** > **Add role assignment**
5. Select the **Cognitive Services User** role
6. Assign it to yourself (or the user/service principal that will run the samples)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity]

    • change: Replaced "you must grant yourself" with "please grant yourself" in the introductory sentence.
    • rationale: The wording softens the instruction, making it more polite and reader-friendly.
    • impact: Enhances the tone of the documentation, improving user engagement and approachability.
  • categories: [Formatting]

    • change: Added two spaces at the end of each step line in the numbered list to enforce line breaks.
    • rationale: In Markdown, two spaces at the end of a line ensure a line break in rendered text, making each step appear on its own line.
    • impact: Improves the readability and presentation of the instructions by ensuring proper separation of each step.

> **Note:** This role assignment is required even if you are the owner of the resource. Without this role, you will not be able to call the Content Understanding API to configure model deployments for prebuilt analyzers.

### Step 2: Deploy Required Models

**⚠️ Important:** The prebuilt analyzers require model deployments. You must deploy these models before using prebuilt analyzers:
- `prebuilt-documentSearch`, `prebuilt-audioSearch`, `prebuilt-videoSearch` require **GPT-4.1-mini** and **text-embedding-3-large**
- Other prebuilt analyzers like `prebuilt-invoice`, `prebuilt-receipt` require **GPT-4.1** and **text-embedding-3-large**
**⚠️ Important:** The prebuilt analyzers require model deployments. Please deploy these models before using prebuilt analyzers:

1. **Deploy GPT-4.1:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **gpt-4.1**
- Complete the deployment with your preferred settings
- `prebuilt-documentSearch`, `prebuilt-audioSearch`, `prebuilt-videoSearch` require **GPT-4.1-mini** and **text-embedding-3-large**
- Other prebuilt analyzers like `prebuilt-invoice` and `prebuilt-receipt` require **GPT-4.1** and **text-embedding-3-large**

1. **Deploy GPT-4.1:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **gpt-4.1**
- Complete the deployment with your preferred settings
- Note the deployment name (by convention, use `gpt-4.1`)

2. **Deploy GPT-4.1-mini:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **gpt-4.1-mini**
- Complete the deployment with your preferred settings
2. **Deploy GPT-4.1-mini:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **gpt-4.1-mini**
- Complete the deployment with your preferred settings
- Note the deployment name (by convention, use `gpt-4.1-mini`)

3. **Deploy text-embedding-3-large:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **text-embedding-3-large**
- Complete the deployment with your preferred settings
3. **Deploy text-embedding-3-large:**
- In Azure AI Foundry, go to **Deployments** > **Deploy model** > **Deploy base model**
- Search for and select **text-embedding-3-large**
- Complete the deployment with your preferred settings
- Note the deployment name (by convention, use `text-embedding-3-large`)

For more information on deploying models, see [Deploy models in Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-openai).
For more information on deploying models, please see [Deploy models in Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-openai).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity]

    • change: Changed "You must deploy these models before using prebuilt analyzers:" to "Please deploy these models before using prebuilt analyzers:"
    • rationale: The phrasing was softened from a directive "must" to a polite request "please," improving readability and tone.
    • impact: Makes the instruction friendlier and clearer to users.
  • categories: [Formatting, Consistency]

    • change: Moved the list of model requirements (prebuilt analyzers and their required models) from bullet points to inline bullet points with double spaces at line ends to preserve spacing.
    • rationale: This aligns the list visually separate from deployment steps and maintains consistent markdown formatting for better readability.
    • impact: Enhances document formatting and makes the requirements easier to find and read.
  • categories: [Formatting, Consistency]

    • change: Added two spaces at the end of list lines and reordered the steps for deployment instructions with consistent indentation and spacing.
    • rationale: Ensures proper markdown rendering with spaced lines and uniform formatting for nested lists across all deployment steps.
    • impact: Improves readability and uniformity of the deployment instructions across sections.
  • categories: [Clarity, Consistency]

    • change: Added "Note the deployment name (by convention, use xxx)" after each deployment step instead of only in the GPT-4.1 step.
    • rationale: Provides consistent important information about deployment naming for all models rather than only the first.
    • impact: Ensures users follow a clear naming convention consistently for all deployments.
  • categories: [Grammar, Clarity]

    • change: Changed "For more information on deploying models, see" to "For more information on deploying models, please see"
    • rationale: Politeness addition to the wording to soften the instruction tone.
    • impact: Creates a more user-friendly and approachable documentation tone.

### Step 3: Configure Environment Variables

Expand Down Expand Up @@ -165,15 +168,14 @@ Choose one of the following options to configure your environment:

#### Set Environment Variables with API Key (Alternative)


1. Copy the sample environment file:

```bash
cp notebooks/.env.sample notebooks/.env
```

2. Edit `notebooks/.env` and set your credentials.
- Replace `<your-resource-name>` and `<your-azure-ai-api-key>` with your actual values. These can be found in your AI Services resource under **Resource Management** > **Keys and Endpoint**.
2. Edit `notebooks/.env` and set your credentials.
- Replace `<your-resource-name>` and `<your-azure-ai-api-key>` with your actual values. These can be found in your AI Services resource under **Resource Management** > **Keys and Endpoint**.
- If you used different deployment names in Step 2, update the deployment variables accordingly:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Formatting]
    • change: Added a leading space before the hyphen in the second bullet point, changing it from a double hyphen (--) to a single indented hyphen (-).
    • rationale: To correct the bullet point formatting, ensuring consistent indentation and proper markdown list structure.
    • impact: Improves the readability and visual clarity of the bullet list in the documentation.


```env
Expand All @@ -184,7 +186,7 @@ Choose one of the following options to configure your environment:
TEXT_EMBEDDING_3_LARGE_DEPLOYMENT=text-embedding-3-large
```

> ⚠️ **Note:** If you skip the token authentication step above, you must set `AZURE_AI_API_KEY` in your `.env` file. Get your API key from Azure Portal by navigating to your Foundry resource > **Resource Management** > **Keys and Endpoint**.
> ⚠️ **Note:** If you skip the token authentication step above, you must set `AZURE_AI_API_KEY` in your `.env` file. You can obtain your API key from Azure Portal by navigating to your Foundry resource > **Resource Management** > **Keys and Endpoint**.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Grammar, Clarity]
    • change: Replaced "Get your API key" with "You can obtain your API key" in the note explaining how to get the API key from Azure Portal.
    • rationale: The revised phrasing improves the formality and clarity of the instruction, making it sound more polite and less imperative.
    • impact: Enhances readability and professionalism of the documentation, making the instruction clearer and more user-friendly.

## Open a Jupyter Notebook and Follow Step-by-Step Guidance

Expand Down Expand Up @@ -217,11 +219,11 @@ Azure AI Content Understanding is a new Generative AI-based [Azure AI service](h

> **Note:** The following samples are currently targeting Preview.2 (API version `2025-05-01-preview`) and will be updated to the GA API version (`2025-11-01`) soon.

- [Azure Search with Content Understanding](https://github.com/Azure-Samples/azure-ai-search-with-content-understanding-python)
- [Azure Search with Content Understanding](https://github.com/Azure-Samples/azure-ai-search-with-content-understanding-python)
- [Azure Content Understanding with OpenAI](https://github.com/Azure-Samples/azure-ai-content-understanding-with-azure-openai-python)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: [Formatting]
    • change: Added two trailing spaces at the end of the line containing the link to "Azure Search with Content Understanding."
    • rationale: In Markdown, two trailing spaces are used to enforce a line break; this ensures that the subsequent list item appears on a new line rather than being rendered inline.
    • impact: Improves the readability and formatting of the list by correctly displaying each item on its own line, enhancing the document’s visual structure.


## Notes

* **Trademarks** - This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos is subject to those third-party’s policies.

* **Data Collection** - The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
* **Data Collection** - The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • categories: Formatting
    • change: Removal of a trailing whitespace line at the end of the paragraph.
    • rationale: To ensure consistent formatting and eliminate unnecessary blank lines.
    • impact: Improves the visual cleanliness and consistency of the documentation without altering its content.

Loading