|
2 | 2 | page_type: sample
|
3 | 3 | languages:
|
4 | 4 | - python
|
| 5 | + - csharp |
5 | 6 | - bicep
|
6 | 7 | products:
|
7 | 8 | - azure
|
8 | 9 | - ai-services
|
9 | 10 | - azure-openai
|
| 11 | + - document-intelligence |
| 12 | + - language-service |
| 13 | + - azure-translator |
10 | 14 | name: Document Processing with Azure AI Samples
|
11 | 15 | description: This collection of samples demonstrates how to use various Azure AI capabilities to build a solution to extract structured data, classify, redact, and analyze documents.
|
12 | 16 | ---
|
@@ -40,31 +44,31 @@ The techniques demonstrated take advantage of various capabilities from each ser
|
40 | 44 |
|
41 | 45 | ## Samples
|
42 | 46 |
|
43 |
| -> [!NOTE] |
44 |
| -> All data extraction samples provide both an accuracy and confidence score for the extracted data. The accuracy score is calculated based on the similarity between the extracted data and the ground truth data. The confidence score is calculated based on OCR analysis confidence and `logprobs` in Azure OpenAI requests. |
45 |
| -
|
46 | 47 | ### Document Classification
|
47 | 48 |
|
48 |
| -| Sample | Description | Example Use Cases | |
49 |
| -| --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | |
50 |
| -| [Classification - Azure OpenAI GPT-4o with Vision](./samples/python/classification/document-classification-gpt-vision.ipynb) | Demonstrates how to use Azure OpenAI GPT-4o and GPT-4o-mini models to classify documents using their built-in vision capabilities. | Processing multiple documents types or documents with varying purposes, such as contracts, legal documents, and emails. | |
51 |
| -| [Classification - Azure AI Document Intelligence + Embeddings](./samples/python/classification/document-classification-text-embeddings.ipynb) | Demonstrates how to use Azure AI Document Intelligence pre-built layout and embeddings models to classify documents based on their content. | Processing multiple documents types or documents with varying purposes, such as contracts, legal documents, and emails. | |
| 49 | +| Sample | Link | Description | Example Use Cases | |
| 50 | +| --------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | |
| 51 | +| Vision-based Classification with Azure OpenAI GPT-4o | [Python](./samples/python/classification/document-classification-gpt-vision.ipynb) \| [.NET](./samples/dotnet/classification/document-classification-gpt-vision.ipynb) | Use Azure OpenAI GPT-4o models to classify documents using their built-in vision capabilities. | Processing multiple documents types or documents with varying purposes, such as contracts, legal documents, and emails. | |
| 52 | +| Semantic Similarity Classification with Vector Embeddings | [Python](./samples/python/classification/document-classification-text-embeddings.ipynb) \| [.NET](./samples/dotnet/classification/document-classification-text-embeddings.ipynb) | Use Azure OpenAI embedding models to convert document text and classify them based on similarity to pre-defined classification lists. | Processing multiple documents types or documents with varying purposes, such as contracts, legal documents, and emails. | |
52 | 53 |
|
53 | 54 | ### Document Redaction
|
54 | 55 |
|
55 |
| -| Sample | Description | Example Use Cases | |
56 |
| -| ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
57 |
| -| [Redaction - Azure AI Document Intelligence + Azure OpenAI GPT-4o](./samples/python/redaction/document-redaction-gpt.ipynb) | Demonstrates how to use Azure AI Document Intelligence pre-built layout and Azure OpenAI GPT models to redact sensitive information from documents using natural language instruction to determine redaction areas. | Need for more complex redaction rules, such as redacting based on context or relationships. Also works well for redacting PII, including names, addresses, and phone numbers. | |
58 |
| -| [Redaction - Azure AI Language PII Native Document Analysis](./samples/python/redaction/document-redaction-language-native-document.ipynb) | Demonstrates how to use Azure AI Language Native Document Analysis features to redact personally identifiable information (PII) from documents. | Redacting sensitive information from documents, such as names, addresses, and phone numbers. | |
| 56 | +| Sample | Link | Description | Example Use Cases | |
| 57 | +| --------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 58 | +| LLM-enabled Redaction with Azure AI Document Intelligence, Azure OpenAI GPT-4o, and Post-Processing | [Python](./samples/python/redaction/document-redaction-gpt.ipynb) | Use Azure AI Document Intelligence `prebuilt-layout` and Azure OpenAI GPT models to redact sensitive information from documents using natural language instruction to determine redaction areas. | Require specific redaction rules, such as redacting based on context or relationships. Also works for redacting PII, including names, addresses, and phone numbers. | |
| 59 | +| Document Redaction with Azure AI Language PII Native Document Analysis | [Python](./samples/python/redaction/document-redaction-language-native-document.ipynb) | Use Azure AI Language Native Document Analysis to redact personally identifiable information (PII) from documents. | Redacting sensitive information from documents, such as names, addresses, and phone numbers. | |
59 | 60 |
|
60 | 61 | ### Document Extraction
|
61 | 62 |
|
62 |
| -| Sample | Description | Example Use Cases | |
63 |
| -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | |
64 |
| -| [Data Extraction - Azure AI Document Intelligence + Azure OpenAI GPT-4o](./samples/python/extraction/text/document-extraction-gpt.ipynb) | Demonstrates how to use Azure AI Document Intelligence pre-built layout and Azure OpenAI GPT models to extract structured data from documents. | Predominantly text-based documents such as invoices, receipts, and forms. | |
65 |
| -| [Data Extraction - Azure AI Document Intelligence + Phi](./samples/python/extraction/text/document-extraction-phi.ipynb) | Demonstrates how to use Azure AI Document Intelligence pre-built layout and Microsoft's Phi small language models to extract structured data from documents. | Predominantly text-based documents such as invoices, receipts, and forms. | |
66 |
| -| [Data Extraction - Azure OpenAI GPT-4o with Vision](./samples/python/extraction/vision/document-extraction-gpt-vision.ipynb) | Demonstrates how to use Azure OpenAI GPT-4o and GPT-4o-mini models to extract structured data from documents using their built-in vision capabilities. | Complex documents with a mix of text and images, including diagrams, signatures, selection marks, etc. such as reports and contracts. | |
67 |
| -| [Data Extraction - Comprehensive Azure AI Document Intelligence + Azure OpenAI GPT-4o with Vision](./samples/python/extraction/multimodal/document-extraction-gpt-text-and-vision.ipynb) | Demonstrates how to improve the accuracy and confidence in extracting structured data from documents by combining Azure AI Document Intelligence and Azure OpenAI GPT-4o models with vision capabilities. | Any structured or unstructured document type. | |
| 63 | +> [!NOTE] |
| 64 | +> All data extraction samples provide both an accuracy and confidence score for the extracted data. The accuracy score is calculated based on the similarity between the extracted data and the ground truth data. The confidence score can be calculated based on OCR analysis confidence and `logprobs` in Azure OpenAI responses. |
| 65 | +
|
| 66 | +| Sample | Link | Description | Example Use Cases | |
| 67 | +| ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | |
| 68 | +| Text-based Extraction with Azure AI Document Intelligence and Azure OpenAI GPT-4o | [Python](./samples/python/extraction/text/document-extraction-gpt.ipynb) \| [.NET](./samples/dotnet/extraction/text/document-extraction-gpt.ipynb) | Use Azure AI Document Intelligence `prebuilt-layout` and Azure OpenAI GPT models to extract structured data from documents using text. | Predominantly text-based documents such as invoices, receipts, and forms. | |
| 69 | +| Text-based Extraction with Azure AI Document Intelligence and Microsoft Phi-4 | [Python](./samples/python/extraction/text/document-extraction-phi.ipynb) \| [.NET](./samples/dotnet/extraction/text/document-extraction-phi.ipynb) | Use Azure AI Document Intelligence `prebuilt-layout` and Microsoft's Phi models to extract structured data from documents using text. | Predominantly text-based documents such as invoices, receipts, and forms. | |
| 70 | +| Vision-based Extraction with Azure OpenAI GPT-4o | [Python](./samples/python/extraction/vision/document-extraction-gpt-vision.ipynb) \| [.NET](./samples/dotnet/extraction/vision/document-extraction-gpt-vision.ipynb) | Use Azure OpenAI GPT-4o models to extract structured data from documents using vision capabilities. | Complex documents with a mix of text and images, including diagrams, signatures, selection marks, etc. such as reports and contracts. | |
| 71 | +| Multi-Modal (Text and Vision) Extraction with Azure AI Document Intelligence and Azure OpenAI GPT-4o | [Python](./samples/python/extraction/multimodal/document-extraction-gpt-text-and-vision.ipynb) \| [.NET](./samples/dotnet/extraction/multimodal/document-extraction-gpt-text-and-vision.ipynb) | Improve the accuracy and confidence in extracting structured data from documents by combining text and images with LLMs. | Any structured or unstructured document type. | |
68 | 72 |
|
69 | 73 | ## Getting Started
|
70 | 74 |
|
|
0 commit comments