Gen AI Intelligent Document Processing (GenAIIDP)

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0

Questions?

Table of Contents

• Introduction
• Key Features
• Architecture Overview
• Quick Start
  • Processing Your First Document
• Updating an Existing Deployment
• Detailed Documentation
  • Core Documentation
  • Processing Patterns
  • Python Development
  • Planning & Operations
• Contributing
• License

Introduction

A scalable, serverless solution for automated document processing and information extraction using AWS services. This system combines OCR capabilities with generative AI to convert unstructured documents into structured data at scale.

IDP.Accelerator.Short.Demo.mp4

White-glove customization, deployment, and integration support for production use cases is also available through AWS Professional Services.

Key Features

• Serverless Architecture: Built entirely on AWS serverless technologies including Lambda, Step Functions, SQS, and DynamoDB
• Modular, pluggable patterns: Pre-built processing patterns using state-of-the-art models and AWS services
• Command Line Interface: Programmatic batch processing with evaluation framework and analytics integration
• Advanced Classification: Support for page-level and holistic document packet classification
• Few Shot Example Support: Improve accuracy through example-based prompting
• Custom Business Logic Integration: Inject custom prompt generation logic via Lambda functions for specialized document processing
• High Throughput Processing: Handles large volumes of documents through intelligent queuing
• Built-in Resilience: Comprehensive error handling, retries, and throttling management
• Cost Optimization: Pay-per-use pricing model with built-in controls
• Comprehensive Monitoring: Rich CloudWatch dashboard with detailed metrics and logs
• Web User Interface: Modern UI for inspecting document workflow status and results
• Human-in-the-Loop (HITL): Amazon A2I integration for human review workflows (Pattern 1 & Pattern 2)
  • Note: When deploying multiple patterns with HITL, reuse existing private workteam ARN due to AWS account limits
• AI-Powered Evaluation: Framework to assess accuracy against baseline data
• Extraction Confidence Assessment: LLM-powered assessment of extraction confidence with multimodal document analysis
• Document Knowledge Base Query: Ask questions about your processed documents

Architecture Overview

The solution uses a modular architecture with nested CloudFormation stacks to support multiple document processing patterns while maintaining common infrastructure for queueing, tracking, and monitoring.

Current patterns include:

• Pattern 1: Packet or Media processing with Bedrock Data Automation (BDA)
• Pattern 2: OCR → Bedrock Classification (page-level or holistic) → Bedrock Extraction
• Pattern 3: OCR → UDOP Classification (SageMaker) → Bedrock Extraction

Quick Start

To quickly deploy the GenAI-IDP solution in your AWS account:

1. Log into the AWS console
2. Choose the Launch Stack button below for your desired region:

Region name	Region code	Launch
US West (Oregon)	us-west-2
US East (N.Virginia)	us-east-1

3. When the stack deploys for the first time, you'll receive an email with a temporary password to access the web UI
4. Use this temporary password for your first login to set up a permanent password

Processing Your First Document

After deployment, choose the processing method that fits your use case:

Method 1: Web UI (Interactive)

1. Open the Web UI URL from CloudFormation stack Outputs
2. Log in and click "Upload Document"
3. Upload a sample document:
   • For Patterns 1 & 2: samples/lending_package.pdf
   • For Pattern 3: samples/rvl_cdip_package.pdf
4. Monitor processing and view results in the dashboard

Method 2: Direct S3 Upload (Simple)

1. Upload to the InputBucket (URL in CloudFormation Outputs)
2. Monitor via Step Functions console
3. Results appear in OutputBucket automatically

Method 3: IDP CLI (Batch/Programmatic)

For batch processing, automation, or evaluation workflows:

# Install CLI
cd idp_cli && pip install -e .

# Process documents
idp-cli run-inference \
    --stack-name <your-stack-name> \
    --dir ./samples/ \
    --monitor

# Download results
idp-cli download-results \
    --stack-name <your-stack-name> \
    --batch-id <batch-id> \
    --output-dir ./results/

See IDP CLI Documentation for:

• CLI-based stack deployment and updates
• Batch document processing
• Complete evaluation workflows with baselines
• Athena and Agent Analytics integration
• CI/CD integration examples

See the Deployment Guide for more detailed testing instructions.

IMPORTANT: If you have not previously done so, you must request access to the following Amazon Bedrock models:

• Amazon: All Nova models, plus Titan Text Embeddings V2
• Anthropic: Claude 3.x models, Claude 4.x models

Updating an Existing Deployment

To update an existing GenAIIDP stack to a new version:

1. Navigate to CloudFormation in the AWS Management Console
2. Select your existing stack
3. Click "Update"
4. Select "Replace current template"
5. Enter the template URL:
   • us-west-2: https://s3.us-west-2.amazonaws.com/aws-ml-blog-us-west-2/artifacts/genai-idp/idp-main.yaml
   • us-east-1: https://s3.us-east-1.amazonaws.com/aws-ml-blog-us-east-1/artifacts/genai-idp/idp-main.yaml
6. Follow the prompts to update your stack, reviewing any parameter changes
7. For detailed instructions, see the Deployment Guide

For testing, use these sample files:

• For Patterns 1 (BDA) and Pattern 2: Use samples/lending_package.pdf
• For Pattern 3 (UDOP): Use samples/rvl_cdip_package.pdf

For detailed deployment and testing instructions, see the Deployment Guide.

Detailed Documentation

Core Documentation

• Architecture - Detailed component architecture and data flow
• Deployment - Build, publish, deploy, and test instructions
• IDP CLI - Command line interface for batch processing and evaluation workflows
• Web UI - Web interface features and usage
• Agent Analysis - Natural language analytics and data visualization feature
• Custom MCP Agent - Integrating external MCP servers for custom tools and capabilities
• Configuration - Configuration and customization options
• Discovery - Pattern-neutral discovery process and BDA blueprint automation
• Classification - Customizing document classification
• Extraction - Customizing information extraction
• Human-in-the-Loop Review - Human review workflows with Amazon A2I
• Assessment - Extraction confidence evaluation using LLMs
• Evaluation Framework - Accuracy assessment system with analytics database and reporting
• Knowledge Base - Document knowledge base query feature
• Monitoring - Monitoring and logging capabilities
• Reporting Database - Analytics database for evaluation metrics and metering data
• Troubleshooting - Troubleshooting and performance guides

Processing Patterns

• Pattern 1: BDA - Packet or Media processing with Bedrock Data Automation (BDA)
• Pattern 2: Textract + Bedrock - OCR with Textract and generative AI with Bedrock
• Pattern 3: Textract + UDOP + Bedrock - OCR with Textract, UDOP Classification, and Bedrock extraction
• Few-Shot Examples - Implementing few-shot examples for improved accuracy

Python Development

• Using Notebooks with IDP Common Library - Guide for using and creating Jupyter notebooks to experiment with the IDP Common Library
• IDP Common Package - Documentation for the core library powering the accelerator

Planning & Operations

• Well-Architected Framework Assessment - Analysis based on AWS Well-Architected Framework
• AWS Services & IAM Roles - AWS services used and IAM role requirements
• Cost Calculator - Framework for estimating solution costs

Contributing

We welcome contributions to the GenAI Intelligent Document Processing accelerator! Whether you're fixing bugs, improving documentation, or proposing new features, your contributions are appreciated.

Please refer to our Contributing Guide for detailed information on:

• Setting up your development environment
• Project structure
• Making and testing changes
• Pull request process
• Coding standards
  • Python code uses ruff for linting
  • UI code uses ESLint (npm run lint to verify)
• Documentation requirements
• Issue reporting guidelines

Thank you to everyone who has contributed to making this project better!

License

This project is licensed under the terms specified in the LICENSE file.