samples/dotnet/extraction/vision/document-extraction-gpt-vision.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Document Extraction with Azure OpenAI GPT-4o (Vision Only)\n",
    "\n",
    "This sample demonstrates how to extract structured data from any document using Azure OpenAI's GPT-4o model with vision capabilities.\n",
    "\n",
    "![Data Extraction](../../../../images/extraction-openai.png)\n",
    "\n",
    "This is achieved by the following process:\n",
    "\n",
    "- Construct a system prompt that defines the instruction for extracting structured data from documents.\n",
    "- Construct a user prompt that includes the specific extraction instruction for the type of document, and each document page as a base64 encoded image.\n",
    "- Use the Azure OpenAI chat completions API with the GPT-4o model to generate a structured output from the content.\n",
    "\n",
    "## Objectives\n",
    "\n",
    "By the end of this sample, you will have learned how to:\n",
    "\n",
    "- Convert a document into a set of base64 encoded images for processing by GPT-4o.\n",
    "- Use prompt engineering techniques to instruct GPT-4o to extract structured data from a type of document.\n",
    "- Use the [Structured Outputs feature](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure) to extract structured data from the document page images using Azure OpenAI's GPT-4o model.\n",
    "- Use the [logprobs](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#request-body:~:text=False-,logprobs,-integer) parameter in an OpenAI request to determine the confidence of the extracted structured output.\n",
    "\n",
    "## Useful Tips\n",
    "\n",
    "- Combine this technique with a [page classification](../../classification/README.md) approach to reduce the number of pages to extract from to only those that match your criteria for extraction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import modules\n",
    "\n",
    "This sample takes advantage of the following .NET dependencies:\n",
    "\n",
    "- **pdf2image-dotnet** for converting a PDF file into a set of images per page.\n",
    "- **Azure.AI.OpenAI** to interface with the Azure OpenAI chat completions API to generate structured extraction outputs using the GPT-4o model.\n",
    "- **Azure.Identity** to securely authenticate with deployed Azure Services using Microsoft Entra ID credentials.\n",
    "\n",
    "The following local components are also used:\n",
    "\n",
    "- [**VehicleInsurancePolicy**](../../modules/samples/models/VehicleInsurancePolicy.csx) to provide the expected structured output JSON schema for vehicle insurance policy documents.\n",
    "- [**OpenAIStructuredOutputsHelpers**](../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx) to generate structured outputs from the OpenAI API and provide parsing functions for the response output.\n",
    "- [**AccuracyEvaluator**](../../modules/samples/evaluation/AccuracyEvaluator.csx) to evaluate the output of the extraction process with expected results.\n",
    "- [**OpenAIConfidence**](../../modules/samples/confidence/OpenAIConfidence.csx) to calculate the confidence of the extraction process based on the `logprobs` response from the OpenAI API request using the `Microsoft.ML.Tokenizers` and `Microsoft.ML.Tokenizers.Data.O200kBase` libraries (the latter is required for tokenizers used for the GPT-4o model).\n",
    "- [**DocumentProcessingResult**](../../modules/samples/models/DocumentProcessingResult.csx) to store the results of the extraction process as a file.\n",
    "- [**AppSettings**](../../modules/samples/AppSettings.csx) to access environment variables from the `.env` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Azure.AI.OpenAI, 2.1.0</span></li><li><span>Azure.Identity, 1.13.2</span></li><li><span>DotNetEnv, 3.1.1</span></li><li><span>Microsoft.ML.Tokenizers, 1.0.2</span></li><li><span>Microsoft.ML.Tokenizers.Data.O200kBase, 1.0.2</span></li><li><span>pdf2image-dotnet, 1.0.0</span></li></ul></div></div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#r \"nuget: Azure.Identity, 1.13.2\"\n",
    "#r \"nuget: Azure.AI.OpenAI, 2.1.0\"\n",
    "#r \"nuget: DotNetEnv, 3.1.1\"\n",
    "#r \"nuget: Microsoft.ML.Tokenizers, 1.0.2\"\n",
    "#r \"nuget: Microsoft.ML.Tokenizers.Data.O200kBase, 1.0.2\"\n",
    "#r \"nuget: pdf2image-dotnet, 1.0.0\"\n",
    "\n",
    "#!import ../../modules/samples/AppSettings.csx\n",
    "#!import ../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx\n",
    "#!import ../../modules/samples/helpers/StopwatchContext.csx\n",
    "#!import ../../modules/samples/models/VehicleInsurancePolicy.csx\n",
    "#!import ../../modules/samples/models/DocumentProcessingResult.csx\n",
    "#!import ../../modules/samples/evaluation/AccuracyEvaluator.csx\n",
    "#!import ../../modules/samples/confidence/OpenAIConfidence.csx\n",
    "\n",
    "using System;\n",
    "using System.IO;\n",
    "using System.Text.Json;\n",
    "using Azure;\n",
    "using Azure.Core;\n",
    "using Azure.Identity;\n",
    "using Azure.AI.OpenAI;\n",
    "using Azure.AI.OpenAI.Chat;\n",
    "using OpenAI;\n",
    "using OpenAI.Chat;\n",
    "using DotNetEnv;\n",
    "using Pdf2Image;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the Azure services\n",
    "\n",
    "To use Azure OpenAI, the SDK is used to create a client instance using a deployed endpoint and authentication credentials.\n",
    "\n",
    "For this sample, the credentials of the Azure CLI are used to authenticate with the deployed services."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string workingDir = Path.GetFullPath(\"../../../../\");\n",
    "AppSettings settings = new AppSettings(new Dictionary<string, string>(Env.Load(Path.Combine(workingDir, \".env\"))));\n",
    "string samplePath = Path.Combine(workingDir, \"samples/dotnet/extraction/vision/\");\n",
    "string sampleName = \"document-extraction-gpt-vision\";\n",
    "\n",
    "DefaultAzureCredential credential = new DefaultAzureCredential(\n",
    "    new DefaultAzureCredentialOptions { \n",
    "        ExcludeWorkloadIdentityCredential = true,\n",
    "        ExcludeAzureDeveloperCliCredential = true,\n",
    "        ExcludeEnvironmentCredential = true,\n",
    "        ExcludeManagedIdentityCredential = true,\n",
    "        ExcludeAzurePowerShellCredential = true,\n",
    "        ExcludeSharedTokenCacheCredential = true,\n",
    "        ExcludeInteractiveBrowserCredential = true\n",
    "    }\n",
    ");\n",
    "\n",
    "AzureOpenAIClient openaiClient = new AzureOpenAIClient(\n",
    "    new Uri(settings.OpenAIEndpoint),\n",
    "    credential\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Establish the expected output\n",
    "\n",
    "To compare the accuracy of the extraction process, the expected output of the extraction process has been defined in the following code block based on each page of a [Vehicle Insurance Policy](../../../assets/vehicle_insurance/policy_1.pdf).\n",
    "\n",
    "> **Note**: More insurance policy examples can be found in the [assets folder](../../../assets/vehicle_insurance). These examples include the PDF file and an associated JSON metadata file that provides the expected structured output. You can add your own scenarios by following the same structure.\n",
    "\n",
    "The expected output has been defined by a human evaluating the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string path = Path.Combine(workingDir, \"samples/assets/vehicle_insurance/\");\n",
    "string metadataFName = \"policy_5.json\"; // Change this to the file you want to evaluate\n",
    "string metadataFPath = Path.Combine(path, metadataFName);\n",
    "\n",
    "Dictionary<string, JsonElement> data = JsonSerializer.Deserialize<Dictionary<string, JsonElement>>(File.ReadAllText(metadataFPath));\n",
    "\n",
    "VehicleInsurancePolicy expected = data[\"expected\"].Deserialize<VehicleInsurancePolicy>(\n",
    "    new JsonSerializerOptions(JsonSerializerDefaults.Web)\n",
    "    {\n",
    "        PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower\n",
    "    }\n",
    ");\n",
    "string pdfFName = data[\"fname\"].GetString();\n",
    "string pdfFPath = Path.Combine(path, pdfFName);\n",
    "\n",
    "AccuracyEvaluator<VehicleInsurancePolicy> insurancePolicyEvaluator = new AccuracyEvaluator<VehicleInsurancePolicy>(matchKeys: new List<string>());"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract data from the document\n",
    "\n",
    "The following code block executes the data extraction process using Azure OpenAI's GPT-4o model using vision capabilities.\n",
    "\n",
    "It performs the following steps:\n",
    "\n",
    "1. Get the document bytes from the provided file path. _Note: In this example, we are processing a local document, however, you can use any document storage location of your choice, such as Azure Blob Storage._\n",
    "2. Use pdf2image to convert the document's pages into images per page as base64 strings.\n",
    "3. Using Azure OpenAI's GPT-4o model and its [Structured Outputs feature](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure), extract a structured data transfer object (DTO) from the content of the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "string systemPrompt = \"You are an AI assistant that extracts data from documents.\";"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "List<ChatMessageContentPart> userContent = new List<ChatMessageContentPart>();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "StringBuilder userTextPromptBuilder = new StringBuilder();\n",
    "userTextPromptBuilder.AppendLine(\"Extract the data from this insurance policy.\");\n",
    "userTextPromptBuilder.AppendLine(\"- If a value is not present, provide null.\");\n",
    "userTextPromptBuilder.AppendLine(\"- Some values must be inferred based on the rules defined in the policy.\");\n",
    "userTextPromptBuilder.AppendLine(\"- Dates should be in the format YYYY-MM-DD.\");\n",
    "\n",
    "string userTextPrompt = userTextPromptBuilder.ToString();\n",
    "\n",
    "userContent.Add(ChatMessageContentPart.CreateTextPart(userTextPrompt));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "StopwatchContext imageSw;\n",
    "\n",
    "using (imageSw = new StopwatchContext())\n",
    "{\n",
    "    var pdfBytes = File.ReadAllBytes(pdfFPath);\n",
    "    var pages = await Pdf2ImageConverter.FromBytesAsync(pdfBytes);\n",
    "\n",
    "    userContent.AddRange(from page in pages select page into pageBytes select BinaryData.FromBytes(pageBytes) into pageData select ChatMessageContentPart.CreateImagePart(pageData, \"image/png\"));\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "ParsedChatCompletion<VehicleInsurancePolicy> completion;\n",
    "\n",
    "StopwatchContext oaiSw;\n",
    "\n",
    "using (oaiSw = new StopwatchContext())\n",
    "{\n",
    "    completion = await openaiClient\n",
    "        .GetChatClient(settings.GPT4OModelDeploymentName)\n",
    "        .CompleteChatAsync(\n",
    "            [\n",
    "                new SystemChatMessage(systemPrompt),\n",
    "                new UserChatMessage(userContent)\n",
    "            ],\n",
    "            new ChatCompletionOptions\n",
    "            {   \n",
    "                ResponseFormat = CreateJsonSchemaFormat<VehicleInsurancePolicy>(\"vehicleInsurancePolicy\", jsonSchemaIsStrict: true),\n",
    "                MaxOutputTokenCount = 4096,\n",
    "                Temperature = 0.1f,\n",
    "                TopP = 0.1f,\n",
    "                IncludeLogProbabilities = true\n",
    "            }\n",
    "        );\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding the Structured Outputs JSON schema\n",
    "\n",
    "Using [`OpenAIJsonSchema.For`](../../modules/samples/helpers/OpenAIStructuredOutputsHelpers.csx), the [VehicleInsurancePolicy](../../modules/samples/models/VehicleInsurancePolicy.csx) data model is converted to a valid OpenAI supported JSON schema when applied to the `ResponseFormat` parameter of the OpenAI chat completions request.\n",
    "\n",
    "The JSON schema is used to instruct the GPT-4o model to generate a strict output that adheres to the structure defined. The approach makes it easier for developers to manage the data structure in code, with helpful descriptions that will be included in the final JSON schema.\n",
    "\n",
    "Demonstrated below, you can see how the Invoice data model is understood by the OpenAI request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\n",
       "  \"type\": \"object\",\n",
       "  \"properties\": {\n",
       "    \"PolicyNumber\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"The policy number of the vehicle insurance policy.\"\n",
       "    },\n",
       "    \"Cost\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsuranceCostDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The cost details of the vehicle insurance policy.\"\n",
       "    },\n",
       "    \"Renewal\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsuranceRenewalDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The renewal details of the vehicle insurance policy.\"\n",
       "    },\n",
       "    \"EffectiveFrom\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"The effective date from which the vehicle insurance policy is valid.\"\n",
       "    },\n",
       "    \"EffectiveTo\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"The effective date until which the vehicle insurance policy is valid.\"\n",
       "    },\n",
       "    \"LastDateToCancel\": {\n",
       "      \"type\": \"string\",\n",
       "      \"description\": \"The last date by which the vehicle insurance policy can be canceled.\"\n",
       "    },\n",
       "    \"Policyholder\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsurancePersonDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The policyholder details of the vehicle insurance policy.\"\n",
       "    },\n",
       "    \"Vehicle\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsuranceVehicleDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The vehicle details of the vehicle insurance policy.\"\n",
       "    },\n",
       "    \"AccidentExcess\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsuranceExcessDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The excess costs for accidents.\"\n",
       "    },\n",
       "    \"FireAndTheftExcess\": {\n",
       "      \"anyOf\": [\n",
       "        {\n",
       "          \"$ref\": \"#/$defs/VehicleInsuranceExcessDetails\"\n",
       "        },\n",
       "        {\n",
       "          \"type\": \"null\"\n",
       "        }\n",
       "      ],\n",
       "      \"description\": \"The excess costs for fire and theft.\"\n",
       "    }\n",
       "  },\n",
       "  \"required\": [\n",
       "    \"PolicyNumber\",\n",
       "    \"Cost\",\n",
       "    \"Renewal\",\n",
       "    \"EffectiveFrom\",\n",
       "    \"EffectiveTo\",\n",
       "    \"LastDateToCancel\",\n",
       "    \"Policyholder\",\n",
       "    \"Vehicle\",\n",
       "    \"AccidentExcess\",\n",
       "    \"FireAndTheftExcess\"\n",
       "  ],\n",
       "  \"additionalProperties\": false,\n",
       "  \"$defs\": {\n",
       "    \"VehicleInsuranceCostDetails\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"AnnualTotal\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"number\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The annual total cost of the vehicle insurance policy.\"\n",
       "        },\n",
       "        \"PayableByDate\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The date by which the vehicle insurance policy must be paid.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"AnnualTotal\",\n",
       "        \"PayableByDate\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"VehicleInsuranceRenewalDetails\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"RenewalNotificationDate\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The date on which the renewal notification is sent.\"\n",
       "        },\n",
       "        \"LastDateToRenew\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The last date before renewal is required.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"RenewalNotificationDate\",\n",
       "        \"LastDateToRenew\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"VehicleInsurancePersonDetails\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"FirstName\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The first name of the person.\"\n",
       "        },\n",
       "        \"LastName\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The last name or surname of the person.\"\n",
       "        },\n",
       "        \"DateOfBirth\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The date of birth of the person.\"\n",
       "        },\n",
       "        \"Address\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The current address of the person.\"\n",
       "        },\n",
       "        \"EmailAddress\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The email address of the person.\"\n",
       "        },\n",
       "        \"TotalYearsOfResidenceInUK\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The total years the person has resided in the UK.\"\n",
       "        },\n",
       "        \"DrivingLicenseNumber\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The driving license number of the person.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"FirstName\",\n",
       "        \"LastName\",\n",
       "        \"DateOfBirth\",\n",
       "        \"Address\",\n",
       "        \"EmailAddress\",\n",
       "        \"TotalYearsOfResidenceInUK\",\n",
       "        \"DrivingLicenseNumber\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"VehicleInsuranceVehicleDetails\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"RegistrationNumber\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The registration number of the vehicle.\"\n",
       "        },\n",
       "        \"Make\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The make of the vehicle.\"\n",
       "        },\n",
       "        \"Model\": {\n",
       "          \"type\": \"string\",\n",
       "          \"description\": \"The model of the vehicle.\"\n",
       "        },\n",
       "        \"Year\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The year of manufacture of the vehicle.\"\n",
       "        },\n",
       "        \"Value\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"number\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The current value of the vehicle.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"RegistrationNumber\",\n",
       "        \"Make\",\n",
       "        \"Model\",\n",
       "        \"Year\",\n",
       "        \"Value\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    },\n",
       "    \"VehicleInsuranceExcessDetails\": {\n",
       "      \"type\": \"object\",\n",
       "      \"properties\": {\n",
       "        \"Compulsory\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The compulsory excess amount.\"\n",
       "        },\n",
       "        \"Voluntary\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The voluntary excess amount.\"\n",
       "        },\n",
       "        \"UnapprovedRepairPenalty\": {\n",
       "          \"anyOf\": [\n",
       "            {\n",
       "              \"type\": \"integer\"\n",
       "            },\n",
       "            {\n",
       "              \"type\": \"null\"\n",
       "            }\n",
       "          ],\n",
       "          \"description\": \"The penalty amount for repairs by an unapproved repairer.\"\n",
       "        }\n",
       "      },\n",
       "      \"required\": [\n",
       "        \"Compulsory\",\n",
       "        \"Voluntary\",\n",
       "        \"UnapprovedRepairPenalty\"\n",
       "      ],\n",
       "      \"additionalProperties\": false\n",
       "    }\n",
       "  }\n",
       "}"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(typeof(VehicleInsurancePolicy).ModelJsonSchema());"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the outputs\n",
    "\n",
    "To provide context for the execution of the code, the following code blocks visualize the outputs of the data extraction process.\n",
    "\n",
    "This includes:\n",
    "\n",
    "- The accuracy of the structured data extraction comparing the expected output with the output generated by Azure OpenAI's GPT-4o model.\n",
    "- The confidence score of the structured data extraction based on the log probability of the output generated by Azure OpenAI's GPT-4o model.\n",
    "- The execution time of the end-to-end process.\n",
    "- The total number of tokens consumed by the GPT-4o model.\n",
    "\n",
    "### Understanding Accuracy vs Confidence\n",
    "\n",
    "When using AI to extract structured data, both confidence and accuracy are essential for different but complementary reasons.\n",
    "\n",
    "- **Accuracy** measures how close the AI model's output is to a ground truth or expected output. It reflects how well the model's predictions align with reality.\n",
    "  - Accuracy ensures consistency in the extraction process, which is crucial for downstream tasks using the data.\n",
    "- **Confidence** represents the AI model's internal assessment of how certain it is about its predictions.\n",
    "  - Confidence indicates that the model is certain about its predictions, which can be a useful indicator for human reviewers to step in for manual verification.\n",
    "\n",
    "High accuracy and high confidence are ideal, but in practice, there is often a trade-off between the two. While accuracy cannot always be self-assessed, confidence scores can and should be used to prioritize manual verification of low-confidence predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Gets the parsed VehicleInsurancePolicy object from the completion response.\n",
    "var insurancePolicy = completion.Parsed;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Determines the accuracy of the extracted data against the expected values.\n",
    "var accuracy = insurancePolicyEvaluator.Evaluate(expected, insurancePolicy);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Determines the confidence of the extracted data using the OpenAI response.\n",
    "var confidence = OpenAIConfidence<VehicleInsurancePolicy>.EvaluateConfidence(insurancePolicy, completion.Origin);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Gets the total execution time of the data extraction process.\n",
    "var totalElapsed = imageSw.Elapsed + oaiSw.Elapsed;\n",
    "\n",
    "// Gets the prompt tokens and completion tokens from the completion response.\n",
    "var promptTokens = completion.Usage.InputTokenCount;\n",
    "var completionTokens = completion.Usage.OutputTokenCount;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Save the output of the data extraction result\n",
    "var extractionResult = new DataProcessingResult<VehicleInsurancePolicy>(\n",
    "    insurancePolicy,\n",
    "    accuracy,\n",
    "    confidence,\n",
    "    promptTokens,\n",
    "    completionTokens,\n",
    "    totalElapsed\n",
    ");\n",
    "\n",
    "var extractionResultJson = JsonSerializer.Serialize(extractionResult, new JsonSerializerOptions { WriteIndented = true });\n",
    "var extractionResultFPath = Path.Combine(samplePath, $\"{sampleName}.{pdfFName}.json\");\n",
    "\n",
    "await File.WriteAllTextAsync(extractionResultFPath, extractionResultJson);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<details open=\"open\" class=\"dni-treeview\"><summary><span class=\"dni-code-hint\"><code>{ Accuracy = 100.00%, Confidence = 98.47%, ExecutionTime = 36.17 seconds, ImagePreprocessingTime = 14.12 seconds, OpenAIExecutionTime = 22.06 seconds, PromptTokens = 9031, CompletionTokens = 243 }</code></span></summary><div><table><thead><tr></tr></thead><tbody><tr><td>Accuracy</td><td><div class=\"dni-plaintext\"><pre>100.00%</pre></div></td></tr><tr><td>Confidence</td><td><div class=\"dni-plaintext\"><pre>98.47%</pre></div></td></tr><tr><td>ExecutionTime</td><td><div class=\"dni-plaintext\"><pre>36.17 seconds</pre></div></td></tr><tr><td>ImagePreprocessingTime</td><td><div class=\"dni-plaintext\"><pre>14.12 seconds</pre></div></td></tr><tr><td>OpenAIExecutionTime</td><td><div class=\"dni-plaintext\"><pre>22.06 seconds</pre></div></td></tr><tr><td>PromptTokens</td><td><div class=\"dni-plaintext\"><pre>9031</pre></div></td></tr><tr><td>CompletionTokens</td><td><div class=\"dni-plaintext\"><pre>243</pre></div></td></tr></tbody></table></div></details><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>PolicyNumber</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999955389061742</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>GB34567890123</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Cost</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>AnnualTotal</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999430670632353</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>750</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>PayableByDate</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9851863339701926</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2024-05-22</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Renewal</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>RenewalNotificationDate</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9643638686184406</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2025-04-20</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>LastDateToRenew</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9645620775435158</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2025-05-04</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>EffectiveFrom</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999813150621809</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2024-05-12</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>EffectiveTo</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9998868484143875</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2025-05-11</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>LastDateToCancel</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9499923556874812</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2024-05-26</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Policyholder</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>FirstName</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.999506184911245</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Laura</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>LastName</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999787870332024</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Bennett</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>DateOfBirth</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9164693744632291</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>1990-09-08</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Address</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9969184892122465</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>18 Elm Grove, Bristol, BS1 5HQ</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>EmailAddress</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9971344534624074</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>LBennett@me.com</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>TotalYearsOfResidenceInUK</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9193956568337379</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>29</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>DrivingLicenseNumber</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9187099947714623</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>BENNEL008099JJ9IT</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Vehicle</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>RegistrationNumber</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.999329606271516</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>IJ90MNO</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Make</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9947576209390623</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Honda</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Model</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9989699093562244</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>Civic</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Year</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999510532874295</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>2019</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Value</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9998745403826832</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>11000</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>AccidentExcess</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>Compulsory</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9995962221495821</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>300</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Voluntary</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999505764962334</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>200</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>UnapprovedRepairPenalty</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9986482962763574</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>250</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>FireAndTheftExcess</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.collections.generic.dictionary-2?view=net-7.0\">System.Collections.Generic.Dictionary&lt;System.String,System.Object&gt;</a></span></td><td><table><thead><tr><th><i>key</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>Compulsory</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999452126087266</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>250</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>Voluntary</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9999526028614667</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>150</pre></div></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>UnapprovedRepairPenalty</pre></div></td><td><table><thead><tr><th><i>key</i></th><th><i>type</i></th><th>value</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\"><pre>confidence</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9982136070546077</pre></div></td></tr><tr><td><div class=\"dni-plaintext\"><pre>value</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.string?view=net-7.0\">System.String</a></span></td><td><div class=\"dni-plaintext\"><pre>250</pre></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><div class=\"dni-plaintext\"><pre>_overall</pre></div></td><td><span><a href=\"https://docs.microsoft.com/dotnet/api/system.double?view=net-7.0\">System.Double</a></span></td><td><div class=\"dni-plaintext\"><pre>0.9846620612937317</pre></div></td></tr></tbody></table><style>\r\n",
       ".dni-code-hint {\r\n",
       "    font-style: italic;\r\n",
       "    overflow: hidden;\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview {\r\n",
       "    white-space: nowrap;\r\n",
       "}\r\n",
       ".dni-treeview td {\r\n",
       "    vertical-align: top;\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "details.dni-treeview {\r\n",
       "    padding-left: 1em;\r\n",
       "}\r\n",
       "table td {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "table tr { \r\n",
       "    vertical-align: top; \r\n",
       "    margin: 0em 0px;\r\n",
       "}\r\n",
       "table tr td pre \r\n",
       "{ \r\n",
       "    vertical-align: top !important; \r\n",
       "    margin: 0em 0px !important;\r\n",
       "} \r\n",
       "table th {\r\n",
       "    text-align: start;\r\n",
       "}\r\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// Display the outputs of the extraction process.\n",
    "var output = new\n",
    "{\n",
    "    Accuracy = $\"{float.Parse(accuracy[\"overall\"].ToString()) * 100:0.00}%\",\n",
    "    Confidence = $\"{float.Parse(confidence[\"_overall\"].ToString()) * 100:0.00}%\",\n",
    "    ExecutionTime = $\"{totalElapsed.TotalSeconds:0.00} seconds\",\n",
    "    ImagePreprocessingTime = $\"{imageSw.Elapsed.TotalSeconds:0.00} seconds\",\n",
    "    OpenAIExecutionTime = $\"{oaiSw.Elapsed.TotalSeconds:0.00} seconds\",\n",
    "    PromptTokens = promptTokens,\n",
    "    CompletionTokens = completionTokens,\n",
    "};\n",
    "\n",
    "display(output);\n",
    "display(confidence);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "name": "python"
  },
  "polyglot_notebook": {
   "kernelInfo": {
    "defaultKernelName": "csharp",
    "items": [
     {
      "aliases": [],
      "name": "csharp"
     }
    ]
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}