<p>In this project, I automated the extraction of ingredient details such as name, unit, quantity, and price from invoices using Azure's Form Recognizer and GPT-4o-mini. The solution leverages state-of-the-art AI models to improve accuracy when data is incomplete or not recognized by default. In this article, I'll explain the choices behind the tools I used, the workflow, and the cost breakdown, making it easy for others to replicate and scale a similar solution.</p>
<h2>Project Overview</h2>
<p>The goal of this project was to streamline the extraction of structured data from inconsistent invoice formats using a combination of <strong>Azure Form Recognizer</strong> and <strong>GPT-4o-mini</strong>. The key data points we extracted were the ingredient names, quantities, units, and prices from the invoice. </p>
<h3>Why Azure Form Recognizer?</h3>
<p>I selected <a href="https://azure.microsoft.com/en-us/services/form-recognizer/">Azure Form Recognizer</a> due to its specialized pre-built models for processing invoices. Form Recognizer offers a straightforward way to extract key fields like <code>Quantity</code>, <code>Description</code>, and <code>UnitPrice</code>. However, when fields are not recognized correctly, GPT-4o-mini fills in the gaps by intelligently interpreting the unstructured data.</p>
<hr>
<h2>Workflow</h2>
<h3>Step 1: Extracting Invoice Data with Azure Form Recognizer</h3>
<p>Azure Form Recognizer is the backbone of this solution, handling the initial extraction of data from PDFs. For invoices, it identifies key columns and fields with high confidence, such as <code>Ingredient</code>, <code>Unit</code>, and <code>Price</code>.</p>
<pre><code class="language-csharp">// Extract items from invoice using Form Recognizer
AnalyzeDocumentOperation operation = await client.AnalyzeDocumentAsync(
    WaitUntil.Completed, 
    "prebuilt-invoice", 
    stream
);
AnalyzeResult result = operation.Value;
</code></pre>
<p>Form Recognizer comes with a <strong>free tier</strong>, allowing you to process up to <strong>500 pages per month at no cost</strong>—perfect for testing and prototyping. For larger-scale operations, there’s a <strong>pay-as-you-go</strong> model: prebuilt invoices cost <strong>$10 per 1,000 pages</strong>, with no volume discount.</p>
<p>If you process high volumes, you can switch to <strong>commitment tiers</strong> and save:</p>
<ul>
<li><strong>$190/month</strong> for <strong>20 000</strong> pages (overage $9.50 per 1 000 pages)  </li>
<li><strong>$900/month</strong> for <strong>100 000</strong> pages (overage $9.00 per 1 000 pages)  </li>
<li><strong>$4 000/month</strong> for <strong>500 000</strong> pages (overage $8.00 per 1 000 pages)</li>
</ul>
<p>Commitment tiers also support disconnected-container deployments for on-premises use.</p>
<p><img alt="Description of image" src="/assets/blog/images/ai-invoice-processing-1.png"></p>
<h3>Step 2: Refining Data Using GPT-4o-Mini</h3>
<p>For cases where Form Recognizer returns low-confidence results—such as when units or ingredients aren't recognized GPT-4o-mini steps in to add context and structure. By batching the invoice data and sending it to GPT-4o-mini, I was able to extract missing or unclear information, ensuring that no ingredient was left behind.</p>
<pre><code class="language-csharp">// GPT-4o-mini prompt to refine invoice data
var prompt = $@"
Extract the ingredient names, units, prices, and quantities from the following content. 
Format the response as a JSON array of objects with properties: 'Guid', 'IngredientName', 'Unit' (with 'UnitName' and 'Amount'), 'PricePerItem', and 'Quantity'.
";
</code></pre>
<h3>Why GPT-4o-Mini?</h3>
<p><strong>GPT-4o-Mini</strong> was chosen for its <strong>strong performance-to-cost ratio</strong>. GPT-4o-mini models are more powerful than GPT-3.5, and while pricing varies based on usage, it typically costs around <strong>$0.15 per 1 million input tokens</strong> and <strong>$0.60 per 1 million output tokens</strong>. Given that invoices are processed in batches, this provides a cost-effective solution that ensures high accuracy.</p>
<h3>Step 3: Confidence-Based Validation</h3>
<p>Each field extracted by Form Recognizer comes with a confidence score. If the confidence score is below a certain threshold (e.g., 40%), GPT-4o-mini is used to provide an alternative interpretation. This ensures the accuracy and reliability of the extracted data.</p>
<pre><code class="language-csharp">// Check confidence score before using data
if (quantityField.Confidence &lt; 0.4) {
    allColumnsAreHighConfidence = false;
}
</code></pre>
<hr>
<h2>Cost Considerations</h2>
<h3>Azure Form Recognizer Costs</h3>
<ul>
<li><strong>Free Tier</strong>: Process up to <strong>500 pages per month</strong> for free, ideal for prototyping.</li>
<li><strong>Standard Pricing</strong>: After the free tier, the cost is <strong>$1.50 per 1,000 pages</strong> for the first <strong>1 million pages</strong>. Beyond that, the price decreases to <strong>$0.60 per 1,000 pages</strong>.</li>
</ul>
<h3>GPT-4o-Mini Costs</h3>
<ul>
<li><strong>Pricing</strong>: Around <strong>$0.15 per 1 million input tokens</strong> and <strong>$0.60 per 1 million output tokens</strong>. This cost is optimized by batching invoice data and using structured prompts to minimize token usage.</li>
</ul>
<p>Both tools provide excellent scalability at a reasonable price, making them ideal for a large-scale invoice processing system.</p>
<hr>
<h2>Key Challenges and Solutions</h2>
<h3>Batch Processing to Avoid Token Limits</h3>
<p>When sending data to GPT-4o-mini, I batch the invoice items to ensure that we do not exceed GPT-4o-mini's token limits, which keeps the operation both cost-effective and efficient.</p>
<pre><code class="language-csharp">var itemBatches = items
    .Select((kv, index) =&gt; new { kv.Key, kv.Value, Batch = index / 10 })
    .GroupBy(x =&gt; x.Batch)
    .ToList();
</code></pre>
<h3>GUID for Tracking</h3>
<p>Each ingredient extracted from the invoice is assigned a unique <code>GUID</code>, allowing us to track it throughout the pipeline. This ensures no ingredient is missed during the transition between Form Recognizer and GPT-4o-mini.</p>
<pre><code class="language-csharp">var guid = Guid.NewGuid().ToString();
</code></pre>
<hr>
<h2>Conclusion</h2>
<p>By combining <strong>Azure Form Recognizer</strong> and <strong>GPT-4o-Mini</strong>, I was able to create a robust and scalable solution for extracting structured data from invoices. This solution is not only cost-efficient but also highly accurate, especially for invoices with varying formats.</p>
<p><strong>Call to Action:</strong> If you're interested in automating your own invoice processing, or would like more details on how to integrate Azure and AI models into your workflows, feel free to contact me!</p>
<hr>
<h3>Next Steps</h3>
<ul>
<li>Explore the potential of <strong>Azure Form Recognizer</strong>'s custom models for other document types.</li>
<li>Continue optimizing prompt engineering for <strong>GPT-4o-Mini</strong> to further reduce token usage while maintaining accuracy.</li>
</ul>
<hr>


Automating invoice data extraction using Azure Form Recognizer and GPT-4o-mini. Learn about the workflow, tool choices, and cost breakdown.

Automating Invoice Data Extraction with Azure Form Recognizer and GPT-4o-mini