agentic invoice design for reliability and performance

This is a design for invoice processing system - the focus is performance, reliability and cost and the workflow is after client made their submission, they expect to be getting json as an output for entry or every item in the invoice and then fill up their web form and then user validate if everything is ok and finally click submit. 

In our agent, we also have a validator but that to ensure we returning legit data. 

Agent is specialized and do not have too many context poisoning. If model validator checks out ok, then we proceed to move to the next - which return json results back to user. 

Another reason reason for this design, lets say our model is down - somewhere somehow - we are able to retain the client submission and help to processes it later.

If for some reason, the default model is not able to decipher the image, model validator can try better model to see if it helps. But this can incur additioan cost. Given that the model can't recognized it, do we want to retry with a larger more expensive model or we should just set our path with a better more expensive model. It really depends on what type of error, we are getting (typical architects). Having said that we will log invoice with bad result - and the reason we are doing this - is we wanted to tells whether a prompt fix or a pre-check (e.g. image quality detection) would be cheaper than paying for the stronger model.

Why we don't use base64 image to upload user request  

Message size limits. Base64 inflates the file by about 33%. Azure Service Bus Standard caps messages at 256 KB (Premium goes to 100 MB), and SQS caps at 256 KB. A single phone-photo invoice or a multi-page PDF will blow past that, so you’d be forced into storage anyway.

Performance (your top priority). Large payloads slow every hop: client → queue → worker, and again on every retry or failover. The queue should carry small, fast messages.

Retries get expensive. In your flow the validator can trigger a retry with the better model. With base64 in the message, each retry re-moves the whole payload. With a reference, the worker just re-reads from storage. 

To save cost, we store a hash of the file to detect duplicate submissions and return cached results instead of paying for another model call.

Reliability. Large messages increase the chance of timeouts, lock expiry, and poison-message and dead-letter problems. They are also harder to inspect or replay when debugging.

In this design the flow would be 

1. Client uploads the invoice directly to blob storage (SAS URL) and gets a blob path.
2. Client submits a request with only {invoiceId, blobPath, clientId, ...}.
3. The message goes to the queue.
4. The queue worker reads the blob once, caches it in memory for the duration of the job, and passes it to the invoice model, the validator, and the failover model if needed.
5. Results are written as JSON, and the user gets them to fill the web form.