What is an AI agent?
An AI agent is software that uses a language model to plan a task with several steps, call tools and hand off at defined checkpoints. A tool call is a request from the agent to another system, such as an ERP lookup or a draft posted to the CRM.
What production looks like
A demo reads one clean document and pulls out the fields. Production has the supplier who sends a photo of a page and the credit note that arrives before the invoice. It has the job code that only one person knows and the month when volume triples. Most of the work is in the exception paths, the review points and the monitoring that shows drift before your customers see it.
How we scope it
A build starts with a written workflow definition that answers four questions:
- What comes in, in every form it arrives in
- What must be true for the output to be correct
- What happens when the agent is not confident
- Who is accountable for the output once it leaves
We tell you before you fund the build if any of those questions has no answer.
Orchestration and tool calls
We build the agent as a graph in LangGraph. Each step is a node, such as extract the fields, look up the record, check the rules or route the case. The graph keeps the state of each case. A case can wait for a reviewer and continue from the same point. Tool calls go through the APIs of your systems of record, such as the ERP, the ledger, the claims platform or the CRM. The agent signs in with a service account that holds only the permissions the workflow needs. The model can run in your own cloud account or on your own servers, as private AI describes.
Confidence thresholds and review queues
Every output carries a confidence score and the checks it passed. A case above the threshold and under the cap goes straight through. A case below it goes to a review queue with the source document, the fields the agent extracted and the reason it stopped. We set each threshold and cap by the cost of an error. A misfiled internal document is cheap and can run without review. A payment to a new bank account cannot be reversed, and it goes to a person every time.
Evaluation
Before go live we build an evaluation harness from your historical cases with known outcomes. Each case holds the input and the output your people agree is correct. pytest runs the checks and scores the agent on each field and each decision. The harness runs again after every change to a prompt, a rule, a threshold or the model. The agent can start in shadow mode, where it drafts each case while your team still does the work, and we compare the two.
Observability and audit trail
Langfuse traces every run, and OpenTelemetry sends the same data to the monitoring you already use. A trace holds the input, each model call, each tool call, the confidence score, the reviewer action and the final output. An alert goes out when accuracy on recent cases drops below the agreed level. The audit trail stays in your environment. The documented rollback sends every case to the manual queue or restores the previous prompt, rules and model.
Use cases
- Financial services: Credit file assembly, AML alert triage, Claims triage, Month-end close, Audit evidence, Advice files, Lease abstraction, KYC and onboarding, Complaints handling, Regulatory reporting, Underwriting submissions, Board reporting
- Construction and industrial: Estimating, Tendering, Quoting, Procurement, Order entry, Accounts payable, Billing
- Both: Document search
In each one the agent gathers the evidence and drafts the case, and a person decides. All use cases are on one page.
In practice
A Sydney builder sends job descriptions to a WhatsApp number. The system drafts an estimate from the builder’s own cost catalogues and past pricing and sends it back for review, in minutes rather than hours. For a meat distributor we built an agent on the product database. It reads text and WhatsApp orders, prices each line and drafts the sales order and the quote. The business now handles more than 100 orders a day and has doubled without adding headcount.
Handover
An operator on your side learns to read the traces, adjust the rules and thresholds, retire a prompt and rerun the evaluation harness. We train that person, write the runbook with them and stay on call while adoption settles.