AI automation, RAG and LLMs, explained for a business that already has software

Most of the businesses that ask us about AI already have software: a store, a booking system, a back office, a support inbox. They are not looking for an AI company. They want the systems they run to do more of the work, and they want to know what the words mean before they spend money on them. This is the plain version of those words, and where each one earns its place.

Large language models (LLMs)

A large language model is software that reads and writes language. It is good at the work that used to need a person reading something — a customer's message, a supplier's invoice, a product specification, an enquiry — and deciding what it says and what should happen next. It is not a database and not a calculator: left to itself it will answer from memory, and its memory is a blend of the public internet. Everything useful that follows is about keeping the model to what it can actually check.

The models come in two kinds. Hosted models from the major providers — OpenAI, Anthropic, Google — are called over an API and billed per use. Open-weight models such as Llama, Mistral and Qwen can be run on your own servers, which matters when your data cannot leave them. We choose per job, and the code keeps the model behind a single switch, because providers retire models faster than businesses retire software.

Prompting, and why fine-tuning is rarely the answer

A prompt is the instruction and context the model is given for each task: what it is for, what it may and may not do, what a good answer looks like, and the facts it should use. Most business work is done with careful prompting plus retrieval, described next. Fine-tuning — training a model further on your own examples — is occasionally worth it for a narrow, high-volume task with a stable format. For most businesses it is the expensive option that gets suggested first and needed last.

Retrieval-augmented generation (RAG)

This is the technique behind most useful business AI, and the name is worse than the idea. Instead of letting the model answer from memory, the system first retrieves the relevant passages from your own material — policies, manuals, orders, tickets, contracts — and hands them to the model with the question. The model then writes an answer from those passages, and can show which ones it used.

Under the hood, your documents are split into pieces and each piece is turned into an embedding: a list of numbers that captures what it is about. Those live in a vector index, which is how the system finds the pieces that match a question even when the wording differs. None of that needs to be visible to your team. What is visible is that the answer cites its source, and that when the source does not exist the system says so instead of inventing one.

Structured outputs and tool use

Language is what the model reads; your systems need data. Structured output means the model is made to return its result in a fixed shape — fields, types, allowed values — so an invoice becomes a record your accounting package will accept, and an enquiry becomes a lead with a budget, a location and a source. Tool use (also called function calling) lets the model ask your systems for something — look up an order, check stock, fetch a listing — rather than guessing. Between them, these are what turn a model from a chat window into a part of your software.

Agents and workflow automation

An agent is a model given a goal, a set of tools and permission to take several steps toward the goal: read the email, look up the order, draft the reply, file the ticket. It is powerful and it is where most of the risk lives, because a mistake in step two becomes the input to step three. The useful versions have narrow goals, tools that can only do what they should, and a person at the points that matter: a reply drafted for someone to send, a refund queued for someone to approve. Workflow automation is the plainer cousin — a fixed sequence with the model doing one language step inside it — and for most businesses it is the better starting point.

Evaluations, guardrails and monitoring

The question every buyer should ask is how do you know it works? The honest answer is an evaluation set: a collection of real cases from your business — real customer messages, real invoices, real enquiries — with the correct result recorded for each. The system is measured against it before launch, and the set is kept, so that when a model is swapped or a prompt is changed the new version can be checked against the same cases.

Guardrails are the limits around the model: what it may talk about, what it must refuse, when its confidence is too low to answer, and what it hands to a person. Monitoring is logging every request and response, so that when something goes wrong you can see exactly what the model saw. Without these three, an AI feature is a demo. With them, it is software.

Where the data goes, and what it costs

Two practical points settle most projects. Data: if a hosted model is used, only what needs to be sent is sent, under the provider's business terms, which do not use your data for training; if your rules require data to stay in your region or on your own servers, an open-weight model is run there instead. Cost: hosted models are billed per token, which is roughly per word processed. The provider account is opened in your name, so you see the bill and control the limit, and the monthly figure is estimated from your real volumes before anything is built.

What we build with all this

The six jobs on our AI integration and development page map directly onto the terms above:

• Support automation — retrieval over your orders and policies, guardrails, hand-off to a person when confidence is low.
• Document extraction — structured output from invoices, purchase orders and forms into the system they belong in.
• Catalogue content — descriptions, attributes and translations generated from your product data and reviewed before publishing.
• Lead qualification — structured output and tool use to read, score and route an enquiry as it lands, with the follow-up drafted for a person to send.
• Search over your own data — retrieval-augmented generation with the source shown and permissions kept.
• AI inside an existing system — a language step added to the store, app or platform you already run, rather than a new tool beside it.

Five things to settle before starting

1. What the work costs you now, in hours or delay or errors, so the result has a number to beat. If a rule or a form would beat it more cheaply, use that.
2. Where your data may go, and whether the model has to run on your own servers.
3. Who pays for usage each month, and what the limit is. It should be you, directly, in your own account.
4. What happens when the model is changed or retired, and whether the evaluation set will be kept so the replacement can be checked.
5. What the contract says about the code and the prompts. For an integration built into your own systems they are normally yours; make sure it is written down.

If you have a piece of work in your business that a person currently reads and decides on, one item at a time, that is the place to start. Describe it in plain language and we will tell you whether it is an AI problem, and if it is, which of the words above it needs.