Glossary

AI glossary

The terms used across our field guides, each linked to the article that explains it.

Air gap: A system with no network path to the internet or to less trusted networks. Models, packages and updates enter only on controlled media through a logged transfer process. Read more
Break-even utilisation: The share of the year that rented GPUs must run before owning the same GPUs costs less. You calculate it from the hardware price, the hourly rental rate and the running costs. Read more
Cross-region inference profile: A named route on Amazon Bedrock that defines a model and the AWS regions that can serve a request for it. The au. profiles route between Sydney and Melbourne. Read more
Data residency: The country or cloud region where data is stored and processed. It says nothing about which laws can reach the data or who holds the keys. Read more
Data sovereignty: The question of which governments can legally compel access to data, which depends on the legal entities that control it rather than on where it is stored. Read more
Data Zone deployment: An Azure OpenAI deployment type that processes prompts anywhere inside a Microsoft-defined zone. The Asia Pacific zone covers several countries, so it does not keep processing in Australia. Read more
Egress control: Rules that decide which outbound connections a server can make. A private LLM stack blocks all outbound traffic from the model servers except to named internal services. Read more
Faithfulness: An evaluation metric that counts how many claims in an answer the retrieved passages support, divided by the total number of claims. It runs from 0 to 1. Read more
GPU utilisation: The share of paid GPU hours that do useful work at your latency target. It decides whether a self-hosted model costs less per token than an API. Read more
Hybrid search: Search that runs a keyword ranking such as BM25 and a vector search on the same question, then merges the two result lists. It finds exact identifiers and passages with similar meaning. Read more
KV cache: GPU memory that stores the key and value vectors for every token in every active request. It grows with the number of users and the length of each request. Read more
LoRA (low-rank adaptation): A fine-tuning method that freezes the original weights and trains small matrices beside chosen layers. QLoRA does the same on a base model loaded in 4-bit precision. Read more
Mixture of experts (MoE): A model design that splits layers into many expert blocks and sends each token through only a few of them. All experts must sit in GPU memory, but each token uses only the active ones. Read more
Open-weight model: A language model whose trained weights you can download and run on your own servers, under the licence the publisher sets. The licence decides who may use it commercially and on what conditions. Read more
Operational control: Who holds the encryption keys and which people can access the systems and data. It is a separate guarantee from residency and sovereignty. Read more
Permission-aware retrieval: Retrieval that returns only the chunks that the person asking can open in the source system. The access groups are stored on each chunk and applied as a filter inside the search query. Read more
Private endpoint: A network interface with a private IP address in your own network that connects to a cloud service. Traffic to the service does not need an internet gateway. AWS calls this PrivateLink. Read more
Quantisation: Storing model weights in fewer bits, such as FP8 or 4-bit formats, to cut memory and increase speed. A quantised model can lose some quality, so it needs its own evaluation. Read more
Reciprocal rank fusion: A method that merges ranked lists by giving each result a score of 1 / (k + rank) in each list and adding the scores. Results that rank well in both lists move to the top. Read more
Reranker: A model that reads the question and one candidate passage together and returns a relevance score. It runs on the top candidates from search, because it costs more compute per passage than a vector comparison. Read more
Safetensors: A file format for model weights that stores tensors without executable code. It replaces pickle files, which can run code when they are loaded. Read more
Tensor parallelism: A way to split each layer of a model across several GPUs. The GPUs exchange data at every layer, so they need a fast link such as NVLink. Read more
VPC (virtual private cloud): A private network inside your own cloud account. GPU instances in private subnets of a VPC have no public IP address. Read more

Next step

We map where your hours go and price the work that is worth automating.

Talk to our team See how we work