Field guide
Private AI and self-hosted LLMs
Private AI means the model, the documents it reads and the logs it writes stay inside an environment you control. That can be your own servers, a VPC in an Australian cloud region or a hosted model under your own cloud account.
This guide covers the decisions in the order they come up: where it runs, what hardware it needs, which model, how retrieval works, how it is secured and evaluated, and when it costs less than an API.
- 01
On-prem, VPC or hybrid: where a private LLM should run
How on-prem, VPC and hybrid LLM deployments differ at each layer, from vLLM serving to egress rules, and which one fits an Australian regulated business.
- 02
Data sovereignty vs data residency for AI in Australia
Residency, sovereignty and operational control are three separate guarantees. How APP 8, APP 11, CPS 234, CPS 230 and the HCF apply to an AI system.
- 03
Amazon Bedrock and Azure OpenAI in Australian regions
Which models run in Sydney, Melbourne and Australia East as of October 2026, how routing options move prompts, and what each provider retains.
- 04
How much GPU memory an LLM needs: a worked sizing guide
Work out GPU memory for model weights and KV cache, estimate throughput from memory bandwidth, and pick a GPU tier for on-prem LLM inference.
- 05
Open-weight LLMs compared: licences, size and fit in 2026
How to pick an open-weight LLM for private use in Australia, by evaluation set, licence terms, GPU footprint, context, languages and tool calling.
- 06
Private RAG stack: parsing, hybrid search and permissions
A reference stack for RAG on your own servers. Parsing, chunking, embeddings, pgvector or Qdrant or Elasticsearch, reranking, ACLs and page citations.
- 07
When to fine-tune a model and when to use retrieval
When to use retrieval, when to fine-tune and in what order, with LoRA and QLoRA explained, upkeep costs, evaluation baselines and a decision checklist.
- 08
LLM evaluation and tracing on your own servers
Build an evaluation set from past cases, gate releases in CI, trace with OpenTelemetry into self-hosted Langfuse and Grafana, and log for audit.
- 09
Securing a self-hosted LLM: controls mapped to Essential Eight
Layered controls for a private LLM stack, from network and identity to model files, prompt injection, output checks and patching, mapped to E8 and CPS 234.
- 10
Air-gapped LLM deployments in Australia
What a true air gap needs for an LLM stack: weight transfer and verification, offline mirrors, no telemetry, local evaluation, licences, and who needs it.
- 11
Buying, leasing or renting GPUs for private AI in Australia
Buy, lease or rent GPUs for private AI in Australia. Which GPU instances each cloud offers locally, the break-even utilisation and the contract risks.
- 12
Self-hosted LLM or API: a break-even model
A transparent cost model for self-hosted LLMs against a per-token API in an Australian region, with a worked break-even and the costs teams leave out.
Next step
We map where your hours go and price the work that is worth automating.