Air-gapped LLM deployments in Australia

An air-gapped AI system has no network path to the internet or to any less trusted network. Every model file, package, container image and licence enters on controlled media through a recorded transfer process. The model server is the easy part. The work is in the supply chain around it: how files get in, how you prove they are the files you meant to bring in, and how you stop software from calling home.

What counts as an air gap

A system behind a firewall with an egress allowlist is a restricted network. The hardening guide covers that design. An air gap has no route out at all. Data moves only by physical media or through a cross domain solution, which is a system that enforces data flow rules between security domains.

The difference shows up in the failure modes. On a restricted network, a tool that tries to download a tokeniser at start-up gets blocked and logs an error. In an air gap, the same tool fails the same way, but you cannot fix it by adding an allowlist entry. You must find every such dependency before the first transfer.

Who needs one in Australia

Most workloads in banks, insurers and accounting firms do not need an air gap. A private deployment in an Australian region or in their own data centre meets their residency and control needs. The article on data sovereignty and residency sets out how to decide.

Defence industry is the main exception. Under the Defence Industry Security Program (DISP), membership is mandatory for entities that “work on classified information or assets (i.e. PROTECTED and above)”. DISP has four levels: Entry level for OFFICIAL and OFFICIAL: Sensitive, Level 1 for PROTECTED, Level 2 for SECRET and Level 3 for TOP SECRET. DISP members must also meet the Essential Eight at Maturity Level 2 across the corporate ICT systems they use to correspond with Defence.

The Information Security Manual (ISM) from ASD sets the controls for moving data into and out of such systems. The ISM Guidelines for data transfers (September 2026 edition) include these controls:

ISM-0657: when manually importing data to systems, the data is scanned for malicious and active content.
ISM-1778: imported data that fails security checks is quarantined until reviewed and approved or not approved.
ISM-0661: human users are held accountable for the data transfers they perform.
ISM-1586: data transfer logs record all imports and exports.
ISM-0660: for SECRET and TOP SECRET systems, transfer logs are fully verified at least monthly.

The ISM Guidelines for media add ISM-0347. For manual transfers between security domains, it requires write-once media unless the destination system can ensure read-only access. ISM-0947 requires rewritable media to be sanitised after each transfer.

Other candidates for an air gap are research data under strict agreements, critical infrastructure control environments and some legal or investigative workloads. For these, the decision comes from the contract or the risk assessment rather than from a named rule.

Model weights: transfer and verification

Model weights are the largest and most sensitive import. A 70 billion parameter model at 16 bit precision needs about 140 GB for weights alone: 70 billion x 2 bytes. A 4 bit quantised copy of the same model needs about 35 GB: 70 billion x 0.5 bytes. Add the tokeniser and configuration files, and plan for at least two candidate models per transfer. One transfer bundle is then in the range of 200 to 400 GB. That size decides the write-once media you buy and the time that the malware scan takes.

The transfer has two sides. On the connected side:

Pin the exact model revision. The Hugging Face CLI downloads a specific commit with --revision.
Prefer safetensors files. The safetensors documentation describes it as a format for storing tensors safely as opposed to pickle. Python’s own pickle documentation warns that malicious pickle data can execute arbitrary code during unpickling.
Write a SHA-256 manifest of every file.
Sign the model directory with a key that your transfer officer controls. The OpenSSF model signing project supports Sigstore and also traditional signing with public keys, certificates or PKCS #11 devices. Sigstore records signing events in a public transparency log that the air gap cannot reach. The key-based method works offline.

On the air-gapped side, scan and quarantine the bundle under ISM-0657 and ISM-1778. Then verify the signature and the manifest before the files reach the model store.

# Connected side
hf download meta-llama/Llama-3.3-70B-Instruct --revision <commit> --local-dir ./bundle/llama-3.3-70b
cd bundle && find . -type f ! -name SHA256SUMS -exec sha256sum {} + > SHA256SUMS
model_signing sign key ./llama-3.3-70b --private-key transfer.priv --signature llama.sig

# Air-gapped side, after scan and quarantine release
sha256sum -c SHA256SUMS
model_signing verify key ./llama-3.3-70b --signature llama.sig --public-key transfer.pub

Packages and container images

A typical LLM stack has hundreds of Python packages, several container images and an operating system package set, and each one needs an offline source.

For Python, build a wheel directory on the connected side and install from it with no index. The pip user guide shows pip install --no-index --find-links=/local/wheels -r requirements.txt. Add hash-checking mode with --require-hashes, and disallow source distributions with --only-binary :all:. The hashes travel with the requirements file, and a changed wheel then fails the install.

For containers, export the images with docker image save, which writes them to a tar archive. Load them into an internal registry inside the air gap, and configure Kubernetes to pull only from that registry. Record image digests in the transfer log alongside the model manifest.

Telemetry and phone-home settings

Many components send usage data by default. In an air gap these calls fail, but you still need them switched off. A failed call can slow start-up, fill logs with errors, or show up as a security alert. The settings for the main components are:

vLLM collects anonymous usage data by default. Set VLLM_NO_USAGE_STATS=1 or DO_NOT_TRACK=1.
The Hugging Face libraries honour HF_HUB_OFFLINE. With it set, “no HTTP calls will be made to the Hugging Face Hub.” DO_NOT_TRACK switches off telemetry across the Hugging Face Python ecosystem.
Grafana sends anonymous usage statistics to stats.grafana.org every 24 hours unless reporting_enabled is false. It checks for new versions every ten minutes unless check_for_updates is false.
Self-hosted Langfuse sends deployment telemetry. For the open source edition, set TELEMETRY_ENABLED=false on all application containers.

# Environment for the inference and evaluation containers
export HF_HUB_OFFLINE=1
export DO_NOT_TRACK=1
export VLLM_NO_USAGE_STATS=1
vllm serve /models/llama-3.3-70b --served-model-name llama-3.3-70b

# grafana.ini
[analytics]
reporting_enabled = false
check_for_updates = false
check_for_plugin_updates = false

Then prove it. Run the full stack in a staging enclave with egress blocked and all DNS queries logged. Any lookup for an external name is a dependency you missed.

Evaluation and monitoring without a vendor

Every evaluation and monitoring tool must run inside the air gap, including the judge model. A Ragas style evaluation harness that scores answers with an LLM must point at a local model endpoint. A hosted judge sends your test questions out of the enclave. Keep the evaluation sets, the expected answers and the score history inside the enclave as well.

Traces go to OpenTelemetry collectors, then to a self-hosted Langfuse for prompt and response traces and to Grafana for latency, throughput and GPU use. The evaluation and observability article covers the metrics. In an air gap, add the transfer log to the same dashboards so that the monthly log verification under the ISM uses the same evidence.

Update cadence through controlled media

Inside an air gap, no component updates itself. A workable pattern is a scheduled transfer, for example monthly, with an emergency path for critical security patches. A bundle carries these items:

Operating system and container security patches
Python wheels for any changed dependency, with hashes
New model versions, if any, with manifests and signatures
Updated evaluation sets, when the business adds test cases

A new model or a new vLLM version goes to a staging environment inside the air gap first. It must pass the same evaluation set as the current version before promotion, and the previous version stays on disk for rollback. Model updates are less frequent than package updates, because each one needs a full evaluation run.

Licences for offline use

An air gap does not remove licence obligations. It changes how you meet them:

Open-weight licences differ. gpt-oss-120b and Mistral Small 3.1 carry the Apache 2.0 licence. If you distribute or make available Llama 3.3 or a product that contains it, the Llama 3.3 licence requires a copy of the agreement and a “Built with Llama” notice. It incorporates an Acceptable Use Policy by reference to a web page. Save a dated copy of that page with each transfer, because staff inside the enclave cannot read it. The guide to choosing open-weight models compares licences in more detail.
GPU software licences can need a licence server. NVIDIA’s Delegated License Service runs on-premises for vGPU software, and node-locked licensing lets a client on an air-gapped network read its licence from a local file. For NLS 3.4 and later, licence server files bound to a DLS instance have a validity of three years. Put the renewal date in the transfer calendar.
Commercial tools can require telemetry. Langfuse says that “Enterprise self-hosted telemetry is used for license compliance and cannot be disabled.” Settle this with the vendor in writing before you buy an enterprise edition for an air gap.

Trade-offs

An air gap costs more in time and people. Hosted frontier models are not available. The system runs on open-weight models that fit your own GPUs, and the GPU sizing guide shows how to size them. Security patches arrive on the transfer cadence rather than the same day. Engineers who work inside the enclave need the right clearances, and debugging takes longer because nobody can search an error message from the console. In return, you remove the network path that most of the other controls exist to protect.

Air gap checklist

What to decide next

First confirm that you need a true air gap. If the driver is a DISP level or an ISM classification, the transfer controls above are the minimum. If the driver is general caution, a restricted network in your own data centre can give most of the protection at lower cost. If you go ahead, run one full dry-run transfer of the whole stack into a staging enclave before you order the production hardware. The dry run finds the hidden dependencies while they are cheap to fix.

We design and build these environments as part of on-prem AI deployment, and our forward deployed engineers can work on site where the enclave is.