Run language models on your own dedicated Apple silicon
For teams that want to run open-weight language models without sending prompts to a third-party AI API. You install the inference software, load the model and put your own authenticated API in front of it.
- 01
Authenticated request
Your app, with an API key
- 02
Model on a dedicated Mac
Open-weight, self-installed
- 03
Response
Returned to your app
What this solves
Prompts and documents leaving your control
With your own inference server, prompts are processed on a Mac only you control.
Large models on limited GPU memory
Apple silicon shares unified memory between CPU and GPU, so the memory you choose is available to the model.
Predictable cost
A fixed monthly price instead of paying per token.
Example workflow
An internal assistant for support staff
- 1
Ollama runs on the Mac and serves a quantised open model.
- 2
A reverse proxy with an API key or mTLS listens on HTTPS; the Ollama port is not public.
- 3
Your internal tool sends a support question and the relevant documents.
- 4
The model generates an answer on the Mac.
- 5
The answer is returned to your tool; logs stay on the Mac or in your own logging.
How to set it up and run it
- 1
Choose memory first
The model weights plus the context cache must fit in unified memory, with room left for macOS.
- 2
Pick a quantised model that fits
Lower-bit quantisation uses less memory and may reduce quality; test with your own prompts.
- 3
Install Ollama, MLX or llama.cpp and download the model weights.
- 4
Bind the inference server to localhost and expose it through a reverse proxy with authentication, or only over a VPN.
- 5
Measure throughput with your own prompts and context length before deciding how many concurrent users one Mac can serve.
How much memory a model needs
The model weights, the context cache and macOS all have to fit in unified memory. Approximate sizes for 4-bit quantised models:
| Model size | Weights at 4-bit (approx.) | Configurations to consider |
|---|---|---|
| 7–8B parameters | Weights at 4-bit (approx.)≈ 5 GB | Configurations to consider16 GB and up |
| 13–14B parameters | Weights at 4-bit (approx.)≈ 8–9 GB | Configurations to consider24 GB and up |
| 27–32B parameters | Weights at 4-bit (approx.)≈ 17–20 GB | Configurations to consider32 GB and up |
| 70B parameters | Weights at 4-bit (approx.)≈ 40 GB | Configurations to consider64 GB (48 GB only with a short context) |
Rules of thumb, not guarantees. Longer contexts, higher-precision quantisation and parallel requests need more memory. By default macOS lets the GPU use only part of total memory. Test with your own model and prompts.
Who does what
vBoxx provides
- A dedicated Mac mini with up to 64 GB unified memory (M5 Pro) for the model and its context
- Hosting in Tier III datacenters in the Netherlands, so prompts are processed on hardware in the EU
- Network: 10 Gigabit Ethernet port · 1 Gbit/s unmetered · public IPv4 + IPv6
- Remote APC power control and optional JetKVM console
You manage
- Inference software, models and model licences
- Authentication, TLS and firewall rules for the API
- Measuring capacity and limiting concurrent requests
- Logging, retention and data protection for prompts and outputs
Before you choose this setup
- This is inference, not training. Training or fine-tuning large models needs far more compute than a Mac mini offers.
- We do not publish tokens-per-second figures. Speed depends on model, quantisation, context length and concurrency; measure it with your own workload.
- Running the model on your own Mac does not by itself make the processing GDPR compliant. That depends on how you handle the data.