Brad Hutchings

Deployment Models for LLMs

The chart below shows four basic deployment models, with examples, advantages, and disadvantages for each. The purple background models are better for privacy.

Brand Name SaaS

Examples:

  • Website: OpenAI.com
  • Website: X.com/grok
  • Website: Perplexity.ai
  • Website: Deepseek.com

Advantages: Available on any connected device, usually have free tier.

Disadvantages: User data may be at risk. User not in control of inputs and outputs. Monthly fee for desired features, models.

User Device

Examples:

  • Software: Brad’s LLMs
  • Software: Ollama
  • Hardware: Laptop or Desktop PC

Advantages: No data leaves the device, free or fixed software cost.

Disadvantages: Large download, large (16+ GB) RAM requirement, not wicked fast without GPU.

Note: Software and model run on the device with no communication to other servers.

Private Local Server

Examples:

  • Software: Brad’s LLMs
  • Software: Ollama
  • Hardware: M4 Mac Mini, Raspberry Pi 5, PC

Advantages: Available on any connected device on your local network or with access to your local network.

Disadvantages: Requires a dedicated device with adequate resources (16+ GB RAM), maintenance, security.

Note: Software and model run on the local server with no communication to other servers.

Private Hosted Server

Examples:

  • Website: KevinFerron.com

Advantages: Available on any connected device, data sent to and from server not retained (by contract).

Disadvantages: Monthly fee for private server.

Note: Software and model run on the private server with no communication to other servers.