Ollama: One-Command Setup and Model Pulls

Ollama is presented as the first and arguably simplest of the 4 ways to run LLMs locally. Getting started is a single install command, and once installed, downloading any of Ollama's supported models is also just one command — no manual weight downloads, no separate quantization step, no wrestling with format conversions before you can actually run something.

Running a model through Ollama is similarly simple: a single command starts an interactive chat session with the downloaded model, running entirely on your own machine. This immediacy — install, pull, run, all in three short commands — is exactly what makes Ollama the natural first stop for engineers who want to try a model locally without committing to a more involved setup.

Beyond interactive CLI use, Ollama exposes a local API server (typically on localhost), which is what makes it useful for actual application development rather than just manual chatting. For programmatic usage, Ollama has an official Python package, and it also has integrations with popular orchestration frameworks like LlamaIndex and CrewAI — meaning you can swap a cloud LLM provider for a locally-running Ollama model in an existing agent or RAG pipeline with relatively little code change, since these frameworks already have first-class Ollama support.

Ollama's core trade-off relative to the other 3 tools in this topic: it prioritizes ease of use and a smooth developer experience over raw inference throughput. For quick local testing, privacy-sensitive prototyping, or building small local-first tools, that trade-off is exactly right — for high-throughput production serving, vLLM's purpose-built serving optimizations (covered later in this topic) are the better fit.

# Ollama's core workflow — install, pull a model, run it, and use
# it programmatically via the Python package.

# 1) Install (one command, varies by OS — see ollama.com)
# 2) Pull a model
#    $ ollama pull llama3.1
# 3) Run it interactively
#    $ ollama run llama3.1

# 4) Programmatic usage via the official Python package
import ollama

response = ollama.chat(
    model="llama3.1",
    messages=[{"role": "user", "content": "Explain what a vector database is, briefly."}],
)
print(response["message"]["content"])

# Ollama also exposes a local REST API (default: http://localhost:11434),
# which is what LlamaIndex/CrewAI integrations talk to under the hood —
# letting you swap a cloud LLM for a local Ollama model with minimal changes