beginner~3h

Spring AI Setup & Hello World

If you're already a Spring developer, you're most of the way to Spring AI already — it's auto-configuration and dependency injection applied to LLM calls, not a new framework paradigm.

Learning objectives

Beginner: Add the Spring AI OpenAI starter to a project and get a working ChatClient hello-world call end to end.
Intermediate: Swap the same application from OpenAI to a locally-running Ollama model by changing only configuration, not business logic.
Advanced: Wire up multiple chat model beans (e.g. OpenAI and Bedrock) in one application and choose between them explicitly at call time.

◆ The problem

Before Spring AI, integrating an LLM into a Java backend meant hand-rolling HTTP calls to each provider's own API shape, managing API keys manually, writing your own retry/timeout logic, and rewriting all of it if you ever switched providers.

Spring AI is a portability layer: a single, provider-agnostic API (ChatClient, EmbeddingModel, VectorStore, and friends) that different provider "starters" implement underneath. Swapping OpenAI for a local Ollama model is meant to be a change of one Maven dependency and a few properties — not a rewrite of your service layer.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>1.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o-mini

@RestController
public class HelloAiController {

    private final ChatClient chatClient;

    public HelloAiController(ChatClient.Builder builder) {
        this.chatClient = builder.build(); // auto-configured for the OpenAI starter on the classpath
    }

    @GetMapping("/hello-ai")
    public String hello(@RequestParam String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }
}

◆ Under the hood

Adding the OpenAI starter triggers Spring Boot auto-configuration to build an OpenAiChatModel bean, wire it into a ChatClient.Builder, and expose that builder for injection — none of which you wrote yourself. This is the exact same auto-configuration pattern that gives you a DataSource bean from a JDBC starter; Spring AI deliberately doesn't introduce a new mental model, just new starters.

▲ Pitfall

Never hardcode an API key in application.properties committed to source control — always resolve it from an environment variable (as above) or, in production, a secrets manager (see Module 18).

💻 Code example

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>1.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

Ollama runs open-weight models (Llama, Mistral, Gemma, and others) entirely on your own machine — no API key, no per-token billing, no data leaving your network. It's the natural first stop for local development and for anything with strict data-residency requirements.

ollama pull llama3.2
ollama run llama3.2

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

spring.ai.ollama.base-url=http://localhost:11434 spring.ai.ollama.chat.options.model=llama3.2

The HelloAiController from §02.2 doesn't change at all — only the starter dependency and properties changed. This is Spring AI's portability promise demonstrated directly, not just claimed.

💻 Code example

ollama pull llama3.2
ollama run llama3.2

services:
  ollama:
    image: ollama/ollama
    ports: ["11434:11434"]
    volumes: ["ollama_data:/root/.ollama"]
volumes:
  ollama_data:

Containerizing Ollama is worth doing the moment more than one developer (or a CI pipeline) needs a consistent local-model environment — same reasoning as containerizing anything else in this book of modules, just applied to a model runtime instead of an application.

💻 Code example

services:
  ollama:
    image: ollama/ollama
    ports: ["11434:11434"]
    volumes: ["ollama_data:/root/.ollama"]
volumes:
  ollama_data:

AWS Bedrock exposes several model families (Anthropic Claude, Amazon Titan, Meta Llama, and others) behind one managed AWS API — useful when you want a hosted model but need it inside your existing AWS security boundary (IAM, VPC, CloudTrail) rather than calling an external provider directly.

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-bedrock-converse</artifactId>
</dependency>

spring.ai.bedrock.converse.chat.options.model=anthropic.claude-3-5-sonnet-20241022-v2:0 spring.ai.bedrock.aws.region=us-east-1

Authentication follows the standard AWS SDK credential chain (environment variables, IAM instance role, etc.) — Spring AI doesn't reinvent AWS auth, it defers to it.

💻 Code example

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-bedrock-converse</artifactId>
</dependency>

A single application can wire up more than one provider at once — common when, say, a cheap/fast model handles simple classification and a stronger model handles complex generation. Spring AI supports this by qualifying multiple ChatModel / ChatClient beans explicitly rather than relying on a single auto-configured default.

@Configuration
public class ChatClientConfig {

    @Bean
    @Qualifier("fastClient")
    public ChatClient fastClient(OpenAiChatModel model) {
        return ChatClient.builder(model).build();
    }

    @Bean
    @Qualifier("reasoningClient")
    public ChatClient reasoningClient(AnthropicChatModel model) {
        return ChatClient.builder(model).build();
    }
}

Full multi-provider production configuration (routing, dynamic model selection at runtime, per-model timeouts) is covered in Module 18 once you have enough context (advisors, tool calling) to make that configuration meaningful.

✓ Quick recap

What actually changes when you swap the OpenAI starter for the Ollama starter? The Maven dependency and a few application.properties entries — your ChatClient-calling code is unchanged. Why would you choose Ollama over a hosted provider for local development? No API key or per-token billing, and no data leaves your machine — useful for development and strict data-residency needs.

💻 Code example

@Configuration
public class ChatClientConfig {

    @Bean
    @Qualifier("fastClient")
    public ChatClient fastClient(OpenAiChatModel model) {
        return ChatClient.builder(model).build();
    }

    @Bean
    @Qualifier("reasoningClient")
    public ChatClient reasoningClient(AnthropicChatModel model) {
        return ChatClient.builder(model).build();
    }
}

Want a visual for this concept?

Generate a diagram tailored to “Spring AI Setup & Hello World” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to ChatClient, Messages & Prompts →← Back to all Spring AI chapters