The rise of generative AI is transforming how we build software—from code generation to intelligent assistants. But integrating AI into your Java/Spring Boot application still feels... foreign.

Enter Spring AI: a project from the Spring team that aims to make AI integration as smooth as any other Spring module—like Spring Data or Spring Security.

Let’s walk through what Spring AI is, what you can build with it, and how to get started in a real Spring Boot application.

🔍 What is Spring AI?

Spring AI is an abstraction layer that helps you integrate LLMs (Large Language Models) like OpenAI, Azure OpenAI, Hugging Face, and more into your Spring Boot application—using familiar idioms like @Service, RestTemplate, and configuration properties.

It provides:

  • 🔄 Prompt templates
  • 📥 Input/output mapping (DTOs)
  • 📚 Embedding & vector store support
  • 📦 RAG-ready integrations (Retriever-Augmented Generation)
  • 🧠 LLM client abstraction

🛠️ Basic Setup

Step 1: Add Spring AI Dependency 

org.springframework.ai
  spring-ai-openai-spring-boot-starter
  0.8.0

Other options:

  • spring-ai-azure-openai-spring-boot-starter
  • spring-ai-ollama-spring-boot-starter (for local models like LLaMA3)
  • spring-ai-huggingface-spring-boot-starter

Step 2: Configure Your AI Provider (OpenAI Example) 

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      base-url: https://api.openai.com/v1

Step 3: Inject & Use the AI Client 

@Service
public class ChatService {

    private final ChatClient chatClient;

    public ChatService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    public String chat(String prompt) {
        ChatResponse response = chatClient.call(prompt);
        return response.getResult().getOutput().getContent();
    }
}

Call chatService.chat("What is the capital of France?") and get an answer.

✨ Prompt Templates (Optional but Powerful)

Create a prompt.st file:

Write a poem about a ${topic} in the style of ${style}.

Then bind and send it:

PromptTemplate template = new PromptTemplate("""
    Write a poem about a ${topic} in the style of ${style}.
""");

Map<String, Object> vars = Map.of("topic", "spring", "style", "Shakespeare");

Prompt prompt = template.create(vars);
ChatResponse response = chatClient.call(prompt);

📚 RAG + Vector Store Support

Spring AI supports Embeddings + Vector DBs (like PGVector, Redis, Milvus, Chroma, etc.) for Retrieval-Augmented Generation.

Example with PostgreSQL + pgvector:

org.springframework.ai
  spring-ai-pgvector-store-spring-boot-starter

You can embed documents, store them, and query by similarity to inject context into LLM prompts—ideal for chatbots or semantic search.

🔗 LangChain4j Compatibility

If you like LangChain but you're in the Java world, Spring AI plays nicely with LangChain4j for more composable workflows.

🌍 Real-World Use Cases

  • 🔎 AI search assistant for documentation
  • 💬 LLM-powered chatbot for support
  • 📝 Text summarizer or document classifier
  • 🤖 Code generation assistant
  • 🧠 Local inference with Ollama or Hugging Face models

🚀 The Future of AI in Spring

Spring AI is still in early development, but the direction is clear: make LLMs as easy to use as RestTemplate or JdbcTemplate. With support for RAG, local models, and full Spring Boot auto-configuration, it's ready for serious experimentation and prototyping.

📦 GitHub & Docs

🧪 Ready to Try It?

Spin up a Spring Boot project, drop in the starter, and start chatting with GPT or your own local model—right from Java. If you want a starter project, let me know—I’ll help scaffold one for you.