Getting Started with Spring AI Advisors

The Spring AI Advisors API is a powerful framework that provides a structured and extensible way to intercept, modify, and enhance AI-driven interactions in our Spring-based applications. This article explores how Advisors work, their benefits, and how they can be implemented.

1. Core Components

The API is built around key components designed to handle non-streaming and streaming scenarios effectively. For non-streaming interactions, it utilizes CallAroundAdvisor and CallAroundAdvisorChain, while for streaming scenarios, StreamAroundAdvisor and StreamAroundAdvisorChain are employed. The API also incorporates AdvisedRequest to represent the unsealed prompt request and AdvisedResponse to encapsulate the resulting chat completion response.

Together, these components enable the creation of robust and flexible chat systems capable of addressing diverse requirements.

1.1 Understanding the Advisor Workflow

The Advisor system works like a chain, where each Advisor in the sequence gets a chance to handle both the incoming request and the outgoing response. Here is a simple overview:

1. AdvisedRequest Creation
   An AdvisedRequest is created from the user’s prompt, along with an empty AdvisorContext.
2. Request Processing
   Each Advisor in the chain processes the AdvisedRequest, potentially modifying it. The Advisor can either:
   • Forward the execution to the next Advisor in the chain, or
   • Block the request by not invoking the next entity.
3. Final Advisor
   The last Advisor sends the modified request to the Chat Model.
4. Response Handling
   The Chat Model’s response is passed back through the chain as an AdvisedResponse, a combination of:
   • The original ChatResponse.
   • The AdvisorContext from the input path of the chain.
5. Response Augmentation
   Each Advisor can process or modify the AdvisedResponse.
6. Final Response
   The augmented ChatResponse is returned to the client.

2. Implementing Advisors

Spring AI provides built-in Advisors for common tasks, including managing chat memory, enhancing prompts, and implementing Retrieval-Augmented Generation (RAG). Below are some of their implementations:

2.1 Chat Memory Advisors

ChatMemoryAdvisor offers an efficient set of Advisor implementations that enhance the functionality of chat systems. These Advisors maintain a detailed record of past interactions, seamlessly incorporating the conversation history into new chat prompts.

2.1.1 Using MessageChatMemoryAdvisor

The MessageChatMemoryAdvisor can be used to manage conversation history, ensuring the chat retains context across multiple interactions. This advisor is useful for creating dynamic, context-aware chat applications. Below is an example implementation:

@Service
public class ChatService {
    
    private final ChatClient chatClient;

    public ChatService(@Qualifier("openAiChatModel") ChatModel chatModel) {
        // Initialize an in-memory chat memory
        ChatMemory chatMemory = new InMemoryChatMemory();
        MessageChatMemoryAdvisor memoryAdvisor = new MessageChatMemoryAdvisor(chatMemory);

        // Build the ChatClient with the advisor
        this.chatClient = ChatClient.builder(chatModel)
                .defaultAdvisors(memoryAdvisor)
                .build();
    }

    public String addItemAndFetchHistory(String item) {
        // Send a prompt to add an item and return the chat history
        return chatClient.prompt()
                .user("Add this item to the list and show all items: " + item)
                .call()
                .content();
    }
}

Example Controller to Interact with the Chat Service

@RestController
@RequestMapping("/chat")
public class ChatController {

    private final ChatService chatService;

    @Autowired
    public ChatController(ChatService chatService) {
        this.chatService = chatService;
    }

    @PostMapping("/addItem")
    public String addItemToChatMemory(@RequestParam String item) {
        return chatService.addItemAndFetchHistory(item);
    }
}

The ChatService encapsulates the chat logic by initializing a ChatClient with an InMemoryChatMemory and a MessageChatMemoryAdvisor, ensuring the chat memory is preserved across interactions. The addItemAndFetchHistory method processes a user-provided item, sends it to the chat client, and returns the updated chat history.

The ChatController exposes a REST endpoint /addItem to handle user requests, delegating the processing to the service. The MessageChatMemoryAdvisor preserves the conversation context by adding items to the chat memory and ensuring each response includes all previously added items for smooth interaction flow.

We can test the application using curl or any HTTP client tool.

Add an item:

curl -X POST "http://localhost:8080/chat/addItem?item=Apples"

Response:

Apples

Add another item:

curl -X POST "http://localhost:8080/chat/addItem?item=Bananas"

Apples, Bananas

Add one more item:

curl -X POST "http://localhost:8080/chat/addItem?item=Oranges"

Response:

Apples, Bananas, Oranges

2.1.2 Using PromptChatMemoryAdvisor

The PromptChatMemoryAdvisor updates the system prompt with the current memory to keep the conversation flowing smoothly. It adds new items to the memory, so each response includes the full conversation history, making sure the chat model manages context effectively. Below is an implementation of this functionality as a service and controller within a Spring Boot application.

@Service
public class ChatService2 {

    private final ChatClient chatClient;

    public ChatService2(ChatModel chatModel) {
        ChatMemory chatMemory = new InMemoryChatMemory();
        PromptChatMemoryAdvisor promptChatMemoryAdvisor = new PromptChatMemoryAdvisor(chatMemory);
        this.chatClient = ChatClient.builder(chatModel)
                .defaultAdvisors(promptChatMemoryAdvisor)
                .build();
    }

    public String addItemAndFetchHistory(String item) {
        return chatClient.prompt()
                .user("Add this name to a list and return all the values: " + item)
                .call()
                .content();
    }
}

The Chat Service sets up a ChatClient with an InMemoryChatMemory to manage chat history and integrates a PromptChatMemoryAdvisor as the default advisor, enabling dynamic updates to the system prompt with the current memory context.

2.2 Using the SafeGuardAdvisor

AI models are built to assist users, but they can sometimes be misused for unintended purposes. To address this, the SafeGuardAdvisor ensures that AI models do not generate harmful or inappropriate content by blocking certain inputs or adjusting responses to align with ethical guidelines.

Here is how we can use the SafeGuardAdvisor to implement safeguards in a Spring Boot application:

@RestController
public class LearningController {

    private final ChatClient chatClient;

    public LearningController(ChatClient.Builder chatClient) {
        // Define forbidden topics or terms
        List<String> forbiddenTopics = List.of("violence", "hate speech", "illegal activities");
        
        // Configure SafeGuardAdvisor with a custom message and priority
        SafeGuardAdvisor safeGuardAdvisor = new SafeGuardAdvisor(
            forbiddenTopics, 
            "I'm sorry, I cannot assist with that topic.", 
            1
        );
        
        // Initialize ChatClient with the SafeGuardAdvisor
        this.chatClient = chatClient.defaultAdvisors(safeGuardAdvisor).build();
    }

    // Expose an endpoint to interact with the chat model
    @GetMapping("/ask")
    public String askAI(
        @RequestParam(value = "question", defaultValue = "What is Spring AI?") String question
    ) {
        return this.chatClient.prompt()
                              .user(question)
                              .call()
                              .content();
    }
}

The SafeGuardAdvisor operates by first defining a list of forbidden topics to block inappropriate or harmful content. It intercepts user prompts and, when a forbidden term is detected, replaces the model’s response with a predefined advisory message.

Users send their questions through the /ask endpoint, and the advisor checks every response to make sure it follows the set rules, ensuring safe and appropriate AI interactions. we can run the following curl command to test if the SafeGuardAdvisor intercepts the request for a forbidden topic like “violence“:

curl --location 'http://localhost:8080/ask?question=Tell%20me%20about%20violence'

2.3 Using the QuestionAnswerAdvisor

The QuestionAnswerAdvisor enables the integration of contextual knowledge stored in a vector database with the chat model. By leveraging this advisor, the chat system can provide answers based on preloaded or dynamically updated information.

Here’s how to configure and use the QuestionAnswerAdvisor:

@Service
public class KnowledgeBasedChatService {

    private final ChatClient chatClient;
    private final VectorStore vectorDatabase;

    public KnowledgeBasedChatService(ChatClient.Builder chatClientBuilder, VectorStore vectorDatabase) {
        // Initialize a VectorStore with predefined knowledge
        this.vectorDatabase = vectorDatabase;
        Document initialKnowledge = new Document("Water boils at 100 degrees Celsius under normal atmospheric pressure.");
        List<Document> preparedDocuments = new TokenTextSplitter().apply(List.of(initialKnowledge));
        this.vectorDatabase.add(preparedDocuments);

        // Set up the QuestionAnswerAdvisor using the vector database
        QuestionAnswerAdvisor knowledgeAdvisor = new QuestionAnswerAdvisor(vectorDatabase);

        // Build the ChatClient with the advisor
        this.chatClient = chatClientBuilder.defaultAdvisors(knowledgeAdvisor).build();
    }

    public String handleKnowledgeQuery(String question) {
        return chatClient.prompt()
                         .user(question)
                         .call()
                         .content();
    }
}

In the above code, the VectorStore holds the knowledge, such as “Water boils at 100 degrees Celsius under normal atmospheric pressure,” which is prepared using the TokenTextSplitter and added to the vector database. The QuestionAnswerAdvisor is then configured with this database to incorporate the stored knowledge into the chat model’s responses. When a user submits a query, the handleKnowledgeQuery method processes it and leverages the advisor to deliver answers based on the stored information.

2.4 Implementing a Custom Advisor

In this section, we will demonstrate how to create a custom advisor for logging purposes. The advisor will log information about both the incoming user request and the outgoing response, providing insights into the communication between the user and the AI model. We will create an ActivityLoggingAdvisor class that implements the CallAroundAdvisor interface.

The advisor will log the user’s input before the request is processed and log the response content after the request is processed.

@Component
public class ActivityLoggingAdvisor implements CallAroundAdvisor {

    private final static Logger logger = LoggerFactory.getLogger(ActivityLoggingAdvisor.class);

    @Override
    public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) {
        // Log the user's input before processing the request
        advisedRequest = this.logBeforeRequest(advisedRequest);

        // Continue with the next advisor in the chain
        AdvisedResponse advisedResponse = chain.nextAroundCall(advisedRequest);

        // Log the response content after processing the request
        logAfterResponse(advisedResponse);

        return advisedResponse;
    }

    private AdvisedRequest logBeforeRequest(AdvisedRequest advisedRequest) {
        // Log the prompt sent by the user
        logger.info("User Prompt: " + advisedRequest.userText());
        return advisedRequest;
    }

    private void logAfterResponse(AdvisedResponse advisedResponse) {
        // Log the response received from the AI model
        logger.info("AI Response: " + advisedResponse.response()
          .getResult()
          .getOutput()
          .getContent());
    }

    @Override
    public String getName() {
        return "ActivityLoggingAdvisor";
    }

    @Override
    public int getOrder() {
        return 0;
    }
}

Here is how it works:

• aroundCall(): The aroundCall method logs the request and the response in sequence, providing an easy way to monitor all interactions.
• logBeforeRequest(): This method logs the user’s input prompt before the request is processed.
• logAfterResponse(): This method logs the response content returned by the AI model after the request is processed.

Now, we will integrate the ActivityLoggingAdvisor into a Spring service. This service will interact with the AI model and use the advisor to log both the prompt and the AI response.

@Service
public class ChatService3 {

    private final ChatClient chatClient;

    @Autowired
    public ChatService3(ChatClient.Builder chatClientBuilder, ActivityLoggingAdvisor activityLoggingAdvisor) {
        this.chatClient = chatClientBuilder
                .defaultAdvisors(activityLoggingAdvisor)  // Add the custom logging advisor
                .build();
    }

    public String initiateChat(String userQuery) {
        return this.chatClient.prompt()
                .user(userQuery)
                .call()
                .content();
    }
}

The controller method below allows users to send queries to the AI model via the /inquire endpoint. The conversation flow will be logged by the ActivityLoggingAdvisor.

    @GetMapping("/inquire")
    public String askAI(@RequestParam(value = "question", defaultValue = "What is the capital of France?") String question) {
        return chatService3.initiateChat(question);
    }

To interact with this controller, we would use the following request:

curl --location 'http://localhost:8080/inquire?question=What%20is%20the%20capital%20of%20France?'

3. Conclusion

In this article, we explored the use of various advisors in a Spring AI application, focusing on how they enhance the functionality and customization of chat interactions. We covered the implementation of advisors such as PromptChatMemoryAdvisor, SafeGuardAdvisor, MessageChatMemoryAdvisor, and QuestionAnswerAdvisor, showcasing their roles in ensuring safe interactions, logging contextual data, and responding to user queries with enriched knowledge from a vector store.

4. Download the Source Code

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design

and many more ....

I agree to the Terms and Privacy Policy

Sign up

Thank you!

We will contact you soon.