Sessions
Understanding Julep Sessions and state management
Overview
Sessions in Julep are the backbone of stateful interactions between users and agents. They maintain the context and history of conversations, enabling personalized and coherent interactions over extended periods. Whether it’s handling ongoing customer support inquiries or having a conversation with a user, sessions ensure that the agent retains necessary information to provide meaningful responses.
Components
Sessions are comprised of several key components that work together to manage state and context:
- Session ID: A unique identifier (
uuid7) for each session. - User: The individual or entity interacting with the agent, represented by its id.
- Agent: The AI entity interacting with the user within the session, represented by its id.
- History: The history of the conversation, which the agent uses to generate relevant responses.
- System Template: A specific system prompt template that sets the background for this session.
- Situation: A description of the current situation for the session.
- Metadata: Additional data associated with the session, such as user preferences, session preferences, and other relevant information.
Session Configuration Options
When creating a session, you can leverage the following configuration options to tailor the experience:
| Option | Type | Description | Default |
|---|---|---|---|
agent_id | string | The ID of the agent to associate with the session | Required |
user_id | string | The ID of the user interacting with the agent | Required |
context_overflow | truncate | adaptive | Strategy for handling context overflow: truncate cuts off the oldest context; adaptive adjusts dynamically | adaptive |
metadata | object | Additional metadata for the session (e.g., user preferences) | null |
situation | string | null | Description of the current situation for the session | null |
system_template | string | A specific system prompt template that sets the background for this session | Default template provided by Julep |
render_templates | boolean | Whether to render system and assistant messages as Jinja templates | true |
token_budget | integer | null | Threshold value for the adaptive context functionality | null |
auto_run_tools | boolean | Whether to auto-run tools and send the tool results to the model when available | false |
forward_tool_calls | boolean | Whether to forward tool calls directly to the model | false |
recall_options | RecallOptions | Options for recalling context or certain data during the session | null |
Recall Options
When configuring a session, you can specify recall options to control how context or certain data is recalled during the session. Below are the available options:
| Option | Type | Description | Default |
|---|---|---|---|
mode | hybrid | vector | text | The mode to use for the search. | vector |
num_search_messages | integer | The number of search messages to use for the search. | 4 |
max_query_length | integer | The maximum query length to use for the search. | 1000 |
alpha | float (0.0 - 1.0) | The weight to apply to BM25 vs Vector search results. 0 => pure BM25; 1 => pure vector. | 0.7 |
confidence | float (-1.0 - 1.0) | The confidence cutoff level. | 0.6 |
limit | integer (1 - 50) | The limit of documents to return. | 10 |
lang | en-US | The language to be used for text-only search. Support for other languages coming soon. | en-US |
metadata_filter | object | Metadata filter to apply to the search. | {} |
mmr_strength | float (0.0 - 1.0) | MMR Strength (mmr_strength = 1 - mmr_lambda). | 0 |
The System Template is a specific system prompt written as a Jinja template that sets the foundational context and instructions for the agent within a session. It defines the background, directives, and any relevant information that the agent should consider when interacting with the user. For more details on Jinja templates, refer to the Jinja documentation.
How to Use Sessions
Sessions are integral to maintaining a continuous and coherent interaction between users and agents. Here’s how to create and manage sessions using Julep’s SDKs.
Creating a Session
Here are examples of how to create a session using the SDKs:
Chatting in a Session
Once a session is created, you can engage in a conversation by sending messages to the agent within that session.
Relationship to Other Concepts
This section will help you understand how sessions relate to other concepts in Julep.
Agents
Agents operate within sessions to provide personalized and context-aware interactions. While an agent defines the behavior and capabilities, a session maintains the state and context of interactions between the agent and the user. In other words, the history of a conversation is tied to a session, rather than an agent.
Example:
In this example, the agent David is used in two different sessions, each with a different situation. The agent’s behavior and responses are tailored to the specific situation of each session, and the history of messages in session1 and session2 are separate.
Users
When a user (or more) is added to a session, the session will be able to access information about the user such as name, and about in order to personalize the interaction. Check out the system_template to see how the user’s info is being accessed.
This is how you can create a user and associate it with a session:
In this example, the user John Doe is associated with the agent Mark Lee in the session. The session will use the user’s information to personalize the interaction, such as using the user’s name in the system prompt.
Tools
Sessions have the ability to use Tools. If an agent has a tool, and the LLM decides to use it, the tool will be run and the result will be automatically sent to the LLM for further processing.
Example:
If the agent that’s associated with the session has a tool called fetch_weather, and the LLM decides to use it, the tool will be run and the result will be automatically sent to the LLM for further processing.
Documents
When chatting in a session, the session can automatically search for documents that are associated with any of the agents and/or users that participate in the session. You can control whether the session should search for documents when chatting using the recall option of the chat method, which is set to True by default. You can also set the session’s recall_options when creating the session to control how the session should search for documents.
When running the above code with an agent that has documents about Julep, the session will search for documents that are relevant to the conversation and return them in the response.docs field.
This example is taken from the crawling-and-rag cookbook. Check it out here.
Best Practices
Session Management
- 1. Reuse Sessions: Reuse existing sessions for returning users to maintain continuity in interactions.
- 2. Session Cleanup: Regularly clean up inactive sessions to manage resources efficiently.
- 3. Context Overflow Strategy: Choose an appropriate context overflow strategy (e.g., “adaptive”) to handle long conversations without losing important information.
Personalization
- 1. Leverage Metadata: Use session metadata to store and retrieve user preferences, enhancing personalized interactions.
- 2. Maintain Context: Ensure that the context within sessions is updated and relevant to provide coherent and context-aware responses.
Performance Optimization
- 1. Efficient Searches: Optimize search queries within sessions to retrieve relevant documents quickly.
- 2. Manage Token Usage: Monitor and manage token usage to ensure efficient use of resources, especially in long sessions.
Next Steps
- Agent Tools - Learn about tools and how to use them with agents
- Agent Tasks - Learn about tasks and how to use them with agents
- Agent Docs - Learn about docs and how to use them with agents