Sessions

​

Overview

Sessions in Julep are the backbone of stateful interactions between users and agents. They maintain the context and history of conversations, enabling personalized and coherent interactions over extended periods. Whether it’s handling ongoing customer support inquiries or having a conversation with a user, sessions ensure that the agent retains necessary information to provide meaningful responses.

​

Components

Sessions are comprised of several key components that work together to manage state and context:

• Session ID: A unique identifier (uuid7) for each session.
• User: The individual or entity interacting with the agent, represented by its id.
• Agent: The AI entity interacting with the user within the session, represented by its id.
• History: The history of the conversation, which the agent uses to generate relevant responses.
• System Template: A specific system prompt template that sets the background for this session.
• Situation: A description of the current situation for the session.
• Metadata: Additional data associated with the session, such as user preferences, session preferences, and other relevant information.

​

Session Configuration Options

When creating a session, you can leverage the following configuration options to tailor the experience:

​

Recall Options (RAG Search)

When configuring a session, you can specify recall options to control how context or certain data is recalled during the session. Below are the available options based on search mode:

​

System Template

​

How to Use Sessions

Sessions are integral to maintaining a continuous and coherent interaction between users and agents. Here’s how to create and manage sessions using Julep’s SDKs.

​

Creating a Session

Here are examples of how to create a session using the SDKs:

client = Julep(api_key="YOUR_API_KEY")
session = client.sessions.create(
    agent="agent_id",
    user="user_id",
    context_overflow="adaptive",
)

​

Chatting in a Session

Once a session is created, you can engage in a conversation by sending messages to the agent within that session.

client = Julep(api_key="YOUR_API_KEY")
response = client.sessions.chat(
    session_id="SESSION_ID",
    messages=[
        {
            "role": "user", # or "system"
            "content": "YOUR_MESSAGE"
        }
    ]
)
print("Agent's response:", response.choices[0].message.content)

​

Relationship to Other Concepts

This section will help you understand how sessions relate to other concepts in Julep.

​

Agents

Agents operate within sessions to provide personalized and context-aware interactions. While an agent defines the behavior and capabilities, a session maintains the state and context of interactions between the agent and the user. In other words, the history of a conversation is tied to a session, rather than an agent.

Example:

agent = client.agents.create(
    name="David",
    about="A news reporter",
    model="gpt-4o-mini",
    instructions=["Keep your responses concise and to the point.", "If you don't know the answer, say 'I don't know'"],
    metadata={"channel": "FOX News"},
)

session1 = client.sessions.create(agent=agent.id, user="user_id", situation="The user is interested in the latest news about the stock market.")
session2 = client.sessions.create(agent=agent.id, user="user_id", situation="The user is interested in political news in the United States.")

​

Users

When a user (or more) is added to a session, the session will be able to access information about the user such as name, and about in order to personalize the interaction. Check out the system_template to see how the user’s info is being accessed.

This is how you can create a user and associate it with a session:

client = Julep(api_key="YOUR_API_KEY")
user = client.users.create(name="John Doe", about="A 21 year old man who is a student at MIT.")
agent = client.agents.create(name="Mark Lee", about="A 49 year old man who is a retired software engineer.")
session = client.sessions.create(agent_id=agent.id, user_id=user.id)

​

Tools

Sessions have the ability to use Tools. When auto_run_tools is set to true (available in chat calls), if an agent has a tool and the LLM decides to use it, the tool will be executed automatically and the result will be sent back to the LLM for further processing. When auto_run_tools is false (default), tool calls are returned in the response without execution.

Example:

If the agent that’s associated with the session has a tool called fetch_weather, and the LLM decides to use it:

• With auto_run_tools=true: The tool executes automatically and returns weather data to the LLM
• With auto_run_tools=false: The tool call is returned in the response for manual execution

# With automatic tool execution
response = client.sessions.chat(
    session_id="session_id",
    messages=[
        {
            "role": "user",
            "content": "What is the weather in San Francisco?"
        }
    ],
    auto_run_tools=True,  # Tools execute automatically
    recall_tools=True     # Tool calls/results included in message history
)

print("Agent's response:", response.choices[0].message.content)
# Agent's response: The weather in San Francisco is 70 degrees and it's sunny. Humidity is 50%, and the wind speed is around 10 mph.

# Without automatic tool execution (default)
response = client.sessions.chat(
    session_id="session_id",
    messages=[
        {
            "role": "user",
            "content": "What is the weather in San Francisco?"
        }
    ],
    auto_run_tools=False  # Default - returns tool calls without execution
)

# Check if tool calls were made
if response.choices[0].message.tool_calls:
    print("Tool calls:", response.choices[0].message.tool_calls)
    # You need to execute these tools manually and send results back

​

Documents

When chatting in a session, the session can automatically search for documents that are associated with any of the agents and/or users that participate in the session. You can control whether the session should search for documents when chatting using the recall option of the chat method, which is set to True by default. You can also set the session’s recall_options when creating the session to control how the session should search for documents.

# Create a session with custom recall options
client.sessions.create(
    agent=agent.id,
    user=user.id,
    recall=True,
    recall_options={
        "mode": "vector", # or "hybrid", "text"
        "num_search_messages": 4, # number of messages to search for documents
        "max_query_length": 1000, # maximum query length
        "alpha": 0.7, # weight to apply to BM25 vs Vector search results (ranges from 0 to 1)
        "confidence": 0.6, # confidence cutoff level (ranges from -1 to 1)
        "limit": 10, # limit of documents to return
        "lang": "en-US", # language to be used for text-only search
        "metadata_filter": {}, # metadata filter to apply to the search
        "mmr_strength": 0, # MMR Strength (ranges from 0 to 1)
    }
)

# Chat in the session
response = client.sessions.chat(
    session_id=session.id,
    messages=[
        {
            "role": "user",
            "content": "Tell me about Julep"
        }
    ],
    recall=True
)
print("Agent's response:", response.choices[0].message.content)
print("Searched Documents:", response.docs)

​

Best Practices

​

Next Steps

• Agent Tools - Learn about tools and how to use them with agents
• Agent Tasks - Learn about tasks and how to use them with agents
• Agent Docs - Learn about docs and how to use them with agents