Understanding Julep Sessions and state management
uuid7
) for each session.Option | Type | Description | Default |
---|---|---|---|
agent | UUID | None | The ID of the agent to associate with the session | None |
user | UUID | None | The ID of the user interacting with the agent | None |
context_overflow | truncate | adaptive | None | Strategy for handling context overflow: truncate cuts off the oldest context; adaptive adjusts dynamically | None |
metadata | object | None | Additional metadata for the session (e.g., user preferences) | None |
system_template | str | None | A specific system prompt template that sets the background for this session | None |
render_templates | StrictBool | Whether to render system and assistant messages as Jinja templates | True |
token_budget | int | None | Threshold value for the adaptive context functionality | None |
auto_run_tools | StrictBool | Whether to automatically execute tools and send the results back to the model when available | False |
forward_tool_calls | StrictBool | Whether to forward tool calls directly to the model | False |
recall_options | object | None | Options for different RAG search modes (VectorDocSearch, TextOnlyDocSearch, HybridDocSearch) in the session | VectorDocSearch |
Parameter | Type | Description | Default |
---|---|---|---|
mode | Literal["vector"] | The mode to use for the search (must be “vector”) | "vector" |
lang | str | The language for text search (other languages coming soon) | en-US |
limit | int | The limit of documents to return (1-50) | 10 |
max_query_length | int | The maximum query length (100-10000 characters) | 1000 |
metadata_filter | object | Metadata filter to apply to the search | {} |
num_search_messages | int | The number of search messages to use for the search (1-50) | 4 |
confidence | float | The confidence cutoff level (-1 to 1) | 0.5 |
mmr_strength | float | MMR Strength (mmr_strength = 1 - mmr_lambda) (0 to 1) | 0.5 |
recall_options
is not explicitly set (for instance, it is None
), vector
search mode is used with default parameters.trigram_similarity_threshold
parameter - higher values (e.g., 0.8) require closer matches while lower values (e.g., 0.3) are more lenient. For more details on the advanced search capabilities, see the Documents (RAG) section.Default System Template
session1
and session2
are separate.name
, and about
in order to personalize the interaction. Check out the system_template
to see how the user’s info is being accessed.
This is how you can create a user and associate it with a session:
auto_run_tools
is set to true
(available in chat calls), if an agent has a tool and the LLM decides to use it, the tool will be executed automatically and the result will be sent back to the LLM for further processing. When auto_run_tools
is false
(default), tool calls are returned in the response without execution.
Example:
If the agent that’s associated with the session has a tool called fetch_weather
, and the LLM decides to use it:
auto_run_tools=true
: The tool executes automatically and returns weather data to the LLMauto_run_tools=false
: The tool call is returned in the response for manual executionrecall
option of the chat
method, which is set to True
by default. You can also set the session’s recall_options
when creating the session to control how the session should search for documents.
response.docs
field.Output
crawling-and-rag
cookbook. Check it out here.