​
Overview

Julep provides a robust chat system with various features for dynamic interaction with agents. Here’s an overview of the key components and functionalities.

​
Chat Input

Messages

An array of input messages representing the conversation so far

Tools

(Advanced) Additional tools provided for this specific interaction

Tool Choice

Specifies which tool the agent should use

Memory Access

Controls how the session accesses history and memories

Chat Settings

Various settings to control the behavior of the chat

​
Chat Settings

  • model: Identifier of the model to be used
  • stream: Indicates if the server should stream the response as it’s generated
  • stop: Up to 4 sequences where the API will stop generating further tokens
  • seed: For deterministic sampling
  • max_tokens: The maximum number of tokens to generate
  • logit_bias: Modify the likelihood of specified tokens appearing in the completion
  • response_format: Control the format of the response (e.g., JSON object)
  • agent: Agent ID to use (for multi-agent sessions)
  • temperature
  • top_p
  • frequency_penalty
  • presence_penalty

​
Chat Response

  • Content-Type: text/event-stream
  • Body: A stream of ChatOutputChunk objects

Both response types include:

  • usage: Token usage statistics
  • jobs: Background job IDs spawned from this interaction
  • docs: Documents referenced for this request (for citation purposes)

​
Finish Reasons

stop

Natural stop point or provided stop sequence reached

length

Maximum number of tokens specified in the request was reached

content_filter

Content was omitted due to a flag from content filters

tool_calls

The model called a tool

​
Advanced Features

1

Tool Integration

The chat API allows for the use of tools, enabling the agent to perform actions or retrieve information during the conversation.

2

Multi-agent Sessions

You can specify different agents within the same session using the agent parameter in the chat settings.

3

Response Formatting

Control the output format, including options for JSON responses with specific schemas.

4

Memory and Recall

Configure how the session accesses and stores conversation history and memories.

5

Document References

The API returns information about documents referenced during the interaction, useful for providing citations or sources.

These features provide developers with a powerful and flexible system for creating sophisticated, context-aware chat interactions that integrate seamlessly with other Julep components.

​
Support

If you need help with further questions in Julep:

LifecyclePython Expression