Agents - Julep

Overview

Agents are conceptual entities that encapsulate all the configurations and settings of an LLM, enabling it to adopt unique personas and execute distinct tasks within an application.

Components

Agents are made up of several components. Think of components as the building blocks of an agent required to perform a task. Here are the key components associated with an agent:

Instructions - Agent configuration that can be provided as either a single string or an array of strings.
Metadata - Key-value pair that can be used to categorize and filter agents.
Tools - Functions that can be used by an agent to perform tasks. Julep supports a wide range of tools, including custom tools, which are functions that can be used by an agent to perform tasks.
Docs - A collection of documents that can be used by an agent to retrieve information. Docs can be associated with an agent and can be used to retrieve or search information from the agent’s context.

Agent Configuration Options

When creating an agent, you can leverage the following configuration options:

Option	Type	Description	Default
`name`	`string`	The name of your agent	Required
`canonical_name`	`string`	A unique identifier for your agent, following the pattern `[a-zA-Z][a-zA-Z0-9_]*`	`null`
`project`	`string`	The canonical name of the project this agent belongs to	`"default"`
`about`	`string`	A brief description of what your agent does	`""`
`model`	`string`	The language model your agent uses (e.g., “gpt-4-turbo”, “gemini-nano”)	`""`
`instructions`	`string` \| `list[string]`	Specific tasks or behaviors expected from the agent	`[]`
`metadata`	`object`	Key-value pairs for additional information about your agent	`null`
`default_settings`	`object`	Default configuration settings for the agent. See supported parameters for details.	`null`
`default_system_template`	`string`	Default system template for all sessions created by this agent.	See default system template

The System Template is a specific system prompt written as a Jinja template that sets the foundational context and instructions for the agent within a session. It defines the background, directives, and any relevant information that the agent should consider when interacting with the user.For a comprehensive guide on system templates including available variables, customization options, and advanced usage patterns, see the System Templates documentation. For more details on Jinja templates, refer to the Jinja documentation.

Default System Template

Python

{%- if agent.name -%}
You are {{ agent.name }}.
{%- endif -%}

{%- if agent.about -%}
About you: {{ agent.about }}.
{%- endif -%}

{%- if user -%}
You are talking to a user
  {%- if user.name -%}
    and their name is {{ user.name }}
    {%- if user.about -%}
      . About the user: {{ user.about }}.
    {%- else -%}
      .
    {%- endif -%}
  {%- endif -%}
{%- endif -%}

{{ NEWLINE }}

{%- if session.situation -%}
Situation: {{ session.situation }}
{%- endif -%}

{{ NEWLINE + NEWLINE }}

{%- if agent.instructions -%}
Instructions:
  {%- if agent.instructions is string -%}
    {{ agent.instructions }}
  {%- else -%}
    {%- for instruction in agent.instructions -%}
      - {{ instruction }}
    {%- endfor -%}
  {%- endif -%}
{{ NEWLINE }}
{%- endif -%}

{%- if docs -%}
Relevant documents:
  {%- for doc in docs -%}
    {{ doc.title }}
    {%- if doc.content is string -%}
      {{ doc.content }}
    {%- else -%}
      {%- for snippet in doc.content -%}
        {{ snippet }}
      {%- endfor -%}
    {%- endif -%}
    ---
  {%- endfor -%}
{%- endif -%}

How to Use Agents

In Julep, how you use agents is very important. The YAML below shows the anatomy of an agent.

YAML

name: "My Agent"
model: "claude-3.5-sonnet"
project: "data-analytics"
about: "A helpful AI assistant that specializes in data analysis"
instructions: "You are a helpful AI assistant that specializes in data analysis"
metadata:
  type: "data-analysis"
tools:
  - name: "calculate_total"
    description: "Calculate the total of a list of numbers"
    function:
      parameters:
        type: "object"
        properties:
          numbers:
            type: "array"
            items:
              type: "number"

Creating an Agent

To create an agent, you can use the create method in the Python or Node.js SDK.

from julep import Julep

client = Julep(api_key="your_api_key")

agent = client.agents.create(
  name="My Agent",
  model="claude-3.5-sonnet",
  about="A helpful AI assistant that specializes in data analysis",
  instructions="You are a helpful AI assistant that specializes in data analysis",
  metadata={"type": "data-analysis"},
  tools=[
    {
      "name": "calculate_total",
      "description": "Calculate the total of a list of numbers",
      "function": {
        "parameters": {"type": "object", "properties": {"numbers": {"type": "array", "items": {"type": "number"}}}}
      }
    }
  ]
)

Using default_settings with response_format

You can configure your agent to always return structured JSON responses by setting response_format in the default_settings:

agent = client.agents.create(
  name="Data Analyst",
  model="gpt-4o-mini",
  about="An agent that analyzes data and returns structured insights",
  instructions="Always provide analysis results in a structured format",
  default_settings={
    "temperature": 0.3,
    "response_format": {
      "type": "json_object"
    }
  }
)

Check out the API reference here or SDK reference (Python here or JavaScript here for more details on different operations you can perform on agents.

Relationship To Other Concepts

This section will help you understand how agents relate to other concepts in Julep.

Projects

Agents belong to exactly one project, which helps organize related resources together. When creating an agent, you can specify which project it belongs to using the project parameter. If not specified, the agent will be assigned to the “default” project. For example:

# Create an agent in a specific project
agent = client.agents.create(
  name="Customer Support Bot",
  project="support-platform"
)

For more information about projects, see Projects.

Tools

Agents can be associated with different types of tools available in Julep to enable them to perform operations. These tools associated with an agent can also be leveraged by a task associated with the agent. For example:

# Create an agent
agent = client.agents.create(name="My Agent")

# Associate a tool with the agent
client.agents.tools.create(
  agent_id=AGENT_UUID,
  **{
    "name": "computer",
    "type": "computer_20241022",
    "computer_20241022": {
        "display_height_px": 768,
        "display_width_px": 1024,
        "display_number": 1,
    },
  }
)

Sessions

Agents can be used in sessions to enable real-time, interactive conversations. While tasks are designed for automated workflows, sessions provide a way to have stateful, continuous interactions with an agent. You can create multiple sessions with the same agent or multiple agents in a session, each session maintaining its own conversation history and context. This makes sessions ideal for scenarios requiring ongoing dialogue or human-in-the-loop interactions. For example:

# Create an agent
agent = client.agents.create(name="My Agent")

# Create a session with the agent
session = client.sessions.create(agent=agent.id)

Best Practices

Agent Design

1. Clear, Focused Purposes: Give agents clear, focused purposes rather than making them generalists
2. Descriptive Names: Use descriptive names that reflect the agent’s primary function
3. Concise Instructions: Keep instructions concise but specific
4. Specialized Agents: Break complex tasks into multiple specialized agents rather than one complex agent

Configuration Management

1. Conservative Model Settings: Start with conservative model settings (temperature, top_p) and adjust as needed
2. Metadata: Use metadata effectively for organization and filtering
3. Tools: Define tools that are specific to the agent’s purpose

Resource Management

1. Reuse Agents: Reuse agents across similar tasks instead of creating new ones
2. Clean Up: Clean up unused agents and their associated resources
3. Monitor Token Usage: Monitor token usage and adjust context windows appropriately

Avoid giving agents more capabilities than they need. Each additional tool or permission increases the complexity and potential security surface area.

Next Steps

Agent Tools - Learn about tools and how to use them with agents
Agent Tasks - Learn about tasks and how to use them with agents
Agent Sessions - Learn about sessions and how to use them with agents
Agent Docs - Learn about docs and how to use them with agents

​Overview

​Components

​Agent Configuration Options

​How to Use Agents

​Creating an Agent

​Using default_settings with response_format

​Relationship To Other Concepts

​Projects

​Tools

​Sessions

​Best Practices

Agent Design

Configuration Management

Resource Management

​Next Steps

​See Examples

Overview

Components

Agent Configuration Options

How to Use Agents

Creating an Agent

Using default_settings with response_format

Relationship To Other Concepts

Projects

Tools

Sessions

Best Practices

Next Steps

See Examples