# FAQ Source: https://docs.julep.ai/FAQ Frequently Asked Questions about Julep Platform # Julep Platform FAQ This comprehensive FAQ document covers all aspects of the Julep platform, from architecture to troubleshooting. The information is organized by category for easy navigation. ## Table of Contents 1. [Architecture & System Design](#architecture--system-design) 2. [Data Model & Storage](#data-model--storage) 3. [Task Execution & Workflow](#task-execution--workflow) 4. [Agents API](#agents-api) 5. [Worker System & Integration](#worker-system--integration) 6. [Development & Deployment](#development--deployment) 7. [Performance & Optimization](#performance--optimization) 8. [Security & Compliance](#security--compliance) 9. [Advanced Use Cases & Patterns](#advanced-use-cases--patterns) 10. [Troubleshooting & Common Issues](#troubleshooting--common-issues) *** ## Architecture & System Design ### Q: What is the overall system architecture of Julep, including all core components and their interactions? Julep is a distributed system built on a microservices architecture designed to orchestrate complex AI workflows. The main components include: * **Client Applications**: Initiate requests to the Julep system * **Gateway**: Entry point for all API requests, handles authentication and load balancing (implemented using Traefik) * **Agents API**: Provides REST endpoints for managing agents, tasks, sessions, and documents; initiates workflows in Temporal * **Temporal Workflow Engine**: Orchestrates durable workflow execution, retries, and state management * **Worker System**: Executes workflows and activities defined in Temporal by polling for tasks * **LiteLLM Proxy**: Provides a unified interface for interacting with various LLM providers * **Memory Store**: Provides persistent storage using PostgreSQL/TimescaleDB for relational data and vector embeddings * **Integration Service**: Enables connections with external tools and APIs ### Q: How does Julep handle distributed task execution and what role does Temporal play in the architecture? Julep handles distributed task execution primarily through the Temporal Workflow Engine: * **Workflow Orchestration**: Temporal ensures durable execution of workflows, handling retries and maintaining state across failures * **Task Queues**: The Agents API initiates workflows by sending requests to Temporal, which places them on task queues like `julep-task-queue` * **Worker Execution**: Workers poll Temporal for tasks and execute activities like LLM calls, tool operations, and data interactions * **State Management**: Temporal persists workflow execution state in PostgreSQL, ensuring long-running processes can recover from failures ### Q: What are the key design decisions behind separating agents-api, memory-store, integrations-service, and other components? The separation follows microservices principles: * **Modularity and Independent Scaling**: Each service can be developed, deployed, and scaled independently * **Separation of Concerns**: Each component handles specific functionalities: * `agents-api`: Manages agent definitions, tasks, sessions, and orchestrates workflows * `memory-store`: Handles all data persistence including relational data and vector embeddings * `integrations-service`: Provides standardized interface for external tool usage * `llm-proxy`: Centralizes LLM interactions with a unified API * **Resilience**: Failures in one service are isolated from others * **Technology Flexibility**: Different services can use different technologies if needed ### Q: How does the gateway component route requests between different services? The Gateway component uses Traefik and routes requests based on defined rules: * Requests to `/api/*` are routed to the Agents API service * Requests to `/tasks-ui/*` go to the Temporal UI service * `/v1/graphql` requests are directed to the Hasura service * In multi-tenant setups, it enforces JWT-based authentication and forwards `X-Developer-Id` headers for resource isolation ### Q: What is the role of the blob-store and how does it integrate with S3-compatible storage? The blob-store is used for persistent storage of large data, specifically for Temporal workflow data when `USE_BLOB_STORE_FOR_TEMPORAL` is enabled. It integrates with S3-compatible storage through environment variables: * `S3_ENDPOINT`, `S3_ACCESS_KEY`, and `S3_SECRET_KEY` for connection * `BLOB_STORE_BUCKET` defines the bucket name * `BLOB_STORE_CUTOFF_KB` sets the size threshold for blob storage ### Q: How does the llm-proxy (LiteLLM) handle different language model providers? LiteLLM provides a unified interface to multiple LLM providers: * Supports providers like OpenAI, Anthropic, Gemini, Groq, and OpenRouter * Configuration defined in `litellm-config.yaml` with model names, parameters, and API keys * Handles response patching for consistency (e.g., changing `finish_reason` from "eos" to "stop") * Tracks token usage and costs in PostgreSQL * Implements request caching using Redis and supports parallel forwarding ### Q: What are the scalability patterns and limitations of the current architecture? Julep's architecture supports scalability through: * **Agents API**: Horizontal scaling via multiple instances, configurable with `GUNICORN_WORKERS` * **Worker**: Multiple workers with concurrency control using `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` * **Memory Store**: PostgreSQL connection pooling with `POOL_MAX_SIZE` * **LiteLLM**: Request caching and parallel forwarding * **Temporal**: Durable workflow execution that scales to handle concurrent tasks ### Q: How does Julep ensure high availability and fault tolerance across services? High availability is achieved through: * Temporal's durable execution model with automatic retries * Microservices architecture allowing independent service failures * PostgreSQL for persistent state storage * Worker pools for distributed task execution * Connection pooling and retry mechanisms *** ## Data Model & Storage ### Q: What is the complete data model including relationships between Agents, Tasks, Tools, Sessions, Entries, and Executions? The core entities and their relationships: * **Developer**: Manages Agents, Users, and owns Tasks * **Agent**: Has Tasks, defines Tools, owns Docs, participates in Sessions * **User**: Owns Docs and participates in Sessions * **Task**: Contains WorkflowSteps and is executed as Executions * **Execution**: Logs Transitions and tracks task execution state * **Session**: Contains Entries (conversation history) * **Entry**: Individual messages within a Session * **Tool**: Capabilities available to Agents * **Doc**: Documents with embeddings for knowledge base All entities use UUIDs for identification and include `developer_id` for multi-tenancy. ### Q: How does the memory-store handle vector embeddings and similarity search? The memory store provides vectorized document storage: * Documents have an `embeddings` field stored in `docs_embeddings_store` table * Supports three search types: * **Vector Search**: `search_docs_by_embedding` for semantic similarity * **Text Search**: `search_docs_by_text` using PostgreSQL full-text search * **Hybrid Search**: `search_docs_hybrid` combining both approaches * Uses cosine similarity for vector comparisons * Implements Maximum Marginal Relevance (MMR) for result diversity ### Q: What PostgreSQL and TimescaleDB features are leveraged for time-series data? While PostgreSQL is the primary database, specific TimescaleDB features are not explicitly detailed in the codebase. The system uses: * Standard PostgreSQL timestamps (`created_at`, `updated_at`) for temporal data * Time-based filtering in queries (e.g., `list_entries` with date ranges) * No explicit TimescaleDB-specific features documented ### Q: How are agent instructions and task definitions stored and versioned? * **Agent Instructions**: Stored as `string` or `array[string]` in the Agent entity * **Task Definitions**: Stored as Task entities with fields like `name`, `description`, `input_schema`, `main` (workflow steps), and `tools` * **Versioning**: Handled through UUID changes - altering a UUID creates a new entity while preserving the original * Timestamps (`created_at`, `updated_at`) provide implicit version tracking ### Q: What is the schema for storing conversation history and context? Conversation history uses two main entities: * **Session**: Contains `id`, `user`, `agent`, `situation` (context), `system_template`, and `metadata` * **Entry**: Contains `id`, `session_id`, `role` (user/assistant/system), `content` (string or JSON), `source`, and `timestamp` * Sessions group entries and maintain context across conversations ### Q: How does Julep handle data partitioning and archiving for long-running agents? The codebase does not contain explicit information about data partitioning or archiving strategies. The system uses: * Multi-tenancy through `developer_id` filtering * Pagination support for large datasets * Time-based filtering capabilities * No documented automatic archiving policies ### Q: What are the indexing strategies for optimizing query performance? Indexing strategies include: * Trigram indexes for text search (indicated by `trigram_similarity_threshold` parameter) * Vector indexes for embedding similarity searches * Document chunking and embedding storage for efficient retrieval * Use of prepared statements for query optimization *** ## Task Execution & Workflow ### Q: How does the TaskExecutionWorkflow handle complex multi-step operations? The TaskExecutionWorkflow orchestrates multi-step operations by: * Processing different WorkflowStep types through dedicated handlers * Managing state transitions between steps * Integrating with Temporal for durability and reliability * Using `handle_step` method to process each step type * Evaluating expressions within steps using `eval_step_exprs` ### Q: What are all the possible workflow step types and their configurations? **Basic Steps:** * **PromptStep**: Sends prompts to LLMs and handles responses * **ToolCallStep**: Executes tool calls (functions, integrations, APIs, system operations) * **EvaluateStep**: Evaluates expressions and returns results * **SetStep**: Sets values in execution state * **GetStep**: Retrieves values from execution state * **LogStep**: Logs messages during execution * **ReturnStep**: Returns a value and completes workflow * **ErrorWorkflowStep**: Raises an error and fails the workflow * **SleepStep**: Pauses execution for specified duration * **WaitForInputStep**: Pauses for external input **Control Flow Steps:** * **IfElseWorkflowStep**: Conditional branching based on condition * **SwitchStep**: Multi-way branching based on case evaluation * **ForeachStep**: Iterates over collections * **MapReduceStep**: Maps function over items with optional parallelism * **YieldStep**: Yields execution to another workflow * **ParallelStep**: Executes steps in parallel (not yet implemented) ### Q: How does the state machine handle transitions between different execution states? The state machine tracks execution through: * **States**: `queued`, `starting`, `running`, `succeeded`, `failed`, `cancelled` * **Transitions**: Record state changes with types: `init`, `step`, `finish`, `error`, `cancelled` * Each transition includes `output`, `current`, and `next` workflow steps * Transitions stored in `execution_transitions` table * `create_execution_transition` function records all state changes ### Q: What happens when a task fails midway through execution? When a task fails: * Execution status transitions to "failed" * An "error" transition is created with the error message * The `error` field of the Execution object is populated * The system tracks the failure point and error details * Workflow implements retry policies for retryable errors * Non-retryable errors cause immediate failure ### Q: How are conditional branches and loops implemented in workflows? **Conditional Branches:** * `IfElseWorkflowStep`: Evaluates condition and executes "then" or "else" branch * `SwitchStep`: Multi-way branching with multiple cases **Loops:** * `ForeachStep`: Iterates over collections processing each item * `MapReduceStep`: Maps functions over collections with optional parallel execution These are processed by dedicated handlers like `_handle_IfElseWorkflowStep` and `_handle_ForeachStep`. ### Q: What is the retry and error handling strategy for failed steps? * **Error Classification**: Errors classified as retryable or non-retryable * **Retry Policy**: `DEFAULT_RETRY_POLICY` applied to retryable errors * **Max Retries**: Workflow fails if max retries exceeded * **Error Transitions**: "error" transitions record failure states * **Last Error Tracking**: `last_error` attribute stores recent errors ### Q: How does Julep handle long-running tasks and prevent timeouts? Julep uses Temporal's timeout mechanisms: * `schedule_to_close_timeout` and `heartbeat_timeout` for activities * Activity and workflow heartbeats ensure progress tracking * Large data handling via `RemoteObject` pattern to optimize memory * Blob storage for data exceeding size thresholds ### Q: What are the mechanisms for task cancellation and cleanup? * **Workflow Cancellation**: Handled through Temporal's cancellation features * **State Management**: Execution can transition to "cancelled" state * **Persistence**: All transitions persisted for state restoration * **Cleanup**: Status updates and transition history provide cleanup context *** ## Agents API ### Q: What are all the endpoints available in the Agents API and their use cases? The Agents API provides these endpoints: * `/agents`: Create, retrieve, update, delete agent definitions * `/tasks`: Define and execute tasks, retrieve workflow definitions * `/sessions`: Manage conversational sessions and conversation history * `/executions`: Track task executions and monitor status * `/docs`: Handle document storage, search, and retrieval with embeddings * `/tools`: Define and manage agent tools * `/users`: Manage user accounts and authentication * `/responses`: OpenAI-compatible interface for LLM responses ### Q: How does session management work and what data is maintained per session? Session management maintains conversation state: * **Session Data**: `id`, `agent_id`, `user_id`, `created_at`, situation context * **Entries**: Individual conversation turns with role, content, and timestamps * Sessions created with `julep.sessions.create` linking agent and user * Messages added via `julep.sessions.chat` with role and content * Pagination support for retrieving conversation history ### Q: What are the different types of tools an agent can use and how are they configured? Tool types available: * **Web Search Tool**: Performs web searches with domain filtering * **Function Tools**: OpenAI-compatible function calling format * **System Tools**: Internal Julep resources (e.g., `create_julep_session`, `session_chat`) * **Integration Tools**: External services (e.g., BrowserBase, email providers) Configuration includes name, type, description, and type-specific parameters. ### Q: How does document storage and retrieval work for agent knowledge bases? Documents (`Doc` entities) provide agent knowledge: * Stored in PostgreSQL with embeddings in `docs_embeddings_store` * Owned by either Agent or User * Three search methods: text-based, embedding-based, hybrid * Document operations: create, retrieve, list, search * Supports metadata filtering and pagination ### Q: What are the authentication and authorization mechanisms for the API? Two authentication modes: **Single-Tenant Mode:** * Uses `AGENTS_API_KEY` for authentication * Default developer ID assumed * `X-Auth-Key` header required **Multi-Tenant Mode:** * JWT-based authentication via Gateway * `X-Developer-Id` header for resource isolation * Developer-specific data access control * JWT must contain sub, email, exp, iat claims ### Q: How are agent instructions processed and validated? * Instructions stored as string or array of strings * Validated by Pydantic models from OpenAPI schemas * Included in agent's `default_system_template` * Dynamic rendering supports both single and array formats * TypeSpec definitions ensure consistent structure ### Q: What are the rate limiting and quota management strategies? Rate limiting and quota management are planned features: * `max_free_sessions` and `max_free_executions` environment variables defined * Implementation details not yet available * Listed as future enhancement in roadmap ### Q: How does the API handle streaming responses for real-time interactions? Streaming support is currently planned but not implemented: * Open Responses API designed for OpenAI compatibility * `stream` configuration option available * Full streaming implementation pending *** ## Worker System & Integration ### Q: How does the worker system integrate with different LLM providers? The Worker integrates with LLMs through LiteLLM Proxy: * LiteLLM acts as unified interface to various providers * Supports OpenAI, Anthropic, Gemini, Groq, OpenRouter * Worker makes LLM calls via LiteLLM Proxy * Configuration in `litellm-config.yaml` * Handles authentication, routing, and caching ### Q: What are the system activities available and how do they work? System activities include: * **LLM Calls**: Through LiteLLM Proxy * **Tool Operations**: Via Integration Service * **Data Operations**: Reading/writing to Memory Store * **PG Query Step**: Direct PostgreSQL queries * Activities invoked with `StepContext` and appropriate definitions ### Q: How does Julep handle tool execution and external API calls? Tool execution handled by Integration Service: * **Integration Tools**: Connect to external services with provider/method specs * **System Tools**: Operate on internal Julep resources * `ToolCallStep` defines tool and arguments * Worker invokes Integration Service for execution * Examples: email sending, browser automation, document search ### Q: What is the sandboxing mechanism for Python expression evaluation? Python expression sandboxing: * `validate_py_expression` function validates expressions * Identifies expressions starting with `$`, `_`, or containing `{{` * Checks for syntax errors, undefined names, unsafe operations * Limited scope with allowed names: `_`, `inputs`, `outputs`, `state`, `steps` * Prevents dunder attribute access and unapproved function calls ### Q: How are integration credentials managed and secured? Integration credentials managed through: * Credentials stored in `setup` parameters of tool definitions * API keys provided in task definition YAML * Environment variables for service-level credentials * No explicit encryption details documented ### Q: What are the patterns for building custom integrations? Custom integrations defined as Tool entities: * Type set to `integration` * Specify `provider`, `method`, and `setup` parameters * Include provider-specific configuration (API keys, endpoints) * Used within Task workflows as ToolCallStep * Examples: Browserbase, email providers, Cloudinary ### Q: How does the browser automation integration work? Browser automation workflow: 1. Create Julep session for AI agent 2. Create browser session using Browserbase 3. Store session info (browser\_session\_id, connect\_url) 4. Perform actions via `perform_browser_action` tool 5. Interactive loop with agent planning and execution 6. Screenshot capture for visual feedback ### Q: What are the performance optimizations for worker pools? Worker performance optimizations: * `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` controls concurrency * `TEMPORAL_MAX_ACTIVITIES_PER_SECOND` limits activity rate * `GUNICORN_WORKERS` for integration service scaling * Connection pooling and timeout configurations * LiteLLM caching and parallel forwarding *** ## Development & Deployment ### Q: What is the recommended development workflow for building with Julep? Development workflow uses Docker Compose with watch mode: * Changes in source directories trigger automatic sync/restart * `agents-api`: Watches `./agents_api` and `gunicorn_conf.py` * `worker`: Watches `./agents_api` and `Dockerfile.worker` * `integrations`: Watches its directory for changes * Lock file or Dockerfile changes trigger rebuilds ### Q: How should developers set up their local environment for testing? Local setup primarily uses Docker: 1. Create project directory: `mkdir julep-responses-api` 2. Download and edit `.env` file 3. Download Docker Compose file 4. Run containers: `docker compose up --watch` 5. Verify with `docker ps` Alternative CLI installation available via `npx` or `uvx`. ### Q: What are the deployment options and best practices? Deployment options: **Single-Tenant Mode:** * All users share context * `SKIP_CHECK_DEVELOPER_HEADERS=True` * Requires `AGENTS_API_KEY` **Multi-Tenant Mode:** * Isolated resources per developer * `AGENTS_API_MULTI_TENANT_MODE: true` * JWT validation via Gateway **Best Practices:** * Use environment variables for configuration * Implement layered security (API keys, JWT tokens) * Enable independent component scaling * Configure appropriate connection pools ### Q: How does the TypeSpec code generation work and when to use it? TypeSpec code generation: * TypeSpec files define data models (e.g., `models.tsp`) * `scripts/generate_openapi_code.sh` generates code * Creates Pydantic models and OpenAPI schemas * Edit TypeSpec files, not generated code * Regenerate after model changes ### Q: What are the testing strategies for agents and workflows? Testing strategies include: * Unit tests for entities (agents, docs, sessions, etc.) * Integration tests simulating scenarios * Workflow tests using `unittest.mock.patch` * Test fixtures for consistent data * Ward framework for test organization ### Q: How to debug failed task executions and trace through workflows? Debugging approach: * Check Execution status and error field * List transitions to trace execution flow * Examine transition outputs and types * Use LogStep for workflow logging * Error transitions indicate failure points ### Q: What are the monitoring and observability features? Limited monitoring information available: * Execution state and transition logging * No explicit monitoring features documented * Prometheus and Grafana mentioned in architecture * Detailed observability features not specified ### Q: How to handle database migrations in production? Database migration information not explicitly documented: * PostgreSQL used as primary database * Migration files exist in memory-store * Production migration process not detailed *** ## Performance & Optimization ### Q: What are the performance characteristics of different operations? Performance characteristics: * MapReduceStep supports sequential or parallel execution * API response times reduced by 15% (per changelog) * Parallel processing improves collection operations * Connection pooling optimizes database access * Caching planned for future optimization ### Q: How does Julep handle concurrent agent executions? Concurrent execution handled through: * TaskExecutionWorkflow with Temporal orchestration * Worker pools for distributed execution * State isolation per execution * Temporal manages workflow concurrency * StepContext provides execution isolation ### Q: What are the caching strategies employed across the system? Current caching: * LiteLLM Proxy implements request caching * Redis used for LiteLLM cache storage * Web Search Tool has result caching * Advanced caching mechanisms planned ### Q: How to optimize memory usage for large conversation histories? Memory optimization strategies: * Pagination with `limit` and `offset` parameters * Maximum limit of 1000 entries per request * `search_window` for time-based filtering (default 4 weeks) * Token counting per entry * RemoteObject pattern for large data ### Q: What are the bottlenecks in the current architecture? Potential bottlenecks: * Large data retrieval without proper pagination * Complex database queries on large tables * Multi-tenancy query filtering overhead * Temporal workflow state management at scale * JSON aggregation in history queries ### Q: How does connection pooling work for database access? Connection pooling configuration: * `connection_lifetime`: 600 seconds * `idle_timeout`: 180 seconds * `max_connections`: 50 * `retries`: 1 * `use_prepared_statements`: true * `POOL_MAX_SIZE` configurable (default: CPU count, max 10) ### Q: What are the best practices for writing efficient task definitions? Best practices: * Define clear input schemas * Use modular workflow steps * Implement proper error handling * Avoid infinite loops in recursive patterns * Use appropriate tool types * Monitor execution status and transitions * Leverage parallel processing where applicable *** ## Security & Compliance ### Q: How does Julep handle sensitive data and ensure data privacy? Data privacy ensured through: * Multi-tenant architecture with resource isolation * Developer-specific data access via `developer_id` * `X-Developer-Id` header for request routing * Separate data storage per developer ### Q: What are the security measures for multi-tenant deployments? Multi-tenant security: * JWT-based authentication at Gateway * JWT validation with required claims (sub, email, exp, iat) * `X-Developer-Id` header enforcement * Developer ID verification against database * Resource isolation by developer ### Q: How are API keys and secrets managed throughout the system? Secret management: * Environment variables for service credentials * API keys in tool setup parameters * `AGENTS_API_KEY` and `JWT_SHARED_KEY` for auth * LLM provider keys as environment variables * No explicit encryption details provided ### Q: What audit logging capabilities are available? Audit logging: * Currently limited implementation * Listed as planned feature in roadmap * Usage tracking for LLM calls (tokens and costs) * Comprehensive audit logging pending ### Q: How does Julep ensure secure execution of user-provided code? Secure code execution through: * Task-based workflow system with YAML definitions * Controlled tool invocation with explicit permissions * Python expression validation * Structured workflow steps * No arbitrary code execution ### Q: What are the network security considerations for deployment? Network security: * API key authentication for single-tenant * JWT tokens for multi-tenant * Gateway-level authentication * HTTPS/TLS support implied * Service-to-service communication within network *** ## Advanced Use Cases & Patterns ### Q: What are examples of complex multi-agent workflows? **Browser Use Assistant:** * Session initialization * Browser session creation * Interactive agent-browser loop * Screenshot feedback * Goal-oriented task completion **Email Assistant:** * Email input processing * Query generation * Documentation search * Response generation * Automated email sending **Video Processing:** * Natural language instructions * Cloudinary integration * Transformation generation * Video processing execution ### Q: How to implement human-in-the-loop patterns? Human-in-the-loop not explicitly documented: * `WaitForInputStep` provides pause mechanism * Session-based interactions allow user input * No dedicated approval workflow patterns * Can be built using existing primitives ### Q: What are the patterns for building conversational agents with memory? Conversational memory patterns: * Session and Entry entities maintain history * `previous_response_id` links responses * Session metadata and custom templates * Persistent conversation state * Context maintained across interactions ### Q: How to implement custom tool integrations? Custom tool implementation: 1. Define Tool entity with type `integration` 2. Specify provider, method, setup parameters 3. Use `@function_tool` decorator for functions 4. Add to agent's tool list 5. Invoke via ToolCallStep in workflows ### Q: What are the best practices for handling structured data extraction? Structured data handling: * Tools accept and return structured JSON * Evaluate steps process tool outputs * Response objects contain structured data * Pydantic models ensure data validation * TypeSpec defines consistent schemas ### Q: How to build agents that can learn and adapt over time? Learning/adaptation features limited: * Agent instructions can be updated * Metadata field allows dynamic information * No explicit learning mechanisms * Adaptation through instruction updates * Memory through conversation history ### Q: What are the patterns for building agents that can collaborate? Agent collaboration not documented: * Current model focuses on single agents * No inter-agent communication patterns * Sessions link one agent to users * Collaboration would require custom implementation *** ## Troubleshooting & Common Issues ### Q: What are the most common errors and how to resolve them? **Python Expression Errors:** * Syntax errors: Fix malformed expressions * Undefined names: Use allowed names only * Unsafe operations: Avoid dunder attributes * Runtime errors: Check for division by zero * Unsupported features: Avoid lambdas, walrus operator **Schema Validation Errors:** * Pydantic validation failures * Adjust JSON to match expected schema **Integration Errors:** * ApplicationError for missing tools * Verify tool definitions and availability ### Q: How to debug issues with task execution? Debugging steps: 1. Check Execution status field 2. Review error messages in Execution object 3. List and examine transitions 4. Check transition outputs and types 5. Use LogStep for additional logging 6. Trace workflow step progression ### Q: What are the common performance problems and solutions? Performance issues and solutions: * **Large collections**: Use MapReduceStep parallelism * **Rate limits**: Implement SleepStep delays * **Memory usage**: Paginate large result sets * **Database queries**: Ensure proper indexing * **Concurrent execution**: Configure worker pools ### Q: How to handle edge cases in agent conversations? Edge case handling: * `IfElseWorkflowStep` for conditional logic * `SwitchStep` for complex branching * `WaitForInputStep` for user input * `ErrorWorkflowStep` for invalid states * Proper error handling in workflows ### Q: What are the known limitations and workarounds? **Expression Limitations:** * No set comprehensions, lambdas, walrus operator * Use alternative Python constructs **Backwards Compatibility:** * Old expression formats supported * Use `$` prefix for consistency **ParallelStep Not Implemented:** * Use MapReduceStep with parallelism instead **Streaming Not Available:** * Planned feature, not yet implemented **Rate Limiting:** * Basic environment variables defined * Full implementation pending *** ## Additional Resources * [System Architecture Wiki](https://deepwiki.com/wiki/julep-ai/julep#2) * [Data Model Wiki](https://deepwiki.com/wiki/julep-ai/julep#2.2) * [TaskExecutionWorkflow Wiki](https://deepwiki.com/wiki/julep-ai/julep#3) * [Workflow Steps Wiki](https://deepwiki.com/wiki/julep-ai/julep#3.1) This FAQ is generated from the Julep platform documentation and codebase analysis. For the most up-to-date information, please refer to the official Julep documentation and repository. # Agentic Patterns Source: https://docs.julep.ai/advanced/agentic-patterns Learn about common patterns and best practices for building Julep agents ## Overview This guide covers common patterns and best practices for building effective Julep agents, inspired by a blog written by Anthropic on [Building effective agents](https://www.anthropic.com/research/building-effective-agents). ## Core Workflow Patterns ### 1. Prompt Chaining Break tasks into sequential steps, where each step’s output feeds the next. Can be used for: * Content generation and subsequent checks * Translation/localization in multiple stages * Ensuring quality between discrete transformations Prompt Chaining Workflow **Example implementation:** ```yaml theme={"dark"} main: # Step 1: Generate initial content - prompt: role: system content: >- $ f'''Generate marketing copy for product X based on the following: target_audience: {_.audience} product_features: {_.features} keywords: {_.seo_keywords}''' unwrap: true # Step 2: Quality check gate - evaluate: quality_check: $ _.content # Step 3: Translation - prompt: role: system content: Translate the approved content to Spanish unwrap: true ``` ### 2. Routing Pattern Act as a “traffic controller” by detecting request types and sending them to the correct handler. Ideal when inputs are diverse or require specialized expertise. Steps generally include: 1. Classification of incoming data 2. Routing to handler modules 3. Specialized processing 4. (Optional) Aggregation of results Routing Workflow **Example implementation:** ```yaml [expandable] theme={"dark"} main: # Classification step - prompt: role: system content: Classify the input query type unwrap: true # Route based on classification - switch: - case: $ _.classification == "technical_support" then: - tool: handle_technical arguments: ... - case: $ _.classification == "billing" then: - tool: handle_billing arguments: ... - case: $ _.classification == "general" then: - tool: handle_general arguments: ... ``` ### 3. Parallelization Pattern Execute subtasks concurrently by either dividing the workload (sectioning) or collecting multiple perspectives (voting). Keys to consider: • Parallel processes for performance or redundancy • Syncing results and handling errors • Aggregating diverse outputs Parallelization Workflow **Example implementations:** 1. Sectioning: ```yaml [expandable] theme={"dark"} tools: - name: aggregate_results type: ... # depends on the specific needs # Custom workflow to run a subtask run_subtask: - .... # Main workflow main: - prompt: role: system content: > $ f'''Break this task into multiple subtasks. Here is the task: {_.task}''' unwrap: true - over: $ _.subtasks do: - workflow: run_subtask arguments: ... - tool: aggregate_results arguments: results: $ _ ``` 2. Voting: ```yaml [expandable] theme={"dark"} tools: - name: perform_voting description: Perform voting on the results of running the task instances, and return the majority best result. type: ... # depends on the specific needs # Custom workflow to run a subtask run_subtask: - .... # Main workflow main: - over: $ _.main_tasks do: - workflow: run_subtask # Run the same task multiple times (given that the `run_subtask` workflow is non-deterministic) arguments: ... - tool: perform_voting arguments: results: $ _ - evaluate: final_result: $ _ ``` ### 4. Orchestrator-Workers Pattern Use a central “orchestrator” that delegates subtasks to multiple “worker” agents and integrates their outputs. Best for: * Large or dynamic multi-step tasks * Coordinating various specialized capabilities * Flexible task distribution Orchestrator-Workers Workflow **Example implementation:** ```yaml theme={"dark"} main: # Orchestrator planning - prompt: role: system content: $ f'''Break down the task into subtasks. Here is the task: {_.task}''' unwrap: true # Worker delegation - foreach: in: $ _.subtasks do: tool: assign_worker arguments: task: $ _ ``` ### 5. Evaluator-Optimizer Pattern Create iterative feedback loops to refine outputs until they meet preset criteria. Suitable for: • Content refinement • Code reviews • Detailed document enhancements General flow: 1. Generate an initial result 2. Evaluate against criteria 3. Provide improvement feedback 4. Optimize/retry until goals are met Evaluator-Optimizer Workflow **Example implementation:** ```yaml [expandable] theme={"dark"} tools: - name: score_content description: Score the content based on the criteria. Returns a json object with a score between 0 and 1, and a feedback string. type: function function: parameters: type: object properties: content: type: string description: Content to score # Subworkflow to evaluate content evaluate_content: - tool: score_content arguments: content: $ _.content - if: $ _.score < 0.5 # If the content does not meet the criteria, improve it then: - workflow: improve_content arguments: content: $ steps[0].input.content # steps[0].input is the main input of this workflow feedback: $ _.feedback # _ is the output of the score_content tool call else: evaluate: final_content: $ steps[0].input.content # Subworkflow to improve content improve_content: - prompt: role: system content: $ f'''Improve the content based on the feedback. Here is the feedback: {_.feedback}''' unwrap: true - workflow: evaluate_content arguments: content: $ _ main: # Initial generation - prompt: role: system content: $ f'''Generate initial content. Here is the task: {_.task}''' unwrap: true # Evaluation loop - loop: while: $ not _.meets_criteria do: - tool: evaluate_content - tool: improve_content ``` **Explanation:** 1. The `evaluate_content` subworkflow: * Takes content as input and scores it using a scoring tool * If the score is below 0.5, it triggers the improvement workflow * Uses a special variable (`_`) to manage content and feedback between workflows * Returns the final content once quality criteria are met 2. The `improve_content` subworkflow: * Receives content and feedback from the evaluation * Uses an LLM to improve the content based on specific feedback * Automatically triggers another evaluation cycle by calling evaluate\_content The main workflow ties these together by: * Generating initial content from a task description * Running a continuous loop that alternates between evaluation and improvement * Only completing when content meets the defined quality criteria This creates a powerful feedback loop where content is repeatedly refined based on specific feedback until it reaches the desired quality level. The pattern is particularly useful for tasks requiring high accuracy or quality, such as content generation, code review, or document analysis. ## Best Practices ## Conclusion These patterns represent proven approaches from production implementations. Choose and adapt them based on your specific use case requirements and complexity needs. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Architecture Deep Dive Source: https://docs.julep.ai/advanced/architecture-deep-dive Understand the core architecture and components of Julep ## Overview Think of Julep as a platform that combines both client-side and server-side components to help you build advanced AI agents. Here's a mental model to help you understand it: 1. Your Application Code * Use the Julep SDK in your application to define agents, tasks, and workflows * SDK provides functions and classes for easy setup and management of components 2. Julep Backend Service * SDK communicates with Julep backend over the network * Backend handles task execution, session state, document storage, and workflow orchestration 3. Integration with Tools and APIs * Integrate external tools and services within your workflows * Backend facilitates integrations for web searches, database access, third-party API calls, and more To help you visualize this, here's a diagram: ## Core Components

The main orchestrator of your application, backed by foundation models like GPT4 or Claude.

Users can be associated with sessions and are used to scope memories formed by agents.

The main workhorse for Julep apps.

Programmatic interfaces that foundation models can "call" with inputs.

Collections of text snippets indexed into a vector database.

Github Actions-style workflows for complex operations.

Encrypted key-value pairs for securely storing sensitive information like API keys and credentials.

 ## Infrastructure Components

The document store provides a vector database for semantic search along with a document management system. It supports various file types and handles automatic indexing and retrieval of documents.

The task execution engine handles distributed task processing, state management, error handling with retries, and supports parallel execution of tasks.

The secrets store provides encrypted storage for sensitive information like API keys, credentials, and tokens. It uses AES-256 encryption and developer-scoped access controls to securely manage confidential data.

The API layer provides RESTful API endpoints, SDK support, authentication and authorization capabilities, and handles rate limiting and quotas.

 ## Data Flow

The data flow in Julep is divided into three main components, each handling specific aspects of request processing and response generation.

 * SDK or API calls initiate requests * Authentication and validation * Request routing * Task decomposition * Step execution * State management * Tool integration * Result aggregation * Error handling * Client notification * State persistence ## Security
  1. **1.** API key-based access
  2. **2.** Role-based permissions
  3. **3.** Session management
  4. **4.** Token validation
  1. **1.** Encryption at rest
  2. **2.** Secure communication
  3. **3.** Data isolation
  4. **4.** Access controls
  5. **5.** Encrypted secrets
  1. **1.** Audit logging
  2. **2.** Performance metrics
  3. **3.** Error tracking
  4. **4.** Usage analytics
 ## Conclusion The modular design enables you to customize and extend functionality as needed, while the security-first approach ensures your data and operations remain protected. Whether you're building a simple chatbot or a complex AI workflow, this architecture provides the foundation you need. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Chat Features in Julep Source: https://docs.julep.ai/advanced/chat Learn about the robust chat system and its various features for dynamic interaction with agents ## Overview Julep provides a robust chat system with various features for dynamic interaction with agents. Here's an overview of the key components and functionalities. ## Features The chat API allows for the use of tools, enabling the agent to perform actions or retrieve information during the conversation. You can specify different agents within the same session using the `agent` parameter in the chat settings. Control the output format, including options for JSON responses with specific schemas. Configure how the session accesses and stores conversation history and memories. The API returns information about documents referenced during the interaction, useful for providing citations or sources.

Prerequisites for Using Chat API

 ## Input Structure * **Messages**: An array of input messages representing the conversation so far. * **Tools**: (Advanced) Additional tools provided for this specific interaction. * **Tool Choice**: Specifies which tool the agent should use. * **Memory Access**: Controls how the session accesses history and memories.(`recall` parameter) * **Additional Parameters**: Various parameters to control the behavior of the chat. You can find more details in the [Additional Parameters](#additional-parameters) section. Here's an example of how a typical message object might be structured in a chat interaction: ```python Python theme={"dark"} """ Attributes for the Message object: role (Literal["user", "assistant", "system", "tool"]): The role of the message sender. tool_call_id (str | None): Optional identifier for a tool call associated with this message. content (Annotated[str | list[str] | list[Content | ContentModel7 | ContentModel] | None, Field(...)]): The main content of the message, which can be a string, a list of strings, or a list of content models. name (str | None): Optional name associated with the message. tool_calls (list[ChosenFunctionCall | ChosenComputer20241022 | ChosenTextEditor20241022 | ChosenBash20241022] | None): List of tool calls generated during the message creation, if any. """ # Example of a simple message structure messages = [{"role": "user", "content": "Your query here"}] ```

This object represents a message in the chat system, detailing the structure and types of data it can hold.

 ## Additional Parameters | Parameter | Type | Description | Default | | -------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | `stream` | `bool` | Indicates if the server should stream the response as it's generated. | `False` | | `stop` | `list[str]` | Up to 4 sequences where the API will stop generating further tokens. | `[]` | | `seed` | `int` | If specified, the system will make a best effort to sample deterministically for that particular seed value. | `None` | | `max_tokens` | `int` | The maximum number of tokens to generate in the chat completion. | `None` | | `logit_bias` | `dict[str, float]` | Modify the likelihood of specified tokens appearing in the completion. | `None` | | `response_format` | `str` | Response format (set to `json_object` to restrict output to JSON). | `None` | | `agent` | `UUID` | Agent ID of the agent to use for this interaction. (Only applicable for multi-agent sessions) | `None` | | `repetition_penalty` | `float` | Number between 0 and 2.0. 1.0 is neutral and values larger than that penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | `None` | | `length_penalty` | `float` | Number between 0 and 2.0. 1.0 is neutral and values larger than that penalize number of tokens generated. | `None` | | `min_p` | `float` | Minimum probability compared to leading token to be considered. | `None` | | `frequency_penalty` | `float` | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | `None` | | `presence_penalty` | `float` | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | `None` | | `temperature` | `float` | What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | `None` | | `top_p` | `float` | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top\_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. | `1.0` | | `recall` | `bool` | Whether to use the document (RAG) search or not | `True` | | `save` | `bool` | Whether this interaction should be stored in the session history or not | `True` | | `remember` | `bool` | DISABLED: Whether this interaction should form new memories or not (will be enabled in a future release) | `False` | | `model` | `str` | The model to use for the chat completion. | `None` | | `metadata` | `dict[str, Any]` | Custom metadata that can be passed to the system template for dynamic behavior. See [System Templates](/advanced/system-templates) for details. | `None` | | `auto_run_tools` | `bool` | Whether to automatically execute tools and send the results back to the model (requires tools on the agent) | `False` | | `recall_tools` | `bool` | Whether to include tool requests and responses when recalling messages from history | `True` | ## Usage Here's an example of how to use the chat API in Julep using the SDKs: To use the Chat endpint, you always have to create a session first. ```python Python [expandable] theme={"dark"} # Create a session with custom recall options client.sessions.create( agent=agent.id, user=user.id, recall=True, recall_options={ "mode": "hybrid", # or "vector", "text" "num_search_messages": 4, # number of messages to search for documents "max_query_length": 1000, # maximum query length "alpha": 0.7, # weight to apply to BM25 vs Vector search results (ranges from 0 to 1) "confidence": 0.6, # confidence cutoff level (ranges from -1 to 1) "limit": 10, # limit of documents to return "lang": "en-US", # language to be used for text-only search "metadata_filter": {}, # metadata filter to apply to the search "mmr_strength": 0, # MMR Strength (ranges from 0 to 1) } ) # Chat in the session response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": "Tell me about Julep" } ], recall=True ) print("Agent's response:", response.choices[0].message.content) print("Searched Documents:", response.docs) # Chat with metadata for dynamic system prompts response_with_metadata = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": "Help me with Python" } ], metadata={ "language": "Spanish", "expertise_level": "beginner", "custom_instructions": "Use simple examples" } ) # The metadata will be available in the system template as {{metadata.language}}, etc. # Chat with structured JSON response response_json = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": "Analyze the sentiment of: 'I love using Julep, it makes AI development so much easier!'" } ], response_format={"type": "json_object"}, # When using response_format, instruct the model to return JSON settings={ "temperature": 0.3 } ) # Response will be valid JSON that can be parsed import json sentiment_data = json.loads(response_json.choices[0].message.content) ``` ```javascript Node.js [expandable] theme={"dark"} client.sessions.create({ agent: agent.id, user: user.id, recall: true, recall_options: { mode: "hybrid", // or "vector", "text" num_search_messages: 4, // number of messages to search for documents max_query_length: 1000, // maximum query length alpha: 0.7, // weight to apply to BM25 vs Vector search results (ranges from 0 to 1) confidence: 0.6, // confidence cutoff level (ranges from -1 to 1) limit: 10, // limit of documents to return lang: "en-US", // language to be used for text-only search metadata_filter: {}, // metadata filter to apply to the search mmr_strength: 0, // MMR Strength (ranges from 0 to 1) } }); // Chat in the session const response = await client.sessions.chat({ session_id: session.id, messages: [ { role: "user", content: "Tell me about Julep" } ], recall: true }); // Chat with metadata for dynamic system prompts const responseWithMetadata = await client.sessions.chat({ session_id: session.id, messages: [ { role: "user", content: "Help me with Python" } ], metadata: { language: "Spanish", expertise_level: "beginner", custom_instructions: "Use simple examples" } }); // The metadata will be available in the system template as {{metadata.language}}, etc. // Chat with structured JSON response const responseJson = await client.sessions.chat({ session_id: session.id, messages: [ { role: "user", content: "Analyze the sentiment of: 'I love using Julep, it makes AI development so much easier!'" } ], response_format: { type: "json_object" }, // When using response_format, instruct the model to return JSON settings: { temperature: 0.3 } }); // Response will be valid JSON that can be parsed const sentimentData = JSON.parse(responseJson.choices[0].message.content); ``` To learn more about the Session object, check out the [Session](/concepts/sessions) page. Check out the [API reference](/api-reference/sessions/chat) or SDK reference ([Python](/sdks/python/reference#sessions) or [JavaScript](/sdks/nodejs/reference#sessions)) for more details on different operations you can perform on sessions. ## Response * **Content-Type**: `application/json` * **Body**: A `MessageChatResponse` object containing the full generated message(s) * **Content-Type**: `text/event-stream` * **Body**: A stream of `ChatOutputChunk` objects sent as Server-Sent Events (SSE) To enable streaming, set `stream: true` in your chat request: ```python Python theme={"dark"} from julep import AsyncClient # Streaming requires async client async_client = AsyncClient(api_key="your-api-key") response = await async_client.sessions.chat( session_id=session.id, messages=[{"role": "user", "content": "Tell me a story"}], stream=True ) async for chunk in response: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="") ``` ```javascript Node.js theme={"dark"} const response = await client.sessions.chat({ session_id: session.id, messages: [{ role: "user", content: "Tell me a story" }], stream: true }); for await (const chunk of response) { process.stdout.write(chunk.choices[0].delta.content); } ``` Streaming is not available when `auto_run_tools=true`. The system will return an error if both options are enabled. Both types of responses include the following fields: * `id`: The unique identifier for the chat response * `choices`: An object of generated message completions containing: * `role`: The role of the message (e.g. "assistant", "user", etc.) * `id`: Unique identifier for the message * `content`: list of actual message content * `created_at`: Timestamp when the message was created * `name`: Optional name associated with the message * `tool_call_id`: Optional ID referencing a tool call * `tool_calls`: Optional list of tool calls made during message generation * `created_at`: When this resource was created as UTC date-time * `docs`: List of document references used for this request, intended for citation purposes * `jobs`: List of UUIDs for background jobs that may have been initiated as a result of this interaction * `usage`: Statistics on token usage for the completion request ## Automatic Tool Calling Julep supports automatic tool execution during chat interactions. This feature allows tools to be executed seamlessly without manual intervention, making conversations more fluid and responsive. ### How It Works 1. **When `auto_run_tools=true`**: * The model identifies when a tool should be used based on the conversation * The tool is automatically executed by Julep's backend * Results are fed back to the model to continue the conversation * The entire process happens in a single API call 2. **When `auto_run_tools=false` (default)**: * The model returns tool call requests in the response * Your application must execute the tools manually * Results need to be sent back in a follow-up message ### Example with Automatic Tool Execution ```python Python theme={"dark"} # Agent with a weather tool agent = client.agents.create( name="Weather Assistant", tools=[ { "name": "get_weather", "type": "integration", "integration": {"provider": "weather"} } ] ) # Chat with automatic tool execution response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": "What's the weather like in Tokyo and Paris?" } ], auto_run_tools=True # Tools execute automatically ) # The response already includes the weather data print(response.choices[0].message.content) # Output: "In Tokyo, it's currently 22°C with clear skies. # In Paris, it's 15°C with light rain." ``` ```javascript Node.js theme={"dark"} // Agent with a weather tool const agent = await client.agents.create({ name: "Weather Assistant", tools: [ { name: "get_weather", type: "integration", integration: {provider: "weather"} } ] }); // Chat with automatic tool execution const response = await client.sessions.chat({ session_id: session.id, messages: [ { role: "user", content: "What's the weather like in Tokyo and Paris?" } ], auto_run_tools: true // Tools execute automatically }); // The response already includes the weather data console.log(response.choices[0].message.content); // Output: "In Tokyo, it's currently 22°C with clear skies. // In Paris, it's 15°C with light rain." ``` ### Tool History Management The `recall_tools` parameter controls whether tool calls and their results are included when recalling conversation history: * **`recall_tools=true` (default)**: Tool interactions are preserved in the conversation history * **`recall_tools=false`**: Tool calls and results are excluded from recalled messages This is useful when you want to maintain a cleaner conversation history without the technical details of tool executions. ## Finish Reasons Natural stop point or provided stop sequence reached Maximum number of tokens specified in the request was reached Content was omitted due to a flag from content filters The model called a tool ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Files (Multimedia) Source: https://docs.julep.ai/advanced/files Learn about file handling and management in Julep ## Overview Julep’s Files feature supports uploading, storing, retrieving, and integrating files across Julep components. It handles various file types and formats. ## File Properties Each file in Julep has the following properties: ```typescript theme={"dark"} id: string; // Unique identifier name: string; // Name of the file content: string; // Base64 encoded content created_at: string; // Creation timestamp size: number; // File size in bytes hash: string; // File hash for verification description?: string; // Optional description mime_type?: string; // Optional MIME type ``` ## Basic Operations ### Creating Files Here's how you can create files using the SDKs: ```python Python theme={"dark"} from julep import Julep client = Julep() file = client.files.create( name="example.pdf", content="base64_encoded_content", description="Sample document", mime_type="application/pdf" ) ``` ```javascript Node.js theme={"dark"} import Julep from '@julep/sdk'; const client = new Julep(); const file = await client.files.create({ name: "example.pdf", content: "base64_encoded_content", description: "Sample document", mime_type: "application/pdf" }); ``` ### Retrieving Files Here's how you can retrieve files using the SDKs: ```python Python theme={"dark"} file = client.files.get("file_id") ``` ```javascript Node.js theme={"dark"} const file = await client.files.get("file_id"); ``` ### Deleting Files ```python Python theme={"dark"} client.files.delete("file_id") ``` ```javascript Node.js theme={"dark"} await client.files.delete("file_id"); ``` ## API Reference For complete API documentation, see [Files API Reference](https://dev.julep.ai/api/docs#tag/files). ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Execution Lifecycle Source: https://docs.julep.ai/advanced/lifecycle Understanding the Lifecycle of Task when executed in Julep ## Overview Understanding the lifecycle of a Julep task is crucial for building effective workflows. Let's explore this through the lens of a [Trip Plan task](/tutorials/trip-planning) - a practical example that showcases key aspects of task execution. The Trip Plan task is an ideal example because it demonstrates: 1. Parallel execution of workflow steps 2. Integration with multiple external APIs: * [Weather API](/integrations/communicationdata/weather) for current conditions * [Brave Search API](/integrations/search/brave) for attractions 3. Data aggregation from various sources 4. AI-powered personalization of results Below is a visual representation of how this task flows through Julep's execution lifecycle: Let's examine each phase of the execution lifecycle using this task as our guide. This is a simplified representation of the execution lifecycle. In practice, Julep handles many more details and edge cases to ensure robust and efficient task execution. Out here in this section, we will be focusing on the main workflow steps of the Trip Plan task and how these steps are executed in Julep. ### 1. Input Schema First, we define what inputs our task expects: ```yaml theme={"dark"} input_schema: type: object properties: locations: type: array items: type: string ``` This schema specifies that our task expects an array of location strings (e.g., `["New York", "London", "Paris"]`). ### 2. Tools Configuration Next, we define the external tools our task will use: ```yaml theme={"dark"} - name: weather type: integration integration: provider: weather setup: openweathermap_api_key: {openweathermap_api_key} - name: internet_search type: integration integration: provider: brave setup: brave_api_key: {brave_api_key} ``` We're using two integrations: * The `weather` integration to fetch current weather conditions * The `brave` search integration to find tourist attractions ### 3. Main Workflow Steps #### Step 1: Fetch Weather Data ```yaml theme={"dark"} - over: $ steps[0].input.locations map: tool: weather arguments: location: $ _ ``` This step: * Iterates over each location in the input array * Calls the weather API for each location * The `_` represents the current location in the iteration #### Step 2: Search for Tourist Attractions ```yaml theme={"dark"} - over: $ steps[0].input.locations map: tool: internet_search arguments: query: $ 'tourist attractions in ' + _ ``` This step: * Iterates over the locations again * Searches for tourist attractions in each location * Constructs a search query by concatenating "tourist attractions in " with the location #### Step 3: Combine Data ```yaml theme={"dark"} - evaluate: zipped: |- $ list( zip( steps[0].input.locations, [output['result'] for output in steps[0].output], steps[1].output ) ) ``` This step: * Combines the data from previous steps into tuples * Each tuple contains: (location, weather\_data, attractions\_data) * Uses Python's `zip` function to align the data #### Step 4: Generate Itineraries ```yaml [expandable] theme={"dark"} - over: $ _['zipped'] parallelism: 3 # Inside the map step, each `_` represents the current element in the list # which is a tuple of (location, weather, attractions) map: prompt: - role: system content: >- $ f'''You are {agent.name}. Your task is to create a detailed itinerary for visiting tourist attractions in some locations. The user will give you the following information for each location: - The location - The current weather condition - The top tourist attractions''' - role: user content: >- $ f'''Location: {_[0]} Weather: {_[1]} Attractions: {_[2]}''' unwrap: true ``` This step: * Processes up to 3 locations in parallel * For each location tuple: * Sends a prompt to the AI model * Includes location, weather, and attraction data * Generates a personalized itinerary #### Step 5: Format Final Output ```yaml theme={"dark"} - evaluate: final_plan: |- $ '\\n---------------\\n'.join(activity for activity in _) ``` This step: * Combines all itineraries into a single output * Separates each itinerary with a divider ## Example Output An example output when running this task with an input of `["New York", "London", "Paris", "Tokyo", "Sydney"]`:
Here's a detailed itinerary for visiting top tourist attractions in New York, considering the current weather conditions. ### Day 1: Iconic Landmarks and Observation Decks * **Morning:** * **Top of the Rock Observation Deck:** Start your day with a visit to the Top of the Rock Observation Deck at Rockefeller Plaza. The panoramic 360-degree views from the 70th floor are a must-see. Dress warmly as it feels like 5.2°C outside, and it’s quite windy. * **Link for more info:** [Tripadvisor - Top of the Rock](https://www.tripadvisor.com/Attractions-g60763-Activities-New_York_City_New_York.html) * **Afternoon:** * **St. Patrick’s Cathedral:** Just a short walk from Rockefeller Plaza, explore the stunning architecture of St. Patrick’s Cathedral. The overcast skies will provide a dramatic backdrop for photos. * **Fifth Avenue:** Enjoy a leisurely stroll along Fifth Avenue, visiting iconic stores and landmarks. * **Evening:** * **Times Square:** Experience the vibrant lights and energy of Times Square. The overcast clouds might enhance the brightness of the neon lights. ### Day 2: Culture and History * **Morning:** * **The Museum of Modern Art (MoMA):** Spend your morning exploring MoMA’s vast collection of modern and contemporary art. This indoor activity is perfect for a cloudy day. * **Afternoon:** * **Central Park:** Head to Central Park for a refreshing walk. With 100% cloud cover, it's a great day to explore the park without the harsh sun. Consider visiting the Central Park Zoo or taking a guided tour. * **Evening:** * **Broadway Show:** End your day with a Broadway show. It’s an ideal indoor activity to avoid the chilly weather outside. Book tickets in advance for popular shows. ### Day 3: Historical and Educational * **Morning:** * **Statue of Liberty and Ellis Island:** Take a ferry to visit these iconic sites. Dress warmly for the ferry ride. The cloud cover will provide a unique perspective for photos. * **Afternoon:** * **9/11 Memorial and Museum:** Spend your afternoon reflecting at the 9/11 Memorial and exploring the museum exhibits. * **Evening:** * **Brooklyn Bridge:** Walk across the Brooklyn Bridge and enjoy the city skyline. With the wind speed at 5.81 m/s, be prepared for breezy conditions. ### Additional Tips: * **Clothing:** Wear layers to keep warm, as the temperature feels colder than it actually is. * **Dining:** New York offers a plethora of dining options. Consider trying some local favorites like a classic New York bagel or pizza. * **Transportation:** Utilize the subway for efficient travel across the city. Taxis and ride-sharing services are also readily available. For more details on attractions and guided tours, you can visit [USA Guided Tours](https://usaguidedtours.com/nyc/attraction/) and [I Love NY](https://www.iloveny.com/places-to-go/new-york-city/attractions/). ## Enjoy your trip to New York City! **Day 1: Exploring Iconic London Landmarks** **Morning:** 1. **Buckingham Palace** * Start your day early with a visit to Buckingham Palace. Arrive by 9:30 AM to catch the Changing of the Guard ceremony, which typically starts at 11:00 AM. Enjoy the majestic architecture and the surrounding gardens. * Weather Tip: With the overcast clouds, it might feel chilly, so dress warmly and bring an umbrella just in case. **Midday:** 2\. **Westminster Abbey** * Head towards Westminster Abbey, a short walk from Buckingham Palace. This historic church has been the site of many significant events, including royal weddings and coronations. Spend about 1.5 hours exploring. 3. **Lunch at Borough Market** * Take a tube or walk to Borough Market for a variety of food options. It's a great place to warm up with some hot street food and explore the diverse culinary offerings. **Afternoon:** 4\. **London Eye** * After lunch, head to the London Eye. The overcast sky might not offer the clearest views, but the experience is still worthwhile. Pre-book your tickets to avoid long lines and enjoy a 30-minute ride on this iconic Ferris wheel. **Evening:** 5\. **The Globe Theatre** * As the day winds down, visit Shakespeare's Globe Theatre. If there's a performance, consider attending or simply take a guided tour to learn about the history of this famous theater. **Day 2: Museums and Cultural Exploration** **Morning:** 1. **The British Museum** * Start your second day at the British Museum. Spend a few hours exploring the vast collection of art and antiquities. Highlights include the Rosetta Stone and the Elgin Marbles. **Midday:** 2\. **Lunch near Covent Garden** * Head to Covent Garden for lunch. This area is bustling with restaurants and cafes, perfect for a cozy indoor meal. **Afternoon:** 3\. **The Tower of London** * After lunch, make your way to the Tower of London. Delve into England's rich history and see the Crown Jewels. Allocate about 2-3 hours for this visit. **Evening:** 4\. **Tower Bridge** * Conclude your day with a walk across Tower Bridge. The views of the Thames River and the cityscape are beautiful, even on a cloudy evening. **Day 3: Leisure and Local Experiences** **Morning:** 1. **Natural History Museum** * Begin your last day at the Natural History Museum. It's a family-friendly museum with fascinating exhibits, including dinosaur skeletons and a model of a blue whale. **Midday:** 2\. **Lunch at South Kensington** * Enjoy a relaxed lunch in South Kensington. There are plenty of options, from casual cafes to high-end dining. **Afternoon:** 3\. **Hyde Park** * Spend your afternoon strolling through Hyde Park. Visit the Serpentine Galleries if you're interested in contemporary art. The park's natural beauty is a peaceful retreat amidst the city hustle. **Evening:** 4\. **Dinner and a Show in the West End** * End your London adventure with a memorable dinner followed by a theater show in the West End. Book your tickets in advance for popular shows. **Additional Tips:** * Always check the attraction websites for any specific COVID-19 guidelines or changes in operating hours. * London’s public transport system is efficient; consider getting an Oyster card for convenient travel. * Don't forget to dress in layers to adapt to the chilly weather, and always have your camera ready to capture memories. *** Here’s a detailed itinerary for your visit to Paris, considering the current snowy weather conditions and top attractions: ### Day 1: Embrace the Iconic Landmarks **Morning: Eiffel Tower** * **Time:** 9:00 AM * **Details:** Begin your day with a visit to the Eiffel Tower. Even in the snow, the tower offers stunning views of Paris. Dress warmly and enjoy hot chocolate from nearby cafes. * **Link for more info:** [Tripadvisor - Things to Do in Paris](https://www.tripadvisor.com/Attractions-g187147-Activities-Paris_Ile_de_France.html) **Afternoon: Louvre Museum** * **Time:** 1:00 PM * **Details:** Spend your afternoon indoors at the Louvre Museum. With its vast collection of art and history, it's a perfect way to escape the cold. Consider taking a guided tour to make the most of your visit. * **Link for more info:** [U.S. News Travel - Things To Do](https://travel.usnews.com/Paris_France/Things_To_Do/) **Evening: Seine River Cruise** * **Time:** 6:00 PM * **Details:** End your day with a magical Seine River cruise. The snow adds a picturesque touch to the illuminated landmarks. Ensure to book a heated cruise for comfort. ### Day 2: Explore Cultural and Historical Treasures **Morning: Notre-Dame Cathedral** * **Time:** 9:30 AM * **Details:** Visit the iconic Notre-Dame Cathedral. Although some areas may be under restoration, its architecture and history are worth experiencing. Warm clothing is essential as the interior can be chilly. **Afternoon: Musée d'Orsay** * **Time:** 1:30 PM * **Details:** Head to the Musée d'Orsay, renowned for its Impressionist masterpieces. This indoor activity is ideal for escaping the cold and enjoying world-class art. **Evening: Montmartre and Sacré-Cœur** * **Time:** 5:00 PM * **Details:** Wander through the charming streets of Montmartre and visit the Sacré-Cœur Basilica. The view of Paris in the snow is breathtaking. Enjoy a cozy dinner at a local bistro in Montmartre. ### Day 3: Discover Hidden Gems and Local Flavors **Morning: Le Marais District** * **Time:** 10:00 AM * **Details:** Explore Le Marais, known for its vibrant street art, boutiques, and cafes. Enjoy a leisurely breakfast and shop for unique souvenirs. **Afternoon: Palais Garnier (Opera House)** * **Time:** 2:00 PM * **Details:** Tour the opulent Palais Garnier. Its stunning interiors are a must-see, especially when it's snowy outside. **Evening: Moulin Rouge Show** * **Time:** 8:00 PM * **Details:** Conclude your trip with a classic Parisian experience at the Moulin Rouge. Book in advance to secure a good seat and enjoy the legendary cabaret performance. ### Additional Tips: * **Weather Preparation:** Wear layers, waterproof boots, and carry an umbrella. The snow and cold wind can be intense. * **Dining:** Indulge in warm, hearty French cuisine at local cafes and restaurants. Try dishes like French onion soup, coq au vin, and tarte Tatin. * **Transport:** Use public transportation to avoid the snowy streets, and consider purchasing a Paris Visite pass for unlimited travel. ## Enjoy your snowy adventure in Paris! **Tokyo Itinerary** **Day 1: Arrival and Exploration of Historical and Cultural Sites** * **Morning:** * **Asakusa District**: Begin your day with a visit to the historic Asakusa district. Explore the iconic Senso-ji Temple, Tokyo's oldest temple. Enjoy the traditional market streets like Nakamise Street for some shopping and snacks. * **Afternoon:** * **Tokyo National Museum**: Head to Ueno Park and visit the Tokyo National Museum. Discover Japan’s extensive collection of art and antiquities. This is a great spot to dive into Japanese history and culture. * **Evening:** * **Dinner in Ueno**: Explore the local dining options around Ueno and enjoy a traditional Japanese dinner. **Day 2: Modern Tokyo and Unique Experiences** * **Morning:** * **Ghibli Museum**: Start your day with a magical visit to the Ghibli Museum in Mitaka. Perfect for fans of Studio Ghibli's animated films, this museum offers a whimsical look into the creative world of Hayao Miyazaki. * **Afternoon:** * **Shibuya and Harajuku**: Head towards the bustling areas of Shibuya and Harajuku. Witness the famous Shibuya Crossing and explore the trendy shops of Harajuku, especially Takeshita Street. * **Evening:** * **Golden Gai**: Conclude your day in the vibrant Golden Gai district. This area is renowned for its narrow alleys filled with small bars and eateries. Experience the unique nightlife of Tokyo here. **Day 3: Relax and Explore Green Spaces** * **Morning:** * **Shinjuku Gyoen National Garden**: Spend a peaceful morning strolling through the beautiful Shinjuku Gyoen, one of Tokyo's largest and most beautiful parks. It's a perfect spot for relaxation and enjoying nature. * **Afternoon:** * **Meiji Shrine**: Visit the Meiji Shrine, located in a forested area near Harajuku and Shibuya. It's a serene place to learn about Shinto traditions and enjoy the tranquil setting. * **Evening:** * **Tokyo Tower or Skytree**: End your trip with a visit to either Tokyo Tower or Tokyo Skytree for a panoramic view of the city. It's an unforgettable way to see Tokyo illuminated at night. **Weather Considerations:** * With the current weather of few clouds and a mild temperature around 10.32°C, it is advisable to wear layers and carry a light jacket for comfort during outdoor activities. * Humidity is high (92%), so be prepared for a slightly damp feeling and consider moisture-wicking clothing. **Additional Tips:** * Always check the opening hours of attractions and book tickets in advance where necessary. * Use Tokyo’s efficient public transport to move around easily. * Consider visiting the websites linked in the attraction descriptions for more detailed information and current updates. *** Here's a detailed itinerary for exploring some of Sydney's top tourist attractions with the current weather conditions in mind. With clear skies and pleasant temperatures, it's a perfect day to explore the outdoors and enjoy what Sydney has to offer. ### Morning **9:00 AM - Sydney Opera House** * Begin your day with a visit to the iconic Sydney Opera House. Take a guided tour to learn about its history and architecture. Tours are available in multiple languages. * **Link:** [Top attractions in Sydney | Sydney.com](https://www.sydney.com/things-to-do/attractions) **11:00 AM - Royal Botanic Garden Sydney** * Just a short walk from the Opera House, enjoy a leisurely stroll through the Royal Botanic Garden. The clear skies will offer beautiful views of the diverse plant life and the Sydney Harbour. ### Afternoon **12:30 PM - Lunch at Opera Bar** * Head back to Opera Bar for lunch. Enjoy a refreshing cocktail with stunning views of the Sydney Harbour Bridge and the waterfront. * **Link:** [Top attractions in Sydney | Sydney.com](https://www.sydney.com/things-to-do/attractions) **2:00 PM - Sydney Harbour Bridge** * After lunch, take a scenic walk across the Sydney Harbour Bridge. If you're up for it, consider the BridgeClimb for breathtaking panoramic views of the city. **4:00 PM - The Rocks** * Explore The Rocks, one of Sydney's most historic areas. Wander through the cobbled streets, visit the local markets, and perhaps enjoy a cup of coffee at a nearby café. ### Evening **6:00 PM - Darling Harbour** * Make your way to Darling Harbour for the evening. Here you can visit attractions such as the SEA LIFE Sydney Aquarium or simply enjoy the lively atmosphere by the waterfront. **8:00 PM - Dinner at a Local Restaurant** * Conclude your day with dinner at one of Darling Harbour's many restaurants. Choose from a variety of cuisines while enjoying the vibrant night scene. ### Additional Suggestions * If you're interested in more unique experiences, consider visiting some of the attractions listed on [Time Out Sydney](https://www.timeout.com/sydney/attractions/tourist-attractions-that-dont-suck), which includes thrilling adventures and scenic tours. With clear skies and mild temperatures, this itinerary offers a balanced mix of cultural, historical, and scenic experiences. Enjoy your visit to Sydney!
## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Local Setup Source: https://docs.julep.ai/advanced/localsetup Learn how to run Julep locally ## Overview Juelp is designed to be run locally with the help of Docker. This guide will help you set up Julep locally. ## Prerequisites * [Docker](https://docs.docker.com/get-docker/) ## Setup Instructions ### 1. Clone the Repository Clone the repository from your preferred source: ```bash theme={"dark"} git clone ``` ### 2. Navigate to the Root Directory Change to the root directory of the project: ```bash theme={"dark"} cd ``` ### 3. Set Up Environment Variables * Create a `.env` file in the root directory. * Refer to the [`.env.example`](https://github.com/julep-ai/julep/blob/dev/.env.example) file for a list of required variables. * Ensure that all necessary variables are set in the `.env` file. ### 4. Create a Docker Volume for Backup Create a Docker volume named `grafana_data`, `memory_store_data`, `temporal-db-data`, `prometheus_data` and `seadweedfs_data`: ```bash theme={"dark"} docker volume create grafana_data docker volume create memory_store_data docker volume create temporal-db-data docker volume create prometheus_data docker volume create seaweedfs_data ``` The volumes are used to store the data for the Grafana, Memory Store (Timescale DB), Temporal DB, Prometheus, and SeadweedFS, and Memories respectively. ### 5. Run the Project using Docker Compose You can run the project in two different modes: **Single Tenant** or **Multi-Tenant**. Choose one of the following commands based on your requirement: #### Single-Tenant Mode Run the project in single-tenant mode: ```bash theme={"dark"} docker compose --env-file .env --profile temporal-ui --profile single-tenant --profile self-hosted-db --profile blob-store --profile temporal-ui-public up --build --force-recreate --watch ``` > **Note:** In single-tenant mode, you can interact with the SDK directly without the need for the API KEY. #### Multi-Tenant Mode Run the project in multi-tenant mode: ```bash theme={"dark"} docker compose --env-file .env --profile temporal-ui --profile multi-tenant --profile self-hosted-db --profile blob-store --profile temporal-ui-public up --build --force-recreate --watch ``` > **Note:** In multi-tenant mode, you need to generate a JWT token locally that act as an API KEY to interact with the SDK. ### 6. Generate a JWT Token (Only for Multi-Tenant Mode) To generate a JWT token, `jwt-cli` is required. install it from [here](https://github.com/mike-engel/jwt-cli). Use the following command and replace `JWT_SHARED_KEY` with the corresponding key from your `.env` file to generate a JWT token: ```bash theme={"dark"} jwt encode --secret JWT_SHARED_KEY --alg HS512 --exp=$(date -j -v +10d +%s) --sub '00000000-0000-0000-0000-000000000000' '{}' ``` > **Note:** This command generates a JWT token that will be valid for 10 days. adjust the expiration date as needed. ### 7. Access and Interact * **Temporal UI**: You can access the Temporal UI through the specified port in your `.env` file. * **API Interactions**: Depending on the chosen mode, interact with the setup using the provided endpoints. ### Troubleshooting * Ensure that all required Docker images are available. * Check for missing environment variables in the `.env` file. * Use the `docker compose logs` command to view detailed logs for debugging. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Multi-Agent Multi-User Sessions Source: https://docs.julep.ai/advanced/multi-agent-multi-user-sessions Learn how to manage complex multi-agent systems with multiple users ## Overview Julep supports different types of sessions based on the number of agents and users involved. This flexibility allows for complex interactions and use cases. ## Types of Sessions
  1. No user
  2. Single user
  3. Multiple users
  1. No user
  2. Single user
  3. Multiple users
 ## Behavior in Multi-Agent/User Sessions ### User Behavior
  1. No user data is retrieved
  2. (Upcoming) Memories are not mined from the session
  1. Docs, metadata, memories, etc. are retrieved for all users in the session
  2. Messages can be added for each user by referencing them by name in the `ChatML` messages
  3. (Upcoming) Memories mined in the background are added to the corresponding user's scope
 ### Agent Behavior
  1. Works as expected
  1. When a message is received by the session, each agent is called one after another in the order they were defined in the session
  2. You can specify which `agent` to use in a request, in which case, only that agent will be used
 This multi-agent/user capability allows for sophisticated scenarios such as: * Collaborative problem-solving with multiple AI agents * Group conversations with multiple users and agents * Specialized agents working together on complex tasks ## Conclusion By supporting these various configurations, Julep provides a flexible framework for building diverse and powerful AI applications. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # New Syntax (Important) Source: https://docs.julep.ai/advanced/new-syntax Learn about the new workflow syntax and how to migrate your existing workflows ## Overview We've updated our workflow syntax to make data flow more explicit and consistent. These changes improve readability and make workflows more maintainable. While this update requires modifications to existing workflows, it provides a more robust foundation for workflow development. ## 1. Input/Output Data References The most significant change is how we reference input and output data between workflow steps. The new syntax makes the data flow more explicit by using the `steps` keyword. ### Old Syntax: * `inputs[x]` - Referenced input data from step x * `outputs[x]` - Referenced output data from step x * `_x` - Shorthand for input data from step x ### New Syntax: * `steps[x].input` - References input data from step x * `steps[x].output` - References output data from step x * `_x` - Has been removed ### Example #### Old Syntax ```yaml theme={"dark"} - evaluate: input_data: inputs[0].a output_data: outputs[0].b shorthand: _0.c ``` #### New Syntax ```yaml theme={"dark"} - evaluate: input: $ steps[0].input.a output: $ steps[0].output.b shorthand: $ steps[0].input.c ``` Notice the addition of the `$` prefix in the new syntax. This indicates that the value should be treated as a Python expression. We'll cover this in detail in the next section. ### Key Changes: * `inputs[0]` becomes `steps[0].input` * `outputs[0]` becomes `steps[0].output` * `_0` becomes `steps[0].input` * Added `$` prefix to indicate Python expressions ## 2. Template Syntax Changes Inside `prompt` Steps Another significant change is the removal of Jinja templates (`{{ }}`) in `log` steps and `prompt` steps. Instead, we now use Python f-strings with the `$` prefix for dynamic content. ### Old Syntax (Using Jinja): ```yaml theme={"dark"} - prompt: - role: user content: Write me an article about the topic: {{inputs[0].topic}} ``` ### New Syntax (Using f-strings): ```yaml theme={"dark"} - prompt: - role: user content: $ f"Write me an article about the topic: {steps[0].input.topic}" - role: user content: this is only text, no need for dollar sign prefix ``` For multiple lines, you can use a multiline f-string: ```yaml theme={"dark"} - prompt: - role: user content: |- $ f""" Write me an article about the topic: {steps[0].input.topic}. The article should be {steps[0].input.article_length} words long. """ ``` The `$` prefix is only needed when the value contains a Python expression or f-string. Plain text content doesn't require the prefix. ## 3. When to Use the \$ Prefix The `$` prefix should be used in two scenarios: 1. When referencing step data (inputs/outputs) 2. When using Python expressions or f-strings #### Examples: ```yaml theme={"dark"} steps: - prompt: # Needs $ because it uses an f-string - role: user content: $ f"The temperature is {steps[0].input.temp}°C" # No $ needed - plain text - role: user content: Tell me about the weather - evaluate: # Needs $ because it references step data input: $ steps[0].output # Needs $ because it's a Python expression condition: $ len(steps[0].input.text) > 100 ``` # Python Expression Source: https://docs.julep.ai/advanced/python-expression Learn how to use Python expressions in Julep task definitions ## Overview Julep tasks support Python expressions for dynamic value computation and data manipulation. This guide explains how to use them effectively. ## The Special `_` Variable The underscore `_` is a special variable that serves three different purposes depending on where it's used: 1. **First Step Input**: In the first step of a task, `_` contains the execution input: ```yaml theme={"dark"} # If task is executed with input `{"topic": "AI"}` - evaluate: topic: $ _.topic # Accesses the input "AI" ``` 2. **Previous Step Output**: In any subsequent step, `_` contains the output from the previous step: ```yaml theme={"dark"} - evaluate: results: $ _.split('\n') # Splits previous step's output into lines ``` 3. **Iterator Value**: In `foreach` and `map` steps, `_` represents the current item being iterated. ```yaml theme={"dark"} # If the task is executed with input `{"questions": ["What is AI?", "What is Julep?"]}` - foreach: in: $ _.questions do: - wait_for_input: info: "message": $ _ # _ is each question ``` ## The Special `$` Variable The `$` variable is used to differentiate between a Python expression and a string. When using the `$` variable, the expression is evaluated as a Python expression. ```yaml theme={"dark"} - evaluate: topic: $ _.topic # Accesses the input "AI" ``` ```yaml theme={"dark"} - prompt: - role: user content: |- $ f''' Please answer the following question: {_.question} ''' ``` To learn more about how to use the `$` variable, please refer to the [New Syntax](/advanced/new-syntax) section. ## Where Python Expressions Are Used Python expressions can be used in various task steps. For a complete list of step types and their syntax, refer to the [Step Types](/advanced/types-of-task-steps) page. Common places include: * `evaluate` steps * Tool `arguments` * `if` conditions * `foreach` and `map` iterations ## Available Functions and Libraries The following Python functions and libraries are available for use in expressions: ### Basic Python Builtins * `abs`, `all`, `any`, `bool`, `dict`, `enumerate` * `float`, `int`, `len`, `list`, `map`, `max`, `min` * `round`, `set`, `str`, `sum`, `tuple`, `zip`, `reduce` ### Safe Versions of Functions * `range`: `def safe_range(*args)` Safely creates a range object with size limits (max 1,000,000 elements). * `load_json`: `def safe_json_loads(s: str) -> Any` (Deprecated in favor of `json.loads`) Safely parses a JSON string with size limits. * `load_yaml`: `def safe_yaml_load(s: str) -> Any` (Deprecated in favor of `yaml.safe_load`) Safely parses a YAML string with size limits. * `dump_json`: `def dump_json(obj: Any, *, **kwargs) -> str` (Deprecated in favor of `json.dumps`) Safely serializes an object to a JSON string. * `dump_yaml`: `def dump_yaml(obj: Any, **kwargs) -> str` (Deprecated in favor of `yaml.dump`) Safely serializes an object to a YAML string. * `extract_json`: `def safe_extract_json(string: str) -> Any` Safely extracts and parses JSON from text. ### Regex and NLP Functions * `search_regex`: `def search_regex(pattern: str, string: str) -> Optional[re2.Match]` Searches for a regex pattern in a string. * `match_regex`: `def match_regex(pattern: str, string: str) -> bool` Checks if a regex pattern matches a string. * `chunk_doc`: `def chunk_doc(string: str) -> list[str]` Chunks a string into sentences. * `nlp` pipelines. Example using these functions in an evaluate step: ````yaml theme={"dark"} - evaluate: # Parse JSON string with size limit of 1MB data: $ json.loads(_.json_string) # Parse YAML string with size limit of 1MB config: $ yaml.safe_load(_.yaml_string) # Extract JSON from text that might contain markdown code blocks extracted_content: $ extract_json('Here is some JSON: ```json\n{"key": "value"}\n```') ```` ### CSV Functions * `reader`: `def reader(data: str, dialect="excel", delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting=0, strict: bool = False) -> csv._reader` Creates a CSV reader. * `writer`: `def writer(data: str, dialect="excel", delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting=0, strict: bool = False) -> csv._writer` Creates a CSV writer. * `DictReader`: `class DictReader(data: str, fieldnames=None, restkey=None, restval=None, dialect="excel", *args, **kwds)` Create an object that operates like a regular reader but maps the information in each row to a dict. * `DictWriter`: `class DictWriter(data: str, fieldnames, restval="", extrasaction="raise", dialect="excel", *args, **kwds)` Create an object which operates like a regular writer but maps dictionaries onto output rows. * `register_dialect`: `def register_dialect(name: str, dialect: type[Dialect] = ..., *, delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting: _QuotingType = 0, strict: bool = False) -> csv._reader` Associate dialect with name. * `unregister_dialect`: `def unregister_dialect(name: str) -> None` Delete the dialect associated with name from the dialect registry. * `get_dialect`: `def get_dialect(name: str) -> Dialect` Return the dialect associated with name. * `list_dialects`: `def list_dialects() -> list[str]` Return the names of all registered dialects. * `field_size_limit`: `def field_size_limit(new_limit: int = ...) -> int` Returns the current maximum field size allowed by the parser. * `Dialect`: `class Dialect()` The Dialect class is a container class whose attributes contain information for how to handle doublequotes, whitespace, delimiters, etc. * `excel`: `class excel()` The excel class defines the usual properties of an Excel-generated CSV file. * `excel_tab`: `class excel_tab()` The excel\_tab class defines the usual properties of an Excel-generated TAB-delimited file. * `unix_dialect`: `class unix_dialect()` The unix\_dialect class defines the usual properties of a CSV file generated on UNIX systems, i.e. using '\n' as line terminator and quoting all fields. * `Sniffer`: `class Sniffer` The Sniffer class is used to deduce the format of a CSV file. * `QUOTE_ALL` - Instructs writer objects to quote all fields. * `QUOTE_MINIMAL` - Instructs writer objects to only quote those fields which contain special characters such as delimiter, quotechar or any of the characters in lineterminator. * `QUOTE_NONNUMERIC` - Instructs writer objects to quote all non-numeric fields. Instructs reader objects to convert all non-quoted fields to type float. * `QUOTE_NONE` - Instructs writer objects to never quote fields. * `QUOTE_NOTNULL` - Instructs writer objects to quote all fields which are not None. * `QUOTE_STRINGS` - Instructs writer objects to always place quotes around fields which are strings. * `Error`: `class Error()` Raised by any of the functions when an error is detected. This module tries to mimic `csv` module from the standard Python library. You can find additional information [here](https://docs.python.org/3/library/csv.html) ## Example Usage ```yaml theme={"dark"} name: Data Processing Task main: # Using _ as input - evaluate: csv_data: $ [row for row in csv.reader("a,b,c\n1,2,3")] ``` ### Standard Library Modules * `re`: Regular expressions (using re2) * `json`: JSON operations * `yaml`: YAML operations * `string`: String constants and operations * `datetime`: Date and time operations * `math`: Mathematical functions * `statistics`: Statistical operations * `base64`: Base64 encoding/decoding * `urllib.parse`: URL parsing operations * `random`: Random number generation * `time`: Time operations * `csv`: CSV operations For the complete list of available functions and their safe implementations, refer to the [utils.py](https://github.com/julep-ai/julep/blob/main/agents-api/agents_api/activities/utils.py) file in the source code. ## Example Usage Here's a practical example combining different aspects of Python expressions: ```yaml theme={"dark"} name: Data Processing Task main: # Using _ as input - evaluate: topics: $ _.topics # Access input topics # Using _ as previous output - evaluate: filtered_topics: $ [t for t in _.topics if len(t) > 3] # Using _ in foreach - foreach: in: $ _.filtered_topics do: - tool: web_search arguments: query: $ 'Latest news about ' + _ # _ is each topic ``` ### Custom Implementations #### Humanization (Alpha) The `humanize_text_alpha` function transforms text using multiple techniques such as: * Back-translation. * Rewriting using non-public LLMs. * Stylistic modifications such as homoglyphs and em dashes. Breakdown of the function: Function signature: `humanize_text_alpha(text: str, threshold: float = 90, src_lang: str = "english", target_lang: str = "german", use_homoglyphs: bool = True, use_em_dashes: bool = True, grammar_check: bool = False, max_tries: int = 10) -> str` Parameters: * `text`: The text to be humanized. * `threshold`: The threshold for the humanization (0-100). * `src_lang`: The source language of the original text. * `target_lang`: The target language used in the back-translation technique. * `use_homoglyphs`: Whether to use homoglyphs, a technique that replaces certain characters with similar looking characters to trick AI detection classifiers. * `use_em_dashes`: Whether to use em dashes, i.e. adding dashes between longer words to break the tokens. * `grammar_check`: Whether to run another prompt after back-translation to check for correct grammar. * `max_tries`: The maximum number of tries to humanize the text. Example usage: ```yaml theme={"dark"} - evaluate: humanized_text: $ humanize_text_alpha(_.text, threshold=30, target_lang="german") ``` #### Markdown to HTML The `markdown_to_html` function converts markdown text to HTML. Breakdown of the function: Function signature: `markdown_to_html(markdown_text: str) -> str` Parameters: * `markdown_text`: The markdown text to be converted to HTML. Example usage: ```yaml theme={"dark"} - evaluate: html_text: $ markdown_to_html(_.markdown_text) ``` #### HTML to Markdown The `html_to_markdown` function converts HTML text to markdown. Breakdown of the function: Function signature: `html_to_markdown(html_text: str) -> str` Parameters: * `html_text`: The HTML text to be converted to markdown. Example usage: ```yaml theme={"dark"} - evaluate: markdown_text: $ html_to_markdown(_.html_text) ``` ## Security All Python expressions are executed in a sandboxed environment with:

Limited available functions to prevent unsafe operations

Maximum string length restrictions to prevent memory issues

Collection size limits to prevent resource exhaustion

Execution time limits to prevent infinite loops

 This ensures safe execution while providing necessary functionality for task workflows. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Render Endpoint in Julep Source: https://docs.julep.ai/advanced/render Learn about the render endpoint for previewing chat inputs before sending them to the model ## Overview Julep provides a render endpoint that allows you to preview how chat inputs will be processed before actually sending them to the model. This is useful for debugging, testing templates, and understanding how the system processes your messages. ## Purpose The render endpoint processes chat inputs exactly as the chat endpoint would, but stops short of sending them to the model. See how templates in your messages and system prompts will be rendered with the current environment variables. Preview which documents will be retrieved from your knowledge base for a given input. Examine how tools will be formatted and made available to the model. Useful for debugging complex prompts and understanding the exact input that would be sent to the model. * The render endpoint uses the same input format as the chat endpoint but returns the processed messages without actually calling the model. * To use the render endpoint, you need to create a session first. To learn more about the session object, check out the [Session](/concepts/sessions) page. ## Input Structure The render endpoint accepts the same input structure as the chat endpoint: * **Messages**: An array of input messages representing the conversation so far. * **Tools**: (Advanced) Additional tools provided for this specific interaction. * **Tool Choice**: Specifies which tool the agent should use. * **Memory Access**: Controls how the session accesses history and memories (`recall` parameter). * **Additional Parameters**: Various parameters to control the behavior of the rendering. Here's an example of how a typical message object might be structured in a render request: ```python Python theme={"dark"} """ Attributes for the Message object: role (Literal["user", "assistant", "system", "tool"]): The role of the message sender. tool_call_id (str | None): Optional identifier for a tool call associated with this message. content (Annotated[str | list[str] | list[Content | ContentModel7 | ContentModel] | None, Field(...)]): The main content of the message, which can be a string, a list of strings, or a list of content models. name (str | None): Optional name associated with the message. tool_calls (list[ChosenFunctionCall | ChosenComputer20241022 | ChosenTextEditor20241022 | ChosenBash20241022] | None): List of tool calls generated during the message creation, if any. """ # Example of a simple message structure messages = [{"role": "user", "content": "Your query here"}] ```

This object represents a message in the chat system, detailing the structure and types of data it can hold.

 ## Additional Parameters The render endpoint accepts the same parameters as the chat endpoint: | Parameter | Type | Description | Default | | ----------------- | ----------- | --------------------------------------------------------------------------------------------- | ------- | | `model` | `str` | The model to use for validation (though no actual model call is made). | `None` | | `agent` | `UUID` | Agent ID of the agent to use for this interaction. (Only applicable for multi-agent sessions) | `None` | | `recall` | `bool` | Whether previous memories and docs should be recalled or not. | `True` | | `response_format` | `str` | Response format specification (used for validation only). | `None` | | `temperature` | `float` | Not used in rendering but validated for format. | `None` | | `top_p` | `float` | Not used in rendering but validated for format. | `1.0` | | `max_tokens` | `int` | Not used in rendering but validated for format. | `None` | | `stop` | `list[str]` | Not used in rendering but validated for format. | `[]` | ## Response Structure The render endpoint returns a `RenderResponse` object with the following structure: ```json theme={"dark"} { "messages": [ // Array of processed messages that would be sent to the model ], "docs": [ // Array of document references that would be used for this request ], "tools": [ // Array of formatted tools that would be available to the model ] } ``` The render response includes: * `messages`: The fully processed messages, including rendered templates and system messages. * `docs`: List of document references that would be used for this request, intended for citation purposes. * `tools`: The formatted tools that would be available to the model. ## Usage Here's an example of how to use the render endpoint in Julep using the SDKs: To use the render endpoint, you always have to create a session first. ```python Python [expandable] theme={"dark"} # Create a session with custom recall options client.sessions.create( agent=agent.id, user=user.id, recall=True, recall_options={ "mode": "hybrid", # or "vector", "text" "num_search_messages": 4, # number of messages to search for documents "max_query_length": 1000, # maximum query length "alpha": 0.7, # weight to apply to BM25 vs Vector search results (ranges from 0 to 1) "confidence": 0.6, # confidence cutoff level (ranges from -1 to 1) "limit": 10, # limit of documents to return "lang": "en-US", # language to be used for text-only search "metadata_filter": {}, # metadata filter to apply to the search "mmr_strength": 0, # MMR Strength (ranges from 0 to 1) } ) # Render the chat input without sending to the model render_response = client.sessions.render( session_id=session.id, messages=[ { "role": "user", "content": "Tell me about Julep" } ], recall=True ) print("Processed messages:", render_response.messages) print("Retrieved documents:", render_response.docs) print("Available tools:", render_response.tools) ``` ```javascript Node.js [expandable] theme={"dark"} client.sessions.create({ agent: agent.id, user: user.id, recall: true, recall_options: { mode: "hybrid", // or "vector", "text" num_search_messages: 4, // number of messages to search for documents max_query_length: 1000, // maximum query length alpha: 0.7, // weight to apply to BM25 vs Vector search results (ranges from 0 to 1) confidence: 0.6, // confidence cutoff level (ranges from -1 to 1) limit: 10, // limit of documents to return lang: "en-US", // language to be used for text-only search metadata_filter: {}, // metadata filter to apply to the search mmr_strength: 0, // MMR Strength (ranges from 0 to 1) } }); // Render the chat input without sending to the model const renderResponse = await client.sessions.render({ session_id: session.id, messages: [ { role: "user", content: "Tell me about Julep" } ], recall: true }); ``` ## Use Cases Test how your templates will be rendered with the current environment variables. Preview which documents will be retrieved for a given query. Verify that tools are properly configured before sending to the model. Test how system prompts will be processed and combined with user messages. ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Secrets Management Source: https://docs.julep.ai/advanced/secrets-management Advanced techniques for managing sensitive information in Julep # Secrets Management This guide covers advanced topics for managing secrets in Julep, including security architecture, best practices, rotation policies, and integration patterns. ## Security Architecture Secrets in Julep are stored with a layered security approach: 1. **Application-level Validation**: Secrets are validated before being stored 2. **Database Encryption**: Secrets are stored encrypted using PostgreSQL's pgcrypto extension with AES-256 3. **Access Control**: Secrets are scoped to developers and only accessible within their resources 4. **Master Key Security**: A separate master encryption key secures all stored secrets The encryption process works as follows: * When a secret is created, its value is encrypted using the master key * The encrypted value is stored in the database's `value_encrypted` column * When a secret is accessed, the value is decrypted using the master key * The master key is stored as an environment variable, separate from the database ## Creating Effective Secret Names Secrets should have descriptive names that follow these conventions: * Use snake\_case formatting * Begin with a letter and contain only alphanumeric characters and underscores * Use a prefix to indicate the service (e.g., `aws_secret_key`, `stripe_api_key`) * Be specific enough to understand the purpose (e.g., `gmail_oauth_token` vs `email_token`) ## Secret Rotation Best Practices Regular rotation of secrets is a security best practice: 1. Create a new secret with a temporary name 2. Update your services to use the new secret 3. Once confirmed working, delete the old secret 4. Update the new secret's name to the standard name For automated rotation: ```python [expandable] theme={"dark"} from julep import Julep import uuid client = Julep(api_key="your_api_key") # Generate temporary name temp_name = f"stripe_key_rotation_{uuid.uuid4().hex[:8]}" # Create new secret with temp name client.secrets.create( name=temp_name, value="sk_new_value...", description="New Stripe API key (rotation)", metadata={"rotation_date": "2025-05-10"} ) # Test the new key (implement your validation logic here) # ... # If valid, delete old secret and rename new one client.secrets.delete(name="stripe_api_key") client.secrets.update( name=temp_name, new_name="stripe_api_key", description="Stripe API key", metadata={"last_rotated": "2025-05-10"} ) ``` ## Using Secrets with Different Tool Types ### API Tools For HTTP-based tools, reference secrets in the headers or authentication: ```yaml theme={"dark"} steps: - kind: tool_call tool: api_service operation: fetch_data arguments: url: "https://api.example.com/data" headers: Authorization: "$ f'Bearer {secrets.api_token}'" ``` ### Database Connections For database tools, use secrets for connection credentials: ```yaml theme={"dark"} steps: - kind: tool_call tool: database operation: query arguments: query: "SELECT * FROM users LIMIT 10" secrets: db_username: "my_db_user" db_password: "my_db_password" ``` ### AI Service Integration For AI services that require API keys: ```yaml theme={"dark"} steps: - kind: prompt model: "$ secrets.preferred_model" provider: "$ secrets.provider_name" prompt: "Generate creative ideas for a marketing campaign" api_key: "$ secrets.openai_api_key" ``` ## Managing Secrets for Multi-Environment Deployments For applications deployed across development, staging, and production environments: 1. Use consistent naming conventions with environment prefixes: * `dev_stripe_key`, `staging_stripe_key`, `prod_stripe_key` 2. Use metadata to tag secrets by environment: ```python theme={"dark"} client.secrets.create( name="stripe_api_key", value="sk_test_...", metadata={"environment": "production"} ) ``` 3. Filter secrets by environment when listing: ```python theme={"dark"} prod_secrets = client.secrets.list( metadata={"environment": "production"} ) ``` ## Secret Templating For complex configurations that require multiple secrets: ```yaml theme={"dark"} steps: - kind: tool_call tool: database operation: query arguments: connection_string: "$ f'mongodb+srv://{secrets.db_username}:{secrets.db_password}@{secrets.db_host}/{secrets.db_name}'" ``` ## Securing LLM API Keys with Secrets Julep automatically looks for LLM API keys in your secrets store based on the provider name. Use these naming conventions for automatic lookup: | Provider | Secret Name | | ------------ | ---------------------- | | OpenAI | `OPENAI_API_KEY` | | Anthropic | `ANTHROPIC_API_KEY` | | Google | `GOOGLE_API_KEY` | | Azure OpenAI | `AZURE_OPENAI_API_KEY` | | Cohere | `COHERE_API_KEY` | Example of setting up an LLM API key: ```python theme={"dark"} client.secrets.create( name="OPENAI_API_KEY", value="sk-...", description="OpenAI API key for GPT-4 access" ) ``` ## Audit and Monitoring Best practices for security monitoring: 1. Regularly audit secret access and usage 2. Track changes to secrets via the `updated_at` timestamp 3. Implement secret expiration for highly sensitive data 4. Use metadata to track last review or rotation dates Example audit script: ```python theme={"dark"} from julep import Julep from datetime import datetime, timedelta client = Julep(api_key="your_api_key") # Find secrets not rotated in over 90 days old_threshold = datetime.now() - timedelta(days=90) secrets = client.secrets.list() for secret in secrets.items: if secret.updated_at < old_threshold: print(f"WARNING: Secret {secret.name} has not been rotated in over 90 days") ``` ## Troubleshooting Common issues when working with secrets: 1. **Secret Not Found**: Check that the secret name matches exactly, including case 2. **Permission Errors**: Verify the developer ID has access to the secret 3. **Encryption Errors**: Ensure the master key is correctly set in the environment 4. **Reference Errors**: Ensure the secret reference syntax is correct in expressions and templates ## Next Steps * [Using Secrets in Julep](/guides/using-secrets) - Step-by-step guide for using secrets * [Integration Patterns](/guides/advanced/integration-patterns) - Learn how to use secrets with integrations * [API Reference](/api-reference#tag/secrets) - Complete API reference for secrets # System Templates Source: https://docs.julep.ai/advanced/system-templates Learn how to use Jinja2 templates to create dynamic system prompts for your AI agents System templates allow you to create dynamic, context-aware prompts for your AI agents using Jinja2 templating. This guide covers everything you need to know about creating and customizing system templates. ## Overview System templates in Julep are **Jinja2 templates** that define the initial system prompt for your AI agents. They allow you to: * Create dynamic prompts that adapt based on context * Include user information, session data, and other variables * Implement conditional logic for different scenarios * Maintain consistency across agent interactions * Inject custom metadata at the message level for dynamic behavior ## Template Hierarchy Julep uses a three-level hierarchy for system templates: 1. **Agent Default Template**: Defined when creating an agent 2. **Session Template**: Can override the agent's default template for specific sessions 3. **Chat Metadata**: Can inject dynamic variables at the individual message level The session template takes precedence over the agent's default template. Chat metadata is available in both cases. ## Default System Template When you create an agent without specifying a custom template, Julep uses this default template: ```jinja theme={"dark"} {%- if agent.name -%} You are {{agent.name}}.{{" "}} {%- endif -%} {%- if agent.about -%} About you: {{agent.about}}.{{" "}} {%- endif -%} {%- if user.name -%} You are talking to {{user.name}}.{{" "}} {%- endif -%} {%- if user.about -%} About the user: {{user.about}}.{{" "}} {%- endif -%} {{agent.instructions}} {%- if tools -%} You have access to these tools: {%- for tool in tools -%} {%- if tool.type == "function" -%} - {{tool.function.name}} {%- if tool.function.description -%}: {{tool.function.description}}{%- endif -%} {%- else -%} - {{ 0/0 }} {# Error: Other tool types aren't supported yet #} {%- endif -%} {%- endfor -%} {%- endif -%} {%- if docs -%} Relevant documents: {%- for doc in docs -%} {{doc.title}} {%- if doc.content -%}: {{doc.content}} {%- endif -%} {%- endfor -%} {%- endif -%} ``` ## Template Variables The following variables are available in your system templates: ### Core Variables | Variable | Type | Description | | -------------------- | ------------ | ------------------------------------- | | `agent` | Object | The agent's configuration and details | | `agent.name` | String | The agent's name | | `agent.about` | String | Description of the agent | | `agent.instructions` | Array/String | Agent's instructions | | `user` | Object | Information about the current user | | `user.name` | String | The user's name | | `user.about` | String | Description of the user | | `session` | Object | Current session information | | `session.situation` | String | The session's context/situation | ### Dynamic Variables | Variable | Type | Description | | ---------- | ------ | --------------------------------------- | | `tools` | Array | Available tools for the agent | | `docs` | Array | Relevant documents from searches | | `metadata` | Object | Custom metadata passed in chat requests | ## Creating Custom Templates ### Basic Example Here's a simple custom template for a customer service agent: ```python python theme={"dark"} agent = client.agents.create( name="Customer Support Agent", model="gpt-4", about="A helpful customer service representative", instructions=[ "Be polite and professional", "Always offer to help further" ], default_system_template=""" You are {{agent.name}}, a customer service representative. {%- if metadata.priority == "high" -%} ⚠️ HIGH PRIORITY CUSTOMER - Provide expedited service {%- endif -%} {%- if metadata.customer_tier == "premium" -%} Premium customer - Offer enhanced support options {%- endif -%} Your guidelines: {{agent.instructions}} {%- if user.name -%} Customer name: {{user.name}} {%- endif -%} """ ) ``` ```javascript javascript theme={"dark"} const agent = await client.agents.create({ name: "Customer Support Agent", model: "gpt-4", about: "A helpful customer service representative", instructions: [ "Be polite and professional", "Always offer to help further" ], default_system_template: ` You are {{agent.name}}, a customer service representative. {%- if metadata.priority == "high" -%} ⚠️ HIGH PRIORITY CUSTOMER - Provide expedited service {%- endif -%} {%- if metadata.customer_tier == "premium" -%} Premium customer - Offer enhanced support options {%- endif -%} Your guidelines: {{agent.instructions}} {%- if user.name -%} Customer name: {{user.name}} {%- endif -%} ` }); ``` ### Advanced Example with Conditional Logic ```python python theme={"dark"} template = """ {%- if agent.name -%} You are {{agent.name}}. {%- endif -%} {# Mood-based personality adjustment #} {%- if metadata.mood == "friendly" -%} Be warm, conversational, and use a friendly tone. {%- elif metadata.mood == "professional" -%} Maintain a formal, business-appropriate tone. {%- elif metadata.mood == "playful" -%} Feel free to be creative and add appropriate humor. {%- else -%} Respond in a balanced, helpful manner. {%- endif -%} {# Language preferences #} {%- if metadata.language -%} Respond in {{metadata.language}}. {%- endif -%} {# Expertise areas #} {%- if metadata.expertise -%} You are an expert in: {%- for skill in metadata.expertise -%} - {{skill}} {%- endfor -%} {%- endif -%} {# Include instructions #} {{agent.instructions}} {# Tool availability #} {%- if tools -%} Available tools: {%- for tool in tools -%} - {{tool.function.name}}: {{tool.function.description}} {%- endfor -%} {%- endif -%} """ # Using the template with metadata response = client.sessions.chat( session_id=session.id, messages=[ {"role": "user", "content": "Help me with Python"} ], metadata={ "mood": "friendly", "language": "English", "expertise": ["Python", "Machine Learning", "Data Science"] } ) ``` ```javascript javascript theme={"dark"} const template = ` {%- if agent.name -%} You are {{agent.name}}. {%- endif -%} {# Mood-based personality adjustment #} {%- if metadata.mood == "friendly" -%} Be warm, conversational, and use a friendly tone. {%- elif metadata.mood == "professional" -%} Maintain a formal, business-appropriate tone. {%- elif metadata.mood == "playful" -%} Feel free to be creative and add appropriate humor. {%- else -%} Respond in a balanced, helpful manner. {%- endif -%} {# Language preferences #} {%- if metadata.language -%} Respond in {{metadata.language}}. {%- endif -%} {# Expertise areas #} {%- if metadata.expertise -%} You are an expert in: {%- for skill in metadata.expertise -%} - {{skill}} {%- endfor -%} {%- endif -%} {# Include instructions #} {{agent.instructions}} {# Tool availability #} {%- if tools -%} Available tools: {%- for tool in tools -%} - {{tool.function.name}}: {{tool.function.description}} {%- endfor -%} {%- endif -%} `; // Using the template with metadata const response = await client.sessions.chat({ sessionId: session.id, messages: [ { role: "user", content: "Help me with Python" } ], metadata: { mood: "friendly", language: "English", expertise: ["Python", "Machine Learning", "Data Science"] } }); ``` ## Using Chat Metadata The `metadata` field in chat requests allows you to pass dynamic variables that can be used in your system templates. This enables message-level customization without modifying the agent or session. ### Example: Dynamic Instructions ```python python theme={"dark"} # Create an agent with a metadata-aware template agent = client.agents.create( name="Adaptive Assistant", default_system_template=""" You are {{agent.name}}. {%- if metadata.instructions -%} Special instructions for this conversation: {{metadata.instructions}} {%- endif -%} {%- if metadata.constraints -%} Important constraints: {%- for constraint in metadata.constraints -%} - {{constraint}} {%- endfor -%} {%- endif -%} """ ) # Use metadata to modify behavior per message response = client.sessions.chat( session_id=session.id, messages=[{"role": "user", "content": "Write a story"}], metadata={ "instructions": "Write in the style of Edgar Allan Poe", "constraints": [ "Keep it under 200 words", "Include a raven", "End with a twist" ] } ) ``` ```javascript javascript theme={"dark"} // Create an agent with a metadata-aware template const agent = await client.agents.create({ name: "Adaptive Assistant", default_system_template: ` You are {{agent.name}}. {%- if metadata.instructions -%} Special instructions for this conversation: {{metadata.instructions}} {%- endif -%} {%- if metadata.constraints -%} Important constraints: {%- for constraint in metadata.constraints -%} - {{constraint}} {%- endfor -%} {%- endif -%} ` }); // Use metadata to modify behavior per message const response = await client.sessions.chat({ sessionId: session.id, messages: [{ role: "user", content: "Write a story" }], metadata: { instructions: "Write in the style of Edgar Allan Poe", constraints: [ "Keep it under 200 words", "Include a raven", "End with a twist" ] } }); ``` ## Session vs Agent Templates You can override an agent's default template at the session level: ```python python theme={"dark"} # Agent with default template agent = client.agents.create( name="Multi-purpose Assistant", default_system_template="You are a helpful assistant." ) # Session with custom template session = client.sessions.create( agent_id=agent.id, system_template=""" You are a technical writing assistant. Focus on clarity, accuracy, and proper documentation structure. {%- if metadata.doc_type -%} You are writing a {{metadata.doc_type}}. {%- endif -%} """ ) ``` ```javascript javascript theme={"dark"} // Agent with default template const agent = await client.agents.create({ name: "Multi-purpose Assistant", default_system_template: "You are a helpful assistant." }); // Session with custom template const session = await client.sessions.create({ agentId: agent.id, system_template: ` You are a technical writing assistant. Focus on clarity, accuracy, and proper documentation structure. {%- if metadata.doc_type -%} You are writing a {{metadata.doc_type}}. {%- endif -%} ` }); ``` Use session templates when you need different behavior for the same agent in different contexts (e.g., customer service vs internal support). ## Jinja2 Features System templates support the full range of Jinja2 features: ### Conditionals ```jinja theme={"dark"} {%- if user.subscription_level == "premium" -%} You have access to advanced features. {%- elif user.subscription_level == "basic" -%} Some features may be limited. {%- else -%} Welcome! Consider upgrading for more features. {%- endif -%} ``` ### Loops ```jinja theme={"dark"} {%- if metadata.topics -%} Focus on these topics: {%- for topic in metadata.topics -%} - {{topic}} {%- endfor -%} {%- endif -%} ``` ### Filters ```jinja theme={"dark"} Agent name in uppercase: {{agent.name|upper}} Word count limit: {{metadata.word_limit|default(500)}} ``` ### Comments ```jinja theme={"dark"} {# This is a comment and won't appear in the output #} ``` ## Best Practices Avoid overly complex templates. If your template is becoming too large, consider breaking the logic into different agents or sessions. When using metadata, choose clear, descriptive keys: * ✅ `metadata.customer_tier` * ❌ `metadata.ct` Always handle cases where variables might be missing: ```jinja theme={"dark"} {{metadata.language|default("English")}} ``` Test your templates with various metadata combinations to ensure they render correctly in all scenarios. ## Common Patterns ### Multi-language Support ```jinja theme={"dark"} {%- if metadata.language == "es" -%} Eres un asistente útil. Responde en español. {%- elif metadata.language == "fr" -%} Vous êtes un assistant utile. Répondez en français. {%- else -%} You are a helpful assistant. Respond in English. {%- endif -%} ``` ### Role-based Access ```jinja theme={"dark"} {%- if metadata.user_role == "admin" -%} You have full access to all operations and sensitive information. {%- elif metadata.user_role == "user" -%} Provide general assistance. Do not share sensitive information. {%- endif -%} ``` ### Context-aware Behavior ```jinja theme={"dark"} {%- if metadata.context == "code_review" -%} Focus on code quality, security issues, and best practices. {%- elif metadata.context == "debugging" -%} Help identify and fix issues. Ask clarifying questions. {%- elif metadata.context == "learning" -%} Explain concepts clearly with examples. Be patient and thorough. {%- endif -%} ``` ## Debugging Templates To debug template rendering issues: 1. **Check Variable Availability**: Ensure all referenced variables exist 2. **Validate Jinja2 Syntax**: Use a Jinja2 linter or validator 3. **Test Incrementally**: Add template features one at a time 4. **Use the Render Endpoint**: Test template rendering without making chat requests ```python python theme={"dark"} # Test template rendering render_result = client.sessions.render( session_id=session.id, messages=[{"role": "user", "content": "Test"}], metadata={"test_var": "test_value"} ) # Inspect the rendered system message print(render_result.messages[0]["content"]) ``` ```javascript javascript theme={"dark"} // Test template rendering const renderResult = await client.sessions.render({ sessionId: session.id, messages: [{ role: "user", content: "Test" }], metadata: { test_var: "test_value" } }); // Inspect the rendered system message console.log(renderResult.messages[0].content); ``` ## Related Topics * [Agents](/concepts/agents) - Learn more about agent configuration * [Sessions](/concepts/sessions) - Understand session management * [Tasks](/concepts/tasks) - Create complex workflows with templates * [Tool Integration](/integrations/tools) - Add tools referenced in templates System templates are rendered server-side before being sent to the language model. This ensures security and consistency across all API clients. # Types of Task Steps Source: https://docs.julep.ai/advanced/types-of-task-steps Learn about different types of task steps and their use ## Overview In Julep broadly speaking there are two types of steps:

These steps control the flow of the task. They are used to create conditional logic, loops, and parallel execution.

These steps are used to get and set values in the task.

These steps are used to iterate over a collection.

These steps are used to create conditional logic.

These steps are used to control the flow of the task.

 The steps defined out here are in the YAML format. You can learn more about the YAML format [here](https://yaml.org/spec/1.2.2/). ## Control Flow Steps ### Prompt Step Send messages to the AI model: ```yaml YAML theme={"dark"} # Simple prompt - prompt: What is your name? # Multi-message prompt - prompt: - role: system content: You are a helpful assistant - role: user content: "Hello!" # Prompt with settings - prompt: - role: user content: Generate a creative story settings: model: "claude-3.5-sonnet" temperature: 0.8 # Prompt with automatic tool execution - prompt: What's the weather in San Francisco? auto_run_tools: true # Tools execute automatically if needed # Prompt with JSON response format - prompt: - role: system content: You are a helpful assistant that always responds in JSON format. - role: user content: List 3 interesting facts about space. Format as JSON with 'facts' array. settings: model: "gpt-4o-mini" temperature: 0.7 response_format: type: json_object # Prompt with JSON schema for structured output - prompt: - role: system content: You are a data extraction specialist. - role: user content: Extract product information from the description. settings: response_format: type: json_schema json_schema: name: product_info schema: type: object properties: product_name: type: string features: type: array items: type: string price: type: number required: ["product_name", "features", "price"] ``` When `auto_run_tools` is set to `true` in a prompt step, any tools available to the agent will be automatically executed if the model decides to use them. The results are then fed back to the model to continue processing. This is particularly useful for creating autonomous workflows where the agent can gather information and make decisions without manual intervention. The `response_format` setting allows you to request structured output from the model. There are two main options: * `type: json_object` - Ensures the model responds with valid JSON * `type: json_schema` - Enforces a specific JSON structure defined by a schema When using `response_format`, make sure to instruct the model to produce JSON in your prompt (via system or user message) for best results. Response format support varies by model provider - check the [supported models documentation](/integrations/supported-models) for compatibility. In the prompt step we offer a bunch of Python functions to help you manipulate data. Here is a list of the functions you can use: * Standard library modules: * `re`: Regular expressions (safe against ReDoS) * `json`: JSON encoding/decoding * `yaml`: YAML parsing/dumping * `string`: String constants and operations * `datetime`: Date and time operations * `math`: Mathematical functions * `statistics`: Statistical operations * `base64`: Base64 encoding/decoding * `urllib`: URL parsing operations * `random`: Random number generation * `time`: Time operations * Constants: * `NEWLINE`: Newline character * `true`: Boolean true * `false`: Boolean false * `null`: None value ### Tool Call Step Execute tools defined in the task: ```yaml YAML theme={"dark"} # Simple tool call - tool: web_search arguments: query: Latest AI news # Tool call with complex arguments - tool: process_data arguments: input_data: $ _.previous_result options: format: "json" validate: true ``` ### Evaluate Step Perform calculations or data manipulation: ```yaml YAML theme={"dark"} # Simple evaluation - evaluate: count: $ len(_.results) # Multiple evaluations - evaluate: total: $ sum(_.numbers) average: $ _.total / len(_.numbers) formatted: $ f'Average: {_.average:.2f}' ``` In the evaluate step we offer a bunch of Python functions to help you manipulate data. Check out the [Python Expressions](/advanced/python-expression#available-functions-and-libraries) for more information. ### Wait for Input Step Pause workflow for user input: ```yaml YAML theme={"dark"} # Simple input request - wait_for_input: info: message: "Please provide your name" # Input with validation - wait_for_input: info: message: "Enter your age" validation: type: "number" minimum: 0 maximum: 150 ``` ### Subworkflow Step Executing a subworkflow from a main workflow: ```yaml YAML [expandable] theme={"dark"} # Subworkflow subworkflow: - evaluate: main_workflow_input: $ _.content # you can use steps[0].input.content to access the input of the subworkflow - return: result: "This is the subworkflow" # Main workflow main: # Step 0: Evaluate step - evaluate: result: "This is the main workflow" # Step 1: Call the subworkflow - workflow: subworkflow # name of the subworkflow arguments: # input to the subworkflow content: $ _.result # you can use steps[0].output.result to access the result of the previous step # Step 2: Evaluate step - evaluate: subworkflow_result: $ steps[1].output.result # this will be the result of the subworkflow ``` * The `arguments` passed from the main workflow to the subworkflow are available in the `steps[0].input` of the subworkflow. * The `result` of the subworkflow is available in the `steps[1].output.result` of the main workflow. * The `Input/Output Data References` between steps is exclusive to the workflow they are defined in. A workflow cannot reference the `input` or `output` of another workflow between the steps. To learn more about `Input/Output Data References` [click here](/advanced/new-syntax#1-input-output-data-references). Self recursion is allowed in a subworkflow but not in a main workflow. ## Key-Value Steps ### Get Step Retrieve values from storage: ```yaml YAML theme={"dark"} # Get a single value - get: user_preference # Get multiple values - get: - preference1 - preference2 ``` ### Set Step Store values for later use: ```yaml YAML theme={"dark"} # Set a single value - set: user_name: John # Set multiple values - set: count: $ len(_.results) has_data: $ _.count > 0 ``` Values stored using the set step are added to the workflow's global `state` object, which can be accessed anywhere in the workflow using `state.variable_name`. For example: ```yaml YAML theme={"dark"} # Access previously set values - evaluate: greeting: $ f"Hello, {state.user_name}!" data_status: $ f"Has data: {state.has_data}, Count: {state.count}" ``` Each subworkflow has its own isolated `state` object. Values set in one subworkflow are not accessible from other subworkflows or the parent workflow. ### Label Step Label a step to make it easier to identify and access those values later in any step: ```yaml YAML theme={"dark"} # Step 0: Set a single value - set: user_name: John label: get_user_name # Step 1: Set multiple values - set: count: $ len(_.results) has_data: $ _.count > 0 label: get_count_and_has_data ``` In any steps following the label step, you can access the values set in the label step using the `$ steps['label_name'].input.attribute_name` or `$ steps['label_name'].output.attribute_name` syntax. For example: ```yaml YAML theme={"dark"} - evaluate: user_name: $ steps['get_user_name'].output.user_name - evaluate: count: $ steps['get_count_and_has_data'].output.count has_data: $ steps['get_count_and_has_data'].output.has_data ``` ## Iteration Steps ### Foreach Step Iterate over a collection: ```yaml YAML theme={"dark"} # Simple foreach - foreach: in: $ _.items do: log: $ f'Processing {_}' # Foreach with complex processing - foreach: in: $ _.documents do: tool: analyze arguments: text: $ _.content evaluate: results: $ _ + [_.analysis] ``` ### Map-Reduce Step Process collections in parallel: ```yaml YAML theme={"dark"} # Simple map-reduce - over: $ _.urls map: tool: fetch_content arguments: url: $ _ reduce: $ results + [_] # Map-reduce with parallelism - over: $ _.queries map: tool: web_search arguments: query: $ _ parallelism: 5 # Number of parallel steps to execute ``` * By default the `parallelism` if not mentioned is 100. If mentioned, it is the maximum number of steps that can run in parallel concurrently. * When using `over` step, the `map` step is executed for each value in the collection. * The `reduce` step is executed after the `map` step. ## Conditional Steps ### If-Else Step Conditional execution: ```yaml YAML [expandable] theme={"dark"} # Simple if - if: $ _.count > 0 then: log: Found results # If-else - if: $ _.score > 0.8 then: log: High score else: log: Low score # If-else with multiple conditions - if: $ 1 > 0 then: if: $ 2 > 1 then: evaluate: x: y else: evaluate: x: z ``` ### Switch Step Multiple condition handling: ```yaml YAML theme={"dark"} # Switch statement - switch: - case: $ _.category == "A" then: - log: Category A - case: $ _.category == "B" then: - log: Category B - case: $ _ # Default case then: - log: Unknown category ``` ## Other Control Flow ### Sleep Step Pause execution: ```yaml YAML theme={"dark"} # Sleep for duration - sleep: seconds: 30 # Sleep with different units - sleep: minutes: 5 # hours: 1 # days: 1 ``` ### Return Step Return values from workflow: ```yaml YAML theme={"dark"} # Simple return - return: $ _.result # Structured return - return: data: $ _.processed_data metadata: count: $ _.count timestamp: $ datetime.now().isoformat() ``` ### Log Step Log messages or specific values: ```yaml YAML theme={"dark"} - log: $ f'Processing completed for item {item_id}' ``` ### Error Step Handle errors by specifying an error message: ```yaml YAML theme={"dark"} - error: Invalid input provided ``` ## Example: Complex Workflow Here's an example combining various step types: ```yaml YAML [expandable] theme={"dark"} # yaml-language-server: $schema=https://raw.githubusercontent.com/julep-ai/julep/refs/heads/dev/src/schemas/create_task_request.json name: Multi-Step Task Demonstration description: A demonstration of multi-step task processing with research and summarization capabilities. ################################################################################ ############################# INPUT SCHEMA ##################################### ################################################################################ input_schema: type: object properties: topic: type: string description: The topic to research and summarize. ################################################################################ ############################# TOOLS ############################################ ################################################################################ # Describing the tools that will be used in the workflow tools: - name: web_search type: integration integration: provider: brave setup: brave_api_key: "YOUR_BRAVE_API_KEY" ################################################################################ ############################# MAIN WORKFLOW #################################### ################################################################################ main: # Step 0: Generate initial research questions - prompt: - role: system content: >- $ f''' You are a research assistant. Your task is to formulate three specific research questions about the given topic: {steps[0].input.topic}''' unwrap: true # Step 1: Web search for each question - foreach: in: $ _.split('\\n') do: tool: web_search arguments: query: $ _ # Step 2: Extract relevant information - evaluate: relevant_info: $ [output for output in _] # Step 3: Process and summarize information - if: $ len(_.relevant_info) >= 3 then: prompt: - role: system content: >- $ f''' Summarize the following information about {steps[0].input.topic}: {_.relevant_info}''' unwrap: true else: prompt: - role: system content: >- $ f''' Not enough information gathered. Please provide a brief overview of {steps[0].input.topic} based on your knowledge.''' unwrap: true # Step 4: Record the summary - log: >- $ f''' Summary for {steps[0].input.topic}: {_}''' # Step 5: Prepare final output - return: summary: $ _ topic: $ steps[0].input.topic ``` ## Best Practices
  • Group related steps logically
  • Use comments to explain complex steps
  • Keep step chains focused and manageable
  • Use if-else for error conditions
  • Provide fallback options
  • Log important state changes
  • Use parallel execution when possible
  • Optimize data passing between steps
  • Cache frequently used values
 ## Support If you need help with further questions in Julep: * Join our [Discord community](https://discord.com/invite/JTSBGRZrzj) * Check the [GitHub repository](https://github.com/julep-ai/julep) * Contact support at [hey@julep.ai](mailto:hey@julep.ai) # Create Agent Source: https://docs.julep.ai/api-reference/agents/create-agent https://api.julep.ai/api/openapi.json post /agents # Create Agent Tool Source: https://docs.julep.ai/api-reference/agents/create-agent-tool https://api.julep.ai/api/openapi.json post /agents/{agent_id}/tools # Create Or Update Agent Source: https://docs.julep.ai/api-reference/agents/create-or-update-agent https://api.julep.ai/api/openapi.json post /agents/{agent_id} # Delete Agent Source: https://docs.julep.ai/api-reference/agents/delete-agent https://api.julep.ai/api/openapi.json delete /agents/{agent_id} # Delete Agent Tool Source: https://docs.julep.ai/api-reference/agents/delete-agent-tool https://api.julep.ai/api/openapi.json delete /agents/{agent_id}/tools/{tool_id} # Get Agent Details Source: https://docs.julep.ai/api-reference/agents/get-agent-details https://api.julep.ai/api/openapi.json get /agents/{agent_id} # List Agent Tools Source: https://docs.julep.ai/api-reference/agents/list-agent-tools https://api.julep.ai/api/openapi.json get /agents/{agent_id}/tools # List Agents Source: https://docs.julep.ai/api-reference/agents/list-agents https://api.julep.ai/api/openapi.json get /agents # List Models Source: https://docs.julep.ai/api-reference/agents/list-models https://api.julep.ai/api/openapi.json get /agents/models List all available models that can be used with agents. Returns: ListModelsResponse: A list of available models # Patch Agent Source: https://docs.julep.ai/api-reference/agents/patch-agent https://api.julep.ai/api/openapi.json patch /agents/{agent_id} # Patch Agent Tool Source: https://docs.julep.ai/api-reference/agents/patch-agent-tool https://api.julep.ai/api/openapi.json patch /agents/{agent_id}/tools/{tool_id} # Update Agent Source: https://docs.julep.ai/api-reference/agents/update-agent https://api.julep.ai/api/openapi.json put /agents/{agent_id} # Update Agent Tool Source: https://docs.julep.ai/api-reference/agents/update-agent-tool https://api.julep.ai/api/openapi.json put /agents/{agent_id}/tools/{tool_id} # Bulk Delete Agent Docs Source: https://docs.julep.ai/api-reference/docs/bulk-delete-agent-docs https://api.julep.ai/api/openapi.json delete /agents/{agent_id}/docs Bulk delete documents owned by an agent based on metadata filter # Bulk Delete User Docs Source: https://docs.julep.ai/api-reference/docs/bulk-delete-user-docs https://api.julep.ai/api/openapi.json delete /users/{user_id}/docs Bulk delete documents owned by a user based on metadata filter # Create Agent Doc Source: https://docs.julep.ai/api-reference/docs/create-agent-doc https://api.julep.ai/api/openapi.json post /agents/{agent_id}/docs # Create User Doc Source: https://docs.julep.ai/api-reference/docs/create-user-doc https://api.julep.ai/api/openapi.json post /users/{user_id}/docs Creates a new document for a user. Parameters: user_id (UUID): The unique identifier of the user associated with the document. data (CreateDocRequest): The data to create the document with. x_developer_id (UUID): The unique identifier of the developer associated with the document. Returns: Doc: The created document. # Delete Agent Doc Source: https://docs.julep.ai/api-reference/docs/delete-agent-doc https://api.julep.ai/api/openapi.json delete /agents/{agent_id}/docs/{doc_id} # Delete User Doc Source: https://docs.julep.ai/api-reference/docs/delete-user-doc https://api.julep.ai/api/openapi.json delete /users/{user_id}/docs/{doc_id} # Embed Source: https://docs.julep.ai/api-reference/docs/embed https://api.julep.ai/api/openapi.json post /embed # Get Doc Source: https://docs.julep.ai/api-reference/docs/get-doc https://api.julep.ai/api/openapi.json get /docs/{doc_id} # List Agent Docs Source: https://docs.julep.ai/api-reference/docs/list-agent-docs https://api.julep.ai/api/openapi.json get /agents/{agent_id}/docs # List User Docs Source: https://docs.julep.ai/api-reference/docs/list-user-docs https://api.julep.ai/api/openapi.json get /users/{user_id}/docs # Search Agent Docs Source: https://docs.julep.ai/api-reference/docs/search-agent-docs https://api.julep.ai/api/openapi.json post /agents/{agent_id}/search Searches for documents associated with a specific agent. Parameters: x_developer_id (UUID): The unique identifier of the developer associated with the agent. search_params (TextOnlyDocSearchRequest | VectorDocSearchRequest | HybridDocSearchRequest): The parameters for the search. agent_id (UUID): The umnique identifier of the agent associated with the documents. Returns: DocSearchResponse: The search results. # Search User Docs Source: https://docs.julep.ai/api-reference/docs/search-user-docs https://api.julep.ai/api/openapi.json post /users/{user_id}/search Searches for documents associated with a specific user. Parameters: x_developer_id (UUID): The unique identifier of the developer associated with the user. search_params (TextOnlyDocSearchRequest | VectorDocSearchRequest | HybridDocSearchRequest): The parameters for the search. user_id (UUID): The unique identifier of the user associated with the documents. Returns: DocSearchResponse: The search results. # Create Task Execution Source: https://docs.julep.ai/api-reference/executions/create-task-execution https://api.julep.ai/api/openapi.json post /tasks/{task_id}/executions # Get Execution Details Source: https://docs.julep.ai/api-reference/executions/get-execution-details https://api.julep.ai/api/openapi.json get /executions/{execution_id} # List Execution Transitions Source: https://docs.julep.ai/api-reference/executions/list-execution-transitions https://api.julep.ai/api/openapi.json get /executions/{execution_id}/transitions # Stream Execution Status Source: https://docs.julep.ai/api-reference/executions/stream-execution-status https://api.julep.ai/api/openapi.json get /executions/{execution_id}/status.stream SSE endpoint that streams the status of a given execution_id by polling the latest_executions view. # Stream Transitions Events Source: https://docs.julep.ai/api-reference/executions/stream-transitions-events https://api.julep.ai/api/openapi.json get /executions/{execution_id}/transitions.stream # Update Execution Source: https://docs.julep.ai/api-reference/executions/update-execution https://api.julep.ai/api/openapi.json put /executions/{execution_id} # Create File Source: https://docs.julep.ai/api-reference/files/create-file https://api.julep.ai/api/openapi.json post /files # Delete File Source: https://docs.julep.ai/api-reference/files/delete-file https://api.julep.ai/api/openapi.json delete /files/{file_id} # Get File Source: https://docs.julep.ai/api-reference/files/get-file https://api.julep.ai/api/openapi.json get /files/{file_id} # List Files Source: https://docs.julep.ai/api-reference/files/list-files https://api.julep.ai/api/openapi.json get /files # Check Health Source: https://docs.julep.ai/api-reference/healthz/check-health https://api.julep.ai/api/openapi.json get /healthz # Get Job Status Source: https://docs.julep.ai/api-reference/jobs/get-job-status https://api.julep.ai/api/openapi.json get /jobs/{job_id} # Create Project Source: https://docs.julep.ai/api-reference/projects/create-project https://api.julep.ai/api/openapi.json post /projects # List Projects Source: https://docs.julep.ai/api-reference/projects/list-projects https://api.julep.ai/api/openapi.json get /projects # Create Developer Secret Source: https://docs.julep.ai/api-reference/secrets/create-developer-secret https://api.julep.ai/api/openapi.json post /secrets Create a new secret for a developer. Args: developer_id: ID of the developer creating the secret secret: Secret to create Returns: The created secret Raises: HTTPException: If a secret with this name already exists (409 Conflict) # Delete Developer Secret Source: https://docs.julep.ai/api-reference/secrets/delete-developer-secret https://api.julep.ai/api/openapi.json delete /secrets/{secret_id} Delete a secret. Args: secret_id: ID of the secret to delete x_developer_id: ID of the developer who owns the secret Returns: The deleted secret Raises: HTTPException: If the secret doesn't exist # List Developer Secrets Source: https://docs.julep.ai/api-reference/secrets/list-developer-secrets https://api.julep.ai/api/openapi.json get /secrets List all secrets for a developer. Args: x_developer_id: ID of the developer whose secrets to list limit: Maximum number of secrets to return offset: Number of secrets to skip Returns: List of secrets # Update Developer Secret Source: https://docs.julep.ai/api-reference/secrets/update-developer-secret https://api.julep.ai/api/openapi.json put /secrets/{secret_id} Update a developer secret. Args: developer_id: ID of the developer who owns the secret secret_id: ID of the secret to update data: New secret data Returns: The updated secret Raises: HTTPException: If the secret doesn't exist or doesn't belong to the developer # Chat Source: https://docs.julep.ai/api-reference/sessions/chat https://api.julep.ai/api/openapi.json post /sessions/{session_id}/chat Initiates a chat session. Routes to different implementations based on feature flags: - If auto_run_tools_chat feature flag is enabled, uses the new auto-tools implementation - Otherwise, uses the legacy implementation Parameters: developer (Developer): The developer associated with the chat session. session_id (UUID): The unique identifier of the chat session. chat_input (ChatInput): The chat input data. background_tasks (BackgroundTasks): The background tasks to run. x_custom_api_key (Optional[str]): The custom API key. mock_response (Optional[str]): Mock response for testing. connection_pool: Connection pool for testing purposes. Returns: ChatResponse or StreamingResponse: The chat response or streaming response. # Create Or Update Session Source: https://docs.julep.ai/api-reference/sessions/create-or-update-session https://api.julep.ai/api/openapi.json post /sessions/{session_id} # Create Session Source: https://docs.julep.ai/api-reference/sessions/create-session https://api.julep.ai/api/openapi.json post /sessions # Delete Session Source: https://docs.julep.ai/api-reference/sessions/delete-session https://api.julep.ai/api/openapi.json delete /sessions/{session_id} # Get Session Source: https://docs.julep.ai/api-reference/sessions/get-session https://api.julep.ai/api/openapi.json get /sessions/{session_id} # Get Session History Source: https://docs.julep.ai/api-reference/sessions/get-session-history https://api.julep.ai/api/openapi.json get /sessions/{session_id}/history # List Sessions Source: https://docs.julep.ai/api-reference/sessions/list-sessions https://api.julep.ai/api/openapi.json get /sessions # Patch Session Source: https://docs.julep.ai/api-reference/sessions/patch-session https://api.julep.ai/api/openapi.json patch /sessions/{session_id} # Render Source: https://docs.julep.ai/api-reference/sessions/render https://api.julep.ai/api/openapi.json post /sessions/{session_id}/render Renders a chat input. Routes to different implementations based on feature flags: - If auto_run_tools_chat feature flag is enabled, uses the new auto-tools implementation - Otherwise, uses the legacy implementation Parameters: developer (Developer): The developer associated with the chat session. session_id (UUID): The unique identifier of the chat session. chat_input (ChatInput): The chat input data. Returns: RenderResponse: The rendered chat input. # Update Session Source: https://docs.julep.ai/api-reference/sessions/update-session https://api.julep.ai/api/openapi.json put /sessions/{session_id} # Create Or Update Task Source: https://docs.julep.ai/api-reference/tasks/create-or-update-task https://api.julep.ai/api/openapi.json post /agents/{agent_id}/tasks/{task_id} # Create Task Source: https://docs.julep.ai/api-reference/tasks/create-task https://api.julep.ai/api/openapi.json post /agents/{agent_id}/tasks # Get Execution Transition Source: https://docs.julep.ai/api-reference/tasks/get-execution-transition https://api.julep.ai/api/openapi.json get /executions/{execution_id}/transitions/{transition_id} # Get Task Details Source: https://docs.julep.ai/api-reference/tasks/get-task-details https://api.julep.ai/api/openapi.json get /tasks/{task_id} # List Task Executions Source: https://docs.julep.ai/api-reference/tasks/list-task-executions https://api.julep.ai/api/openapi.json get /tasks/{task_id}/executions # List Tasks Source: https://docs.julep.ai/api-reference/tasks/list-tasks https://api.julep.ai/api/openapi.json get /agents/{agent_id}/tasks # Decode Payloads Source: https://docs.julep.ai/api-reference/temporal/decode-payloads https://api.julep.ai/api/openapi.json post /temporal/decode Decodes a list of payloads from the request body. Args: req (Request): The request containing the payloads to decode. Returns: dict: A dictionary containing the decoded payloads, including any errors encountered. # Create Or Update User Source: https://docs.julep.ai/api-reference/users/create-or-update-user https://api.julep.ai/api/openapi.json post /users/{user_id} # Create User Source: https://docs.julep.ai/api-reference/users/create-user https://api.julep.ai/api/openapi.json post /users # Delete User Source: https://docs.julep.ai/api-reference/users/delete-user https://api.julep.ai/api/openapi.json delete /users/{user_id} # Get User Details Source: https://docs.julep.ai/api-reference/users/get-user-details https://api.julep.ai/api/openapi.json get /users/{user_id} # List Users Source: https://docs.julep.ai/api-reference/users/list-users https://api.julep.ai/api/openapi.json get /users # Patch User Source: https://docs.julep.ai/api-reference/users/patch-user https://api.julep.ai/api/openapi.json patch /users/{user_id} # Update User Source: https://docs.julep.ai/api-reference/users/update-user https://api.julep.ai/api/openapi.json put /users/{user_id} # Agents Source: https://docs.julep.ai/concepts/agents Understanding Julep Agents and their capabilities ## Overview Agents are conceptual entities that encapsulate all the configurations and settings of an LLM, enabling it to adopt unique personas and execute distinct tasks within an application. ## Components Agents are made up of several components. Think of components as the building blocks of an agent required to perform a task. Here are the key components associated with an agent: * Instructions - Agent configuration that can be provided as either a single string or an array of strings. * Metadata - Key-value pair that can be used to categorize and filter agents. * Tools - Functions that can be used by an agent to perform tasks. Julep supports a wide range of tools, including custom tools, which are functions that can be used by an agent to perform tasks. * Docs - A collection of documents that can be used by an agent to retrieve information. Docs can be associated with an agent and can be used to retrieve or search information from the agent's context. ### Agent Configuration Options When creating an agent, you can leverage the following configuration options: | Option | Type | Description | Default | | ------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | | `name` | `string` | The name of your agent | Required | | `canonical_name` | `string` | A unique identifier for your agent, following the pattern `[a-zA-Z][a-zA-Z0-9_]*` | `null` | | `project` | `string` | The canonical name of the project this agent belongs to | `"default"` | | `about` | `string` | A brief description of what your agent does | `""` | | `model` | `string` | The language model your agent uses (e.g., "gpt-4-turbo", "gemini-nano") | `""` | | `instructions` | `string` \| `list[string]` | Specific tasks or behaviors expected from the agent | `[]` | | `metadata` | `object` | Key-value pairs for additional information about your agent | `null` | | `default_settings` | `object` | Default configuration settings for the agent. See [supported parameters](/integrations/supported-models#supported-parameters) for details. | `null` | | `default_system_template` | `string` | Default system template for all sessions created by this agent. | See [default system template](/concepts/agents#default-system-template) | The **System Template** is a specific system prompt written as a Jinja template that sets the foundational context and instructions for the agent within a session. It defines the background, directives, and any relevant information that the agent should consider when interacting with the user. For a comprehensive guide on system templates including available variables, customization options, and advanced usage patterns, see the [System Templates](/advanced/system-templates) documentation. For more details on Jinja templates, refer to the [Jinja documentation](https://jinja.palletsprojects.com/). ```python Python [expandable] theme={"dark"} {%- if agent.name -%} You are {{ agent.name }}. {%- endif -%} {%- if agent.about -%} About you: {{ agent.about }}. {%- endif -%} {%- if user -%} You are talking to a user {%- if user.name -%} and their name is {{ user.name }} {%- if user.about -%} . About the user: {{ user.about }}. {%- else -%} . {%- endif -%} {%- endif -%} {%- endif -%} {{ NEWLINE }} {%- if session.situation -%} Situation: {{ session.situation }} {%- endif -%} {{ NEWLINE + NEWLINE }} {%- if agent.instructions -%} Instructions: {%- if agent.instructions is string -%} {{ agent.instructions }} {%- else -%} {%- for instruction in agent.instructions -%} - {{ instruction }} {%- endfor -%} {%- endif -%} {{ NEWLINE }} {%- endif -%} {%- if docs -%} Relevant documents: {%- for doc in docs -%} {{ doc.title }} {%- if doc.content is string -%} {{ doc.content }} {%- else -%} {%- for snippet in doc.content -%} {{ snippet }} {%- endfor -%} {%- endif -%} --- {%- endfor -%} {%- endif -%} ``` ## How to Use Agents In Julep, how you use agents is very important. The YAML below shows the anatomy of an agent. ```yaml YAML theme={"dark"} name: "My Agent" model: "claude-3.5-sonnet" project: "data-analytics" about: "A helpful AI assistant that specializes in data analysis" instructions: "You are a helpful AI assistant that specializes in data analysis" metadata: type: "data-analysis" tools: - name: "calculate_total" description: "Calculate the total of a list of numbers" function: parameters: type: "object" properties: numbers: type: "array" items: type: "number" ``` ### Creating an Agent To create an agent, you can use the `create` method in the Python or Node.js SDK. ```python Python theme={"dark"} from julep import Julep client = Julep(api_key="your_api_key") agent = client.agents.create( name="My Agent", model="claude-3.5-sonnet", about="A helpful AI assistant that specializes in data analysis", instructions="You are a helpful AI assistant that specializes in data analysis", metadata={"type": "data-analysis"}, tools=[ { "name": "calculate_total", "description": "Calculate the total of a list of numbers", "function": { "parameters": {"type": "object", "properties": {"numbers": {"type": "array", "items": {"type": "number"}}}} } } ] ) ``` ```javascript Node.js theme={"dark"} import { Julep } from '@julep/sdk'; const client = new Julep({ apiKey: 'your_api_key' }); const agent = await client.agents.create({ name: "My Agent", model: "claude-3.5-sonnet", about: "A helpful AI assistant that specializes in data analysis", instructions: "You are a helpful AI assistant that specializes in data analysis", metadata: {"type": "data-analysis"}, tools: [ { "name": "calculate_total", "description": "Calculate the total of a list of numbers", "function": { "parameters": {"type": "object", "properties": {"numbers": {"type": "array", "items": {"type": "number"}}}} } } ] }); ``` ### Using default\_settings with response\_format You can configure your agent to always return structured JSON responses by setting `response_format` in the `default_settings`: ```python Python theme={"dark"} agent = client.agents.create( name="Data Analyst", model="gpt-4o-mini", about="An agent that analyzes data and returns structured insights", instructions="Always provide analysis results in a structured format", default_settings={ "temperature": 0.3, "response_format": { "type": "json_object" } } ) ``` ```javascript Node.js theme={"dark"} const agent = await client.agents.create({ name: "Data Analyst", model: "gpt-4o-mini", about: "An agent that analyzes data and returns structured insights", instructions: "Always provide analysis results in a structured format", default_settings: { temperature: 0.3, response_format: { type: "json_object" } } }); ``` Check out the API reference [here](/api-reference/agents/create-agent) or SDK reference (Python [here](/sdks/python/reference#agents) or JavaScript [here](/sdks/nodejs/reference#agents) for more details on different operations you can perform on agents. ## Relationship To Other Concepts This section will help you understand how agents relate to other concepts in Julep. ### Projects Agents belong to exactly one project, which helps organize related resources together. When creating an agent, you can specify which project it belongs to using the `project` parameter. If not specified, the agent will be assigned to the "default" project. For example: ```python Python theme={"dark"} # Create an agent in a specific project agent = client.agents.create( name="Customer Support Bot", project="support-platform" ) ``` ```javascript Node.js theme={"dark"} // Create an agent in a specific project const agent = await client.agents.create({ name: "Customer Support Bot", project: "support-platform" }); ``` For more information about projects, see [Projects](/concepts/projects). ### Tools Agents can be associated with different types of tools available in Julep to enable them to perform operations. These tools associated with an agent can also be leveraged by a task associated with the agent. For example: ```python Python theme={"dark"} # Create an agent agent = client.agents.create(name="My Agent") # Associate a tool with the agent client.agents.tools.create( agent_id=AGENT_UUID, **{ "name": "computer", "type": "computer_20241022", "computer_20241022": { "display_height_px": 768, "display_width_px": 1024, "display_number": 1, }, } ) ``` ```javascript Node.js theme={"dark"} // Create an agent const agent = await client.agents.create({name: "My Agent"}); // Associate a tool with the agent await client.agents.tools.create(agent.id, { name: "computer", type: "computer_20241022", computer_20241022: { display_height_px: 768, display_width_px: 1024, display_number: 1, }, }); ``` ### Sessions Agents can be used in sessions to enable real-time, interactive conversations. While tasks are designed for automated workflows, sessions provide a way to have stateful, continuous interactions with an agent. You can create multiple sessions with the same agent or multiple agents in a session, each session maintaining its own conversation history and context. This makes sessions ideal for scenarios requiring ongoing dialogue or human-in-the-loop interactions. For example: ```python Python theme={"dark"} # Create an agent agent = client.agents.create(name="My Agent") # Create a session with the agent session = client.sessions.create(agent=agent.id) ``` ```javascript Node.js theme={"dark"} // Create an agent const agent = await client.agents.create({name: "My Agent"}); // Create a session with the agent const session = await client.sessions.create({agent: agent.id}); ``` ## Best Practices
  • **1. Clear, Focused Purposes**: Give agents clear, focused purposes rather than making them generalists
  • **2. Descriptive Names**: Use descriptive names that reflect the agent's primary function
  • **3. Concise Instructions**: Keep instructions concise but specific
  • **4. Specialized Agents**: Break complex tasks into multiple specialized agents rather than one complex agent
  • **1. Conservative Model Settings**: Start with conservative model settings (temperature, top\_p) and adjust as needed
  • **2. Metadata**: Use metadata effectively for organization and filtering
  • **3. Tools**: Define tools that are specific to the agent's purpose
  • **1. Reuse Agents**: Reuse agents across similar tasks instead of creating new ones
  • **2. Clean Up**: Clean up unused agents and their associated resources
  • **3. Monitor Token Usage**: Monitor token usage and adjust context windows appropriately
 Avoid giving agents more capabilities than they need. Each additional tool or permission increases the complexity and potential security surface area. ## Next Steps * [Agent Tools](/concepts/tools) - Learn about tools and how to use them with agents * [Agent Tasks](/concepts/tasks) - Learn about tasks and how to use them with agents * [Agent Sessions](/concepts/sessions) - Learn about sessions and how to use them with agents * [Agent Docs](/concepts/docs) - Learn about docs and how to use them with agents ## See Examples * [Hello Agent notebook](https://github.com/julep-ai/julep/blob/main/cookbooks/basics/01-Hello-Agent.ipynb) * [Companion Agent notebook](https://github.com/julep-ai/julep/blob/main/cookbooks/advanced/09-companion-agent.ipynb) # Documents (RAG) Source: https://docs.julep.ai/concepts/docs Working with documents in Julep ## Overview Documents in Julep provide a way to store and retrieve information that can be used by agents. This section covers how to work with documents effectively. ## Components Documents in Julep consist of several key components that enable efficient storage and retrieval of information. * **Title**: The title component helps identify and organize documents. * **Content**: The textual content of the document. * **Embeddings** (automatically generated): The vector representations of text that enable semantic search capabilities. Generated using the `text-embedding-3-large` model from OpenAI. * **Metadata**: Metadata provides additional context and filtering capabilities for documents. ### Docs Configuration Options When creating a doc, the following attributes can be specified: | **Field** | **Type** | **Description** | **Default** | | ---------- | -------- | ---------------------------------------------------------------------- | ----------- | | `title` | `string` | The title of the document. | Required | | `content` | `string` | The content of the document. | Required | | `metadata` | `object` | Additional metadata for the document, such as preferences or settings. | `null` | ## How to Use Docs ### Creating a Doc Documents are attached to either an agent or a user. This is how you can create a doc using Julep's SDKs. **Example:** ```python Python theme={"dark"} # Creating a doc client.agents.docs.create( agent_id=agent.id, title="Old World", content="The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa. In Europe, the era was marked by the rise of feudalism, the power of the Catholic Church, and the cultural blossoming of the Renaissance towards its end. Asia witnessed the flourishing of the Silk Road, facilitating trade and cultural exchange, and the rise of powerful empires such as the Mongol Empire, which at its height, stretched from Europe to Asia. Meanwhile, Africa saw the growth of influential kingdoms and empires like Mali, known for its wealth and the legendary pilgrimage of Mansa Musa, and the spread of Islam across the continent, which played a crucial role in shaping its cultural and social landscapes.", metadata={"source": "https://en.wikipedia.org/wiki/Medieval_period"}, ) ``` ```javascript Node.js theme={"dark"} // Creating a doc const doc = await client.agents.docs.create({ agent_id: "agent_id", title: "Old World", content: "The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa. In Europe, the era was marked by the rise of feudalism, the power of the Catholic Church, and the cultural blossoming of the Renaissance towards its end. Asia witnessed the flourishing of the Silk Road, facilitating trade and cultural exchange, and the rise of powerful empires such as the Mongol Empire, which at its height, stretched from Europe to Asia. Meanwhile, Africa saw the growth of influential kingdoms and empires like Mali, known for its wealth and the legendary pilgrimage of Mansa Musa, and the spread of Islam across the continent, which played a crucial role in shaping its cultural and social landscapes.", metadata: {"source": "https://en.wikipedia.org/wiki/Medieval_period"}, }); ``` To create a user doc, replace `client.agents.docs.create` with `client.users.docs.create`, and the `agent_id` argument with `user_id`. Check out the [API reference](/api-reference/create-user-doc) or SDK reference ([Python](/sdks/python/reference#docs) or [JavaScript](/sdks/nodejs/reference#docs)) for more details on different operations you can perform on docs. ### Getting a Doc To get a doc, you can use the `client.docs.get` method, and pass the doc's ID. **Example:** ```python Python theme={"dark"} # Getting a doc doc = client.docs.get(doc_id="doc_id") print(doc) ``` ```javascript Node.js theme={"dark"} // Getting a doc const doc = await client.docs.get("doc_id"); console.log(doc) ``` When you get a doc, you can access the doc's `title`, `content`, `embeddings`, `metadata` and other attributes. ```[expandable] theme={"dark"} Doc( id='0680b756-9827-7cf2-8000-82ed5daf816e', content=['The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa.'], created_at=datetime.datetime(2023, 4, 25, 11, 43, 37, 513161, tzinfo=datetime.timezone.utc), title='Old World', embedding_dimensions=1024, embedding_model='text-embedding-3-large', embeddings=[ -0.036940154, 0.021077264, -0.03274468, 0.016222501, ... -0.05038565, 0.012776218, ... ], language='english', metadata={"source": "https://en.wikipedia.org/wiki/Medieval_period"}, modality='text' ) ``` ### Chunking In Julep, documents are not automatically chunked. We recommend that developers handle chunking based on their specific use case requirements, as different applications may have unique needs for how documents should be divided. For those who need assistance with chunking, we provide a utility function `chunk_doc` that can be used directly in task steps. For implementation details, you can check the source code for this method in [this file](https://github.com/julep-ai/julep/blob/main/agents-api/agents_api/activities/utils.py). ### Search #### Full Text Search Our full-text search functionality leverages Timescale's powerful indexing to efficiently match user-provided keywords and phrases against document content, delivering relevant results even with variations in text. Key features include prefix and infix searching, morphology processing (stemming and lemmatization), fuzzy searching to handle typos, and exact result counts. **Parameters:** | Parameter | Type | Description | Default | | ------------------------------ | -------- | ------------------------------------------------------------------------------------ | ------------ | | `text` | `str` | The textual query to search within documents. | **Required** | | `metadata_filter` | `object` | Filters to apply based on document metadata. | `None` | | `lang` | `str` | The language to use for full-text search processing. | `'english'` | | `limit` | `int` | The maximum number of documents to return. | `10` | | `trigram_similarity_threshold` | `float` | The threshold for trigram similarity matching (higher values require closer matches) | `0.6` | The default parameters for full-text search are based on our internal benchmarking. These values provide a good starting point, but you may need to adjust them depending on your specific use case to achieve optimal results. **Example:** ```python Python theme={"dark"} # Define the query text_query = "Medieval times in Europe" # Search for docs matched_docs = client.agents.docs.search( agent_id="agent_id", text=text_query, limit=10, # the maximum number of docs to return ) print(matched_docs.model_dump()) ``` ```javascript Node.js theme={"dark"} // Define the query const text_query = "Medieval times in Europe"; // Search for docs const matched_docs = await client.agents.docs.search({ agent_id: "agent_id", text: text_query, limit: 10, // the maximum number of docs to return }); console.log(matched_docs); ``` ``` {'docs': [{'id': '06791e86-6dcf-7816-8000-e4c3b781d05b', 'owner': {'id': '06791e82-46b7-739c-8000-3428f9d4e40f', 'role': 'agent'}, 'snippet': {'content': 'The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa. In Europe, the era was marked by the rise of feudalism, the power of the Catholic Church, and the cultural blossoming of the Renaissance towards its end. Asia witnessed the flourishing of the Silk Road, facilitating trade and cultural exchange, and the rise of powerful empires such as the Mongol Empire, which at its height, stretched from Europe to Asia. Meanwhile, Africa saw the growth of influential kingdoms and empires like Mali, known for its wealth and the legendary pilgrimage of Mansa Musa, and the spread of Islam across the continent, which played a crucial role in shaping its cultural and social landscapes.', 'index': 0, 'embedding': [ -0.036940154, 0.021077264, -0.03274468, 0.016222501, -0.05038565, 0.012776218, ... ...]}, 'distance': 0.6666666865348816, 'metadata': {"source": "https://en.wikipedia.org/wiki/Medieval_period"}, 'title': 'Old World'}], 'time': 0.0522150993347168} ``` Check out the [API reference](/api-reference/search-agent-docs) or SDK reference ([Python](/sdks/python/reference#docs) or [JavaScript](/sdks/nodejs/reference#docs)) for more details on different operations you can perform on docs. #### Embedding (vector) Search Our embedding (vector) search functionality leverages machine learning to convert search queries and documents into numerical vectors, enabling semantic matching based on vector similarity. It utilizes an embedding space where similar vectors indicate related content and employs algorithms like k-nearest neighbors (KNN) to retrieve the most relevant documents. Key features include context awareness, flexibility for natural language queries, multi-modal search across various content types, and effective handling of synonyms. **Parameters:** | Parameter | Type | Description | Default | | ----------------- | ---------- | ------------------------------------------------------------------------------------- | ------------ | | `vector` | `number[]` | The embedding vector representing the semantic meaning of the query. | **Required** | | `limit` | `integer` | The number of top results to return (must be between 1 and 50). | `10` | | `lang` | `string` | The language for the search query. | `en-US` | | `metadata_filter` | `object` | Filters to apply based on document metadata. | None | | `mmr_strength` | `number` | The strength of Maximum Marginal Relevance diversification (must be between 0 and 1). | `0.5` | | `confidence` | `number` | The confidence threshold for embedding similarity (must be between -1 and 1). | `0.5` | The default parameters for embedding search are based on our internal benchmarking. These values provide a good starting point, but you may need to adjust them depending on your specific use case to achieve optimal results. ```python Python theme={"dark"} # Define the vector query vector_query = client.docs.embed(text="Medieval times in Europe").vectors[0] # Search for docs matched_docs = client.agents.docs.search( agent_id="agent_id", vector=vector_query, limit=10, # the maximum number of docs to return confidence=-0.3, # confidence range is -1 to 1 ) print(matched_docs.model_dump()) ``` ```javascript Node.js theme={"dark"} // Define the vector query const embedding_result = client.docs.embed({ text: "Medieval times in Europe" }); const vector_query = embedding_result.vectors[0]; // Search for docs const matched_docs = await client.agents.docs.search({ agent_id: "agent_id", vector: vector_query, limit: 10, // the maximum number of docs to return confidence: -0.3, // confidence range is -1 to 1 }); console.log(matched_docs); ``` ``` {'docs': [{'id': '06791e86-6dcf-7816-8000-e4c3b781d05b', 'owner': {'id': '06791e82-46b7-739c-8000-3428f9d4e40f', 'role': 'agent'}, 'snippet': {'content': 'The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa. In Europe, the era was marked by the rise of feudalism, the power of the Catholic Church, and the cultural blossoming of the Renaissance towards its end. Asia witnessed the flourishing of the Silk Road, facilitating trade and cultural exchange, and the rise of powerful empires such as the Mongol Empire, which at its height, stretched from Europe to Asia. Meanwhile, Africa saw the growth of influential kingdoms and empires like Mali, known for its wealth and the legendary pilgrimage of Mansa Musa, and the spread of Islam across the continent, which played a crucial role in shaping its cultural and social landscapes.', 'index': 0, 'embedding': [ -0.036940154, 0.021077264, -0.03274468, 0.016222501, -0.05038565, 0.012776218, -0.034123193, -0.005039564, ... ...]}, 'distance': 0.9322964182738759, 'metadata': {"source": "https://en.wikipedia.org/wiki/Medieval_period"}, 'title': 'Old World'}], 'time': 0.04166412353515625} ``` Check out the [API reference](/api-reference/search-agent-docs) or SDK reference ([Python](/sdks/python/reference#docs) or [JavaScript](/sdks/nodejs/reference#docs)) for more details on different operations you can perform on docs. #### Hybrid Search Our hybrid search functionality combines multiple search techniques to deliver highly relevant and accurate search results. Julep's hybrid search uses a three-pronged approach that leverages: 1. **Full-text search** - Traditional keyword-based search using PostgreSQL's tsquery/tsrank 2. **Vector search** - Semantic search using embeddings for contextual understanding 3. **Trigram search** - Fuzzy text matching using character n-grams for typo tolerance By combining these approaches, hybrid search ensures that queries are matched not only on exact terms but also understood in context and tolerant of variations, providing more nuanced and precise results. This comprehensive approach enhances search performance, improves result relevance, and accommodates a wider range of search queries. ##### Trigram Search Features Julep's trigram search capabilities include: * **Fuzzy Matching** - Handles typos, spelling variations, and morphological differences * **Similarity Scoring** - Combines trigram similarity with Levenshtein distance for accurate matching * **Word-Level Analysis** - Matches individual meaningful words against target content * **Adaptive Weighting** - Adjusts fuzzy matching strength based on query length * **Performance Optimization** - Uses PostgreSQL's GIN indexes and materialized CTEs for efficient processing ##### How It Works 1. When a search query is submitted, Julep runs both full-text search and trigram-based fuzzy search in parallel. 2. Traditional full-text search results are prioritized (returned first). 3. Trigram search then finds documents that full-text search might miss due to minor variations or typos. 4. The system integrates these results with vector-based search results using Distribution-Based Score Fusion (DBSF). 5. Results are ranked and returned based on a combination of all three search approaches. **Parameters:** | **Parameter** | **Type** | **Description** | **Default** | | ------------------------------ | ------------- | --------------------------------------------------------------------------------------------------- | ------------------ | | `text` | `str` | The textual query to search within documents. | **Required** | | `vector` | `List[float]` | The embedding vector representing the semantic meaning of the query. | **Required** | | `alpha` | `float` | The weight assigned to embedding-based results versus text-based results (must be between 0 and 1). | `0.5` | | `confidence` | `float` | The confidence threshold for embedding similarity (must be between -1 and 1). | `0.5` | | `metadata_filter` | `object` | Filters to apply based on document metadata. | `None` | | `limit` | `int` | The number of top results to return. | `3` | | `lang` | `str` | The language to use for full-text search processing. | `english_unaccent` | | `mmr_strength` | `float` | The strength of Maximum Marginal Relevance diversification (must be between 0 and 1). | `0.5` | | `trigram_similarity_threshold` | `float` | The threshold for trigram similarity matching (must be between 0 and 1). | `0.6` | | `k_multiplier` | `int` | Controls how many intermediate results to fetch before final scoring. | `7` | The default parameters for hybrid search are based on our internal benchmarking. These values provide a good starting point, but you may need to adjust them depending on your specific use case to achieve optimal results. **Example:** ```python Python theme={"dark"} # Define the query text_query = "Medieval times in Europe" # Embed the query using the `docs.embed` method embedded_query = client.docs.embed(text=text_query).vectors[0] # Search for docs matched_docs = client.agents.docs.search( agent_id="agent_id", text=text_query, alpha=0.5, # the weight of the embedding query vector=embedded_query, confidence=-0.3, # confidence range is -1 to 1 ) print(matched_docs.model_dump()) ``` ```javascript Node.js theme={"dark"} // Define the query const text_query = "Medieval times in Europe"; // Embed the query using the `docs.embed` method const embedding_result = await client.docs.embed({ text: text_query }); const embedded_query = embedding_result.vectors[0]; // Search for docs const matched_docs = await client.agents.docs.search({ agent_id: "agent_id", text: text_query, vector: embedded_query, confidence: -0.3, // confidence range is -1 to 1 alpha: 0.5, // the weight of the embedding query }); console.log(matched_docs); ``` ``` {'docs': [{'id': '06791e86-6dcf-7816-8000-e4c3b781d05b', 'owner': {'id': '06791e82-46b7-739c-8000-3428f9d4e40f', 'role': 'agent'}, 'snippet': {'content': 'The medieval period, spanning roughly from the 5th to the late 15th century, was a time of significant transformation across Europe, Asia, and Africa. In Europe, the era was marked by the rise of feudalism, the power of the Catholic Church, and the cultural blossoming of the Renaissance towards its end. Asia witnessed the flourishing of the Silk Road, facilitating trade and cultural exchange, and the rise of powerful empires such as the Mongol Empire, which at its height, stretched from Europe to Asia. Meanwhile, Africa saw the growth of influential kingdoms and empires like Mali, known for its wealth and the legendary pilgrimage of Mansa Musa, and the spread of Islam across the continent, which played a crucial role in shaping its cultural and social landscapes.', 'index': 0, 'embedding': [ -0.036940154, 0.021077264, -0.03274468, 0.016222501, -0.05038565, 0.012776218, -0.034123193, -0.005039564, ... ...]}, 'distance': 0.13411101466087272, 'metadata': {"source": "https://en.wikipedia.org/wiki/Medieval_period"}, 'title': 'Old World'}], 'time': 0.0514223575592041} ``` Check out the [API reference](/api-reference/search-agent-docs) or SDK reference ([Python](/sdks/python/reference#docs) or [JavaScript](/sdks/nodejs/reference#docs)) for more details on different operations you can perform on docs. ## Filtering While search is carried on based on the textual and/or semantic content of the documents, you can also filter the documents based on their metadata. **Example:** ```python Python theme={"dark"} # Filter docs based on metadata dev_client.agents.docs.list( agent_id="agent_id", metadata_filter={"source": "wikipedia"}, ) print(docs.items) ``` ```javascript Node.js theme={"dark"} // Filter docs based on metadata const docs = await client.agents.docs.list( agentId, { metadata_filter: {"source": "wikipedia"}, } ); console.log(docs.items); ``` Check out the [API reference](/api-reference/list-agent-docs) or SDK reference ([Python](/sdks/python/reference#docs) or [JavaScript](/sdks/nodejs/reference#docs)) for more details on different operations you can perform on docs. ## Relationship to Other Concepts ### Sessions Sessions have access to search, retrieve and reference agents and users documents inside chat conversations. Read more about it [here](/concepts/sessions#documents). ### Tasks By leveraging [System Tools](/concepts/tools#system-tools), Julep [Tasks](/concepts/tasks) have the ability to create, search, filter and read documents. **Example:** ```yaml [expandable] theme={"dark"} input_schema: type: object properties: user_id: type: string description: The id of the user to list documents for tools: - name: "list_user_docs" description: "List all documents for the current user" type: "system" system: resource: user subresource: doc operation: list main: # Step that lists all documents for the current user - tool: "list_user_docs" arguments: user_id: $ _.user_id # Step that evaluates the textual contents of all documents for the current user - evaluate: all_user_docs_contents: $ [doc.content for doc in _.items] ``` Checkout [this cookbook](https://colab.research.google.com/github/julep-ai/julep/blob/dev/cookbooks/11-generate-user-personas.ipynb) that leverages Julep's docs, system tools and tasks to create content-rich user personas. ## Best Practices
  • **1. Metadata**: Use consistent and descriptive metadata to enhance document retrieval and filtering.
  • **1. Version Control**: Maintain version control for documents to track changes and updates over time.
  • **1. Access Control**: Ensure sensitive information is protected and access to documents is properly managed.
  • **1. Chunking Strategies**: Implement efficient chunking strategies to optimize document processing and retrieval.
  • **1. Update**: Regularly update document content and metadata to keep information current and relevant.
 ## Advanced Search Features ### Trigram-Enhanced Fuzzy Matching Julep's advanced fuzzy matching capability is built on PostgreSQL's `pg_trgm` extension and enhanced with additional similarity techniques. This allows for resilient document retrieval that can handle variations in text, including: * Typos and spelling errors * Morphological variations * Term order differences * Incomplete terms #### Similarity Mechanisms Julep uses a multi-layered approach to determine text similarity: 1. **Basic Trigram Similarity** - Uses PostgreSQL's built-in trigram functions to match documents based on character-level n-grams. 2. **Enhanced Similarity** - Combines trigram matching with Levenshtein distance calculations to provide better accuracy, especially for shorter text segments: ```sql theme={"dark"} -- 70% trigram, 30% Levenshtein for shorter strings RETURN 0.7 * trgm_sim + 0.3 * norm_lev; ``` 3. **Word-Level Similarity** - Breaks text into individual words and finds the best match for each meaningful word: ```sql theme={"dark"} -- Only process meaningful words (longer than 2 chars) IF length(words1[i]) > 2 THEN -- Find best match in target content best_match := GREATEST(best_match, similarity(words1[i], words2[j])); ``` 4. **Comprehensive Similarity** - Adaptively weights different similarity metrics based on query characteristics: ```sql theme={"dark"} -- Weight factor based on query length - shorter queries need more help word_weight float := CASE WHEN length(query) < 10 THEN 0.4 WHEN length(query) < 20 THEN 0.3 ELSE 0.2 END; ``` #### Tuning Search Behavior You can customize Julep's fuzzy search behavior through several parameters: * **Similarity Threshold** (`trigram_similarity_threshold`) - Controls the minimum similarity score required for a document to match: * Higher values (e.g., 0.8) require closer matches, reducing false positives but may miss relevant documents with variations * Lower values (e.g., 0.3) are more lenient, catching more variations but potentially including less relevant results * Default: 0.6 for hybrid search, 0.3 for text-only search * **Alpha Weight** (`alpha`) - Balances the importance of vector-based semantic search vs. text-based search: * Higher values prioritize semantic/embedding matches * Lower values prioritize text and trigram matches * Default: 0.7 (70% weight to embeddings) * **Search Language** (`lang`) - Affects tokenization, stemming, and other text processing operations: * Default: 'english\_unaccent' which handles accent/diacritic-insensitive matching #### Implementation Details Julep's trigram search is implemented using: 1. **Database Indexes** - GIN indexes on document title and content for efficient trigram operations: ```sql theme={"dark"} CREATE INDEX IF NOT EXISTS idx_docs_title_trgm ON docs USING GIN (title gin_trgm_ops); CREATE INDEX IF NOT EXISTS idx_docs_content_trgm ON docs USING GIN (content gin_trgm_ops); ``` 2. **Materialized CTEs** - Improves performance for complex query operations: ```sql theme={"dark"} WITH tsv_results AS MATERIALIZED (...) ``` 3. **Runtime Optimizations** - Selective application of more expensive calculations: ```sql theme={"dark"} -- Only compute Levenshtein for reasonable length strings (performance) IF length(text1) <= 50 AND length(text2) <= 50 THEN ``` 4. **Distribution-Based Score Fusion** - Combines results from different search methods: ```sql theme={"dark"} -- Aggregate all text/embedding scores into arrays aggregated AS ( SELECT array_agg(text_score ORDER BY rn) AS text_scores, array_agg(embedding_score ORDER BY rn) AS embedding_scores ``` These technologies combine to provide a sophisticated fuzzy search capability that significantly improves document retrieval compared to traditional search methods. ## Next Steps * [Sessions](/concepts/sessions) - Learn about sessions and how documents are used in chat conversations. * [Tools](/concepts/tools) - Learn about tools and how they can be used to fill documents with content. * [Tasks](/concepts/tasks) - Learn about tasks and how to use documents inside tasks. * [Cookbooks](https://github.com/julep-ai/julep/tree/main/cookbooks) - Check out cookbooks to see how Julep can be used in real-world scenarios. # Executions Source: https://docs.julep.ai/concepts/execution Understanding Task Executions and Their Lifecycle ## Overview Executions in Julep represent instances of tasks that have been initiated with specific inputs. They embody the lifecycle of a task, managing its progression through various states from initiation to completion. Understanding executions is crucial for effectively managing and monitoring the behavior of your AI agents and their workflows. ## Components Executions are comprised of several key components that work together to manage and monitor the state of a task: * **Execution ID**: A unique identifier for each execution instance. * **Task ID**: The identifier of the task being executed. * **Input**: The inputs provided to the task at the time of execution. * **Status**: The current state of the execution (e.g., queued, running, succeeded). * **Output**: The result produced by the execution upon completion. * **Transitions**: The sequence of state changes that the execution undergoes. * **Transition Count**: The number of transitions that have occurred in this execution. ### Execution Configuration options | Option | Type | Description | Default | | ------------------ | -------- | ---------------------------------------------- | ------------ | | `task_id` | `string` | The ID of the task to execute | **Required** | | `input` | `object` | The input to the task | **Required** | | `metadata` | `object` | Additional metadata for the execution instance | `null` | | `transition_count` | `number` | The number of transitions in this execution | `null` | ## Lifecycle of an Execution An execution follows a well-defined lifecycle, transitioning through various states from start to finish. Understanding these states helps in monitoring and managing task executions effectively. ### Execution Statuses Executions can exist in one of the following statuses: | **Status** | **Description** | | ---------------- | ------------------------------------------------------------- | | `queued` | The execution is queued and waiting to start. | | `starting` | The execution is starting. | | `running` | The execution is currently running. | | `awaiting_input` | The execution is suspended and awaiting user input to resume. | | `succeeded` | The execution has completed successfully. | | `failed` | The execution has failed due to an error. | | `cancelled` | The execution has been cancelled by the user or system. | ### Execution State Machine The state transitions of an execution are governed by a state machine that ensures proper progression and handling of different scenarios. ```mermaid theme={"dark"} stateDiagram-v2 [*] --> queued queued --> starting queued --> cancelled starting --> cancelled starting --> failed starting --> running running --> running running --> awaiting_input running --> cancelled awaiting_input --> running awaiting_input --> cancelled running --> succeeded failed --> [*] succeeded --> [*] cancelled --> [*] ``` ## Execution State Transitions Executions in Julep follow a specific state transition model. The transitions are governed by both the execution status and the transition type: * **Init**: The execution is initialized. * **Start**: The execution begins. * **Step**: A step within the execution is executed. * **Wait**: The execution is waiting for an external input. * **Resume**: The execution resumes after waiting. * **Finish**: The execution completes successfully. * **Error**: The execution encounters an error. * **Cancel**: The execution is cancelled. ### Transition Types | **Transition Type** | **Description** | | ------------------- | ---------------------------------------------- | | `init` | Initializes the execution. | | `start` | Starts the execution process. | | `step` | Executes a step within the task. | | `wait` | Pauses execution waiting for external input. | | `resume` | Resumes execution after a wait. | | `finish` | Marks the execution as successfully completed. | | `error` | Marks the execution as failed due to an error. | | `cancel` | Cancels the execution. | ## Creating an Execution To create an execution for a specific task, use the following method in the SDKs. ```python Python theme={"dark"} from julep import Julep import yaml client = Julep(api_key="YOUR_API_KEY") # Execute the task (assuming the task is already created) execution = client.executions.create( task_id="task_id", input={ "parameter1": "value1", "parameter2": "value2" } ) print(f"Execution ID: {execution.id}") ``` ```javascript Node.js theme={"dark"} const { Julep } = require('@julep/sdk'); const client = new Julep({ apiKey: 'YOUR_API_KEY' }); // Execute the task (assuming the task is already created) const execution = await client.executions.create({ task_id: "task_id", input: { parameter1: 'value1', parameter2: 'value2' } }); console.log("Execution ID:", execution.id); ``` Check out the API reference [here](/api-reference/executions) or SDK reference (Python [here](/sdks/python/reference#executions) or JavaScript [here](/sdks/nodejs/reference#executions) for more details on different operations you can perform on executions. ## Monitoring an Execution After initiating an execution, it's essential to monitor its progress and handle its completion or failure appropriately. ```python Python theme={"dark"} import time from julep import Julep client = Julep(api_key="YOUR_API_KEY") execution_id = "YOUR_EXECUTION_ID" while True: result = client.executions.get(execution_id) print(f"Status: {result.status}") print(f"Current output: {result.output}") if result.status in ["succeeded", "failed", "cancelled"]: if result.status == "succeeded": print("Execution succeeded with output:", result.output) else: print("Execution ended with status:", result.status) break time.sleep(5) # Wait for 5 seconds before polling again ``` ```javascript Node.js theme={"dark"} const { Julep } = require('@julep/sdk'); const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const executionId = 'YOUR_EXECUTION_ID'; while (true) { const result = await client.executions.get(executionId); console.log(`Status: ${result.status}`); console.log(`Current output: ${result.output}`); if (["succeeded", "failed", "cancelled"].includes(result.status)) { if (result.status === "succeeded") { console.log("Execution succeeded with output:", result.output); } else { console.log("Execution ended with status:", result.status); } break; } await new Promise(resolve => setTimeout(resolve, 5000)); // Wait for 5 seconds before polling again } ``` To view more details about the status of the execution and how it is transitioning between states, you can use list the transitions of an execution. Example: ```python Python theme={"dark"} from julep import Julep client = Julep(api_key="YOUR_API_KEY") execution_id = "YOUR_EXECUTION_ID" transitions = client.executions.transitions.list(execution_id) print(transitions.items) ``` ```javascript Node.js theme={"dark"} const { Julep } = require('@julep/sdk'); const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const executionId = 'YOUR_EXECUTION_ID'; const transitions = await client.executions.transitions.list(executionId); console.log(transitions.items); ``` Check out the API reference [here](/api-reference/executions) or SDK reference (Python [here](/sdks/python/reference#executions) or JavaScript [here](/sdks/nodejs/reference#executions) for more details on different operations you can perform on executions. ## Streaming Execution Status Updates ### Using the raw SSE endpoint You can subscribe to real-time status updates using the Server-Sent Events (SSE) endpoint. Each event conforms to the `ExecutionStatusEvent` schema and includes the following fields: * **execution\_id**: The UUID of the execution. * **status**: The current execution status. * **updated\_at**: ISO 8601 timestamp of the update. * **error**: Error message if the execution failed. * **transition\_count**: Number of transitions that have occurred. * **metadata**: Arbitrary metadata for the event. ```bash theme={"dark"} curl -X GET 'https://api.julep.ai/api/executions/{execution_id}/status.stream' \ -H 'Authorization: Bearer $JULEP_API_KEY' ``` You'll be getting events that look like this: ```bash theme={"dark"} data: {"execution_id":"068306ff-e0f3-7fe9-8000-0013626a759a","status":"starting","updated_at":"2025-05-23T12:54:24.565424Z","error":null,"transition_count":1,"metadata":{}} data: {"execution_id":"068306ff-e0f3-7fe9-8000-0013626a759a","status":"running","updated_at":"2025-05-23T12:54:30.903484Z","error":null,"transition_count":2,"metadata":{}} data: {"execution_id":"068306ff-e0f3-7fe9-8000-0013626a759a","status":"succeeded","updated_at":"2025-05-23T12:56:12.054067Z","error":null,"transition_count":3,"metadata":{}} ``` ### Using the Python SDK `AsyncClient` ```python Python theme={"dark"} from julep import AsyncClient client = AsyncClient(api_key="YOUR_API_KEY") execution_id = "YOUR_EXECUTION_ID" # Subscribe to the live status stream (async generator) status_stream = await client.executions.status.stream(execution_id=execution_id) # Consume events in real-time using async for async for event in status_stream: print("Execution status:", event.status, "updated at", event.updated_at) ``` This approach relies on Python's async / await syntax. Make sure to: 1. Use `AsyncClient` not `Client`. 2. `await client.executions.status.stream(...)` to obtain the async generator. 3. Iterate with `async for` to consume events. ## Updating/Cancelling an Execution To update or cancel an execution, you can use the `change_status` method in the SDKs. Example: ```python Python theme={"dark"} from julep import Julep client = Julep(api_key="YOUR_API_KEY") execution_id = "YOUR_EXECUTION_ID" # To cancel an execution client.executions.change_status(execution_id=execution_id, status="cancelled") # To resume an execution with specific input client.executions.change_status( execution_id=execution_id, status="running", input={ "parameter1": "value1", "parameter2": "value2" } ) ``` ```javascript Node.js theme={"dark"} const { Julep } = require('@julep/sdk'); const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const executionId = 'YOUR_EXECUTION_ID'; // To cancel an execution await client.executions.changeStatus(executionId, 'cancelled'); // To resume an execution with specific input await client.executions.changeStatus( executionId, 'running', { input: { "parameter1": "value1", "parameter2": "value2" } } ); ``` Check out the API reference [here](/api-reference/executions) or SDK reference (Python [here](/sdks/python/reference#executions) or JavaScript [here](/sdks/nodejs/reference#executions) for more details on different operations you can perform on executions. ## Best Practices
  • **1. Execution Statuses**: Ensure your application gracefully handles all possible execution statuses, including `failed` and `cancelled`.
  • **1. Polling Interval**: Choose an appropriate polling interval to balance responsiveness and API usage.
  • **1. Logging**: Maintain detailed logs of execution statuses and outputs for auditing and debugging purposes.
 ## Next Steps * [Checkout the Tutorial](/tutorials) - Learn how to use executions in a tutorial * [Checkout the Execution Lifecycle](/advanced/lifecycle) - Learn more about the execution lifecycle # Files Source: https://docs.julep.ai/concepts/files Managing files and attachments in Julep # Files Files in Julep allow agents to work with various types of data including documents, images, audio, and other media. Files are stored securely and can be accessed by agents and tasks as needed. ## Overview Julep's file system provides: * Secure storage for various file types * Unique identifiers for consistent access * Metadata for organization and discovery * Content hashing for integrity verification * Project association for logical grouping ## File Properties Each file in Julep has the following properties: | **Field** | **Type** | **Description** | **Default** | | ------------- | -------- | ------------------------------------------------------ | --------------- | | `name` | `string` | Name of the file | Required | | `project` | `string` | The canonical name of the project this file belongs to | `"default"` | | `content` | `string` | Base64-encoded file content | Required | | `description` | `string` | Description of the file | `""` | | `mime_type` | `string` | MIME type of the file | `null` | | `size` | `number` | Size of the file in bytes (read-only) | Auto-calculated | | `hash` | `string` | Hash of the file content (read-only) | Auto-calculated | | `created_at` | `string` | Creation timestamp (read-only) | Auto-generated | ## Creating Files You can create files using the Julep SDK in Python or JavaScript: ```python Python theme={"dark"} from julep import Julep import base64 client = Julep(api_key="your_api_key") # Read a local file and encode it as base64 with open("path/to/document.pdf", "rb") as file: content = base64.b64encode(file.read()).decode("utf-8") # Create the file in Julep file = client.files.create( name="document.pdf", project="knowledge-base", content=content, description="Important document for reference", mime_type="application/pdf" ) print(f"Created file: {file.id}") ``` ```javascript Node.js theme={"dark"} import { Julep } from '@julep/sdk'; import fs from 'fs'; const client = new Julep({ apiKey: 'your_api_key' }); // Read a local file and encode it as base64 const content = fs.readFileSync('path/to/document.pdf').toString('base64'); // Create the file in Julep const file = await client.files.create({ name: "document.pdf", project: "knowledge-base", content: content, description: "Important document for reference", mime_type: "application/pdf" }); console.log(`Created file: ${file.id}`); ``` ## Managing Files ### Retrieving Files ```python Python theme={"dark"} # Get a specific file by ID file = client.files.get("file_id_here") print(file) # List all files files = client.files.list() for file in files: print(f"{file.name}: {file.description}") # List files with filtering files = client.files.list(metadata_filter={"category": "report"}) ``` ```javascript Node.js theme={"dark"} // Get a specific file by ID const file = await client.files.get("file_id_here"); console.log(file); // List all files const files = await client.files.list(); files.forEach(file => console.log(`${file.name}: ${file.description}`)); // List files with filtering const files = await client.files.list({ metadata_filter: { category: "report" } }); ``` ### Deleting Files ```python Python theme={"dark"} # Delete a file client.files.delete("file_id_here") ``` ```javascript Node.js theme={"dark"} // Delete a file await client.files.delete("file_id_here"); ``` ## Relationship to Other Concepts ### Projects Files belong to exactly one project, which helps organize related resources together. When creating a file, you can specify which project it belongs to using the `project` parameter. If not specified, the file will be assigned to the "default" project. **Example:** ```python Python theme={"dark"} # Create a file in a specific project file = client.files.create( name="marketing-image.jpg", project="product-launch", content=base64_content, mime_type="image/jpeg" ) ``` ```javascript Node.js theme={"dark"} // Create a file in a specific project const file = await client.files.create({ name: "marketing-image.jpg", project: "product-launch", content: base64Content, mime_type: "image/jpeg" }); ``` For more information about projects, see [Projects](/concepts/projects). ### Agents and Tasks Files can be used by both agents and tasks to access and process information. For example, an agent might analyze an image file, or a task might process a document file. ```python Python theme={"dark"} # Create a file file = client.files.create(name="data.csv", content=base64_content) # Reference the file in a task execution execution = client.tasks.executions.create( task_id="task_id_here", input={"file_id": file.id} ) ``` ```javascript Node.js theme={"dark"} // Create a file const file = await client.files.create({ name: "data.csv", content: base64Content }); // Reference the file in a task execution const execution = await client.tasks.executions.create({ task_id: "task_id_here", input: { file_id: file.id } }); ``` ## Best Practices
  • Group related files in the same project
  • Use consistent naming conventions
  • Include descriptive metadata for better discovery
  • Keep file sizes reasonable for faster processing
  • Consider chunking large files into smaller ones
  • Be mindful of the content limits when Base64 encoding
  • Avoid storing sensitive information in files
  • Regularly audit and clean up unused files
  • Validate file types before uploading
 ## Next Steps * [Projects](/concepts/projects) - Learn about organizing resources with projects * [Agents](/concepts/agents) - Learn how agents can work with files * [Tasks](/concepts/tasks) - Learn how to use files in task workflows # Projects Source: https://docs.julep.ai/concepts/projects Organizational units for grouping related resources ## Overview Projects are organizational units that allow you to group related agents, users, files, and other resources together. They provide a way to manage resources at a higher level and maintain clean separation between different use cases or applications. Projects provide several benefits: * **Organization**: Group related resources together logically * **Management**: Administer resources at the project level * **Isolation**: Keep resources separate between different applications * **Deployment**: Deploy multiple templates or configurations without conflicts Each developer automatically gets a "default" project that contains all existing resources. This ensures backward compatibility with existing applications while providing the benefits of project organization. ## Project Properties A project includes: * `id`: Unique identifier * `canonical_name`: Machine-readable name (unique per developer) * `name`: Human-readable display name * `metadata`: Custom attributes for the project ## Project Relationships Projects have a one-to-many relationship with resources: * Each agent belongs to exactly one project * Each user belongs to exactly one project * Each file belongs to exactly one project When creating resources, you can specify which project they belong to using the `project` field. If not specified, resources are automatically assigned to the "default" project. ## Default Project Every developer automatically receives a "default" project when they first interact with the API. This project: * Cannot be deleted * Has a canonical name of "default" * Contains all previously created resources (pre-dating the projects feature) * Serves as a fallback for resources created without a project specification ## API Operations Projects support standard CRUD operations: * `GET /projects`: List all projects for a developer * `POST /projects`: Create a new project * `GET /projects/{id}`: Retrieve a specific project * `PUT /projects/{id}`: Update a project * `PATCH /projects/{id}`: Partially update a project * `DELETE /projects/{id}`: Delete a project (except the default project) ## Usage Examples ### Creating a Project ```json theme={"dark"} POST /projects { "name": "Customer Support Bot", "canonical_name": "support-bot", "metadata": { "description": "Resources for our customer support chatbot", "team": "customer-success" } } ``` ### Creating a Resource in a Project When creating a resource such as an agent, specify the project canonical name: ```json theme={"dark"} POST /agents { "name": "Support Assistant", "project": "support-bot", "instructions": [ "You are a helpful customer support assistant." ], "model": "gpt-4-turbo" } ``` ## Future Capabilities Projects are designed with future extensibility in mind. Planned enhancements include: * Project-specific API keys for more granular access control * Usage tracking and billing at the project level * Enhanced permissions and role-based access control * Cross-project resource sharing capabilities ## Best Practices * Use meaningful canonical names for projects that reflect their purpose * Group resources by logical application or team * Use metadata to add custom attributes for filtering and organization * Create separate projects for development, staging, and production environments # Secrets Source: https://docs.julep.ai/concepts/secrets Securely store and manage sensitive information for your LLM applications # Secrets Secrets allow you to securely store and manage sensitive information like API keys, credentials, and other confidential data that your agents and tasks need to access external services. ## What are Secrets? Secrets are encrypted key-value pairs that can be referenced in your tasks and used by your agents without exposing the actual values in your code or configuration files. This helps maintain security and separates sensitive data from your application logic. ## Key Features * **Encrypted Storage**: All secrets are encrypted at rest using industry-standard AES-256 encryption * **Access Control**: Secrets are scoped to developers and cannot be accessed across developer accounts * **Named References**: Reference secrets by name in tasks and tools instead of hardcoding values * **Versioning**: Track when secrets were created and updated * **Metadata Support**: Add custom metadata to organize and categorize your secrets ## Common Use Cases * Storing API keys for external services (OpenAI, Google, AWS, etc.) * Managing database credentials * Securing authentication tokens * Storing sensitive configuration values * Managing encrypted communication channels ## Secrets vs. Environment Variables While environment variables are commonly used for configuration, secrets provide several advantages: * **Encrypted Storage**: Environment variables are stored in plain text, while secrets are encrypted * **Access Management**: Environment variables are global, while secrets have access controls * **Audit Trail**: Secrets maintain creation and update timestamps * **Organized Management**: The secrets API provides a structured way to manage sensitive data ## Working with Secrets ### Creating Secrets Secrets can be created through the API, SDK, or CLI: ```python Python theme={"dark"} secret = client.secrets.create( name="stripe_api_key", value="sk_test_...", description="Stripe API key for payment processing", metadata={"environment": "production", "owner": "payments-team"} ) print(f"Created secret: {secret.name}") ``` ```bash CLI theme={"dark"} julep secrets create --name "openai_api_key" --value "sk-..." --description "OpenAI API key" ``` ```bash API theme={"dark"} curl https://dev.julep.ai:443/api/secrets \ --request POST \ --header 'Content-Type: application/json' \ --data '{ "metadata": {}, "name": "openai_api_key", "description": "", "value": "sk-..." }' ``` #### Required Parameters * `name`: The name of the secret (must be a valid identifier) * `value`: The secret value to encrypt and store #### Optional Parameters * `description`: A description of what the secret is used for * `metadata`: A dictionary of metadata to associate with the secret ### Using Secrets in Tasks Once created, secrets can be referenced in your tasks: ```yaml theme={"dark"} steps: - kind: tool_call tool: openai operation: completion arguments: prompt: "Summarize this text" secret_name: openai_api_key ``` ### Using Secrets in Expressions You can also reference secrets in expressions: ```yaml theme={"dark"} steps: - kind: prompt model: gpt-4 prompt: "Generate a summary" template_variables: api_url: "$ f'https://api.example.com/v1?key={secrets.api_key}'" ``` ## Secret Names and Conventions Secret names must: * Begin with a letter * Contain only alphanumeric characters and underscores * Be unique within a developer account Good naming conventions: * Use descriptive names like `stripe_api_key` instead of just `api_key` * Include service names in the key name * Use consistent formatting (snake\_case recommended) ## Next Steps * [Using Secrets in Julep](/guides/using-secrets) - Step-by-step guide for using secrets * [Secrets Management](/advanced/secrets-management) - Advanced guide for managing secrets * [API Reference](/api-reference#tag/secrets) - Complete API reference for secrets # Sessions Source: https://docs.julep.ai/concepts/sessions Understanding Julep Sessions and state management ## Overview Sessions in Julep are the backbone of stateful interactions between users and agents. They maintain the context and history of conversations, enabling personalized and coherent interactions over extended periods. Whether it's handling ongoing customer support inquiries or having a conversation with a user, sessions ensure that the agent retains necessary information to provide meaningful responses. ## Components Sessions are comprised of several key components that work together to manage state and context: * **Session ID**: A unique identifier (`uuid7`) for each session. * **User**: The individual or entity interacting with the agent, represented by its id. * **Agent**: The AI entity interacting with the user within the session, represented by its id. * **History**: The history of the conversation, which the agent uses to generate relevant responses. * **System Template**: A specific system prompt template that sets the background for this session. * **Situation**: A description of the current situation for the session. * **Metadata**: Additional data associated with the session, such as user preferences, session preferences, and other relevant information. ### Session Configuration Options When creating a session, you can leverage the following configuration options to tailor the experience: | Option | Type | Description | Default | | -------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------- | ----------------- | | `agent` | `UUID` \| `None` | The ID of the agent to associate with the session | `None` | | `user` | `UUID` \| `None` | The ID of the user interacting with the agent | `None` | | `context_overflow` | `truncate` \| `adaptive` \| `None` | Strategy for handling context overflow: `truncate` cuts off the oldest context; `adaptive` adjusts dynamically | `None` | | `metadata` | `object` \| `None` | Additional metadata for the session (e.g., user preferences) | `None` | | `system_template` | `str` \| `None` | A specific system prompt template that sets the background for this session | `None` | | `render_templates` | `StrictBool` | Whether to render system and assistant messages as Jinja templates | `True` | | `token_budget` | `int` \| `None` | Threshold value for the adaptive context functionality | `None` | | `auto_run_tools` | `StrictBool` | Whether to automatically execute tools and send the results back to the model when available | `False` | | `forward_tool_calls` | `StrictBool` | Whether to forward tool calls directly to the model | `False` | | `recall_options` | `object` \| `None` | Options for different RAG search modes (VectorDocSearch, TextOnlyDocSearch, HybridDocSearch) in the session | `VectorDocSearch` | ### Recall Options (RAG Search) When configuring a session, you can specify recall options to control how context or certain data is recalled during the session. Below are the available options based on search mode: | Parameter | Type | Description | Default | | --------------------- | ------------------- | ---------------------------------------------------------- | ---------- | | `mode` | `Literal["vector"]` | The mode to use for the search (must be "vector") | `"vector"` | | `lang` | `str` | The language for text search (other languages coming soon) | `en-US` | | `limit` | `int` | The limit of documents to return (1-50) | `10` | | `max_query_length` | `int` | The maximum query length (100-10000 characters) | `1000` | | `metadata_filter` | `object` | Metadata filter to apply to the search | `{}` | | `num_search_messages` | `int` | The number of search messages to use for the search (1-50) | `4` | | `confidence` | `float` | The confidence cutoff level (-1 to 1) | `0.5` | | `mmr_strength` | `float` | MMR Strength (mmr\_strength = 1 - mmr\_lambda) (0 to 1) | `0.5` | | Parameter | Type | Description | Default | | ------------------------------ | ----------------- | ----------------------------------------------------------------------- | -------- | | `mode` | `Literal["text"]` | The mode to use for the search (must be "text") | `"text"` | | `lang` | `str` | The language for text search (other languages coming soon) | `en-US` | | `limit` | `int` | The limit of documents to return (1-50) | `10` | | `max_query_length` | `int` | The maximum query length (100-10000 characters) | `1000` | | `metadata_filter` | `object` | Metadata filter to apply to the search | `{}` | | `num_search_messages` | `int` | The number of search messages to use for the search (1-50) | `4` | | `trigram_similarity_threshold` | `float` | The threshold for trigram similarity matching (must be between 0 and 1) | `0.6` | | Parameter | Type | Description | Default | | ------------------------------ | ------------------- | ------------------------------------------------------------------------------ | ------------------ | | `mode` | `Literal["hybrid"]` | The mode to use for the search (must be "hybrid") | `"hybrid"` | | `lang` | `str` | The language for text search (other languages coming soon) | `english_unaccent` | | `limit` | `int` | The limit of documents to return (1-50) | `10` | | `max_query_length` | `int` | The maximum query length (100-10000 characters) | `1000` | | `metadata_filter` | `object` | Metadata filter to apply to the search | `{}` | | `num_search_messages` | `int` | The number of search messages to use for the search (1-50) | `4` | | `alpha` | `float` | Weight between text-based and vector-based search (0=pure text, 1=pure vector) | `0.5` | | `confidence` | `float` | The confidence cutoff level (-1 to 1) | `0.5` | | `mmr_strength` | `float` | MMR Strength (mmr\_strength = 1 - mmr\_lambda) (0 to 1) | `0.5` | | `trigram_similarity_threshold` | `float` | The threshold for trigram similarity matching (must be between 0 and 1) | `0.6` | | `k_multiplier` | `int` | Controls how many intermediate results to fetch before final scoring | `7` | * When `recall_options` is not explicitly set (for instance, it is `None`), `vector` search mode is used with default parameters. * The default parameters for each search mode are based on our internal benchmarking. These values provide a good starting point, but you may need to adjust them depending on your specific use case to achieve optimal results. **Hybrid Search with Trigram Support** Julep's hybrid search combines multiple search techniques: 1. **Traditional full-text search** using PostgreSQL's tsquery/tsrank for keyword matching 2. **Vector-based semantic search** using embeddings for contextual understanding 3. **Trigram fuzzy matching** for handling typos, spelling variations, and morphological differences The trigram search capability uses PostgreSQL's pg\_trgm extension enhanced with Levenshtein distance calculations to provide resilient document retrieval even when search terms contain variations or errors. This is especially useful for natural language queries that may contain typos or alternative word forms. You can control the fuzzy matching behavior using the `trigram_similarity_threshold` parameter - higher values (e.g., 0.8) require closer matches while lower values (e.g., 0.3) are more lenient. For more details on the advanced search capabilities, see the [Documents (RAG)](/concepts/docs#advanced-search-features) section. ### System Template The **System Template** is a specific system prompt written as a Jinja template that sets the foundational context and instructions for the agent within a session. It defines the background, directives, and any relevant information that the agent should consider when interacting with the user. For a comprehensive guide on system templates including available variables, customization options, and advanced usage patterns, see the [System Templates](/advanced/system-templates) documentation. For more details on Jinja templates, refer to the [Jinja documentation](https://jinja.palletsprojects.com/). ```python [expandable] theme={"dark"} {%- if agent.name -%} You are {{agent.name}}.{{" "}} {%- endif -%} {%- if agent.about -%} About you: {{agent.about}}.{{" "}} {%- endif -%} {%- if user -%} You are talking to a user {%- if user.name -%}{{" "}} and their name is {{user.name}} {%- if user.about -%}. About the user: {{user.about}}.{%- else -%}.{%- endif -%} {%- endif -%} {%- endif -%} {{NEWLINE}} {%- if session.situation -%} Situation: {{session.situation}} {%- endif -%} {{NEWLINE+NEWLINE}} {%- if agent.instructions -%} Instructions:{{NEWLINE}} {%- if agent.instructions is string -%} {{agent.instructions}}{{NEWLINE}} {%- else -%} {%- for instruction in agent.instructions -%} - {{instruction}}{{NEWLINE}} {%- endfor -%} {%- endif -%} {{NEWLINE}} {%- endif -%} {%- if docs -%} Relevant documents:{{NEWLINE}} {%- for doc in docs -%} {{doc.title}}{{NEWLINE}} {%- if doc.content is string -%} {{doc.content}}{{NEWLINE}} {%- else -%} {%- for snippet in doc.content -%} {{snippet}}{{NEWLINE}} {%- endfor -%} {%- endif -%} --- {%- endfor -%} {%- endif -%} ``` ## How to Use Sessions Sessions are integral to maintaining a continuous and coherent interaction between users and agents. Here's how to create and manage sessions using Julep's SDKs. ### Creating a Session Here are examples of how to create a session using the SDKs: ```python Python theme={"dark"} client = Julep(api_key="YOUR_API_KEY") session = client.sessions.create( agent="agent_id", user="user_id", context_overflow="adaptive", ) ``` ```javascript Node.js theme={"dark"} const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const session = await client.sessions.create({ agent_id: "agent_id", user_id: "user_id", context_overflow: "adaptive", }); ``` Check out the [API reference](/api-reference/sessions) or SDK reference ([Python](/sdks/python/reference#sessions) or [JavaScript](/sdks/nodejs/reference#sessions)) for more details on different operations you can perform on sessions. ### Chatting in a Session Once a session is created, you can engage in a conversation by sending messages to the agent within that session. ```python Python theme={"dark"} client = Julep(api_key="YOUR_API_KEY") response = client.sessions.chat( session_id="SESSION_ID", messages=[ { "role": "user", # or "system" "content": "YOUR_MESSAGE" } ] ) print("Agent's response:", response.choices[0].message.content) ``` ```javascript Node.js theme={"dark"} const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const response = await client.sessions.chat({ session_id: "SESSION_ID", messages: [ { role: "user", content: "YOUR_MESSAGE" } ] }); console.log("Agent:", response.choices[0].message.content); ``` ## Relationship to Other Concepts This section will help you understand how sessions relate to other concepts in Julep. ### Agents Agents operate within sessions to provide personalized and context-aware interactions. While an agent defines the behavior and capabilities, a session maintains the state and context of interactions between the agent and the user. In other words, the history of a conversation is tied to a session, rather than an agent. Example: ```python theme={"dark"} agent = client.agents.create( name="David", about="A news reporter", model="gpt-4o-mini", instructions=["Keep your responses concise and to the point.", "If you don't know the answer, say 'I don't know'"], metadata={"channel": "FOX News"}, ) session1 = client.sessions.create(agent=agent.id, user="user_id", situation="The user is interested in the latest news about the stock market.") session2 = client.sessions.create(agent=agent.id, user="user_id", situation="The user is interested in political news in the United States.") ``` In this example, the agent David is used in two different sessions, each with a different situation. The agent's behavior and responses are tailored to the specific situation of each session, and the history of messages in `session1` and `session2` are separate. ### Users When a user (or more) is added to a session, the session will be able to access information about the user such as `name`, and `about` in order to personalize the interaction. Check out the `system_template` to see how the user's info is being accessed. This is how you can create a user and associate it with a session: ```python Python theme={"dark"} client = Julep(api_key="YOUR_API_KEY") user = client.users.create(name="John Doe", about="A 21 year old man who is a student at MIT.") agent = client.agents.create(name="Mark Lee", about="A 49 year old man who is a retired software engineer.") session = client.sessions.create(agent_id=agent.id, user_id=user.id) ``` ```javascript Node.js theme={"dark"} const client = new Julep({ apiKey: 'YOUR_API_KEY' }); const user = await client.users.create({ name: "John Doe", about: "A 21 year old man who is a student at MIT." }); const agent = await client.agents.create({ name: "Mark Lee", about: "A 49 year old man who is a retired software engineer." }); const session = await client.sessions.create({ agent_id: agent.id, user_id: user.id }); ``` In this example, the user John Doe is associated with the agent Mark Lee in the session. The session will use the user's information to personalize the interaction, such as using the user's name in the system prompt. ### Tools Sessions have the ability to use [Tools](/concepts/tools). When `auto_run_tools` is set to `true` (available in chat calls), if an agent has a tool and the LLM decides to use it, the tool will be executed automatically and the result will be sent back to the LLM for further processing. When `auto_run_tools` is `false` (default), tool calls are returned in the response without execution. Example: If the agent that's associated with the session has a tool called `fetch_weather`, and the LLM decides to use it: * With `auto_run_tools=true`: The tool executes automatically and returns weather data to the LLM * With `auto_run_tools=false`: The tool call is returned in the response for manual execution ```python Python theme={"dark"} # With automatic tool execution response = client.sessions.chat( session_id="session_id", messages=[ { "role": "user", "content": "What is the weather in San Francisco?" } ], auto_run_tools=True, # Tools execute automatically recall_tools=True # Tool calls/results included in message history ) print("Agent's response:", response.choices[0].message.content) # Agent's response: The weather in San Francisco is 70 degrees and it's sunny. Humidity is 50%, and the wind speed is around 10 mph. # Without automatic tool execution (default) response = client.sessions.chat( session_id="session_id", messages=[ { "role": "user", "content": "What is the weather in San Francisco?" } ], auto_run_tools=False # Default - returns tool calls without execution ) # Check if tool calls were made if response.choices[0].message.tool_calls: print("Tool calls:", response.choices[0].message.tool_calls) # You need to execute these tools manually and send results back ``` ```javascript Node.js theme={"dark"} // With automatic tool execution const response = await client.sessions.chat({ session_id: "session_id", messages: [ { role: "user", content: "What is the weather in San Francisco?" } ], auto_run_tools: true, // Tools execute automatically recall_tools: true // Tool calls/results included in message history }); console.log("Agent:", response.choices[0].message.content); // Agent's response: The weather in San Francisco is 70 degrees and it's sunny. Humidity is 50%, and the wind speed is around 10 mph. // Without automatic tool execution (default) const response2 = await client.sessions.chat({ session_id: "session_id", messages: [ { role: "user", content: "What is the weather in San Francisco?" } ], auto_run_tools: false // Default - returns tool calls without execution }); // Check if tool calls were made if (response2.choices[0].message.tool_calls) { console.log("Tool calls:", response2.choices[0].message.tool_calls); // You need to execute these tools manually and send results back } ``` ### Documents When chatting in a session, the session can automatically search for documents that are associated with any of the agents and/or users that participate in the session. You can control whether the session should search for documents when chatting using the `recall` option of the `chat` method, which is set to `True` by default. You can also set the session's `recall_options` when creating the session to control how the session should search for documents. ```python Python [expandable] theme={"dark"} # Create a session with custom recall options client.sessions.create( agent=agent.id, user=user.id, recall=True, recall_options={ "mode": "vector", # or "hybrid", "text" "num_search_messages": 4, # number of messages to search for documents "max_query_length": 1000, # maximum query length "alpha": 0.7, # weight to apply to BM25 vs Vector search results (ranges from 0 to 1) "confidence": 0.6, # confidence cutoff level (ranges from -1 to 1) "limit": 10, # limit of documents to return "lang": "en-US", # language to be used for text-only search "metadata_filter": {}, # metadata filter to apply to the search "mmr_strength": 0, # MMR Strength (ranges from 0 to 1) } ) # Chat in the session response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": "Tell me about Julep" } ], recall=True ) print("Agent's response:", response.choices[0].message.content) print("Searched Documents:", response.docs) ``` ```javascript Node.js [expandable] theme={"dark"} client.sessions.create({ agent: agent.id, user: user.id, recall: true, recall_options: { mode: "vector", // or "hybrid", "text" num_search_messages: 4, // number of messages to search for documents max_query_length: 1000, // maximum query length alpha: 0.7, // weight to apply to BM25 vs Vector search results (ranges from 0 to 1) confidence: 0.6, // confidence cutoff level (ranges from -1 to 1) limit: 10, // limit of documents to return lang: "en-US", // language to be used for text-only search metadata_filter: {}, // metadata filter to apply to the search mmr_strength: 0, // MMR Strength (ranges from 0 to 1) } }); // Chat in the session const response = await client.sessions.chat({ session_id: session.id, messages: [ { role: "user", content: "Tell me about Julep" } ], recall: true }); ``` When running the above code with an agent that has documents about Julep, the session will search for documents that are relevant to the conversation and return them in the `response.docs` field. ```[expandable] theme={"dark"} Agent's response: Julep is a comprehensive platform designed for creating production-ready AI systems and agents. Here are the key aspects of Julep: Core Features: 1. Complete Infrastructure Layer - Provides infrastructure between LLMs and software - Built-in support for long-term memory - Multi-step process management - State management capabilities 2. AI Agent Development - Creates persistent AI agents that remember past interactions - Supports complex task execution - Enables multi-step workflows - Includes built-in tools and integrations 3. Production-Ready Features - Automatic retries for failed steps - Message resending capabilities - Task recovery systems - Real-time monitoring - Error handling - Automatic scaling and load balancing 4. Development Approach - Uses 8-Factor Agent methodology - Treats prompts as code with proper versioning - Provides clear tool interfaces - Offers model independence to avoid vendor lock-in - Includes structured reasoning capabilities - Maintains ground truth examples for validation Available Resources: - Documentation: https://docs.julep.ai/ - API Playground: https://dev.julep.ai/api/docs - Python SDK: https://github.com/julep-ai/python-sdk/blob/main/README.md - JavaScript SDK: https://github.com/julep-ai/node-sdk/blob/main/README.md - Various use case examples and cookbooks You can explore different use cases through their cookbooks, including: - User Profiling - Email Assistant - Trip Planner - Document Management - Website Crawler - Multi-step Tasks - Advanced Chat Interactions For additional support or to learn more: - Discord Community: https://discord.gg/2EUJzJU2Yt - Book a Demo: https://calendly.com/ishita-julep - Dev Support: hey@julep.ai Searched Documents: ### Build faster using Julep API ### Deploy multi-step workflows easily with ### built-in tools and ### State management that is ### production ready from day one Keep me informed ### Build faster using Julep API ### Deploy multi-step workflows easily with built-in tools and State management that is production ready from day one Keep me informed Keep me informed Dev Resources [Docs](https://docs.julep.ai/) [Get API Key](https://dashboard-dev.julep.ai/) [API Playground](https://dev.julep.ai/api/docs) [Python SDK](https://github.com/julep-ai/python-sdk/blob/main/README.md) [Javascript SDK](https://github.com/julep-ai/node-sdk/blob/main/README.md) [Integration List](https://github.com/julep-ai/julep/tree/dev?tab=readme-ov-file#integrations) Usecases [User Profiling](https://github.com/julep-ai/julep/blob/dev/cookbooks/09-User_Management_and_Personalization.py) [Email Assistant (Mailgun)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/00-Devfest-Email-Assistant.ipynb) [Trip Planner (Weather + Wiki)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/04-TripPlanner_With_Weather_And_WikiInfo.ipynb) [Doc Management](https://github.com/julep-ai/julep/blob/dev/cookbooks/10-Document_Management_and_Search.py) [Website Crawler (Spider)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/01-Website_Crawler_using_Spider.ipynb) [Multi-step Tasks](https://github.com/julep-ai/julep/blob/dev/cookbooks/06-Designing_Multi-Step_Tasks.py) [Advanced Chat](https://github.com/julep-ai/julep/blob/dev/cookbooks/11-Advanced_Chat_Interactions.py) Company [About](./about) [Contact Us](./contact) [Book Demo](https://calendly.com/ishita-julep) Socials [Github](https://github.com/julep-ai/julep) [LinkedIn](https://www.linkedin.com/company/julep-ai) [Twitter](https://x.com/julep_ai) [Dev.to](https://dev.to/julep) [Hugging Face](https://huggingface.co/julep-ai) [Youtube](https://www.youtube.com/@julep_ai) Built by Engineers, for Engineers ©Julep AI Inc 2024 Dev Resources [Docs](https://docs.julep.ai/) [Get API Key](https://dashboard-dev.julep.ai/) [API Playground](https://dev.julep.ai/api/docs) [Python SDK](https://github.com/julep-ai/python-sdk/blob/main/README.md) [Javascript SDK](https://github.com/julep-ai/node-sdk/blob/main/README.md) [Integration List](https://github.com/julep-ai/julep/tree/dev?tab=readme-ov-file#integrations) Usecases [User Profiling](https://github.com/julep-ai/julep/blob/dev/cookbooks/09-User_Management_and_Personalization.py) [Email Assistant (Mailgun)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/00-Devfest-Email-Assistant.ipynb) [Trip Planner (Weather + Wiki)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/04-TripPlanner_With_Weather_And_WikiInfo.ipynb) [Doc Management](https://github.com/julep-ai/julep/blob/dev/cookbooks/10-Document_Management_and_Search.py) [Website Crawler (Spider)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/01-Website_Crawler_using_Spider.ipynb) [Multi-step Tasks](https://github.com/julep-ai/julep/blob/dev/cookbooks/06-Designing_Multi-Step_Tasks.py) [Advanced Chat](https://github.com/julep-ai/julep/blob/dev/cookbooks/11-Advanced_Chat_Interactions.py) Company [About](./about) [Contact Us](./contact) [Book Demo](https://calendly.com/ishita-julep) Socials [Github](https://github.com/julep-ai/julep) [LinkedIn](https://www.linkedin.com/company/julep-ai) [Twitter](https://x.com/julep_ai) [Dev.to](https://dev.to/julep) [Hugging Face](https://huggingface.co/julep-ai) [Youtube](https://www.youtube.com/@julep_ai) Built by Engineers, for Engineers ©Julep AI Inc 2024 Dev Resources [Docs](https://docs.julep.ai/) [Get API Key](https://dashboard-dev.julep.ai/) [API Playground](https://dev.julep.ai/api/docs) [Python SDK](https://github.com/julep-ai/python-sdk/blob/main/README.md) [Javascript SDK](https://github.com/julep-ai/node-sdk/blob/main/README.md) [Integration List](https://github.com/julep-ai/julep/tree/dev?tab=readme-ov-file#integrations) Usecases [User Profiling](https://github.com/julep-ai/julep/blob/dev/cookbooks/09-User_Management_and_Personalization.py) [Email Assistant (Mailgun)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/00-Devfest-Email-Assistant.ipynb) [Trip Planner (Weather + Wiki)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/04-TripPlanner_With_Weather_And_WikiInfo.ipynb) [Doc Management](https://github.com/julep-ai/julep/blob/dev/cookbooks/10-Document_Management_and_Search.py) [Website Crawler (Spider)](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/01-Website_Crawler_using_Spider.ipynb) [Multi-step Tasks](https://github.com/julep-ai/julep/blob/dev/cookbooks/06-Designing_Multi-Step_Tasks.py) [Advanced Chat](https://github.com/julep-ai/julep/blob/dev/cookbooks/11-Advanced_Chat_Interactions.py) Company [About](./about) [Contact Us](./contact) [Book Demo](https://calendly.com/ishita-julep) Socials [Github](https://github.com/julep-ai/julep) [LinkedIn](https://www.linkedin.com/company/julep-ai) [Twitter](https://x.com/julep_ai) [Dev.to](https://dev.to/julep) [Hugging Face](https://huggingface.co/julep-ai) [Youtube](https://www.youtube.com/@julep_ai) Built by Engineers, for Engineers ©Julep AI Inc 2024 This chunk provides an overview of resources and support options offered by Julep AI, including documentation, development tools, community links, and contact information for booking demos or reaching out for support. It highlights features such as the Julep API, multi-step workflow deployment, and various use cases relevant for developers. ``` This example is taken from the `crawling-and-rag` cookbook. Check it out [here](https://github.com/julep-ai/julep/blob/dev/cookbooks/advanced/10-crawling-and-rag.ipynb). ## Best Practices
  • **1. Reuse Sessions**: Reuse existing sessions for returning users to maintain continuity in interactions.
  • **2. Session Cleanup**: Regularly clean up inactive sessions to manage resources efficiently.
  • **3. Context Overflow Strategy**: Choose an appropriate context overflow strategy (e.g., "adaptive") to handle long conversations without losing important information.
  • **1. Leverage Metadata**: Use session metadata to store and retrieve user preferences, enhancing personalized interactions.
  • **2. Maintain Context**: Ensure that the context within sessions is updated and relevant to provide coherent and context-aware responses.
  • **1. Efficient Searches**: Optimize search queries within sessions to retrieve relevant documents quickly.
  • **2. Manage Token Usage**: Monitor and manage token usage to ensure efficient use of resources, especially in long sessions.
 ## Next Steps * [Agent Tools](/concepts/tools) - Learn about tools and how to use them with agents * [Agent Tasks](/concepts/tasks) - Learn about tasks and how to use them with agents * [Agent Docs](/concepts/docs) - Learn about docs and how to use them with agents # Tasks Source: https://docs.julep.ai/concepts/tasks Understanding Julep Tasks and workflows ## Overview Tasks are GitHub Actions-style workflows that define multi-step actions in Julep. Think of them as recipes that tell an agent exactly how to accomplish a goal. For example, a task might outline the steps to "Summarize a Research Paper" or "Debug a Code Issue".