Learn how to use Python expressions in Julep task definitions
_
Variable_
is a special variable that serves three different purposes depending on where itβs used:
_
contains the execution input:_
contains the output from the previous step:foreach
and map
steps, _
represents the current item being iterated.$
Variable$
variable is used to differentiate between a Python expression and a string. When using the $
variable, the expression is evaluated as a Python expression.
$
variable, please refer to the New Syntax section.evaluate
stepsarguments
if
conditionsforeach
and map
iterationsabs
, all
, any
, bool
, dict
, enumerate
float
, int
, len
, list
, map
, max
, min
round
, set
, str
, sum
, tuple
, zip
, reduce
range
: def safe_range(*args)
Safely creates a range object with size limits (max 1,000,000 elements).
load_json
: def safe_json_loads(s: str) -> Any
(Deprecated in favor of json.loads
)
Safely parses a JSON string with size limits.
load_yaml
: def safe_yaml_load(s: str) -> Any
(Deprecated in favor of yaml.safe_load
)
Safely parses a YAML string with size limits.
dump_json
: def dump_json(obj: Any, *, **kwargs) -> str
(Deprecated in favor of json.dumps
)
Safely serializes an object to a JSON string.
dump_yaml
: def dump_yaml(obj: Any, **kwargs) -> str
(Deprecated in favor of yaml.dump
)
Safely serializes an object to a YAML string.
extract_json
: def safe_extract_json(string: str) -> Any
Safely extracts and parses JSON from text.
search_regex
: def search_regex(pattern: str, string: str) -> Optional[re2.Match]
Searches for a regex pattern in a string.
match_regex
: def match_regex(pattern: str, string: str) -> bool
Checks if a regex pattern matches a string.
chunk_doc
: def chunk_doc(string: str) -> list[str]
Chunks a string into sentences.
nlp
pipelines.
reader
: def reader(data: str, dialect="excel", delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting=0, strict: bool = False) -> csv._reader
Creates a CSV reader.
writer
: def writer(data: str, dialect="excel", delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting=0, strict: bool = False) -> csv._writer
Creates a CSV writer.
DictReader
: class DictReader(data: str, fieldnames=None, restkey=None, restval=None, dialect="excel", *args, **kwds)
Create an object that operates like a regular reader but maps the information in each row to a dict.
DictWriter
: class DictWriter(data: str, fieldnames, restval="", extrasaction="raise", dialect="excel", *args, **kwds)
Create an object which operates like a regular writer but maps dictionaries onto output rows.
register_dialect
: def register_dialect(name: str, dialect: type[Dialect] = ..., *, delimiter: str = ",", quotechar: str | None = '"', escapechar: str | None = None, doublequote: bool = True, skipinitialspace: bool = False, lineterminator: str = "\r\n", quoting: _QuotingType = 0, strict: bool = False) -> csv._reader
Associate dialect with name.
unregister_dialect
: def unregister_dialect(name: str) -> None
Delete the dialect associated with name from the dialect registry.
get_dialect
: def get_dialect(name: str) -> Dialect
Return the dialect associated with name.
list_dialects
: def list_dialects() -> list[str]
Return the names of all registered dialects.
field_size_limit
: def field_size_limit(new_limit: int = ...) -> int
Returns the current maximum field size allowed by the parser.
Dialect
: class Dialect()
The Dialect class is a container class whose attributes contain information for how to handle doublequotes, whitespace, delimiters, etc.
excel
: class excel()
The excel class defines the usual properties of an Excel-generated CSV file.
excel_tab
: class excel_tab()
The excel_tab class defines the usual properties of an Excel-generated TAB-delimited file.
unix_dialect
: class unix_dialect()
The unix_dialect class defines the usual properties of a CSV file generated on UNIX systems, i.e. using β\nβ as line terminator and quoting all fields.
Sniffer
: class Sniffer
The Sniffer class is used to deduce the format of a CSV file.
QUOTE_ALL
- Instructs writer objects to quote all fields.
QUOTE_MINIMAL
- Instructs writer objects to only quote those fields which contain special characters such as delimiter, quotechar or any of the characters in lineterminator.
QUOTE_NONNUMERIC
- Instructs writer objects to quote all non-numeric fields. Instructs reader objects to convert all non-quoted fields to type float.
QUOTE_NONE
- Instructs writer objects to never quote fields.
QUOTE_NOTNULL
- Instructs writer objects to quote all fields which are not None.
QUOTE_STRINGS
- Instructs writer objects to always place quotes around fields which are strings.
Error
: class Error()
Raised by any of the functions when an error is detected.
csv
module from the standard Python library. You can find additional information here
re
: Regular expressions (using re2)json
: JSON operationsyaml
: YAML operationsstring
: String constants and operationsdatetime
: Date and time operationsmath
: Mathematical functionsstatistics
: Statistical operationsbase64
: Base64 encoding/decodingurllib.parse
: URL parsing operationsrandom
: Random number generationtime
: Time operationscsv
: CSV operationshumanize_text_alpha
function transforms text using multiple techniques such as:
humanize_text_alpha(text: str, threshold: float = 90, src_lang: str = "english", target_lang: str = "german", use_homoglyphs: bool = True, use_em_dashes: bool = True, grammar_check: bool = False, max_tries: int = 10) -> str
Parameters:
text
: The text to be humanized.threshold
: The threshold for the humanization (0-100).src_lang
: The source language of the original text.target_lang
: The target language used in the back-translation technique.use_homoglyphs
: Whether to use homoglyphs, a technique that replaces certain characters with similar looking characters to trick AI detection classifiers.use_em_dashes
: Whether to use em dashes, i.e. adding dashes between longer words to break the tokens.grammar_check
: Whether to run another prompt after back-translation to check for correct grammar.max_tries
: The maximum number of tries to humanize the text.markdown_to_html
function converts markdown text to HTML.
Breakdown of the function:
Function signature:
markdown_to_html(markdown_text: str) -> str
Parameters:
markdown_text
: The markdown text to be converted to HTML.html_to_markdown
function converts HTML text to markdown.
Breakdown of the function:
Function signature:
html_to_markdown(html_text: str) -> str
Parameters:
html_text
: The HTML text to be converted to markdown.Limited available functions to prevent unsafe operations
Maximum string length restrictions to prevent memory issues
Collection size limits to prevent resource exhaustion
Execution time limits to prevent infinite loops