DUST

Core Blocks

An overview of Dust's core blocks. You'll learn about their functionality and usage.

Dust apps are composed of Blocks which are executed sequentially. Each block produces outputs and can refer to the outputs of previously executed blocks. We invite you to review the sections on Inputs and Execution to better understand the execution logic of a Dust app.

For each block we describe its specification parameters and configuration parameters. Please refer to the Blocks section for more details on the difference between the two.

Input block

The input block is the entry point to a Dust app and receives the arguments on which the app is executed, in the form of JSON objects. During the design phase of the app it is associated to a Dataset of examples used to iterate on the app. When deployed by API, the input block receives the arguments passed to the app by API.

Refer to the Inputs section for more details on how the input block forks the execution of the app in independent execution streams. When run by API, the dataset to which the input block is associated is ignored and replaced by the arguments passed by API.

Specification

  • Name
    dataset
    Type
    dataset
    Description

    The dataset of examples to be executed by the app during the design phase. Ignored and replaced by the arguments passed when run by API.

Data block

The data block outputs the dataset it is associated with. Unlike the input block it does not fork the execution of the app and simply returns the entire dataset as an array of JSON objects.

The data block is generally used in conjunction with an llm block to output a dataset of few-shot examples to prompt models.

Specification

  • Name
    dataset
    Type
    dataset
    Description

    The dataset of examples to be returned by the block as array.

Code block

The code block executes the Javascript code provided by the user. The code must define a function _fun which takes as input an env variable. _fun is executed as the block is run by the app. code blocks are generally used to glue other blocks together by postprocessing previous blocks outputs.

Specification

  • Name
    code
    Type
    javascript
    Description

    The code to be executed. Exposed as a function named _fun taking an env variable as argument.

Properties of the env variable

  • Name
    state
    Type
    object
    Description

    An object whose fields are the name of previously executed blocks and values are the output of the associated blocks. The output of a previously executed EXAMPLE block can be accessed as env.state.EXAMPLE.

  • Name
    input
    Type
    object
    Description

    An object with fields input and index. input is the input object of the current execution stream, null if there is no input block executed yet. index is the index of the current input object in the dataset associated with the input block.

  • Name
    map
    Type
    optional object
    Description

    An optional object set only in the context of a block being executed as part of a map reduce pair. If set, contains a field name set to the name of the current map block and a field iteration which is the index of the element being executed as part of the map reduce.

  • Name
    config
    Type
    object
    Description

    The configuration with which the app is currently run. An object whose keys are block names and values are the associated configuration values.

LLM block

The llm block provides a standardized interface to large language models from multiple providers (OpenAI, Cohere, AI21, ...). It also provides automatic caching mechanisms and a rich templating language (based on Tera, similar go Jinja2) to construct prompts.

It is used to issue completion requests to large language models as part of an app execution.

Specification

  • Name
    prompt
    Type
    string
    Description

    The prompt to generate completions. The prompt can be templated using the Tera templating language (see below for more details on templating).

  • Name
    temperature
    Type
    float
    Description

    The temperature to use when sampling from the model. A higher temperature (eg 1.0) results in more diverse completions, while a lower temperature (eg 0.0) results in more conservative completions.

  • Name
    max_tokens
    Type
    integer
    Description

    The maximum number of tokens to generate from the model. It can be very broadly seen as a word worth of content. The model may decide to stop generation before reaching max_tokens. For OpenAI models, we support -1 as a special value, specifying that we want the model to generate as many tokens as possible given its context size. Using -1 with a model of context size 2048 with a prompt consuming 1024 tokens, is equivalent to specifying a max_tokens of 1024.

  • Name
    stop
    Type
    []string
    Description

    An array of strings that should interrupt completion when sampled from the model.

  • Name
    frequency_penalty
    Type
    float
    Description

    Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

  • Name
    presence_penalty
    Type
    float
    Description

    Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

  • Name
    top_p
    Type
    float
    Description

    An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

  • Name
    top_logprobs
    Type
    integer
    Description

    Include the log probabilities on the top_logprobs most likely tokens, as well as the chosen token. For example, if logprobs is 5, the API will return a list of the 5 most likely tokens' probabilities. Useful for classification tasks.

Configuration

  • Name
    provider_id
    Type
    string
    Description

    The model provider to use. One of openai, cohere, ai21.

  • Name
    model_id
    Type
    string
    Description

    The model to use from the provider specified by provider_id.

  • Name
    use_cache
    Type
    bool
    Description

    Whether to rely on the automated caching mechanism of the llm block. If set to true and a previous request was made with the same specification and configuration parameters, then the cached response is returned.

  • Name
    use_stream
    Type
    bool
    Description

    In the context of running an app by API. Whether to stream each token as they are emitted by the model. Currently only supported for provider_id set to openai.

Prompt templating

The prompt field of the llm block can be templated using the Tera. This is particularly useful to construct few-shot prompts for models. Here's an example of templated model prompt:

{% for e in EXAMPLES %}
EN: {{e.english}}
FR: {{e.french}}
{% endfor %}
EN: {{INPUT.english}}
FR:

In this example, for each object in the output of the EXAMPLES block (which is an array of JSON objects), create a line with the English sentence and French translation. Then, add a final line with the English sentence from the INPUT block output english field. A fully functional example app relying on this prompt can be found here.

The Tera templating language supports a variety of constructs including loops and variable replacement. Please refer to the Tera documentation for more details.

Support for chat-based models (chatGPT)

We also support chat-based models such as OpenAI's gpt-3.5-turbo (Chat API) and gpt-4. When these models are used, as part of the llm block, max_tokens and top_logprobs are ignored, and the content of the templated prompt is passed as initial message with user role. The block response is the content of the assistant role message returned by the model. Note that logprobs and tokens are not available with these models.

Chat block

The chat block provides a standardized interface to chat-based large language models such as OpenAI's gpt-3.5-turbo (Chat API) and gpt-4. It is similar in many respects to an LLM block, it has templated (see LLM block above) intstructions that serve as initial prompt and exposes a messages code block to output previous messages (generally passed to the app as input)

It is used to build chat-based experiences where the model is exposed with previous interactions and generates a new message in response.

Specification

  • Name
    instructions
    Type
    string
    Description

    The instructions are passed to the model as initial system role message (see OpenAI's Chat guide). The instructions can be templated using the Tera templating language (see above for more details on templating).

  • Name
    messages_code
    Type
    javascript
    Description

    A Javascript function _fun which takes a single argument env (see the Code block documentation for details) and returns an array (possibly empty) of messages (but you generally want at least one user role message). Messages should be objects with two fields: role and content. Values should be string. Possible values for role are user, assistant and system (see OpenAI's Chat guide).

  • Name
    temperature
    Type
    float
    Description

    The temperature to use when sampling from the model. A higher temperature (eg 1.0) results in more diverse completions, while a lower temperature (eg 0.0) results in more conservative completions.

  • Name
    stop
    Type
    []string
    Description

    An array of strings that should interrupt completion when sampled from the model.

  • Name
    max_tokens
    Type
    integer
    Description

    The maximum number of tokens to generate from the model for the next message. It can be very broadly seen as a word worth of content. The model may decide to stop generation before reaching max_tokens.

  • Name
    frequency_penalty
    Type
    float
    Description

    Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

  • Name
    presence_penalty
    Type
    float
    Description

    Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

  • Name
    top_p
    Type
    float
    Description

    An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

Configuration

  • Name
    provider_id
    Type
    string
    Description

    The model provider to use. One of openai, cohere, ai21.

  • Name
    model_id
    Type
    string
    Description

    The model to use from the provider specified by provider_id.

  • Name
    use_cache
    Type
    bool
    Description

    Whether to rely on the automated caching mechanism of the llm block. If set to true and a previous request was made with the same specification and configuration parameters, then the cached response is returned.

  • Name
    use_stream
    Type
    bool
    Description

    In the context of running an app by API. Whether to stream each token as they are emitted by the model. Currently only supported for provider_id set to openai.

Map Reduce blocks

The map and reduce blocks are used to execute the set of blocks in between them on each elements of an array, in parallel. The map block takes a block name as from specification argument. If the output of the block referred to is an array, the execution stream will fork on each element of the array. If the output is an object you can use the repeat specification argument to map on the same element repeat times.

The reduce block does not take any argument and has the same name as its associated map block. After executing a reduce block, each output of the blocks in between the map and reduce blocks will be collected in an array accessible from any subsequent block.

Assume we have a MAPREDUCE map block whose from specification argument points to a block whose output is an array of length 4. Assume it is followed by a DUMMY code block and an associated MAPREDUCE reduce block, and finally a FINAL code block. Also assume that the output of the DUMMY block is a simple { "foo": "bar" } object. The DUMMY code block will be executed 4 times in parallel, and as we execute the FINAL block, env.state.DUMMY will contain an array of 4 { "foo": "bar" } objects.

Configuration

  • Name
    from
    Type
    string
    Description

    The name of the block's output to map on. If the output is an array, the execution stream will fork on each element of the array. If the output is an object you the repeat specification argument must be specified.

  • Name
    repeat
    Type
    integer
    Description

    Only valid if the output of the from block is an object. The number of times to repeat the execution of the blocks in between the map and reduce blocks on the same object.

DataSource block

The data_source block provides an interface to query a data source and return chunks that are semantically similar to the query provided. When executed, the query is embedded using the embedding model set on the data source and the resulting enmbedding vector is used to perform semantic search on the data source's chunks.

The output of the data_source block is a list of Document objects. Each document may include one or more Chunks (only those that were returned from the search). The documents are sorted by decreasing order of the max of their retrieved chunks score. Please refer to the data sources overview for more details about how documents are chunked to enable semantic search.

Specification

  • Name
    query
    Type
    string
    Description

    The query to embed and use to perform semantic seach against the data source. The query can be templated using the Tera templating language (see the LLM block documentation for more details on templating).

  • Name
    top_k
    Type
    integer
    Description

    The number of chunks to return. The resulting number of document objects may be smaller if multiple chunks from the same document are among the top_k retrieved chunks.

Configuration

  • Name
    project_id
    Type
    string
    Description

    The internal project_id associated with the data source. Dust's interface presents a data source picker that lets you select your own or other users' data sources without having to explicitly know the associated project_id.

  • Name
    data_source_id
    Type
    string
    Description

    The name of the data source to use in the project_id above.