DUST

Integration Blocks

An overview of Dust's integration blocks. You'll learn about their functionality and usage.

Integration blocks are Blocks used to integrate with external systems, mostly for retrieving data. They can be used to search the web, retrieve content or more generally make API calls.

For each block we describe its specification parameters and configuration parameters. Please refer to the Blocks section for more details on the difference between the two.

cURL block

The curl block is used to make HTTP(s) requests to external systems. As an example you can use it to call into a Replit endpoint to perform advanced computations or to retrieve data from an internal or private API.

Specification

  • Name
    method
    Type
    string
    Description

    The HTTP method to use, one of GET, POST, PUT, PATCH.

  • Name
    url
    Type
    string
    Description

    The URL to use for the request. Only http and https schemes are supported. This parameter can rely on templating to dynamically generate the URL based on previously executed blocks outputs. See the LLM block documentation for more details.

  • Name
    headers_code
    Type
    javascript
    Description

    A Javascript function _fun which takes a single argument env (see the Code block documentation for details) and returns null (no headers) or an object representing the headers to use for the request.

  • Name
    body_code
    Type
    javascript
    Description

    A Javascript function _fun which takes a single argument env (see the Code block documentation for details) and returns null (no body) or a string representing the body to use for the request.

Configuration

  • Name
    use_cache
    Type
    bool
    Description

    Whether to rely on the automated caching mechanism of the block. If set to true and a previous request was made with the same specification parameters, then the cached response is returned.

Search block

The search block is used to search the web. It is currently a wrapper around SerpApi and Serper (we plan to also add Google Programmable Search Engine). It requires setting up the SerpApi and/or Serper provider in your account's provider settings.

Specification

  • Name
    engine
    Type
    optional string
    Description

    The SerpApi engine to use, defaults to google.

  • Name
    query
    Type
    string
    Description

    The query to use for the search. This parameter can rely on templating to dynamically generate the query based on previously executed blocks outputs. See the LLM block documentation for more details.

  • Name
    num
    Type
    optional integer
    Description

    The number of results to return. Defaults to 10.

Configuration

  • Name
    provider_id
    Type
    string
    Description

    The serach provider to use. One of serpapi, serper.

  • Name
    use_cache
    Type
    bool
    Description

    Whether to rely on the automated caching mechanism of the block. If set to true and a previous search was made with the same specification parameters, then the cached response is returned.

Browser block

The browser block is used to scrape content from web pages. It is a wrapper around Browserless.io. It requires setting up the Browserless provider in your account's provider settings.

Specification

  • Name
    url
    Type
    string
    Description

    The URL to scrape. This parameter can rely on templating to dynamically generate the URL based on previously executed blocks outputs. See the LLM block documentation for more details.

  • Name
    selector
    Type
    string
    Description

    The CSS selector to use to select the content to scrape. Use body to scrape the entire page.

  • Name
    timeout
    Type
    optional integer
    Description

    The timeout in ms to use for the scrape request. Defaults to 16000. Passed to underlying Puppeteer's page.goto method. See Browserless scrape method documentation for details.

  • Name
    wait_until
    Type
    optional string
    Description

    One of load, domcontentloaded, networkidle0, networkidle2. Passed to underlying Puppeteer's page.goto method. See Browserless scrape method documentation for details.

  • Name
    wait_for
    Type
    optional string
    Description

    Wait for an event to happen to return content. Accepts one of three options: a function to be ran, a number indicating the time in ms to wait, or a valid selector to wait for. See Browserless scrape method documentation for details.

Configuration

  • Name
    use_cache
    Type
    bool
    Description

    Whether to rely on the automated caching mechanism of the block. If set to true and a previous request was made with the same specification parameters, then the cached response is returned.

  • Name
    error_as_output
    Type
    bool
    Description

    By default the block will error if the scrape request times out, interrupting the flow of execution of the app. If set to true then the block will return the error as an object and execution will continue.