User guidesDeveloper platformLatest updatesSlack Community
Developer platform

Integration Blocks

An overview of Dust's integration blocks. You'll learn about their functionality and usage.

Integration blocks are Blocks used to integrate with external systems, mostly
for retrieving data. They can be used to search the web, retrieve content or more generally make API
calls.
For each block we describe its specification parameters and configuration parameters. Please
refer to the Blocks section for more details on the difference between the two.

cURL block

The cURL block is used to make HTTP(s) requests to external systems. As an example you can use it
to call into a Replit endpoint to perform advanced computations or to retrieve
data from an internal or private API.
To avoid sharing sensitive information, we recommend using Developer secrets to store your API keys and other sensitive information.

Specification

  • method (string): The HTTP method to use, one of GET, POST, PUT, PATCH.
  • url (string): The URL to use for the request. Only http and https schemes are
    supported. This parameter can rely on templating to dynamically generate the
    URL based on previously executed blocks outputs. See the LLM
    block     documentation for more details.
  • headers_code (javascript): A Javascript function _fun which takes a single argument env (see the
    Code block documentation for details) and returns
    null (no headers) or an object representing the headers to use for the
    request.
  • body_code (javascript): A Javascript function _fun which takes a single argument env (see the
    Code block documentation for details) and returns
    null (no body) or a string representing the body to use for the request.

Configuration

  • use_cache (bool): Whether to rely on the automated caching mechanism of the block. If set to
    true and a previous request was made with the same specification
    parameters, then the cached response is returned.

Search block

The search block is used to search the web. It is currently a wrapper around
SerpApi and Serper (we plan to also add Google
Programmable Search Engine). It requires setting up the SerpApi and/or Serper provider in your
account's provider settings.

Specification

  • engine (optional string): The SerpApi engine to use, defaults to google.
  • query (string): The query to use for the search. This parameter can rely on templating to
    dynamically generate the query based on previously executed blocks outputs.
    See the LLM block documentation for more details.
  • num (optional integer): The number of results to return. Defaults to 10.

Configuration

  • provider_id (string): The search provider to use. One of serpapi, serper.
  • use_cache (bool): Whether to rely on the automated caching mechanism of the block. If set to
    true and a previous search was made with the same specification
    parameters, then the cached response is returned.

Browser block

The browser block is used to scrape content from web pages. It is a wrapper around
Browserless.io. It requires setting up the Browserless provider in your
account's provider settings.

Specification

  • url (string): The URL to scrape. This parameter can rely on templating to dynamically
    generate the URL based on previously executed blocks outputs. See the LLM
    block     documentation for more details.
  • selector (string): The CSS selector to use to select the content to scrape. Use body to
    scrape the entire page.
  • timeout (optional integer): The timeout in ms to use for the scrape request. Defaults to 16000. Passed
    to underlying Puppeteer's
    page.goto method. See
    Browserless scrape method
    documentation for details.
  • wait_until (optional string): One of load, domcontentloaded, networkidle0, networkidle2. Passed to
    underlying Puppeteer's page.goto
    method. See Browserless scrape method
    documentation for details.
  • wait_for (optional string): Wait for an event to happen to return content. Accepts one of three options:
    a function to be ran, a number indicating the time in ms to wait, or a valid
    selector to wait for. See Browserless
    scrape method documentation for
    details.

Configuration

  • use_cache (bool): Whether to rely on the automated caching mechanism of the block. If set to
    true and a previous request was made with the same specification
    parameters, then the cached response is returned.
  • error_as_output (bool): By default the block will error if the scrape request times out,
    interrupting the flow of execution of the app. If set to true then the
    block will return the error as an object and execution will continue.