Observable Notebooks
Data loaders

Data loaders are special cells that run “ahead” at build time via an interpreter, rather than “live” when you view a notebook in the browser. Data loaders are useful for preparing static data, ensuring consistency and stability, and improving performance. Think of data loaders as a generalization of database connectors that allow languages besides SQL.

Notebooks currently support Node.js and Python data loaders. We will likely add additional interpreters in the future.

As an example, here is a trivial Python cell that says hello, and reports the current version of Python:

import platform

print(f"Hello from Python {platform.python_version()}!", end="")

The Python cell above uses the text format, and hence its value is displayed as a string. The output is given the name hello, allowing it to be referenced in JavaScript:

hello.toUpperCase()

A variety of formats are supported, including these text-based formats:

  • text - a string
  • json - JSON
  • csv - comma-separated values
  • tsv - tab-separated values
  • xml - XML

And these binary formats:

  • arrow - Apache Arrow IPC
  • parquet - Apache Parquet
  • blob - binary data as a Blob
  • buffer - binary data as an ArrayBuffer

You can also generate images in jpeg, gif, webp, png, and svg format. And you can server-side render HTML content using the html format.

As a more realistic example, below is a Node.js data loader cell that fetches download statistics for Observable Plot from npm.

async function getNpmDownloads(name, {end, start}) {
  const data = [];
  for (let d1 = end, d2; d1 > start; ) {
    d2 = d1;
    d1 = addDate(d1, -365); // fetch a year at a time
    if (d1 < start) d1 = start;
    const response = await fetch(`https://api.npmjs.org/downloads/range/${formatDate(d1)}:${formatDate(addDate(d2, -1))}/${encodeURIComponent(name)}`);
    if (!response.ok) throw new Error(`fetch failed: ${response.status}`);
    const {downloads} = await response.json();
    for (const {downloads: value, day: date} of downloads.reverse()) {
      data.push({date: new Date(date), value});
    }
  }
  for (let i = data.length - 1; i >= 0; --i) {
    if (data[i].value > 0) {
      return data.slice(data[0].value > 0 ? 0 : 1, i + 1); // ignore npm reporting zero for today
    }
  }
  throw new Error("no data found");
}

function formatDate(date) {
  return date.toISOString().slice(0, 10);
}

function addDate(date, n) {
  date = new Date(+date);
  date.setDate(date.getDate() + n);
  return date;
}

process.stdout.write(
  JSON.stringify(
    await getNpmDownloads("@observablehq/plot", {
      start: new Date("2022-09-01"),
      end: new Date("2025-09-01")
    })
  )
);

The output of a data loader cell is automatically saved to a .observable/cache directory on your local file system alongside your notebooks. Data snapshots are stable — the data only updates if you re-run the data loader cell. In Observable Desktop, you can re-run a data loader cell by clicking the Play button, by hitting shift-return, or by clicking on the query age in the cell toolbar. In Notebook Kit, delete the corresponding file from the .observable/cache directory; you can also use continuous deployment, such as GitHub Actions, to refresh data automatically.

The Node.js cell above defines the downloads variable, which we use below to render an area chart with Observable Plot:

Plot.plot({
  width,
  x: {type: "utc"},
  y: {grid: true, label: "downloads"},
  marks: [
    Plot.axisY({label: "Downloads per day"}),
    Plot.areaY(downloads, {x: "date", y: "value", curve: "step"}),
    Plot.tip(downloads, Plot.pointerX({x: "date", y: "value", tip: true}))
  ]
})

Here’s a bit more about data loaders.

Node.js data loaders

Node.js data loaders require Node.js 22.12+ to be installed in one of the following locations:

  • /opt/homebrew/bin/node (Homebrew)
  • /opt/local/bin/node (MacPorts)
  • /usr/local/bin/node (official Node.js installer)
  • /usr/bin/node (operating system)

TypeScript is supported via Node.js’ built-in type stripping. Hence, no type checking is performed, and only erasable TypeScript syntax is supported (for example, enum declarations are not allowed).

To improve security, the Node.js interpreter uses process-based permissions: Node.js cells are only allowed to read files in the same directory as the notebook, with no other permissions. (We may offer a way to relax permissions in the future, but want to encourage safety; let us know if you run into issues.)

Due to the above security restrictions, if you wish to import installed packages, they must be installed within the same directory as the notebook (e.g., if your notebook is in docs, packages must be installed in docs/node_modules with a docs/package.json).

Python data loaders

Python data loaders require Python 3.12+ to be installed in one of the following locations:

  • .venv/bin/python3 (venv)
  • /opt/homebrew/bin/python3 (Homebrew)
  • /opt/local/bin/python3 (MacPorts)
  • /usr/local/bin/python3 (official Python installer)
  • /usr/bin/python3 (operating system)

If you have a virtual environment (.venv) in the same directory as the notebook, it will automatically be used. However, packages are not installed implicitly; you must install packages yourself, typically using pip. (And we recommend using pip freeze to create a requirements.txt.)

✎ Suggest changes to this page