Standard library
Many capabilities that you will find useful are provided in the Observable standard library. This notebook describes those capabilities in brief, and provides links when there are notebooks that go into greater detail. (Older functionality is listed in Deprecated / No Longer Recommended.)
currentUser
The currentUser object is a special addition to the standard library, only available within private notebooks in a workspace. currentUser provides information about the team member currently viewing the page:
DatabaseClient
For most uses, the Data table cell is preferable to creating a database client programmatically. Their results can be accessed as variables just like any other cell.
To programmatically create a database client for a configured database, use DatabaseClient:
client = DatabaseClient("Baby Names")This returns a promise to the client for the database with the specified name. You then can use a query method that returns a promise to the array of results:
names = client.query(
`SELECT name, gender, year, SUM(number) AS number
FROM names
WHERE year > ?
GROUP BY name, gender, year`,
[1920]
)The database client has the following methods available:
| Name | Description |
|---|---|
| databaseClient.query() | Run the specified SQL query, returning a promise to the array of results. |
| databaseClient.queryRow() | Similar to databaseClient.query but returns a promise to a single row of results. |
| client.sql`` | Similar to query(), but allows easy interpolation of variables into the query string |
| databaseClient.explain() | Explains the query plan for the specified query, returning a promise to an HTML element. (PostgreSQL databases only.) |
| databaseClient.describe() | Describes the schema for the table with the specified name, returning a promise to an HTML table. |
client.query(query[, parameters])
Run the specified SQL query, returning a promise to the array of results. (You can either explicitly await the promise as part of a larger expression, or rely on Observable's implicit cell-level await.)
If parameters is specified, it is an array of values to bind to query parameters. The parameter syntax depends on the database type:
- BigQuery can use named (
@age,@name, …) or ordered (repeating?) parameters - Databricks uses numbered parameters (
:1,:2, …) - Mongo SQL uses ordered parameters (repeating `?`)
- MySQL uses ordered parameters (repeating
?) - Oracle can use named (
:age,:name, …) or ordered (repeating?) parameters - PostgreSQL uses numbered parameters (
$1,$2, …) - Snowflake uses numbered parameters (
:1,:2, …) - SQL Server can use named (
@age,@name, …) parameters
Please refer to the respective documentation:
client.queryRow(query[, parameters])
An alternative to client.query that returns a single row instead of an array of results. For example:
client.queryRow(
`SELECT MIN(year) AS year
FROM names
WHERE name = ?`,
["Loki"]
)Returns Object {year: 2007}
client.sql`query`
Run the specified SQL query returning a promise to the array of results. Interpolated expressions are treated as query parameters. (Like with client.query, you can either explicitly await the promise as part of a larger expression, or rely on Observable’s implicit cell-level await.)
year = 1950client.sql`SELECT name, gender, year, SUM(number) AS number
FROM names
WHERE year > ${year}
GROUP BY name, gender, year`client.explain(query[, parameters])
Explains the query plan for the specified query, returning a promise to an HTML element. Currently this is only supported for PostgreSQL databases.
client.explain(
`SELECT MIN(year) AS year
FROM names
WHERE name = ?`,
["Loki"]
)returns SEARCH names.
client.describe([name])
Describes the schema for the table with the specified name, returning a promise to an HTML table. The exact output depends on the database type.
In addition to the returned HTML table, you can access the table schema programmatically as table.value. The returned array contains an object for each of the table's columns.
If you don't specify a table name, client.describe instead describes the available tables in the database.
DOM
The following Document Object Model methods are supported.
DOM.context2d(width, height[, dpi]))
Returns a new canvas context with the specified width and height and the specified device pixel ratio dpi. If dpi is not specified, it defaults to window.devicePixelRatio. To access the context's canvas, use context.canvas. For example, to create a 960x500 canvas:
{
const context = DOM.context2d(960, 500);
return context.canvas;
}To expand the DOM, click the black triangle next to "Object":
Another example: the following code creates a small canvas and adds text:
DOM.context2d is helpful if you are using 2D Canvas (rather than WebGL), because of automatic pixel density scaling.
DOM.uid([name])
Returns a new unique identifier. If name is specified, the identifier.id will be derived from the specified name, which may be useful for debugging. If DOM.uid is called repeatedly with the same name, every returned identifier is still unique (that is, different).
Identifiers are useful in SVG:
- Use identifier.
hreffor IRI references, such as the xlink:href attribute. - Use identifier.
toStringfor functional notation, such as the clip-path presentation attribute.
For example, to clip an attached image (image@1.png) to a circle of radius 320 px:
{
const clip = DOM.uid("clip");
return svg`<svg width="128" height="128" viewBox="0 0 640 640">
<defs>
<clipPath id="${clip.id}">
<circle cx="320" cy="320" r="320"></circle>
</clipPath>
</defs>
<image
clip-path="${clip}"
width="640" height="640"
preserveAspectRatio="xMidYMin slice"
xlink:href="${await FileAttachment("image@1.png").url()}"
></image>
</svg>`;
}Using DOM.uid is strongly recommended over hand-coding as it ensures that your identifiers are still unique if your code is imported into another notebook. Because identifier.href and identifier.toString return absolute rather than local IRIs, it also works well in conjunction with a notebook’s base URL.
FileAttachment
To read in local files, use one of the FileAttachment methods below or a FileReader. See File Attachments for examples and more explanation.
fileAttachment.arrayBuffer()
Returns a promise to the file's contents as an ArrayBuffer.
const city = shapefile.read(await FileAttachment("sf.shp").arrayBuffer());fileAttachment.blob()
Returns a promise to a Blob containing the raw content of the file.
const blob = await FileAttachment("binary-data.dat").blob();fileAttachment.csv({array = false, typed = false} = {})
Returns a promise to the file’s contents, parsed as comma-separated values (CSV) into an array.
const data = await FileAttachment("cars.csv").csv();If array is true, an array of arrays is returned; otherwise, the first row is assumed to be the header row and an array of objects is returned, and the returned array has a data.columns property that is an array of column names. (See d3.csvParseRows.) If typed is true, automatic type inference is applied; only use this feature if you know your data is compatible.
fileAttachment.image()
Returns a promise to a file loaded as an Image. The promise resolves when the image has finished loading, making this useful for reading the image pixels in Canvas, or for loading the image into a WebGL texture. Consider fileAttachment.url if you want to embed an image in HTML or Markdown.
const image = await FileAttachment("sunset.jpg").image();fileAttachment.json()
Returns a promise to the file's contents, parsed as JSON into JavaScript values.
const logs = await FileAttachment("weekend-logs.json").json();fileAttachment.stream()
Returns a promise to a ReadableStream of the file's contents.
const stream = await FileAttachment("metrics.csv").stream();
const reader = stream.getReader();
let done, value;
while (({done, value} = await reader.read()), !done) {
yield value;
}fileAttachment.text()
Returns a promise to the file's contents as a JavaScript string.
const hello = await FileAttachment("hello.txt").text();fileAttachment.tsv({array = false, typed = false} = {})
Returns a promise to the file’s contents, parsed as tab-separated values (CSV) into an array.
const data = await FileAttachment("cars.tsv").tsv();If array is true, an array of arrays is returned; otherwise, the first row is assumed to be the header row and an array of objects is returned, and the returned array has a data.columns property that is an array of column names. (See d3.csvParseRows.) If typed is true, automatic type inference is applied; only use this feature if you know your data is compatible.
fileAttachment.url()
Returns a promise to the URL at which the file may be retrieved.
const url = await FileAttachment("file.txt").url();fileAttachment.xlsx()
Returns an array of sheetnames from a given XLSX file.
workbook = FileAttachment("Laser_Report_2020.xlsx").xlsx()See FileAttachment.xlsx for examples and more explanation.
fileAttachment.zip()
Returns an array of filenames from a given ZIP archive.
dogZip = FileAttachment("Dog_Photos.zip").zip()See FileAttachment.zip for examples and more explanation.
Note that the Files.text(), Files.url(), and Files.buffer() methods in Reading Local Files are deprecated now.
Generators
Use the methods in this section to create generators. (See Introduction to Generators for an explanation of what generators are used for.)
Generators.input(input)
Note
Consider using the viewof operator.
Returns a new generator that yields promises to the current value of the specified input element; each promise resolves when the input element emits an event. (The promise resolves when the event is emitted, even if the value of the input is unchanged.) If the initial value of the input is not undefined, the returned generator’s first yielded value is a resolved promise with the initial value of the input.
The type of event that triggers promise resolution depends on the input type as follows:
- For button, submit, and checkbox inputs, click events.
- For file inputs, change events.
- For all other types, input events.
The resolved value is likewise dependent on the input.type as follows:
- For range and number inputs, input.
valueAsNumber. - For date inputs, input.
valueAsDate. - For checkbox inputs, input.
checked. - For single-file inputs (input.
multipleis falsey), input.files[0]. - For multi-file inputs (input.
multipleis truthy), input.files. - For all other types, input.
value.
The specified input need not be an HTMLInputElement, but it must support the target.addEventListener and target.remoteEventListener methods of the EventTarget interface.
Generators.input is used by Observable’s viewof operator to define the current value of a view, and is based on Generators.observe. Usually you do not use Generators.input directly, but it can be used to define a generator cell exposing the current value of an input, and you can also read the yielded values by hand. For example, to accumulate the first four values:
{
const values = [];
for (const value of Generators.input(element)) {
if (values.push(await value) >= 4) {
return values;
}
}
}Generators.input is lossy and may skip values: if more than one event is emitted before the next promise is pulled from the generator (more than once per animation frame), then the next promise returned by the generator will be resolved with the latest input value, potentially skipping intermediate values. See Generators.queue for a non-debouncing generator.
Generators.observe(initialize)
Returns a generator that yields promises to an observable value, adapting a push-based data source (such as an Observable, an EventEmitter or an EventTarget) to a pull-based one.
The specified initialize function is invoked before Generators.observe returns, being passed a change function; calling change triggers the resolution of the current promise with the passed value. The initialize function may also return a dispose function; this function will be called when the generator is disposed. (See invalidation.)
For example, to observe the current value of a text input element, you might say:
Generators.observe(change => {
// An event listener to yield the element’s new value.
const inputted = () => change(element.value);
// Attach the event listener.
element.addEventListener("input", inputted);
// Yield the element’s initial value.
change(element.value);
// Detach the event listener when the generator is disposed.
return () => element.removeEventListener("input", inputted);
})See also Generators.input.
Generators.observe is typically used to define a generator cell, but you can also read the yielded values by hand. For example, to accumulate the first four values:
{
const generator = Generators.observe(…);
const values = [];
for (const value of generator) {
if (values.push(await value) >= 4) {
return values;
}
}
}Generators.observe is lossy and may skip values: if change is called more than once before the next promise is pulled from the generator (more than once per animation frame), then the next promise returned by the generator will be resolved with the latest value passed to change, potentially skipping intermediate values. See Generators.queue for a non-debouncing generator.
Generators.queue(initialize)
Returns a generator that yields promises to an observable value, adapting a push-based data source (such as an Observable, an EventEmitter or an EventTarget) to a pull-based one.
The specified initialize function is invoked before Generators.queue returns, being passed a change function; calling change triggers the resolution of the current promise with the passed value. The initialize function may also return a dispose function; this function will be called when the generator is disposed. (See invalidation.)
For example, to observe the current value of a text input element, you might say:
Generators.queue(change => {
// An event listener to yield the element’s new value.
const inputted = () => change(element.value);
// Attach the event listener.
element.addEventListener("input", inputted);
// Yield the element’s initial value.
change(element.value);
// Detach the event listener when the generator is disposed.
return () => element.removeEventListener("input", inputted);
})(See also Generators.input.)
Generators.queue is typically used to define a generator cell, but you can also read the yielded values by hand. For example, to accumulate the first four values:
{
const generator = Generators.queue(…);
const values = [];
for (const value of generator) {
if (values.push(await value) >= 4) {
return values;
}
}
}Generators.queue is non-lossy and, as a result, may yield “stale” values: if change is called more than once before the next promise is pulled from the generator (more than once per animation frame), the passed values are queued in order and the generator will return resolved promises until the queue is empty again. See Generators.observe for a debouncing generator.
See Introduction to Generators for examples and more explanation.
Graphviz and the dot template
Observable supports the dot template for convenient use of the Graphviz language to render graphs:
See Graphviz for more examples.
html `string`
Note
The html function is supported, but consider using the HTML cell mode available from the Add Cell menu, which allows you to type HTML code without the html keyword and without needing to enclose the text in backticks.
Use html to create an HTML element. For example, to create an H3 element whose content is "Hello, world!":
html`<h3>Hello, world!`The html function returns the HTML element represented by the specified HTML string literal. The function is intended to be used as a tagged template literal. Leading and trailing whitespace is automatically trimmed.
If the resulting HTML fragment is not a single HTML element or node, it is wrapped in a DIV element. For example, consider the following expression:
html`Hello, <b>world</b>!`This expression is equivalent to the next expression:
html`<div>Hello, <b>world</b>!</div>`If an embedded expression is a DOM element, it is embedded in generated HTML. For example, to embed TeX within HTML:
html`I like ${tex`\KaTeX`} for math.`If an embedded expression is an array, the elements of the array are embedded in the generated HTML. For example, to create a table from an array of values:
Another example:
Inputs
Observable has a wide variety of inputs, such as buttons, sliders, radio buttons, checkboxes, text entry inputs and more. These are part of the Recommended Libraries, but there is a separate page devoted solely to Observable Inputs.
invalidation
To free up resources when a cell is re-evaluated, such as cancelling timers or disposing WebGL contexts, use the `invalidation` promise. This promise resolves when the current cell is re-evaluated: when the cell’s code changes, when it is run using Shift-Enter, or when a referenced input changes. This promise is typically used to dispose of resources that were allocated by the cell. For example, to abort a fetch if the cell is invalidated:
{
const controller = new AbortController;
invalidation.then(() => controller.abort());
const response = await fetch(url, {signal: controller.signal});
return response.json();
}The invalidation promise is provided by the runtime rather than the standard library, because it resolves to a new promise each time a cell is evaluated.
See Invalidation for examples and more explanation.
md `string`
Note
The md function is supported, but consider using the Markdown cell mode available from the Add Cell menu, which allows you to type Markdown code without the md keyword and without needing to enclose the text in backticks.
The md function returns the HTML element represented by the specified Markdown string literal. For example, to create an H3 element whose content is “Hello, world!”:
md`### Hello, world!`The Markdown compiler used is Marked. Leading and trailing whitespace is automatically trimmed.
If an embedded expression is a DOM element, it is embedded in generated HTML. For example, to embed LaTeX within Markdown:
md`My *favorite* number is ${tex`\tau`}.`If an embedded expression is an array, the elements of the array are embedded in the generated HTML. The elements may either be strings, which are interpreted as Markdown, or DOM elements. For example, given the following array of data:
elements = [
{symbol: "Co", name: "Cobalt", number: 27},
{symbol: "Cu", name: "Copper", number: 29},
{symbol: "Sn", name: "Tin", number: 50},
{symbol: "Pb", name: "Lead", number: 82}
]You can then create a table from the array of data elements in Markdown:
now
A reactive variable that returns the current value of Date.now.
To display the current time in a Markdown string:
Promises
Promises.delay(duration[, value])
Returns a promise that resolves with the specified value after the specified duration in milliseconds. For example, to define a cell that increments approximately every second:
i = {
let i = 0;
yield i;
while (true) {
yield Promises.delay(1000, ++i);
}
}If you desire precise synchronization, such as a timer that ticks exactly every second, use Promises.tick instead of Promises.delay.
Promises.tick()
Returns a promise that resolves with the specified value at the next integer multiple of milliseconds since the UNIX epoch. This is much like Promises.delay, except it allows promises to be synchronized. For example, to define a cell that increments every second, on the second:
i = {
let i = 0;
yield i;
while (true) {
yield Promises.tick(1000, ++i);
}
}Or, to define a cell that is an async generator:
i = {
let i = 0;
while (true) {
yield i++;
await Promises.tick(1000);
}
}Promises.when(date[, value])
Returns a promise that resolves with the specified value at the specified date. This method relies on setTimeout, and thus the specified date must be no longer than 2,147,483,647 milliseconds (24.9 days) from now.
require
require.resolve()require.alias()
These are supported, but consider using dynamic import.
require(names...)
Returns a promise of the asynchronous module definition (AMD) with the specified names, loaded from jsDelivr. Each module name can be a package (or scoped package) name optionally followed by the at sign (@) and a semver range. For example, to load d3-array:
d3 = require("d3-array")Or, to load d3-array and d3-color and merge them into a single object:
d3 = require("d3-array", "d3-color")Or, to load d3-array 1.1x:
d3 = require("d3-array@1.1")See d3-require for more information.
require.resolve(name)
Returns a promise to the resolved URL to require the module with the specified name. For example:
require.resolve("d3-array") // "https://cdn.jsdelivr.net/npm/d3-array@2.0.3/dist/d3-array.min.js"require.alias(aliases)
Returns a require function with the specified aliases. For each key in the specified aliases object, any require of that key is substituted with the corresponding value. For example:
React = require("react@16/umd/react.production.min.js")ReactDOM = require("react-dom@16/umd/react-dom.production.min.js")Semiotic = require.alias({"react": React, "react-dom": ReactDOM})("semiotic@1")Or, equivalently:
r = require.alias({
"react": "react@16/umd/react.production.min.js",
"react-dom": "react-dom@16/umd/react-dom.production.min.js",
"semiotic": "semiotic@1"
})Then to require the libraries:
React = r("react")ReactDOM = r("react-dom")Semiotic = r("semiotic")Sample datasets
The standard library includes multiple sample datasets so that all notebooks have data available to work with. These include financial data, statistics, weather information, and scientific data. See Sample Datasets for the full list.
Secrets
Secrets are name-value pairs that you can use in your private notebooks to access private data. By specifying secrets in your Settings, you can enable your notebooks to use API keys or other sensitive data without making the API keys or sensitive data public.
See Introduction to Secrets for examples and more explanation.
SVG `string`
Returns the SVG element represented by the specified SVG string literal. This function is intended to be used as a tagged template literal. Leading and trailing whitespace is automatically trimmed. For example, to create an SVG element whose content is a circle:
svg`<svg width=16 height=16>
<circle cx=8 cy=8 r=4></circle>
</svg>`If the resulting SVG fragment is not a single SVG element, it is wrapped in a <g>/</g> element. For example, consider this expression:
svg`
<circle cx=8 cy=4 r=4></circle>
<circle cx=8 cy=8 r=4></circle>
`The previous expression is equivalent to the next expression:
svg`<g>
<circle cx=8 cy=4 r=4></circle>
<circle cx=8 cy=8 r=4></circle>
</g>`If an embedded expression is a DOM element, it is embedded in generated SVG. If an embedded expression is an array, the elements of the array are embedded in the generated SVG.
The following expression creates an SVG image with text:
svg`<svg width=${width} height=27>
<text y=22>Hello, I am an SVG image!</text>
</svg>`tex `string`
Note
The tex function is supported, but consider using the Mathematical formula available from the Add Cell menu, which allows you to type TeX code without the tex keyword and without needing to enclose the text in backticks.
tex `string` returns the HTML element represented by the specified LaTeX string literal. Implemented by KaTeX.
Below are two examples:
tex.block `string`
Equivalent to tex, but uses KaTeX's display mode to produce a bigger block element rather than a smaller inline element.
tex.block`E = mc^2`tex.options(options)
Returns a function equivalent to tex, but with the specified options.
tex.options({displayMode: true})`E = mc^2`visibility
Visibility is a special kind of promise that resolves when a cell becomes visible. If, for example, you want animation to run only after a cell has scrolled into view, you can do that using visibility().
The visibility function is provided by the runtime rather than the standard library because it resolves to a different function for each cell. An example of using visibility as part of a fadeIn function follows, along with an example of the effect:
function fadeIn(element, visibility) {
element.style.color = "transparent";
element.style.willChange = "color";
element.style.transition = "color 2.5s linear";
visibility().then(() => (element.style.color = "inherit"));
return element;
}The following text will fade in when the page is first scrolled:
fadeIn(md`This text will fade in.`, visibility)See Awaiting visibility for more examples and explanation.
width
A reactive variable that returns the current page width.
For example, to make a rounded rectangle in SVG that resizes to fit the page:
html`<svg width=${width} height=200>
<rect width=${width} height=200 rx=10 ry=10></rect>
</svg>`Recommended libraries
Observable also includes and recommends the following additional libraries (some are third-party).
| Symbol | Name | Version |
|---|---|---|
| _ | Lodash | 4.17.21 |
| aq | Arquero | 4.8.8 |
| Arrow | Apache Arrow | 4.0.1 |
| d3 | D3.js | 7.8.5 |
| dot | Graphviz | 0.2.1 |
| htl | Hypertext Literal | 0.3.1 |
| Inputs | Observable Inputs | 0.10.6 |
| L | Leaflet | 1.9.3 |
| mermaid | Mermaid | 9.1.6 |
| Plot | Observable Plot | 0.6.11 |
| SQLite | SQL.js | 1.7.0 |
| topojson | TopoJSON Client | 3.1.0 |
| vl | Vega, Vega-Lite | 5.22.1, 5.2.0 |
See Recommended libraries for details about each individual library.
Deprecated / No longer recommended
The following methods are deprecated and are no longer recommended. Consider using the suggested substitutions.
DOM methods that have been deprecated
| Name | Suggested substitution |
|---|---|
| DOM.canvas() | Deprecated; use htl.html |
| DOM.download() | Deprecated; use the cell menu or Blob |
| DOM.element() | Deprecated; use htl.html |
| DOM.input() | Deprecated; use htl.html or Inputs |
| DOM.range() | Deprecated; use htl.html or Inputs.range |
| DOM.select() | Deprecated; use htl.html or Inputs.select |
| DOM.svg() | Deprecated; use htl.html |
| DOM.text() | Deprecated; use htl.html |
Files methods that have been deprecated
| Name | Suggested substitution |
|---|---|
| Files.buffer() | Deprecated; use FileAttachment or FileReader |
| Files.text() | Deprecated; use FileAttachment or FileReader |
| Files.url() | Deprecated; use FileAttachment or FileReader |
Generators methods that have been deprecated
| Name | Suggested substitution |
|---|---|
| Generators.disposable() | Deprecated; use invalidation |
| Generators.filter() | Deprecated; use for and yield |
| Generators.map() | Deprecated; use for and yield |
| Generators.range() | Deprecated; use for and yield |
| Generators.valueAt() | Deprecated; use for |
| Generators.worker() | Deprecated; use Worker |