Dominik Moritz

This would be awesome if it dynamically loaded experiments from 3rd party notebooks (e.g. https://observablehq.com/@tomlarkworthy/svg-to-gif)

Tom Larkworthy

I would like to use your benchmar utils to replace my bad ad-hoc solution here https://observablehq.com/d/507ab3f69d80ffb9?ui=classic

However, since it's a sorting benchmark I need to randomize the input before each run. I can imagine other types of benchmarks where preparation, or clean-up after the run, is necessary. How about adding support for optional before/after properties to the experiment objects? They could be callbacks (like run is now) that lets us prepare data or maybe do clean-up afterwards (I don't need the latter but I can imagine there being use-cases for it)

Job van der Zwan

(I'll try quickly adding them myself, doesn't sound too complicated)

I've made a few changes in https://next.observablehq.com/@fil/fcumsum-benchmark :
- show intermediate results
- add the ratio (compared to the best)

Feel free to merge some of them if you like.

That was fast. Awesome! I will merge them back.

Excellent, thanks! I've merged back (note: I put the warmup # below the # of trials, and fixed a typo)

I moved the utils into a new notebook so that the template is lighter. I also fixed a few bugs I found along the way.

We should fix the table's columns widths otherwise the display tends to be jaggy