You can also do the recoloring via SVG filters: https://observablehq.com/@mootari/svg-filter-scales

For canvas the way I do it is to create a uint32 lookup table for all 256 grey values. Since you can have a Uint32Array view of a Uint8ClampedArray, you can simply assign entire colors (instead of RGB values).

(Here's a small helper to create a palette array from CSS colors: https://observablehq.com/@mootari/sibson-interpolation#createPalette)