Is there a reason why for some target histograms the really bright parts of the original image end up "clipping" into different colors? (it's especially visible in clouds) Not a complaint, just curious if the reason behind it would give me a better feeling for the maths involved. Is it maybe related to the use of RGB instead? Would things be different in other color spaces?

Yes, it happens because while it's a fine algorithm which can be applied to images, it's quite a bad fit for the actual use case of style transfer. It's approximate, but the task it seeks is to simply transfer pixels 1:1 to a new image, so if your target image has 50% white and 50% black and your source image is a sunset with mostly sky, it will end up thresholding the sky at some random value. This is made worse by the fact that the algorithm uses no information about the spatial distribution of values. Theo Honohan referred us to a paper which attacks the style transfer via histograms problem much more meaningfully than is addressed here (although I haven't had time to wrap my brain around it just yet): https://graphics.social/@theohonohan/111490607440087628

I'm a big fan of OT :) https://observablehq.com/@fil/sliced-optimal-transport?collection=@fil/optimal-transport

Oh wow, has that been there the whole time?? Nice. <3