Ricky Reusser

Fun Notebooks

I'm a big fan of OT :)
https://observablehq.com/@fil/sliced-optimal-transport?collection=@fil/optimal-transport

Oh wow, has that been there the whole time?? Nice. <3

Is there a reason why for some target histograms the really bright parts of the original image end up "clipping" into different colors? (it's especially visible in clouds) Not a complaint, just curious if the reason behind it would give me a better feeling for the maths involved. Is it maybe related to the use of RGB instead? Would things be different in other color spaces?

Job van der Zwan

Yes, it happens because while it's a fine algorithm which can be applied to images, it's quite a bad fit for the actual use case of style transfer. It's approximate, but the task it seeks is to simply transfer pixels 1:1 to a new image, so if your target image has 50% white and 50% black and your source image is a sunset with mostly sky, it will end up thresholding the sky at some random value. This is made worse by the fact that the algorithm uses no information about the spatial distribution of values.

Theo Honohan referred us to a paper which attacks the style transfer via histograms problem much more meaningfully than is addressed here (although I haven't had time to wrap my brain around it just yet): https://graphics.social/@theohonohan/111490607440087628

yosemite.jpg

Fotothek_df_ps_0002300_Sport_^_Klettersport.jpg

grav-sm.jpg

topo.jpg

ellingwood@1.jpg

webb.jpg

bigcat.jpg

yosemite-2.jpg

nz-sunset.jpg

arthurs-pass.jpg

rainier.jpg

shelf.jpg

cranes.jpg

smith-rock.jpg

dog.jpg

yosemite-3@1.jpg

yosemite-smog.jpg

m61.jpg

ansel-adams.jpg

ansel-adams@1.jpg

Sliced Optimal Transport