FZF seems to be handling this :
https://github.com/junegunn/fzf
--color 'fg:#bbccdd,fg+:#ddeeff,bg:#334455,preview-bg:#223344,border:#778899'
Currently, icdiff only accepts a small subset of all possible colors.
It could also be done with ANSI codes but I would find it even cooler to simply use RGB which is easier to parse for a human.