Making absl::flat_hash_map work in interface boundaries in dynamically loaded libraries

[michaeleisel]

I'm interested in getting abseil to work in interface boundaries in dynamically loaded libraries. The only thing I see preventing this is the H1 function in raw_hash_set.h, which uses a seed. If we remove the seed from that function, e.g. by just removing the ^ seed.seed() part, is that sufficient to make things work? Also, since this looks so easy to change, it would be nice if abseil allowed people to set it for themselves without needing to fork abseil, e.g. with a template parameter on absl::flat_hash_map or a preprocessor define.

[derekmauro]

If we can make Abseil work at interface boundaries in dynamically loaded libraries, that would be great, but there is no chance whatsoever that we will accept any change that allows for disabling the randomization of the seed. We would have to find some other solution.

To understand why, you have to understand just how much CPU and memory we have tied up in our hash tables. We have used the randomization to allow us to roll out steady improvements to the implementation without breaking users dependent on ordering or other characteristics. The amount of money saved in compute costs through these optimizations is staggering (it is an order of magnitude higher than I would have guessed).

I know someone is going to say "just give them a feature flag and say it is unsupported". In the real world, that just doesn't work. People just do what works for them at the time, wind up in a bind, and then complain and plead ignorance when they need to update.

[michaeleisel]

That makes sense, Hyrum's law strikes again :)
Maybe coordination between the libraries could be done with something like shm_open to access a shared file.

[michaeleisel]

Thinking more about this, it seems like on Linux we'd want the value to be consistent after calling fork() but not after calling exec(). Perhaps getauxval(AT_RANDOM) would be good.