Cluster Map Implementation

[boda26]

1. Cluster Map Object Implementation
2. Refresh ClusterMap at the start of each fanout operation; fanout operations use targets from ClusterMap object

[yairgott]

I think that it would be much more efficient if the engine allow subscription to a notification mechanism which is triggered once the slot assignment is changed. @murphyjacob4 , wdyt?

[boda26]

Updated ClusterMap Implementation (significant change)

1. added a configurable ClusterMap expiration timestamp, only refresh when it expires
2. created a slot-to-shard map (map start slot to end slot and ShardInfo), enable efficient slot to shard retrieval
3. shard-level and cluster-level fingerprint
4. complete redesign of CreateNewClusteMap, entirely eliminated GetTargets, building shard map and target vectors altogether with only one CLUSTER SLOTS call to core
5. redesign of GetRandomTargets, randomly picking one node from each shard in pre-computed shard map

[boda26]

Todo: create a SocketAddress struct for ip and port

[boda26]

Separately stored ip and port in ClusterMap

[boda26]

Added unit tests for ClusterMap

[allenss-amazon]

If it's freestanding, you need to add inline. If you make it a member function then you don't need that.

[allenss-amazon]

This doesn't belong in the coordinator namespace. That's an application thing, for vmsdk, I'd just put this into it's own namespace: clustermap remove the nested namespace, i.e., just put it into vmsdk.

[allenss-amazon]

As we discussed before, the coordinator port number stuff belongs elsewhere.

[boda26]

Update 10.30

1. fix unit test compile error on macos
2. change to member functions to avoid large number of parameters
3. use a map to store (start slot, end slot) for each shard owned slots
4. use highway hash for fingerprints
5. SocketAddress struct for ip and port
6. boolean isPrimary, isLocal
7. simpler CHECK error messages

TODO:

1. node info list (4th entry of node array)
2. possible issue when 2 slot range point to same shard, but the ShardInfo differs