Stateful Session cookies incompatible between Envoy Processes.

[bbassingthwaite]

Title: Stateful Session cookies incompatible between Envoy Processes.

Description:
When using Envoy as an Edge Proxy with multiple instances (an NLB in front) AND using the StatefulSession with cookies doesn't work properly.

The cookie is encoded with the expiry time BUT it uses the monotonic clock:

envoy/source/extensions/http/stateful_session/cookie/cookie.cc

Lines 21 to 22 in 21b70d1

	const auto expiry_time = std::chrono::duration_cast<std::chrono::seconds>(
	(time_source_.monotonicTime() + std::chrono::seconds(factory_.ttl_)).time_since_epoch());

When HTTP requests are routed to different Envoy processes, they will have different values for their monotonic clock and the cookie can inadvertently expire sooner then its should, which will cause a new backend to get selected.

I believe the appropriate fix is for time_source_.systemTime() to be used so that two envoy processes can validate each other cookies.

cc @cpakulski

[cpakulski]

Your analysis makes sense. I do not remember exact reasoning behind usage of monotonic time, but am thinking that the design process did not take 1+ Envoys into account. Only single instance. Let me try to dig out the design doc and I will get back.