fix(common/redis): Pipeliner.Do queues every command twice when writeTimeout is set - #4498
Open
paulstadler-mesh wants to merge 1 commit into
Open
fix(common/redis): Pipeliner.Do queues every command twice when writeTimeout is set#4498paulstadler-mesh wants to merge 1 commit into
paulstadler-mesh wants to merge 1 commit into
Conversation
…Timeout is set The writeTimeout branch was missing a return: after queueing the command with a timeout context it fell through and queued the same command AGAIN with the unbounded context. Every pipelined command then executed twice per Exec. Most visible casualty: the state store's transactional CAS EVAL scripts (Multi/TxPipeline) — each transactional write incremented the version hash field by 2, breaking ETag/first-write semantics. With daprd >= 1.17.9 (whose workflow engine plants the first-write flag), Dapr Workflow actor state saves fail on every second write with: ERR user_script:14: failed to set key <app>||...workflow||<id>||metadata On earlier daprd the duplication is silent (versions inflate, the Lua escape hatch masks the CAS) and transactional Redis CPU is ~doubled. Fixes dapr#4464. Regression test drives a pipelined INCR through both clients and asserts single execution with and without writeTimeout. Signed-off-by: Paul Stadler <paul.stadler@meshconnect.com>
5 tasks
Author
|
@mikeee sorry for the direct ping — this one's a 2-line fix (+regression test) for #4464: The CI workflows are sitting at awaiting approval (fork PR) — if you could approve the runs, everything else should go green from there. Happy to make any changes needed. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #4464.
Pipeliner.Do(bothv8client.goandv9client.go) is missing areturnin itswriteTimeout > 0branch: after queueing the command with a timeout context, execution falls through and queues the same command again with the unbounded context. Every command sent through aTxPipelinethen executes twice perExecwhenever the component setswriteTimeoutmetadata.Impact
StateStore.Multipipelines the CASEVALper operation, so a single transactional write runs the script twice — theversionhash field increments by 2 per write, breaking ETag/first-writesemantics.first-writeflag): every second workflow write fails withERR user_script:14: failed to set key …||workflow||…||metadata. On earlier daprd the duplication is silent — versions inflate and transactional Redis CPU is ~doubled.state.redisand pluggable components (shared client).Minimal repro and full analysis in #4464 (e.g. one transactional upsert on a fresh key →
GETreturns ETag2).Fix
Add the missing
return(mirrors the existing pattern inDoWrite/DoReadon the same clients).Testing
New regression test
TestPipelinerDoQueuesOnceWithWriteTimeoutdrives a pipelinedINCRthrough both v8 and v9 clients against miniredis and asserts single execution, with and withoutwriteTimeout(fails on the previous code with counter=2).Checklist