[#4802] Adopt a concurrently created config token instead of failing to start - #4813
Merged
Merged
Conversation
…to start The config token row is created on first use, so instances starting together against a fresh token table all raced to insert it. Only one could win, and the losers rethrew the primary-key violation as UnableToRetrieveIdentifierException without ever re-reading the row the winner had just committed, leaving all but one unable to start. Measured at three failures in four. The insert is now wrapped in a JDBC savepoint. On failure it rolls back to the savepoint, re-reads the config token and adopts the winner's identifier, rethrowing with the original cause only when there is genuinely no row to adopt. The savepoint is what makes the re-read legal on databases that abort the whole transaction on a constraint violation, and it is skipped when the connection auto-commits, since a failed insert cannot then poison anything else. Fixes #4802
schananas
requested review from
MateuszNaKodach,
laura-devriendt-lemon and
zambrovski
and removed request for
a team
July 29, 2026 11:47
Contributor
|
/backport axon-5.3.x |
Contributor
|
Successfully created backport PR for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4802
What changed
initializeConfigTokenrethrew the insert's primary-key violation asUnableToRetrieveIdentifierExceptionwithout re-reading the row the winning instance had just committed. Several instances starting at once against a fresh token table therefore left all but one unable to start. Measured at three failures in four.The insert is now wrapped in a JDBC savepoint. On failure it rolls back to the savepoint, re-reads the config token, and adopts the winner's identifier, rethrowing with the original cause only when there is genuinely no row. The savepoint is what makes the re-read legal on databases such as PostgreSQL that abort the whole transaction on a constraint violation.
Tests
New:
JdbcTokenStoreTest.ConcurrentIdentifierInitialization, a four-thread barrier race. Green on 5 consecutive runs; withJdbcTokenStore.javareverted it fails 3 of 3, 3 of 4 threads throwingUnableToRetrieveIdentifierException-- the three-in-four rate reported on the issue.Full
messagingmodule: green. No existing test needed changing.The fix only works at READ COMMITTED
Measured across four HSQLDB configurations, same 4-thread race:
At SERIALIZABLE the cause is
SQLTransactionRollbackException: transaction rollback: serialization failure-- the whole transaction is dead before the savepoint rollback runs, so the re-read never happens and 3 of 4 processors still fail to start. The same reasoning applies to PostgreSQL REPEATABLE READ and SERIALIZABLE, and to MySQL and MariaDB, whose default is REPEATABLE READ: a consistent read pins the snapshot at the first read, so a plain non-FOR UPDATEre-read cannot see the winner.Not a regression, and not a correctness hazard -- a
40001serialization failure is exactly what a caller retries as a whole transaction. But it is a real limit on where this helps.For the reviewer
The savepoint is deliberately optional:
savepointOrNullreturns null when the connection auto-commits, where a failed statement poisons nothing, and when the driver rejectssetSavepoint. The re-read is still attempted, so both degrade to today's behaviour rather than breaking. The savepoint is never explicitly released, becausereleaseSavepointis unsupported on some drivers.The original
SQLExceptionis preserved as the cause and surfaces only when the re-read finds no row, so a genuine schema or connectivity fault still fails loudly instead of being swallowed as a lost race.Two gaps worth naming. The savepoint half of the diff has no test: HSQLDB does not abort the transaction on a PK violation, so the adopt path works even with
setSavepointthrowing, leaving the property claimed for the savepoint unverified here. And the test asserts no landing evidence -- if the barrier ever serialised so all four threads found the row on the first read, it would pass without exercising the catch block.Not covered here:
JpaTokenStore.getConfighas the same race. After a failedem.flush()the persistence context is rollback-only, so recovering there needs a fresh transaction rather than a re-read. Called out in #4802 as deserving its own issue rather than widened into this one.