v1.21.0

[romange]

Dragonfly v1.21.0

Some prominent changes include:

• Alpha release of SSD Data tiering - enabled with flag --tiered_prefix some/path/basename
• Very basic support of multi-tenancy @3260
• HSETEX now supports NX option, see our docs.
• Added support for JSON.MERGE and for JSON.MSET.
• valkey's replica-announce-ip and --cluster--announc--ip are consolidated via --announce-ip flag.

What's Changed

• feat(server): master stop sending exec opcode to replica by @adiholden in feat(server): master stop sending exec opcode to replica #3289
• chore: optimize zpopminmax operations by @romange in chore: optimize zpopminmax operations #3291
• fix(json_family): Fix error in JsonFamilyTest.MGet by @BagritsevichStepan in fix(json_family): Fix error in JsonFamilyTest.MGet #3285
• fix: define macro WITH_AWS in cmake when flag is ON by @kostasrim in fix: define macro WITH_AWS in cmake when flag is ON #3296
• chore: Add 'memory arena show' command by @romange in chore: Add 'memory arena show' command #3298
• fix: missing logs on pytest failures fix: missing logs on pytest failures #3255 by @BorysTheDev in fix: missing logs on pytest failures #3255 #3272
• chore: refactor compact_object and introduce materialize method by @romange in chore: refactor compact_object and introduce materialize method #3300
• fix(acl): loading interleaved plain and hashed passwords by @kostasrim in fix(acl): loading interleaved plain and hashed passwords #3297
• chore: Add CompactObj Raw methods by @romange in chore: Add CompactObj Raw methods #3303
• chore: On invalid TTL, print the TTL first by @chakaz in chore: On invalid TTL, print the TTL first #3306
• chore: moving functions + renaming in tiered storage by @romange in chore: moving functions + renaming in tiered storage #3304
• fix(tests): Add missing awaits by @chakaz in fix(tests): Add missing awaits #3273
• chore: refactoring around tiered storage by @romange in chore: refactoring around tiered storage #3299
• feat: yield when serialization is in progress by @kostasrim in feat: yield when serialization is in progress #3220
• refactor: Use DbContext, OpArgs and Transaction to access DbSlice by @chakaz in refactor: Use DbContext, OpArgs and Transaction to access DbSlice #3311
• chore: Separate tiered serialization format from object values by @romange in chore: Separate tiered serialization format from object values #3310
• chore: add more community links to README by @Niennienzz in chore: add more community links to README #3308
• fix(cluster): Join on specified attempt id by @chakaz in fix(cluster): Join on specified attempt id #3305
• chore(acl): add test with requirepass and aclfile by @kostasrim in chore(acl): add test with requirepass and aclfile #3312
• chore: skip test_cluster_flushall_during_migration by @kostasrim in chore: skip test_cluster_flushall_during_migration #3316
• fix (pytest): generate unique random dbfilename for tests by @adiholden in fix (pytest): generate unique random dbfilename for tests #3317
• chore: bypass decoding/encoding of the data when performing offloading by @romange in chore: bypass decoding/encoding of the data when performing offloading #3315
• refactor: acl helpers and global tables by @kostasrim in refactor: acl helpers and global tables #3309
• chore(tiering): Fixes by @dranikpg in chore(tiering): Fixes #3225
• fix(migration): Use transactions! by @dranikpg in fix(migration): Use transactions! #3266
• fix: forbid DFLYCLUSTER commads set for emulated cluster mode by @BorysTheDev in fix: forbid DFLYCLUSTER commads set for emulated cluster mode #3307
• feat(namespaces): Initial support for multi-tenant by @chakaz in feat(namespaces): Initial support for multi-tenant #3260
• chore: improve dfly_bench stats by @romange in chore: improve dfly_bench stats #3320
• chore: remove replace_deleted flag from hnswlib by @dranikpg in chore: remove replace_deleted flag from hnswlib #3323
• feat(test): Improve benchmark workflow by @adiholden in feat(test): Improve benchmark workflow #3330
• fix: Proper shutdown sequence with Namespaces by @chakaz in fix: Proper shutdown sequence with Namespaces #3333
• fix(test): copy logs for failed test during TEARDOWN phase by @BorysTheDev in fix(test): copy logs for failed test during TEARDOWN phase #3331
• fix(json_family): fix JSON.STRAPPEND command for JSON legacy mode by @BagritsevichStepan in fix(json_family): fix JSON.STRAPPEND command for JSON legacy mode #3264
• chore(tiering): add protection against overruning memory budget by @romange in chore(tiering): add protection against overruning memory budget #3327
• chore: Add coordinated omission mode by @romange in chore: Add coordinated omission mode #3332
• chore: implement sequential pass without the overlapping traffic by @romange in chore: implement sequential pass without the overlapping traffic #3335
• feat: add an option to flush serialized entries on threshold limit by @kostasrim in feat: add an option to flush serialized entries on threshold limit #3241
• feat(hset_family): Add NX option to HSETEX by @BagritsevichStepan in feat(hset_family): Add NX option to HSETEX #3295
• Fix blocking commands moved error by @BorysTheDev in Fix blocking commands moved error #3334
• fix: ub in RegisterOnChange and regression tests for big values by @kostasrim in fix: ub in RegisterOnChange and regression tests for big values #3336
• fix: Cancel outgoing migration when retrying / closing by @chakaz in fix: Cancel outgoing migration when retrying / closing #3339
• chore: Make KeyIndex iterable by @dranikpg in chore: Make KeyIndex iterable #3326
• fix: AllocationTracker::Remove return value was reversed by @romange in fix: AllocationTracker::Remove return value was reversed #3341
• chore: remove redundant metrics from memory stats by @romange in chore: remove redundant metrics from memory stats #3345
• fix: corruption in replication stream by @romange in fix: corruption in replication stream #3344
• chore: clean up TaskQueue since we do not need multiple fibers for it by @romange in chore: clean up TaskQueue since we do not need multiple fibers for it #3348
• chore: pull helio by @romange in chore: pull helio #3350
• chore: Log connection context when issuing dangerous cmds by @chakaz in chore: Log connection context when issuing dangerous cmds #3352
• chore: small rename and add dcheck on LocalBlockingCounter by @kostasrim in chore: small rename and add dcheck on LocalBlockingCounter #3356
• refactor: reduce number of logs for cluster by @BorysTheDev in refactor: reduce number of logs for cluster #3357
• chore: fixes to dfly_bench by @romange in chore: fixes to dfly_bench #3353
• chore: fix test_parser_memory_stats flakiness by @romange in chore: fix test_parser_memory_stats flakiness #3354
• chore(server): Introduce StringSetWrapper by @dranikpg in chore(server): Introduce StringSetWrapper #3347
• fix: do not upload offload values on a first hit by @romange in fix: do not upload offload values on a first hit #3360
• chore(tiering): Range functions + small refactoring by @dranikpg in chore(tiering): Range functions + small refactoring #3207
• fix: failure in test_cluster_fuzzymigration by @romange in fix: failure in test_cluster_fuzzymigration #3363
• fix(server): Require >=1 args to GETEX by @chakaz in fix(server): Require >=1 args to GETEX #3366
• fix(transaction): Fix namespace access by @dranikpg in fix(transaction): Fix namespace access #3364
• chore: disable compression on big values by @kostasrim in chore: disable compression on big values #3358
• fix: protect OnJournalEntry with ConditionGuard by @kostasrim in fix: protect OnJournalEntry with ConditionGuard #3367
• chore: Introduce CoolQueue by @romange in chore: Introduce CoolQueue #3365
• chore: small fixes around tiering by @romange in chore: small fixes around tiering #3368
• chore: add a test for HeapSize() function by @romange in chore: add a test for HeapSize() function #3349
• chore: simplify computation of used_mem_current by @romange in chore: simplify computation of used_mem_current #3372
• chore: disable cluster_fuzzymigration by @kostasrim in chore: disable cluster_fuzzymigration #3373
• chore(replica): remove unused methods in the Replica class by @BagritsevichStepan in chore(replica): remove unused methods in the Replica class #3374
• fix(transaction): Properly store block cancel status by @dranikpg in fix(transaction): Properly store block cancel status #3371
• chore: Track db_slice table memory instantly by @romange in chore: Track db_slice table memory instantly #3375
• chore: add mem test for big values and default the flag by @kostasrim in chore: add mem test for big values and default the flag #3369
• update: replication_acks_interval flag to 1000 by @kostasrim in update: replication_acks_interval flag to 1000 #3378
• chore: dfly_bench - print ongoing error counts by @romange in chore: dfly_bench - print ongoing error counts #3382
• chore: introduce a cool queue that gradually retires cool items by @romange in chore: introduce a cool queue that gradually retires cool items #3377
• chore: update cached stats inside PollExecution by @romange in chore: update cached stats inside PollExecution #3376
• fix: Fix test_take_over_seeder by @chakaz in fix: Fix test_take_over_seeder #3385
• chore: reenable evictions upon insertion to avoid OOM rejections by @romange in chore: reenable evictions upon insertion to avoid OOM rejections #3387
• fix: remove fiber guard from non atomic section by @kostasrim in fix: remove fiber guard from non atomic section #3381
• chore: do not preempt on db_slice::RegisterOnChange by @kostasrim in chore: do not preempt on db_slice::RegisterOnChange #3388
• fix: disable inline transactions when db_slice has registered callbacks by @kostasrim in fix: disable inline transactions when db_slice has registered callbacks #3391
• fix: test_big_value_serialization_memory_limit shutdown timeout by @kostasrim in fix: test_big_value_serialization_memory_limit shutdown timeout #3390
• chore: set serialization_max_chunk_size to 1 byte by @kostasrim in chore: set serialization_max_chunk_size to 1 byte #3379
• chore: tiered fixes by @romange in chore: tiered fixes #3393
• fix(acl): remove none from acl categories by @kostasrim in fix(acl): remove none from acl categories #3392
• chore: tiering - make Modify work with cool storage by @romange in chore: tiering - make Modify work with cool storage #3395
• fix: Fix unsupported object type rejson-rl in RedisInsight by @BagritsevichStepan in fix: Fix unsupported object type rejson-rl in RedisInsight #3384
• Revert "chore: set serialization_max_chunk_size to 1 byte (chore: set serialization_max_chunk_size to 1 byte #3379)" by @chakaz in Revert "chore: set serialization_max_chunk_size to 1 byte (#3379)" #3398
• chore(tiering): Move cool entry warmup to DbSlice by @dranikpg in chore(tiering): Move cool entry warmup to DbSlice #3397
• fix: reenable macos builds by @romange in fix: reenable macos builds #3399
• chore: Don't print password to log on replica AUTH failure by @chakaz in chore: Don't print password to log on replica AUTH failure #3403
• chore: Support setting the value of replica-priority by @chakaz in chore: Support setting the value of replica-priority #3400
• feat: stabilize non-coordinated omission mode by @romange in feat: stabilize non-coordinated omission mode #3407
• chore: cancel slot migrations on shutdown by @dranikpg in chore: cancel slot migrations on shutdown #3405
• chore: add db_slice lock to protect segments from preemptions by @kostasrim in chore: add db_slice lock to protect segments from preemptions #3406
• chore: increase timeout of regression tests by @kostasrim in chore: increase timeout of regression tests #3412
• fix: crash with NS in multi/exec crash with redis insight #3410 by @BorysTheDev in fix: crash with NS in multi/exec #3410 #3415
• fix: json.merge exception crash by @kostasrim in fix: json.merge exception crash #3409
• fix(connection): Count memchached pipelined commands by @dranikpg in fix(connection): Count memchached pipelined commands #3413
• chore: remove verbose printing of tests by @romange in chore: remove verbose printing of tests #3420
• chore: Tiered fixes by @romange in chore: Tiered fixes #3401
• feat: Support non-root paths for json.merge by @romange in feat: Support non-root paths for json.merge #3419
• chore: skip cluster tests if redis-server wasn't found by @BorysTheDev in chore: skip cluster tests if redis-server wasn't found #3416
• chore: expose metric that shows how many task submitters are blocked by @romange in chore: expose metric that shows how many task submitters are blocked #3427
• chore: optimize SendStringArrInternal even more by @romange in chore: optimize SendStringArrInternal even more #3425
• fix(server): Implement SCRIPT GC command by @dranikpg in fix(server): Implement SCRIPT GC command #3431
• chore: retire TEST_EnableHeartBeat by @romange in chore: retire TEST_EnableHeartBeat #3435
• feat(server): Support replica-announce-ip/port by @chakaz in feat(server): Support replica-announce-ip/port #3421
• test(cluster): Migration replication test by @dranikpg in test(cluster): Migration replication test #3417
• chore: improve replication locks by @romange in chore: improve replication locks #3436
• chore: reorganize EngineShard::Heartbeat by @romange in chore: reorganize EngineShard::Heartbeat #3437
• chore: fix memcached pipeline test by @dranikpg in chore: fix memcached pipeline test #3438
• chore: simplify master replication cancelation interface by @romange in chore: simplify master replication cancelation interface #3439
• chore: reset serialization_max_chunk_size to 0 by @kostasrim in chore: reset serialization_max_chunk_size to 0 #3432
• feat: DEBUG REPLICA PAUSE now pauses fullsync by @romange in feat: DEBUG REPLICA PAUSE now pauses fullsync #3441
• test: fix test_disconnect_replica by @BorysTheDev in test: fix test_disconnect_replica #3442
• fix: cluster_mgr.py to use CLUSTER MYID by @chakaz in fix: cluster_mgr.py to use CLUSTER MYID #3444
• chore: disable serialization_max_chunk_size in regtests by @romange in chore: disable serialization_max_chunk_size in regtests #3445
• fix: properly seriailize meta buffer in SendStringArrInternal by @romange in fix: properly seriailize meta buffer in SendStringArrInternal #3455

Full Changelog: v1.20.0...v1.21.0

---

This discussion was created from the release v1.21.0.

[romange]

v1.21.0 has a bug that prevents it from replicating from the older dragonfly versions. The patch release with a fix that will be released shortly.