Experimental tagging support and other goodies

Bentocache 1.2.0

The main feature of this release is experimental tagging support!
Documentation available here https://bentocache.dev/docs/tagging
If you encounter any bugs with tags, please report them via Github issues !

Minor Changes

• 8bb87b6: Add a new expire method.

  This method is slightly different from delete:

  When we delete a key, it is completely removed and forgotten. This means that even if we use grace periods, the value will no longer be available.

  expire works like delete, except that instead of completely removing the value, we just mark it as expired but keep it for the grace period. For example:

  // Set a value with a grace period of 6 minutes
  await cache.set({
    key: 'hello',
    value: 'world',
    grace: '6m',
  })
  
  // Expire the value. It is kept in the cache but marked as STALE for 6 minutes
  await cache.expire({ key: 'hello' })
  
  // Here, a get with grace: false will return nothing, because the value is stale
  const r1 = await cache.get({ key: 'hello', grace: false })
  
  // Here, a get with grace: true will return the value, because it is still within the grace period
  const r2 = await cache.get({ key: 'hello' })
  
  assert.deepEqual(r1, undefined)
  assert.deepEqual(r2, 'world')

• d513fe2: Add a new E_L2_CACHE_ERROR. Before this commit, error happening when interacting with the L2 cache were not wrapped in a custom error. This is now the case. If needed, you can still access the original error by using the cause property of the E_L2_CACHE_ERROR error.

  import { errors } from 'bentocache'
  
  try {
    await cache.getOrSet({
      key: 'foo',
      factory: getFromDb(),
    })
  } catch (err) {
    if (err instanceof errors.E_L2_CACHE_ERROR) {
      console.error('An error happened while interacting with the L2 cache', err.cause)
    }
  }

• 4d1feb5: Added a super simple circuit breaker system to the L2 Cache :

  • a l2CircuitBreakerDuration parameter to set the duration of the circuit breaker. How many seconds the circuit breaker will stay open.
  • If defined, the circuit breaker will open when a call to our distributed cache fails. It will stay open for l2CircuitBreakerDuration seconds.

  We may introduce more sophisticated circuit breaker system in the future, but for now, this simple system should be enough.

• 6b1f42a: Enhance Factory Context by adding some new props.

  await cache.getOrSet({
    key: 'foo',
    factory: (ctx) => {
      // You can access the graced entry, if any, from the context
      if (ctx.gracedEntry?.value === 'bar') {
        return 'foo'
      }
  
      // You should now use `setOptions` to update cache entry options
      ctx.setOptions({
        tags: ['foo'],
        ttl: '2s',
        skipL2Write: true,
      })
  
      return 'foo'
    },
  })

  setTtl has been deprecated in favor of setOptions and will be removed in the next major version.

• 73ac0fa: Add experimental tagging support. See #53

  await bento.getOrSet({
    key: 'foo',
    factory: getFromDb(),
    tags: ['tag-1', 'tag-2'],
  })
  
  await bento.set({
    key: 'foo',
    tags: ['tag-1'],
  })

  Then, we can delete all entries tagged with tag-1 using:

  await bento.deleteByTags({ tags: ['tag-1'] })

  As this is a rather complex feature, let's consider it experimental for now. Please report any bugs on Github issues

• 6b1f42a: Add skipL2Write and skipBusNotify options.

  await cache.getOrSet({
    key: 'foo',
    skipL2Write: true,
    skipBusNotify: true,
    factory: () => 'foo',
  })

  When enabled, skipL2Write will prevent the entry from being written to L2 cache, and skipBusNotify will prevent any notification from being sent to the bus. You will probably never need to use these options, but they were useful for internal code, so decided to expose them.

• b9db3b5: Rework the logs issued by bentocache to make them much cleaner, more consistent and make debug easier

Patch Changes

• 491d12e: Handle deleteMany with empty keys list

[email protected]

This release primarily brings some huge performance boosts along with a few new features. Simply by installing this new version, you can enjoy up to 15x higher throughput in certain caching scenarios without any additional action required.

In some benchmarks of super common caching use-cases, we are up to 160x faster than cache-manager, the most popular caching library in Node.js ecosystem :

┌──────────────────────────────────┬────────────────────────┬─────────┐
│ Task name                        │ Throughput med (ops/s) │ Samples │
├──────────────────────────────────┼────────────────────────┼─────────┤
│ 'L1 GetOrSet - BentoCache'       │ '1062699 ± 20724'      │ 168254  │
│ 'L1 GetOrSet - CacheManager'     │ '5702 ± 176'           │ 1040    │
│ 'L2 GetOrSet - BentoCache'       │ '1823 ± 64'            │ 347     │
│ 'L2 GetOrSet - CacheManager'     │ '1455 ± 38'            │ 282     │
│ 'Tiered GetOrSet - BentoCache'   │ '1072961 ± 11637'      │ 174603  │
│ 'Tiered GetOrSet - CacheManager' │ '5876 ± 80'            │ 1089    │
│ 'Tiered Get - BentoCache'        │ '1949318 ± 37272'      │ 1805048 │
│ 'Tiered Get - CacheManager'      │ '531350 ± 7789'        │ 496805  │
│ 'Tiered Set - BentoCache'        │ '2171 ± 45'            │ 2143    │
│ 'Tiered Set - CacheManager'      │ '2211 ± 84'            │ 2159    │
└──────────────────────────────────┴────────────────────────┴─────────┘

You can see the benchmarks here

New Features

• 07224ba: Add two new functions in the factory callback context:

  cache.getOrSet({
    key: 'foo',
    factory: ({ skip, fail }) => {
      const item = await getFromDb()
      if (!item) {
        return skip()
      }
  
      if (item.isInvalid) {
        return fail('Item is invalid')
      }
  
      return item
    },
  })

  • Returning skip in a factory will not cache the value, and getOrSet will returns undefined even if there is a stale item in cache.
    It will force the key to be recalculated on the next call.

  • Returning fail in a factory will not cache the value and will throw an error. If there is a stale item in cache, it will be used.

  🔗 https://bentocache.dev/docs/methods#ctxskip

• 2578357: Added a serialize: boolean option to the memory driver.

  If false, It means the data stored in the memory cache will not be serialized/parsed using JSON.stringify and JSON.parse. This allows for a much faster throughput but at the expense of:

  • not being able to limit the size of the stored data, because we can't really know the size of an unserialized object
  • Having inconsistent return between the L1 and L2 cache. The data stored in the L2 Cache will always be serialized because it passes over the network. Therefore, depending on whether the data is retrieved from the L1 and L2, we can have data that does not have the same form. For example, a Date instance will become a string if retrieved from the L2, but will remain a Date instance if retrieved from the L1. So, you should put extra care when using this feature with an additional L2 cache.

  🔗 https://bentocache.dev/docs/cache-drivers#serialize-option

v1.0.0

Upgrade Guide 1.0.0

Breaking Changes

Remove old syntax

We removed the "legacy" syntax and only keep the POJO-one.
For each method, the method signature is a full object, for example :

bento.get({ key: 'foo '})

timeouts

Previously you could define your timeouts like { soft: '200ms', hard: '2s' }. Now you should use the timeout and hardTimeout options like this:

getOrSet({ timeout: '200ms', hardTimeout: '2s' })

You can now also use 0 for timeout which means that, if a stale value is available, then it will be returned immediately, and the factory will run in the background. SWR-like, in short.

Default soft timeout

Now, the default timeout is 0. As explained above, this enables the SWR-like behavior by default, which is a good default for most cases and what most people expect. So make sure to update your code if you were relying on the previous default.

gracePeriod

• gracePeriod is now grace and should be either false or a Duration.
• If you were using the fallbackDuration option, you should now use the graceBackoff option at the root level.

suppressL2Errors

Previously, suppressL2Errors was automatically enabled even when we had just a L2 layer. Which can be confusing, because errors were filtered out. See #25

Now suppressL2Errors is a bit more intelligent and will only be enabled if you have a L1 layer. Unless you explicitly set it to true.

undefined values

Now, undefined values are forbidden in the cache. If you are trying to cache undefined, an error will be thrown. This is a breaking change because it was previously allowed.

If you want to cache something to represent the absence of a value, you can use null instead of undefined.

New Features

onFactoryError

Added an onFactoryError option that allows to catch errors that happen in factories, whether they are executed in background or not.

const result = await cache.getOrSet({
  key: 'foo',
  grace: '5s',
  factory: () => {
    throw new MyError()
  },
  onFactoryError: (error) => {
    // error is an instance of errors.E_FACTORY_ERROR
    // error.cause is the original error thrown by the factory
    // you can also check if the factory was executed in background with error.isBackgroundFactory
    // and also get the key with error.key. Will be `foo` in this case
  },
})

Memory driver options

The memory driver can now accept maxSize and maxEntrySize in human format. For example, maxSize: '1GB' or maxEntrySize: '1MB'.

We use https://www.npmjs.com/package/bytes for parsing so make sure to respect the format accepted by this module.

Re-worked the logic when only L2 is enabled

Previously, we were using a single piece of code to handle both scenarios, where only L2 is enabled and where both L1 and L2 are enabled. This was a bit confusing and led to some bugs.

Now, we have a clear separation between the two cases and we could fix some bugs that were present when only L2 was enabled.

[email protected]

Commits

• refactor: Only one bus instance per named cache. All subsequent namespaces under it use the same bus. (3813247)
• fixed return type for CacheStack namespace function (c772f83)
• optimize: added namespace cache to CacheStack to avoid unnecessary creation of caches and buses (eaef9d8)
• fix: clear message goes to all bus instances since channel name is the same (1699fe6)

What's Changed

• fix: clear message goes to all bus instances + refactor: to optimize bus sync by @gkachru in #39

Full Changelog: https://github.com/Julien-R44/bentocache/compare/[email protected]@1.0.0-beta.12

[email protected]

What's Changed

• fix: BinaryEncoding did not encode/decode the Clear message type by @gkachru in #38

Full Changelog: https://github.com/Julien-R44/bentocache/compare/[email protected]@1.0.0-beta.11

Bug fixes

Commits

• chore: add server playground (7613081)
• docs: add note about DynamoDB TTL configuration (d0024dd)
• chore: update lock file (96c1924)
• chore: add dynamodb in playground (c5e646c)
• Fix: Issue of synchronizing namespaced local memory caches (#37) (f077d30)
• fix: remove ttl usage from getOrSetForever (32943e2)
• fix: use undefined instead of Infinity for memory driver (7267974)
• fix: use specific poppinss/utils import path for exceptions (e00ac08)
• chore: update dependencies (f22c953)
• chore(docs): prometheus dashboard json url updated (#23) (9d66e2f)

What's Changed

• chore(docs): prometheus dashboard json url updated by @joravkumar in #23
• Fix: Issue of synchronizing namespaced local memory caches by @gkachru in #37

New Contributors

• @joravkumar made their first contribution in #23
• @gkachru made their first contribution in #37

Full Changelog: https://github.com/Julien-R44/bentocache/compare/[email protected]@1.0.0-beta.10

Orchid ORM support

Features

Orchid ORM support

We now have first-class support for Orchid ORM thanks to a PR from @bingtsingw!

See https://bentocache.dev/docs/cache-drivers#orchid-orm

Commits

• fix: bus utf8 characters handling (7690bc2)
• chore: update dependencies (35263ec)
• chore: remove obsolete version in compose file (105841f)
• feat: add Orchid ORM driver (#17) (695cd71)
• Fixes missing comma in Knex code example (#18) (1822312)
• chore: add banner (3bb8127)
• chore: add sponsors (517dd97)

What's Changed

• Fixes missing comma in Knex code example by @DilwoarH in #18
• feat: add orchid driver by @bingtsingw in #17

New Contributors

• @DilwoarH made their first contribution in #18
• @bingtsingw made their first contribution in #17

Full Changelog: https://github.com/Julien-R44/bentocache/compare/[email protected]@1.0.0-beta.9

Lot of goodies

Breaking Changes

We've introduced an adapter system for the database store, which allows us to decouple Knex from Bentocache. This change brings a few breaking changes:

• All specific drivers like mysqlDriver, sqliteDriver, etc., have been removed. You should now use knexDriver instead.
• The knex driver won't automatically create the knex instance anymore. You'll need to create the instance yourself and pass it to the adapter.

For more information, check out the documentation here.

Features

Database Adapter

We've added an adapter system to Bentocache, allowing us to decouple from Knex. This means you can now use a different ORM/Query builder such as Prisma, Kysely, Drizzle... You can use the same ORM that's already part of your application.

Kysely Adapter

With the new adapter system, we're introducing a built-in adapter for Kysely for the database store. You can find instructions on how to use Kysely with Bentocache here.

Adaptive Caching

Sometimes, we can't predetermine the TTL to use. For example, when fetching a token from an API that returns the token and its lifespan. It would be great to set the TTL to match the token's lifespan. This wasn't possible with the getOrSet method before. That's why we've introduced adaptive caching:

const authToken = await bento.getOrSet('token', async (options) => {
  const token = await fetchAccessToken();
  options.setTtl(token.expiresIn);
  return token;
});

In short, you can now dynamically set the TTL based on certain parameters from your factory function. For more information, check the documentation.

File Driver Automatic Eviction

Until now, using the file driver for our cache meant that files kept accumulating, never deleting even after the cached values expired. Because, well, File system doesn't have a TTL system.

To address this, we've added an automatic eviction system. The concept is simple: start a worker thread that regularly cleans up files with expired entries.

For more details, see here.

POJOs Methods

Most of the methods now accept arguments in two different ways : either as a single argument or as an object.

If you need to pass some specific options, the object way is probably the best choice since it is more "vertical" and may be easier to read. On the other hand, the single argument may be more concise when you don't need to pass specific options.

Example:

// multiple arguments
await bento.getOrSet('products', () => fetchProducts(), {
  ttl: '5m',
  gracePeriod: { enabled: true, duration: '1m' },
})

// is equivalent to
await bento.getOrSet({
  key: 'products',
  ttl: '5m',
  factory: () => fetchProducts(),
  gracePeriod: { enabled: true, duration: '1m' },
})

Boring Bus

Bentocache Bus has been exported into a generic package here. As a result, Bentocache was refactored to use this new package.

Along the way, we improved the bus, fixed some bugs, and added a retry interval concept for the retry queue. Thanks to @RomainLanz for the work on this package!

Commits

• fix: connection options wrongly passed to redis transport (e92d835)
• chore: add hono example (3785f7b)
• feat: pojo methods syntax (#15) (01d1d4b)
• refactor: move to @boringnode/bus (#14) (ff32759)
• doc: fix invalid url (bb4ea66)
• refactor: remove deprecated dependency (393f316)
• feat: add adaptive caching (1ec8c29)
• refactor: move test_helpers to tests folder (0068d36)
• feat: cleaner worker thread for File Driver (#13) (3a839e7)
• refactor: add Driver suffix to all drivers (b718b95)
• ci: update ci deps (58d4879)
• feat!: add adapter system for database store (#12) (08fff55)
• chore: remove upstash tests (9ed2738)
• chore: ts checks (146ecd4)
• chore: update dependencies (ee94407)

[email protected]

Changes

• Breaking change : updated the test suite API. See new usage here https://bentocache.dev/docs/custom-cache-driver#tests
• Fix when used in another context than Node( Vite for example ), an import to @poppinss/utils was throwing an error

Commits

• chore: update lockfile (22478fb)
• refactor: chainable cache factory method (96a61a6)
• refactor: make CacheFactory setup more explicit (dba0a82)
• chore(docs): handle redirect (da7b486)
• update @poppinss/utils (92e050f)
• chore(docs): update to adonis 6 (d9849a9)
• refactor: test suite api (f0f3764)
• chore: add missing readme (edc023c)
• chore: fix jsdoc position (adb49ae)

New Contributors

• @TiBianMod made their first contribution in #6

Full Changelog: https://github.com/Julien-R44/bentocache/compare/[email protected]@1.0.0-beta.7

[email protected]

Fixes

• Background factories that threw an error could kill the application if no graced value was found.
• Background factories that threw an error wasnt releasing the locks.

Internal

• Migration to a monorepo. We now have the docs website, @bentocache/plugin-prometheus, bentocache and a playground under this same repo