Skip to content

Commit 46252ad

Browse files
authored
Merge branch 'develop' into fix/aggregate-routekeys-nandan
2 parents 5120304 + e3ad51c commit 46252ad

File tree

227 files changed

+6172
-3958
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

227 files changed

+6172
-3958
lines changed

.github/workflows/develop.yml

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -68,15 +68,15 @@ jobs:
6868
build-cake:
6969
needs: build
7070
runs-on: ubuntu-latest
71-
environment: build.cake
71+
# environment: build.cake
7272
env:
7373
# https://github.com/actions/setup-dotnet/blob/main/README.md#environment-variables
7474
# NUGET_PACKAGES: ${{ github.workspace }}/.nuget/packages
7575
DOTNET_INSTALL_DIR: "/usr/share/dotnet" # don't override by /usr/lib/dotnet
76-
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
77-
GITHUB_SHA: ${{ github.sha }}
78-
GITHUB_REF: ${{ github.ref }}
79-
GITHUB_REF_NAME: ${{ github.ref_name }}
76+
# COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
77+
# GITHUB_SHA: ${{ github.sha }}
78+
# GITHUB_REF: ${{ github.ref }}
79+
# GITHUB_REF_NAME: ${{ github.ref_name }}
8080
steps:
8181
- name: .NET Info
8282
run: |
@@ -144,9 +144,12 @@ jobs:
144144
target: Build
145145
- name: Prepare Coveralls
146146
run: |
147-
echo "GITHUB_REF is ${{ env.GITHUB_REF }}"
148-
echo "GITHUB_REF_NAME is ${{ env.GITHUB_REF_NAME }}"
149-
echo "GITHUB_SHA is ${{ env.GITHUB_SHA }}"
147+
# echo "GITHUB_REF is ${{ env.GITHUB_REF }}"
148+
# echo "GITHUB_REF_NAME is ${{ env.GITHUB_REF_NAME }}"
149+
# echo "GITHUB_SHA is ${{ env.GITHUB_SHA }}"
150+
echo "Listing environment variables:"
151+
env | sort
152+
echo ------------ Detect coverage file ------------
150153
coverage_1st_folder=$(ls -d /home/runner/work/Ocelot/Ocelot/artifacts/UnitTests/*/ | head -1)
151154
echo "Detected first folder : $coverage_1st_folder"
152155
coverage_file="${coverage_1st_folder%/}/coverage.cobertura.xml"

.github/workflows/pr.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,12 +82,12 @@ jobs:
8282
build-cake:
8383
needs: build
8484
runs-on: ubuntu-latest
85-
environment: build.cake
86-
env:
87-
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
88-
GITHUB_SHA: ${{ github.sha }}
89-
GITHUB_REF: ${{ github.ref }}
90-
GITHUB_REF_NAME: ${{ github.ref_name }}
85+
# environment: Pull-Request
86+
# env:
87+
# COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
88+
# GITHUB_SHA: ${{ github.sha }}
89+
# GITHUB_REF: ${{ github.ref }}
90+
# GITHUB_REF_NAME: ${{ github.ref_name }}
9191
steps:
9292
- name: .NET Info
9393
run: |

docs/features/caching.rst

Lines changed: 182 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,10 @@
33
Caching
44
=======
55

6-
Ocelot supports some very rudimentary caching at the moment provider by the `CacheManager <https://github.com/MichaCo/CacheManager>`_ project.
6+
[#f1]_ Ocelot currently supports caching on the URL of the downstream service and setting a TTL in seconds to expire the cache.
7+
Users can also clear the cache for a specific region by using Ocelot's :ref:`administration-api`.
8+
9+
Ocelot utilizes some very rudimentary caching at the moment provider by the `CacheManager <https://github.com/MichaCo/CacheManager>`_ project.
710
This is an amazing project that is solving a lot of caching problems. We would recommend using this package to cache with Ocelot.
811

912
The following example shows how to add *CacheManager* to Ocelot so that you can do output caching.
@@ -28,36 +31,96 @@ The second step is to add the following to your `Program`_:
2831
.AddOcelot(builder.Configuration)
2932
.AddCacheManager(x => x.WithDictionaryHandle());
3033
34+
``CacheOptions`` Schema
35+
-----------------------
36+
37+
.. _FileCacheOptions: https://github.com/ThreeMammals/Ocelot/blob/main/src/Ocelot/Configuration/File/FileCacheOptions.cs
38+
39+
Class: `FileCacheOptions`_
40+
41+
The following is the full *caching* configuration, used in both the :ref:`config-route-schema` and the :ref:`config-dynamic-route-schema`.
42+
Not all of these options need to be configured; however, the ``TtlSeconds`` option is mandatory.
43+
44+
.. code-block:: json
45+
46+
"CacheOptions": {
47+
"TtlSeconds": 1, // nullable integer
48+
"Region": "", // string
49+
"Header": "", // string
50+
"EnableContentHashing": false // nullable boolean
51+
}
52+
53+
.. list-table::
54+
:widths: 25 75
55+
:header-rows: 1
56+
57+
* - *Option*
58+
- *Description*
59+
* - ``TtlSeconds``
60+
- Time-To-Live (TTL) in seconds for the cached downstream response, i.e., the absolute expiration timeout starting from when the item is added to the cache.
61+
This option is required. If undefined, it defaults to 0 (zero), which disables caching.
62+
* - ``Region``
63+
- Specifies the cache region to be cleared via Ocelot's :ref:`administration-api`.
64+
See: ``DELETE {adminPath}/outputcache/{region}``
65+
* - ``Header``
66+
- Specifies the header name used for native Ocelot caching control, defaulting to the special ``OC-Cache-Control`` header.
67+
If the header is present, its value is included in the cache key constructed by the ``ICacheKeyGenerator`` service.
68+
Varying header values result in different cache keys, effectively invalidating the cache.
69+
70+
* - ``EnableContentHashing``
71+
- Toggles inclusion of request body hashing in the cache key.
72+
Disabled by default (``false``) due to potential performance impact.
73+
Recommended for POST/PUT routes where request body affects response.
74+
Refer to the :ref:`EnableContentHashing option <caching-enablecontenthashing-option>` section.
75+
76+
The actual ``CacheOptions`` schema with all the properties can be found in the C# `FileCacheOptions`_ class.
77+
3178
Configuration
3279
-------------
3380

34-
Finally, in order to use caching on a route in your route configuration add this setting:
81+
Finally, in order to use caching on a route in your route configuration add these sections:
3582

3683
.. code-block:: json
3784
85+
"CacheOptions": {
86+
"TtlSeconds": 15,
87+
"Region": "europe-central",
88+
"Header": "OC-Cache-Control",
89+
"EnableContentHashing": false // my route has GET verb only, assigning 'true' for requests with body: POST, PUT etc.
90+
},
91+
// Warning! FileCacheOptions section is deprecated! -> use CacheOptions
3892
"FileCacheOptions": {
3993
"TtlSeconds": 15,
4094
"Region": "europe-central",
41-
"Header": "OC-Caching-Control",
95+
"Header": "OC-Cache-Control",
4296
"EnableContentHashing": false // my route has GET verb only, assigning 'true' for requests with body: POST, PUT etc.
4397
}
4498
45-
* In this example ``TtlSeconds`` is set to 15 which means the cache will expire after 15 seconds.
46-
* The ``Region`` represents a region of caching.
47-
The cache for a region can be cleared by calling Ocelot's :ref:`administration-api`.
48-
* If a header name is defined in the ``Header`` property, that header value is looked up by the key (header name) in the ``HttpRequest`` headers, and if the header is found, its value will be included in caching key.
49-
This causes the cache to become invalid due to the header value changing.
99+
* In this example, ``TtlSeconds`` is set to 15, which means the cache will expire 15 seconds after the response is stored.
100+
* The ``Region`` property specifies a cache region. Cache entries within a region can be cleared by calling Ocelot's :ref:`administration-api`.
101+
* If a header name is defined in the ``Header`` property, its value is retrieved from the ``HttpRequest`` headers.
102+
If the header is present, its value is included in the cache key constructed by the ``ICacheKeyGenerator`` service.
103+
Varying header values result in different cache keys, effectively invalidating the cache.
104+
* Finally, ``EnableContentHashing`` is disabled due to the current route using the ``GET`` verb, which does not include a request body.
105+
106+
.. _24.1: https://github.com/ThreeMammals/Ocelot/releases/tag/24.1.0
107+
.. _25.0: https://github.com/ThreeMammals/Ocelot/milestone/12
108+
.. warning::
109+
According to the static :ref:`config-route-schema`, the ``FileCacheOptions`` section has been deprecated!
110+
111+
The `old schema <https://github.com/ThreeMammals/Ocelot/blob/24.1.0/src/Ocelot/Configuration/File/FileRoute.cs#L86-L88>`_ ``FileCacheOptions`` section is deprecated in version `24.1`_!
112+
Use ``CacheOptions`` instead of ``FileCacheOptions``! Note that ``FileCacheOptions`` will be removed in version `25.0`_!
113+
For backward compatibility in version `24.1`_, the ``FileCacheOptions`` section takes precedence over the ``CacheOptions`` section.
50114

51115
.. _caching-enablecontenthashing-option:
52116

53-
``EnableContentHashing`` option
54-
-------------------------------
117+
``EnableContentHashing`` option [#f2]_
118+
--------------------------------------
55119

56-
In version `23.0`_, the new property ``EnableContentHashing`` has been introduced.
57-
Previously, the request body was utilized to compute the cache key.
120+
Previously, in versions prior to `23.0`_, the request body was used to compute the cache key.
58121
However, due to potential performance issues arising from request body hashing, it has been disabled by default.
59122
Clearly, this constitutes a breaking change and presents challenges for users who require cache key calculations that consider the request body (e.g., for the POST method).
60-
To address this issue, it is recommended to enable the option either at the route level or globally in the :ref:`caching-global-configuration` section:
123+
To address this issue, it is recommended to enable the option either at the route level or globally in the ":ref:`Global Configuration <caching-global-configuration>`" section:
61124

62125
.. code-block:: json
63126
@@ -66,29 +129,103 @@ To address this issue, it is recommended to enable the option either at the rout
66129
"EnableContentHashing": true
67130
}
68131
132+
.. rubric:: Ocelot Team Recommendation
133+
134+
Although the community raised concerns about backward compatibility in issue `2234`_, Ocelot team maintains that *caching* performance takes precedence over backward compatibility when migrating from versions prior to `23.0`_.
135+
The proposed option clarifies that ``POST`` requests should **not** be cached; only ``GET`` requests are eligible for caching.
136+
Therefore, ``POST`` and ``GET`` verbs must be separated into distinct routes:
137+
138+
* POST routes with *caching* disabled
139+
* GET routes with *caching* enabled
140+
69141
.. _caching-global-configuration:
70142

71-
Global Configuration
72-
--------------------
143+
Global Configuration [#f3]_
144+
---------------------------
73145

74-
The positive update is that copying route-level properties for each route is no longer necessary, as version `23.3`_ allows for setting their values in the ``GlobalConfiguration``.
75-
This convenience extends to ``Header`` and ``Region`` as well.
76-
However, an alternative is still being sought for ``TtlSeconds``, which must be explicitly set for each route to enable caching.
146+
Copying route-level properties for each static route is no longer necessary, as version `23.3`_ allows these values to be set in the ``GlobalConfiguration`` section.
147+
This convenience applies to ``Header`` and ``Region`` as well.
148+
However, if no global ``TtlSeconds`` value is defined, this option must still be explicitly set per route to enable caching.
149+
As a result, the final configuration for static routes might look like:
150+
151+
.. code-block:: json
152+
:emphasize-lines: 5-7, 12, 18-21
153+
154+
{
155+
"Routes": [
156+
{
157+
"Key": "R0", // optional
158+
"CacheOptions": {
159+
"TtlSeconds": 60 // 1-minute short-term caching
160+
},
161+
// ...
162+
},
163+
{
164+
"Key": "R1", // this route is part of a group
165+
"CacheOptions": {}, // optional due to grouping
166+
// ...
167+
}
168+
],
169+
"GlobalConfiguration": {
170+
"BaseUrl": "https://ocelot.net",
171+
"CacheOptions": {
172+
"RouteKeys": ["R1",], // if undefined or empty array, opts will apply to all routes
173+
"TtlSeconds": 300 // enable global caching for a duration of 5 minutes
174+
},
175+
// ...
176+
}
177+
}
178+
179+
Dynamic routes were not supported in versions prior to `24.1`_.
180+
Starting with version `24.1`_, global *cache options* for :ref:`Dynamic Routing <routing-dynamic>` were introduced.
181+
These global options may also be overridden in the ``DynamicRoutes`` configuration section, as defined by the :ref:`config-dynamic-route-schema`.
182+
183+
.. code-block:: json
184+
:emphasize-lines: 6-8, 17-20
185+
186+
{
187+
"DynamicRoutes": [
188+
{
189+
"Key": "", // optional
190+
"ServiceName": "my-service",
191+
"CacheOptions": {
192+
"TtlSeconds": 60 // 1-minute short-term caching
193+
}
194+
}
195+
],
196+
"GlobalConfiguration": {
197+
"BaseUrl": "https://ocelot.net",
198+
"DownstreamScheme": "http",
199+
"ServiceDiscoveryProvider": {
200+
// required section for dynamic routing
201+
},
202+
"CacheOptions": {
203+
"RouteKeys": [], // or null, no grouping, thus opts apply to all dynamic routes
204+
"TtlSeconds": 300 // enable global caching for a duration of 5 minutes
205+
}
206+
}
207+
}
208+
209+
In this configuration, a 5-minute *caching* duration is applied to all implicit dynamic routes.
210+
However, for the "my-service" service, the *caching* TTL has been explicitly reduced from 5 minutes to 1 minute.
211+
212+
.. note::
213+
214+
1. If the ``RouteKeys`` option is not defined or the array is empty in the global ``CacheOptions``, the global options will apply to all routes.
215+
If the array contains route keys, it defines a single group of routes to which the global options apply.
216+
Routes excluded from this group must specify their own route-level ``CacheOptions``.
217+
218+
2. Prior to version `23.3`_, global ``CacheOptions`` were not available.
219+
Starting with version `24.1`_, global configuration is supported for both static and dynamic routes.
77220

78221
.. Sample
79222
.. -----
80223
81224
.. If you look at the example `here <https://github.com/ThreeMammals/Ocelot/blob/main/test/Ocelot.ManualTest/Program.cs>`_ you can see how the cache manager is setup and then passed into the Ocelot ``AddCacheManager`` configuration method.
82225
.. You can use any settings supported by the **CacheManager** package and just pass them in.
83226
84-
Notes
85-
-----
86-
87-
Ocelot currently supports caching on the URL of the downstream service and setting a TTL in seconds to expire the cache.
88-
You can also clear the cache for a region by calling Ocelot's :ref:`administration-api`.
89-
90-
Your Own Caching
91-
----------------
227+
Custom Caching
228+
--------------
92229

93230
If you want to add your own caching method, implement the following interfaces and register them in DI e.g.
94231

@@ -100,12 +237,29 @@ If you want to add your own caching method, implement the following interfaces a
100237
* ``IOcelotCache<CachedResponse>`` this is for output caching.
101238
* ``IOcelotCache<FileConfiguration>`` this is for caching the file configuration if you are calling something remote to get your config such as Consul.
102239

103-
Future
104-
------
240+
Roadmap
241+
-------
105242

106243
Please dig into the Ocelot source code to find more.
107244
We would really appreciate it if anyone wants to implement `Redis <https://redis.io/>`_, `Memcached <http://www.memcached.org/>`_ etc.
108245
Please, open a new `Show and tell <https://github.com/ThreeMammals/Ocelot/discussions/categories/show-and-tell>`_ thread in `Discussions <https://github.com/ThreeMammals/Ocelot/discussions>`_ space of the repository.
109246

247+
""""
248+
249+
.. [#f1] Historically, *Caching* is one of Ocelot's earliest features, first introduced in version `1.1`_ on February 2, 2017, the initial release of Ocelot.
250+
The "Clear cache region via :ref:`administration-api`" feature was first delivered in pull request `109`_ and released in version `1.4.8`_.
251+
.. [#f2] The ":ref:`EnableContentHashing option <caching-enablecontenthashing-option>`" feature was requested in issue `2059`_ and released in version `23.3`_.
252+
.. [#f3] :ref:`Global Configuration <caching-global-configuration>` for static routes was first introduced in pull request `2058`_ and released in version `23.3`_.
253+
Support for dynamic routes was added in pull request `2331`_ and delivered in version `24.1`_.
254+
255+
.. _109: https://github.com/ThreeMammals/Ocelot/pull/109
256+
.. _2058: https://github.com/ThreeMammals/Ocelot/pull/2058
257+
.. _2059: https://github.com/ThreeMammals/Ocelot/issues/2059
258+
.. _2234: https://github.com/ThreeMammals/Ocelot/issues/2234
259+
.. _2331: https://github.com/ThreeMammals/Ocelot/pull/2331
260+
261+
.. _1.1: https://github.com/ThreeMammals/Ocelot/releases/tag/1.1.0
262+
.. _1.4.8: https://github.com/ThreeMammals/Ocelot/releases/tag/1.4.8
110263
.. _23.0: https://github.com/ThreeMammals/Ocelot/releases/tag/23.0.0
111264
.. _23.3: https://github.com/ThreeMammals/Ocelot/releases/tag/23.3.0
265+
.. _24.1: https://github.com/ThreeMammals/Ocelot/releases/tag/24.1.0

0 commit comments

Comments
 (0)