Skip to content

Commit f58dbbf

Browse files
authored
Add support for return/ThrowBehavior of services (#148)
* Add support for `return/ThrowBehavior` of services # Motivation Currently the `ServiceGroup` cancels the whole group when a service returns or throws. This is fine for most server applications but it doesn't work for CLI tools since they mostly want to use the group to orchestrate the services as long as the user command is handled and then return cleanly from the group. Furthermore, even on server setups one might want to customize the behavior of what happens when a service returns/throws e.g. one might want to shutdown the group after their HTTP server throws so that a telemetry service flushes out all the remaining data. # Modification This PR does a few things: 1. It creates a new `ServiceConfiguration` and `TerminationBehavior` in the `ServiceGroupConfiguration`. Those can be used to declare the services that are run and what happens when the service returns or throws 2. Adds a new `cancellationSignals` to to the `ServiceGroupConfiguration` which allows cancellation to trigger based on a signal 3. Makes sure that any given service is only retained as long as necessary i.e. when a service returns the group is not retaining it anymore. This allows freeing of resources as early as possible 4. Breaking: Removes the `Hashable` conformance on the configuration structs. This was wrong in the first place and something we should just avoid doing in general. # Result We now give the user even more control about the group's behaviors. * Review * Review * Fix CI
1 parent 36f71f6 commit f58dbbf

8 files changed

+1051
-259
lines changed

Sources/ServiceLifecycle/Docs.docc/How to adopt ServiceLifecycle in applications.md

+136-63
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,49 @@
11
# How to adopt ServiceLifecycle in applications
22

3-
``ServiceLifecycle`` aims to provide a unified API that services should adopt to make orchestrating
4-
them in an application easier. To achieve this ``ServiceLifecycle`` is providing the ``ServiceGroup`` actor.
3+
``ServiceLifecycle`` aims to provide a unified API that services should adopt to
4+
make orchestrating them in an application easier. To achieve this
5+
``ServiceLifecycle`` is providing the ``ServiceGroup`` actor.
56

67
## Why do we need this?
78

8-
When building applications we often have a bunch of services that comprise the internals of the applications.
9-
These services include fundamental needs like logging or metrics. Moreover, they also include
10-
services that compromise the application's business logic such as long-running actors.
11-
Lastly, they might also include HTTP, gRPC, or similar servers that the application is exposing.
12-
One important requirement of the application is to orchestrate the various services currently during
13-
startup and shutdown. Furthermore, the application also needs to handle a single service failing.
14-
15-
Swift introduced Structured Concurrency which already helps tremendously with running multiple
16-
async services concurrently. This can be achieved with the use of task groups. However, Structured
17-
Concurrency doesn't enforce consistent interfaces between the services, so it becomes hard to orchestrate them.
18-
This is where ``ServiceLifecycle`` comes in. It provides the ``Service`` protocol which enforces
19-
a common API. Additionally, it provides the ``ServiceGroup`` which is responsible for orchestrating
20-
all services in an application.
9+
When building applications we often have a bunch of services that comprise the
10+
internals of the applications. These services include fundamental needs like
11+
logging or metrics. Moreover, they also include services that compromise the
12+
application's business logic such as long-running actors. Lastly, they might
13+
also include HTTP, gRPC, or similar servers that the application is exposing.
14+
One important requirement of the application is to orchestrate the various
15+
services during startup and shutdown.
16+
17+
Swift introduced Structured Concurrency which already helps tremendously with
18+
running multiple asynchronous services concurrently. This can be achieved with
19+
the use of task groups. However, Structured Concurrency doesn't enforce
20+
consistent interfaces between the services, so it becomes hard to orchestrate
21+
them. This is where ``ServiceLifecycle`` comes in. It provides the ``Service``
22+
protocol which enforces a common API. Additionally, it provides the
23+
``ServiceGroup`` which is responsible for orchestrating all services in an
24+
application.
2125

2226
## Adopting the ServiceGroup in your application
2327

24-
This article is focusing on how the ``ServiceGroup`` works and how you can adopt it in your application.
25-
If you are interested in how to properly implement a service, go check out the article: <doc:How-to-adopt-ServiceLifecycle-in-libraries>.
28+
This article is focusing on how the ``ServiceGroup`` works and how you can adopt
29+
it in your application. If you are interested in how to properly implement a
30+
service, go check out the article:
31+
<doc:How-to-adopt-ServiceLifecycle-in-libraries>.
2632

2733
### How is the ServiceGroup working?
2834

29-
The ``ServiceGroup`` is just a slightly complicated task group under the hood that runs each service
30-
in a separate child task. Furthermore, the ``ServiceGroup`` handles individual services exiting
31-
or throwing unexpectedly. Lastly, it also introduces a concept called graceful shutdown which allows
32-
tearing down all services in reverse order safely. Graceful shutdown is often used in server
33-
scenarios i.e. when rolling out a new version and draining traffic from the old version.
35+
The ``ServiceGroup`` is just a complicated task group under the hood that runs
36+
each service in a separate child task. Furthermore, the ``ServiceGroup`` handles
37+
individual services exiting or throwing. Lastly, it also introduces a concept
38+
called graceful shutdown which allows tearing down all services in reverse order
39+
safely. Graceful shutdown is often used in server scenarios i.e. when rolling
40+
out a new version and draining traffic from the old version (commonly referred
41+
to as quiescing).
3442

3543
### How to use the ServiceGroup?
3644

37-
Let's take a look how the ``ServiceGroup`` can be used in an application. First, we define some
38-
fictional services.
45+
Let's take a look how the ``ServiceGroup`` can be used in an application. First,
46+
we define some fictional services.
3947

4048
```swift
4149
struct FooService: Service {
@@ -53,11 +61,12 @@ public struct BarService: Service {
5361
}
5462
```
5563

56-
The `BarService` is depending in our example on the `FooService`. A dependency between services
57-
is quite common and the ``ServiceGroup`` is inferring the dependencies from the order of the
58-
services passed to the ``ServiceGroup/init(services:configuration:logger:)``. Services with a higher
59-
index can depend on services with a lower index. The following example shows how this can be applied
60-
to our `BarService`.
64+
The `BarService` is depending in our example on the `FooService`. A dependency
65+
between services is quite common and the ``ServiceGroup`` is inferring the
66+
dependencies from the order of the services passed to the
67+
``ServiceGroup/init(configuration:)``. Services with a higher index can depend
68+
on services with a lower index. The following example shows how this can be
69+
applied to our `BarService`.
6170

6271
```swift
6372
@main
@@ -68,9 +77,13 @@ struct Application {
6877

6978
let serviceGroup = ServiceGroup(
7079
// We are encoding the dependency hierarchy here by listing the fooService first
71-
services: [fooService, barService],
72-
configuration: .init(gracefulShutdownSignals: []),
73-
logger: logger
80+
configuration: .init(
81+
services: [
82+
.init(service: fooService),
83+
.init(service: barService)
84+
],
85+
logger: logger
86+
),
7487
)
7588

7689
try await serviceGroup.run()
@@ -80,17 +93,26 @@ struct Application {
8093

8194
### Graceful shutdown
8295

83-
The ``ServiceGroup`` supports graceful shutdown by taking an array of `UnixSignal`s that trigger
84-
the shutdown. Commonly `SIGTERM` is used to indicate graceful shutdowns in container environments
85-
such as Docker or Kubernetes. The ``ServiceGroup`` is then gracefully shutting down each service
86-
one by one in the reverse order of the array passed to the init.
87-
Importantly, the ``ServiceGroup`` is going to wait for the ``Service/run()`` method to return
96+
Graceful shutdown is a concept from service lifecycle which aims to be an
97+
alternative to task cancellation that is not as forceful. Graceful shutdown
98+
rather lets the various services opt-in to supporting it. A common example of
99+
when you might want to use graceful shutdown is in containerized enviroments
100+
such as Docker or Kubernetes. In those environments, `SIGTERM` is commonly used
101+
to indicate to the application that it should shut down before a `SIGKILL` is
102+
sent.
103+
104+
The ``ServiceGroup`` can be setup to listen to `SIGTERM` and trigger a graceful
105+
shutdown on all its orchestrated services. It will then gracefully shut down
106+
each service one by one in reverse startup order. Importantly, the
107+
``ServiceGroup`` is going to wait for the ``Service/run()`` method to return
88108
before triggering the graceful shutdown on the next service.
89109

90-
Since graceful shutdown is up to the individual services and application it requires explicit support.
91-
We recommend that every service author makes sure their implementation is handling graceful shutdown
92-
correctly. Lastly, application authors also have to make sure they are handling graceful shutdown.
93-
A common example of this is for applications that implement streaming behaviours.
110+
Since graceful shutdown is up to the individual services and application it
111+
requires explicit support. We recommend that every service author makes sure
112+
their implementation is handling graceful shutdown correctly. Lastly,
113+
application authors also have to make sure they are handling graceful shutdown.
114+
A common example of this is for applications that implement streaming
115+
behaviours.
94116

95117
```swift
96118
struct StreamingService: Service {
@@ -126,27 +148,32 @@ struct Application {
126148
})
127149

128150
let serviceGroup = ServiceGroup(
129-
services: [streamingService],
130-
configuration: .init(gracefulShutdownSignals: [.sigterm]),
131-
logger: logger
151+
configuration: .init(
152+
services: [.init(service: streamingService)],
153+
gracefulShutdownSignals: [.sigterm],
154+
logger: logger
155+
)
132156
)
133157

134158
try await serviceGroup.run()
135159
}
136160
}
137161
```
138162

139-
The code above demonstrates a hypothetical `StreamingService` with a configurable handler that
140-
is invoked per stream. Each stream is handled in a separate child task concurrently.
141-
The above code doesn't support graceful shutdown right now. There are two places where we are missing it.
142-
First, the service's `run()` method is iterating the `makeStream()` async sequence. This iteration is
143-
not stopped on graceful shutdown and we are continuing to accept new streams. Furthermore,
144-
the `streamHandler` that we pass in our main method is also not supporting graceful shutdown since it
145-
is iterating over the incoming requests.
146-
147-
Luckily, adding support in both places is trivial with the helpers that ``ServiceLifecycle`` exposes.
148-
In both cases, we are iterating an async sequence and what we want to do is stop the iteration.
149-
To do this we can use the `cancelOnGracefulShutdown()` method that ``ServiceLifecycle`` adds to
163+
The code above demonstrates a hypothetical `StreamingService` with a
164+
configurable handler that is invoked per stream. Each stream is handled in a
165+
separate child task concurrently. The above code doesn't support graceful
166+
shutdown right now. There are two places where we are missing it. First, the
167+
service's `run()` method is iterating the `makeStream()` async sequence. This
168+
iteration is not stopped on graceful shutdown and we are continuing to accept
169+
new streams. Furthermore, the `streamHandler` that we pass in our main method is
170+
also not supporting graceful shutdown since it is iterating over the incoming
171+
requests.
172+
173+
Luckily, adding support in both places is trivial with the helpers that
174+
``ServiceLifecycle`` exposes. In both cases, we are iterating an async sequence
175+
and what we want to do is stop the iteration. To do this we can use the
176+
`cancelOnGracefulShutdown()` method that ``ServiceLifecycle`` adds to
150177
`AsyncSequence`. The updated code looks like this:
151178

152179
```swift
@@ -183,18 +210,64 @@ struct Application {
183210
})
184211

185212
let serviceGroup = ServiceGroup(
186-
services: [streamingService],
187-
configuration: .init(gracefulShutdownSignals: [.sigterm]),
188-
logger: logger
213+
configuration: .init(
214+
services: [.init(service: streamingService)],
215+
gracefulShutdownSignals: [.sigterm],
216+
logger: logger
217+
)
189218
)
190219

191220
try await serviceGroup.run()
192221
}
193222
}
194223
```
195224

196-
Now one could ask - Why aren't we using cancellation in the first place here? The problem is that
197-
cancellation is forceful and doesn't allow users to make a decision if they want to cancel or not.
198-
However, graceful shutdown is very specific to business logic often. In our case, we were fine with just
199-
stopping to handle new requests on a stream. Other applications might want to send a response indicating
200-
to the client that the server is shutting down and waiting for an acknowledgment of that message.
225+
Now one could ask - Why aren't we using cancellation in the first place here?
226+
The problem is that cancellation is forceful and doesn't allow users to make a
227+
decision if they want to cancel or not. However, graceful shutdown is very
228+
specific to business logic often. In our case, we were fine with just stopping
229+
to handle new requests on a stream. Other applications might want to send a
230+
response indicating to the client that the server is shutting down and waiting
231+
for an acknowledgment of that message.
232+
233+
### Customizing the behavior when a service returns or throws
234+
235+
By default the ``ServiceGroup`` is cancelling the whole group if the one service
236+
returns or throws. However, in some scenarios this is totally expected e.g. when
237+
the ``ServiceGroup`` is used in a CLI tool to orchestrate some services while a
238+
command is handled. To customize the behavior you set the
239+
``ServiceGroupConfiguration/ServiceConfiguration/returnBehaviour`` and
240+
``ServiceGroupConfiguration/ServiceConfiguration/throwBehaviour``. Both of them
241+
offer three different options. The default behavior for both is
242+
``ServiceGroupConfiguration/ServiceConfiguration/TerminationBehavior/cancelGroup``.
243+
You can also choose to either ignore if a service returns/throws by setting it
244+
to ``ServiceGroupConfiguration/ServiceConfiguration/TerminationBehavior/ignore``
245+
or trigger a graceful shutdown by setting it to
246+
``ServiceGroupConfiguration/ServiceConfiguration/TerminationBehavior/gracefullyShutdownGroup``.
247+
248+
Another example where you might want to use this is when you have a service that
249+
should be gracefully shutdown when another service exits, e.g. you want to make
250+
sure your telemetry service is gracefully shutdown after your HTTP server
251+
unexpectedly threw from its `run()` method. This setup could look like this:
252+
253+
```swift
254+
@main
255+
struct Application {
256+
static func main() async throws {
257+
let telemetryService = TelemetryService()
258+
let httpServer = HTTPServer()
259+
260+
let serviceGroup = ServiceGroup(
261+
configuration: .init(
262+
services: [
263+
.init(service: telemetryService),
264+
.init(service: httpServer, returnBehavior: .shutdownGracefully, throwBehavior: .shutdownGracefully)
265+
],
266+
logger: logger
267+
),
268+
)
269+
270+
try await serviceGroup.run()
271+
}
272+
}
273+
```

0 commit comments

Comments
 (0)