Migrating to a unified gRPCServer package developed by both myself and s-celles - #72
Merged
Conversation
Point lib/gRPCServer.jl at bbfce7d on s-celles-merge: the merged collaboration between csvance and s-celles. It combines both codebases into one package built around a protojl codegen interface (messages, client stubs, and per-service registration functions in one generated module) over the s-celles runtime (GRPCServer, register_method!, ServerContext, typed streams, GRPCError/StatusCode), with a pluggable HTTP/2 backend. The legacy csvance-era runtime API (gRPCRouter, gRPCMethod, handle!, gRPCServiceCallException, GRPC_* constants) is gone, which is why the consumer packages migrate in the follow-up commits.
The merged package carries a new uuid (608c6337) and a 0.3 version, so every Project.toml that pins gRPCServer by uuid is updated and its compat moves from 0.1 to 0.3. gRPCClient's URL source is aligned on v1.1.0-rc2 (matching the root workspace; the rc1/rc2 split was a leftover). The client adds a gRPCServer compat entry for its test target. All environments (root, docs, examples, e2e, export test) share the new uuid so the workspace resolves against the vendored submodule.
…c codec ReactantServerCore no longer depends on gRPCClient or gRPCServer. The two pb files are regenerated with protojl while neither gRPC package is loaded, so they emit only the protobuf message types (no *_Client constructors, no register_*! functions, no import gRPC* lines), and the hand-split client/server stub files are deleted. Core's dependency closure drops gRPCServer and the unregistered gRPCClient (sourced by URL only; not in the General registry), which the gRPC-using packages now carry themselves. The wire codec is parametric over the pb module: decode_infer_request, encode_infer_response, decode_infer_response, encode_repository_index, encode_shm_*, deadline_params, and the tensor builders take a PB::Module first argument, with convenience forms defaulting to Core's own inference module. Consumer packages pass their own generated module, since message structs generated from the same proto are structurally identical but module-local.
The worker generates its own inference and control pb modules under src/proto (protojl with gRPCServer loaded: messages plus the per-service registration functions) and registers its handlers through register_GRPCInferenceService_<Rpc>! and register_ControlService! on the merged API: GRPCServer(host, port; context = InferContext(...)), ctx-first handler signatures, GRPCError(StatusCode.X) in place of gRPCServiceCallException(GRPC_*), a deadline converted back onto the scheduler's time_ns() clock via remaining_time, the single max_message_size cap in place of separate recv/send caps, and the Prometheus admission counters read from the server's own inflight and shed_total atomics. The Core re-export loop skips :inference and :control, which the worker's own generated modules shadow. Tests and the e2e client follow the new module layout.
The gateway generates its own inference and control pb modules under
src/proto (protojl with gRPCClient and gRPCServer loaded, since it both
terminates services and dials workers). Raw Vector{UInt8} forwarding
methods (ModelInfer, SystemSharedMemory register/unregister) and the
typed IsSameIPCNamespace RPC register through the generated
register_GRPCInferenceService_<Rpc>! functions with raw_request /
raw_response flags; ControlService CompactMemory and the gateway's own
GatewayControlService use the generated control module types; outbound
*_Client stubs and messages come from the gateway's modules. Error
mapping moves from gRPCServiceCallException to GRPCError(StatusCode.X),
and the self-check builds its probe from the local generated module.
Tests follow the new module layout.
The client generates its own inference pb module under src/proto (protojl with only gRPCClient loaded: messages plus the *_Client constructors, no server registration, so gRPCServer stays out of the client's [deps]). Requests, responses, and the generated client constructors now come from the client's own module, and the Core codec calls pass the client's module so requests serialize identically on the wire. Tests follow the new module layout.
The GatewayControlService_*_Client constructors and the control message types moved out of ReactantServerCore into the gateway package (Core keeps only the protobuf message types). The operator CLI now loads them from ReactantServerGateway.control.
.pi/skills holds grpcserver-jl and grpcserver-jl-dev installed from ~/Git/gRPCServer.jl/skills. They are local working aids for this repository and must not be committed.
The two projects that use gRPCServer as a runtime dependency (the ReactantServer worker and ReactantServerGateway) now pull it directly from https://github.com/csvance/gRPCServer.jl at the s-celles-merge branch instead of the local submodule path. The branch tip (bbfce7d) is exactly the commit the submodule points at, so the resolved code is identical; the change removes these two projects' reliance on the vendored checkout. The remaining [sources] entries (client test env, examples, e2e, export test envs, docs, root) still use the submodule path.
…ronment Follows the worker and gateway switch: the remaining [sources] entries (root, docs, client test env, e2e, export test envs, and the two example servers) now resolve gRPCServer from github.com/csvance/gRPCServer.jl at the s-celles-merge branch instead of the lib/ submodule path. The example servers' gRPCClient path source pointed at a lib/gRPCClient.jl that never existed; it now uses the same URL source as the rest of the workspace. No [sources] entry references the submodule anymore, so it can be deleted.
Every environment now sources gRPCServer from its GitHub s-celles-merge branch, so the lib/gRPCServer.jl submodule and .gitmodules are deleted. The docker build no longer copies lib/gRPCServer.jl into the image (Pkg resolves it from GitHub during instantiate), and the README, CONTRIBUTING, docs pages, docker README, docker-compose, Makefile, and the CI workflows drop the submodule init steps and the lib path-change filters. lib/ is now empty and the repo has no git submodules left.
Adding consistent [compat] "1.1.0" for gRPCClient
gRPCServer now exposes per-backend constructors (GRPCServerHTTPJl / GRPCServerPureHTTP2 / GRPCServerNghttp2) that fix the backend and validate explicitly-set config keywords against the backend's capabilities, raising UnsupportedFeatureError for anything it cannot honor. The worker's build_grpc_server, the gateway's server build, and the three test mock servers switch to GRPCServerHTTPJl so the HTTPjl backend is guaranteed even if the default ever changes, and any future unsupported kwarg fails loudly at construction instead of being silently ignored. All kwargs these sites pass (context, per-direction message caps, max_concurrent_requests, h2 window sizes) are supported on HTTPjl. max_concurrent_streams is now enforced on HTTPjl (default 100 per connection); neither the worker nor the gateway sets it, and both stay under the cap (worker admission 64, gateway outbound 32 per connection).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.