[Feature Request]: Add Triton Inference Server support for RunInference transform

[SaiShashank12]

What would you like to happen?

Summary

This PR adds support for Triton Inference Server in Apache Beam’s RunInference transform by implementing a TritonModelHandler class.
What does this PR do?
• Implements TritonModelHandler that extends ModelHandler[str, PredictionResult, Model]
• Enables inference on text data using Triton Inference Server models
• Supports batch processing of text strings through the Beam pipeline
• Handles model loading, initialization, and inference execution with Triton server
Key Features
• Model Loading: Initializes Triton server with configurable model repository and model name
• Batch Inference: Processes sequences of text strings efficiently
• Result Handling: Parses JSON responses from Triton and returns structured PredictionResult objects
• Flexible Configuration: Supports custom inference arguments
Use Case
This handler allows users to leverage Triton Inference Server’s optimized inference capabilities within Apache Beam pipelines, particularly useful for:
• Text classification tasks
• Document processing pipelines
• Real-time and batch ML inference workloads

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[SaiShashank12]

.take-issue

[SaiShashank12]

@neilbhutada

[neilbhutada]

Hey, sorry for getting to this right now. But basically, the trick is to use Triton Server's In-process Python C-API.
These are some great places to start that talk about this concept:

• https://www.youtube.com/watch?v=oHx77B5i8vk
• https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/client_guide/python.html
• https://github.com/triton-inference-server/tutorials/tree/main/Triton_Inference_Server_Python_API

I was inspired by what NVIDIA proposed to for Kafka and applied the same concept to Dataflow:

• https://github.com/triton-inference-server/tutorials/tree/main/Triton_Inference_Server_Python_API/examples/kafka-io