Skip to content

Commit 3cffe3f

Browse files
Merge pull request #1147 from roboflow/feature/batch-processing-api
Add Swagger for Batch Processing and Data Staging services
2 parents 0c0c860 + f41ebf7 commit 3cffe3f

File tree

5 files changed

+1936
-4
lines changed

5 files changed

+1936
-4
lines changed

docs/workflows/batch_processing/about.md

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,17 @@ By installing `inference-cli` you gain access to the `inference rf-cloud` comman
6060
managed components of the Roboflow Platform — including `batch-processing` and `data-staging`, the core components of
6161
the Roboflow Batch Processing offering.
6262

63+
!!! hint "Inference CLI setup"
64+
65+
To follow the tutorial you must install `inference-cli` and export your Roboflow API key.
66+
67+
```bash
68+
pip install inference-cli
69+
export ROBOFLOW_API_KEY="YOUR-API-KEY-GOES-HERE"
70+
```
71+
72+
If you struggle to find the API key, check our [guide](https://docs.roboflow.com/api-reference/authentication).
73+
6374
The typical flow of interaction with the CLI is as follows:
6475

6576
First, ingest data into the platform. For images, use the following command:
@@ -89,20 +100,32 @@ Once the data is ingested - you can trigger batch job.
89100
For images, use the following command:
90101

91102
```bash
92-
inference rf-cloud batch-processing process-images-with-workflow --workflow-id <workflow-id> --batch-id <batch-id>
103+
inference rf-cloud batch-processing process-images-with-workflow \
104+
--workflow-id <workflow-id> \
105+
--batch-id <batch-id> \
106+
--machine-type gpu
93107
```
94108

95109
For videos:
96110

97111
```bash
98-
inference rf-cloud batch-processing process-videos-with-workflow --workflow-id <workflow-id> --batch-id <batch-id>
112+
inference rf-cloud batch-processing process-videos-with-workflow \
113+
--workflow-id <workflow-id> \
114+
--batch-id <batch-id> \
115+
--machine-type gpu \
116+
--max-video-fps <your-desired-fps>
99117
```
100118

101119
!!! hint "How would I know `<workflow-id>`?"
102120

103121
Workflow ID can be found in Roboflow App - open Workflow Editor of selected Workflow, hit "Deploy" button
104122
and find identifier in code snippet.
105123

124+
!!! hint "GPU vs CPU"
125+
126+
By default, processing run on CPU device, but if you require extra compute power - use `--machine-type gpu`
127+
option of the above commands.
128+
106129
Command will **display the ID of the job**, which can be used to check the job status:
107130

108131
```bash
@@ -176,7 +199,7 @@ potential costs.
176199
* Certain Workflow blocks requiring access to env variables and local storage (like File Sink and Environment
177200
Secret Store) are blacklisted and will not execute.
178201

179-
* Service only works with Workflows that define **singe** input image parameter.
202+
* Service only works with Workflows that define **single** input image parameter.
180203

181204

182205
## Technical Details of Batch Processing
@@ -191,7 +214,7 @@ for processing (each batch is processed by a single job).
191214
output batch that you can retrieve later. We **advise** using `export` stage outputs, as they are optimized for
192215
network transfer (content is compressed / packed into an archive).
193216

194-
* A running job in the `processing` tage can be aborted using both the UI and CLI.
217+
* A running job in the `processing` stage can be aborted using both the UI and CLI.
195218

196219
* An aborted or failed job can be restarted using the mentioned tools.
197220

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Below you can find API reference for **Data Staging** and **Batch Processing** services. Specifications can
2+
be used to build API integrations.
3+
4+
<swagger-ui src="./batch_processing_swagger.json"/>
5+
6+
<swagger-ui src="./data_staging_swagger.json"/>

0 commit comments

Comments
 (0)