Skip to content

Commit 440263a

Browse files
dmorin1dmorin1
committed
Improvements based on comments
- Add tensor decoding logic - Add all ref to models - Add all ref to tensor-decodr - Change memory layout to be datatype agnostic
1 parent c4ff69b commit 440263a

5 files changed

+104
-76
lines changed

‎tensor-groups/ssd-mobilenet-v1-variant-1-out.md

+35-16
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,6 @@
33
- layer-type: output
44
- use-case: object-detection
55

6-
# Internal References
7-
8-
| Tensor | Link to tensor description |
9-
|--- |--- |
10-
|Count | [iref:count](/tensors/generic-variant-1-out-count.md) |
11-
|Boxes | [iref:boxes](/tensors/ssd-mobilenet-v1-variant-1-out-boxes.md) |
12-
|Classes | [iref:classes](/tensors/ssd-mobilenet-v1-variant-1-out-classes.md) |
13-
|Scores | [iref:scores](/tensors/ssd-mobilenet-v1-variant-1-out-scores.md) |
14-
15-
166
# Description
177

188
Variant of ssd-mobilenet-v1 outputs. This tensor group is contain of four
@@ -21,14 +11,43 @@ a scalar the represent the number of object detection in the other 3 tensors.
2111

2212
SSD-MobileNetV1 Variant-1 Output Tensors:
2313

24-
| Name |Internal Reference |shape |
25-
|--- |--- |--- |
26-
| Count |iref:count |1 x 1 |
27-
| Boxes |iref:boxes |1 x iref:boxes_count x 4 |
28-
| Classes |iref:classes |1 x iref:boxes_count |
29-
| Scores |iref:scores |1 x iref:boxes_count |
14+
| Name |shape |
15+
|--- |--- |
16+
| [count] |1 x 1 |
17+
| [boxes] |1 x [count] x 4 |
18+
| [classes] |1 x [count] |
19+
| [scores] |1 x [count] |
20+
21+
# Tensor Decoding Logic
22+
23+
```
24+
Foreach i in count:
25+
X = boxes[i, 0]
26+
Y = boxes[i, 1]
27+
W = boxes[i, 2]
28+
H = boxes[i, 3]
29+
C = classes[i]
30+
S = scores[i]
31+
detections[i] = [X, Y, W, H, C, S]
32+
```
3033

3134
# External References
3235

3336
* [MobileNets Paper](https://arxiv.org/pdf/1704.04861)
3437
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
38+
39+
# Models
40+
41+
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
42+
43+
# Tensor Decoders
44+
|Framework | Links |
45+
|--- |--- |
46+
|GStreamer | [perm](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/c206ddd9308a3ce529e0d8957b7c165b3a15c932/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c#L36-39) \| [latest](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c?ref_type=heads#L36-39) |
47+
48+
49+
[count]: /tensor-groups/generic-variant-1-out-count.md
50+
[boxes]: /tensors/ssd-mobilenet-v1-variant-1-out-boxes.md
51+
[classes]: /tensors/ssd-mobilenet-v1-variant-1-out-classes.md
52+
[scores]: /tensors/ssd-mobilenet-v1-variant-1-out-scores.md
53+

‎tensors/generic-variant-1-out-count.md

+5-6
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,8 @@ Scalar expressing a count.
1010

1111
## Count Tensor
1212

13-
- iref:count
1413
- tensor-shape: 1 x 1
15-
- tensor-datatype: float32
14+
- tensor-datatype: float32, uint8
1615
- tensor-id: generic-variant-1-count
1716

1817
## Known Aliases
@@ -23,9 +22,9 @@ Count value stored in the tensor.
2322

2423
Memory layout of tensor data:
2524

26-
|Address | Symbol | Value | Comment |
27-
|--- |--- |--- |--- |
28-
| 0x0 | tensor-start | count | complete tensor data |
29-
| 0x4 | tensor-ended | - | - |
25+
|Address| Symbol | Value | Comment |
26+
|--- |--- |--- |--- |
27+
| 0x0 | tensor-start | count | complete tensor data |
28+
| 0x4 | tensor-ended | - | - |
3029

3130

‎tensors/ssd-mobilenet-v1-variant-1-out-boxes.md

+33-30
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,13 @@
66
- part-of-tensor-groups:
77
- [ssd-mobilenet-v1-variant-1-out](/tensor-groups/ssd-mobilenet-v1-variant-1-out.md)
88

9-
# Internal References
10-
11-
| Tensor | Link to tensor description |
12-
|--- |--- |
13-
|Count | [iref:count](/tensors/generic-variant-1-out-count.md) |
14-
15-
169
# Description
1710
Location of objects detected
1811

19-
2012
## Boxes Tensor
2113

22-
- iref:boxes
23-
- tensor-shape: 1 x iref:box_count x 4
24-
- tensor-datatype: float32
14+
- tensor-shape: 1 x [count] x 4
15+
- tensor-datatype: float32, uint8
2516

2617
### Known Aliases
2718
* detection_boxes:0
@@ -30,31 +21,43 @@ Location of objects detected
3021

3122
Scheme: (X: box x coord, Y: box y coord, W: box width, H: box height)
3223

33-
|box-1 | box-1 | box-1 | box-1 | box-2 | box-2 | box-2 | box-2 | ... | box-iref:box_count|box-iref:box_count|box-iref:box_count|box-iref:box_count|
24+
|box-1 | box-1 | box-1 | box-1 | box-2 | box-2 | box-2 | box-2 | ... | [count]|[count]|[count]|[count]|
3425
|--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |
35-
| X | Y | W | H | X | Y | W | H |...| X | Y | Y | H |
26+
| X | Y | W | H | X | Y | W | H |...| X | Y | W | H |
3627

3728
Memory layout of tensor data:
3829

39-
|Address | Symbol |Value | Comment |
40-
|--- |--- |--- |--- |
41-
| - | - | - | - |
42-
|add | tensor-start |x-coord | box-0-tensor-data |
43-
|add + 32bits | tensor-continue |y-coord | box-0-tensor-data |
44-
|add + 64bits | tensor-continue |width | box-0-tensor-data |
45-
|add + 96bits | tensor-continue |height | box-0-tensor-data |
46-
|add + 128bits | tensor-continue |x-coord | box-1-tensor-data |
47-
|add + 160bits | tensor-continue |y-coord | box-1-tensor-data |
48-
|add + 192bits | tensor-continue |width | box-1-tensor-data |
49-
|add + 96bits | tensor-continue |height | box-1-tensor-data |
50-
|... | ... | ... | ... |
51-
|add + ((iref:count - 1) * 32)bits | tensor-continue |x-coord | iref:count-tensor-data |
52-
|add + ((iref:count - 1) * 32)+32bits | tensor-continue |y-coord | iref:count-tensor-data |
53-
|add + ((iref:count - 1) * 32)+64bits | tensor-continue |width | iref:count-tensor-data |
54-
|add + ((iref:count - 1) * 32)+96bits | tensor-continue |height | iref:count-tensor-data |
55-
| - | - | - | - |
30+
|Index | Symbol |Value | Comment |
31+
|--- |--- |--- |--- |
32+
| - | - | - | - |
33+
|0 | |x-coord |tensor-start, box-0-tensor-data |
34+
|1 | |y-coord |tensor-continue, box-0-tensor-data |
35+
|2 | |width |tensor-continue, box-0-tensor-data |
36+
|3 | |height |tensor-continue, box-0-tensor-data |
37+
|4 | |x-coord |tensor-continue, box-1-tensor-data |
38+
|5 | |y-coord |tensor-continue, box-1-tensor-data |
39+
|6 | |width |tensor-continue, box-1-tensor-data |
40+
|7 | |height |tensor-continue, box-1-tensor-data |
41+
|... | ... | ... | ... |
42+
|([count] - 1) x 4 | |x-coord |tensor-continue, [count]-tensor-data |
43+
|([count] - 1) x 4 + 1 | |y-coord |tensor-continue, [count]-tensor-data |
44+
|([count] - 1) x 4 + 2 | |width |tensor-continue, [count]-tensor-data |
45+
|([count] - 1) x 4 + 3 | |height |tensor-continue, [count]-tensor-data |
46+
| - | - | - | - |
5647

5748
# External References
5849

5950
* [MobileNets Paper](https://arxiv.org/pdf/1704.04861)
6051
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
52+
53+
# Models
54+
55+
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
56+
57+
# Tensor Decoders
58+
|Framework | Links |
59+
|--- |--- |
60+
|GStreamer | [perm](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/c206ddd9308a3ce529e0d8957b7c165b3a15c932/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c#L36-39) \| [latest](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c?ref_type=heads#L36-39) |
61+
62+
63+
[count]: /tensors/generic-variant-1-out-count.md

‎tensors/ssd-mobilenet-v1-variant-1-out-classes.md

+20-12
Original file line numberDiff line numberDiff line change
@@ -12,37 +12,45 @@ Classes of object associated with [object location](/tensors/ssd-mobilenet-v1-va
1212

1313
## Classes Tensor
1414

15-
iref:classes
16-
tensor-shape: 1 x iref:box_count
17-
tensor-datatype: float32
15+
tensor-shape: 1 x [count]
16+
tensor-datatype: float32, uint8
1817

1918
### Known Aliases
2019
* detection_classes:0
2120

2221
### Encoding
2322

2423
Scheme:
25-
|det-1 | det-2 | ... | det-iref:count |
24+
|det-1 | det-2 | ... | det-[count] |
2625
|--- |--- |--- |--- |
2726
| object-class | object-class | ... | object-class |
2827

2928

3029
Memory layout of tensor data:
3130

32-
|Address | Symbol |Value | Comment |
33-
|--- |--- |--- |--- |
34-
| - | - | - | - |
35-
|add | tensor-start | det-1-objet-class | tensor-data |
36-
|add+32bits | tensor-continue | det-2-objet-class | tensor-data |
37-
|... | ... | ... | tensor-data |
38-
|add+ ((iref:count - 1) * 32) bits | det-iref:count-objet-class | tensor-data |
39-
|add+ ((iref:count) * 32) bits | - | - |
31+
|Index |Value | Comment |
32+
|--- |--- |--- |
33+
| - | - | - |
34+
|0 | det-1-objet-class | tensor-start, tensor-data |
35+
|1 | det-2-objet-class | tensor-continue, tensor-data |
36+
|... | ... | tensor-continue, tensor-data |
37+
|[count] - 1 | det-[count]-objet-class | tneosr-continue, tensor-data |
38+
|([count]) | - | - |
4039

4140
# External References
4241

4342
* [MobileNets Paper](https://arxiv.org/pdf/1704.04861)
4443
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
4544

45+
# Models
46+
47+
* [ONNX model trained on COCO dataset](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/acc119dd795be5e8c756457dc04507a5d9b8e768/models/ssd_mobilenet_v1_coco.onnx)
48+
49+
# Tensor Decoders
50+
|Framework | Links |
51+
|--- |--- |
52+
|GStreamer | [perm](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/c206ddd9308a3ce529e0d8957b7c165b3a15c932/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c#L36-39) \| [latest](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c?ref_type=heads#L36-39) |
53+
4654

4755

4856

‎tensors/ssd-mobilenet-v1-variant-1-out-scores.md

+11-12
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,8 @@ Confidence levels of [classes](/tensors/ssd-mobilenet-v1-variant-1-out-classes.m
1212

1313
## Scores Tensor
1414

15-
iref: scores
16-
tensor-shape: 1 x iref:box_count
17-
tensor-datatype: float32
15+
tensor-shape: 1 x [count]
16+
tensor-datatype: float32, uint8
1817
tensor-id: out-ssd-mobilenet-v1-scores-variant-1
1918

2019
### Known Aliases
@@ -25,14 +24,14 @@ Number of detection correspond to the value stored in the tensor.
2524

2625
Memory layout of tensor data:
2726

28-
|Address | Symbol |Value | Comment |
29-
|--- |--- |--- |--- |
30-
| - | - | - | - |
31-
|add | tensor-start | det-1-objet-class-score | tensor-data |
32-
|add+32bits | tensor-continue | det-2-objet-class-score | tensor-data |
33-
|... | ... | ... | tensor-data |
34-
|add+ ((iref:count - 1) * 32) bits | tensor-end | det-iref:count-objet-class-score | tensor-data |
35-
|add+ ((iref:count) * 32) bits | - | - | - |
27+
|Index | Symbol |Value | Comment |
28+
|--- |--- |--- |--- |
29+
| - | - | - | - |
30+
|0 | tensor-start | det-1-objet-class-score | tensor-data, start |
31+
|1 i | tensor-continue | det-2-objet-class-score | tensor-data, continue |
32+
|... | ... | ... | tensor-data, continue |
33+
|([count] - 1) | tensor-end | det-[count]-objet-class-score | tensor-data, end |
34+
|[count] | - | - | - |
3635

3736
# External References
3837

@@ -48,7 +47,7 @@ Memory layout of tensor data:
4847
|GStreamer | [perm](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/c206ddd9308a3ce529e0d8957b7c165b3a15c932/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c#L36-39) \| [latest](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-bad/gst/tensordecoders/gstssdobjectdetector.c?ref_type=heads#L36-39) |
4948

5049

51-
50+
[count]: /tensors/generic-variant-1-out-count.md
5251

5352

5453

0 commit comments

Comments
 (0)