Workflow for extracting model training images

[john-]

For my use case I spend most of my time in Viseron fgathering images to train object detection models. To get these images out of Viseron seems like it should be easy but for a few reasons it is not for me. For sure I can list those reasons.

However, it would be great to be able to click on an event image and have Viseron provide the full size screen image without a bounding box (if present).

Anyone else have this need? Do you have a straightforward way of collecting full size images for model training purposes?

[roflcoopter]

Right now the snapshots stored are zoomed in on the object, but i could probably make that configurable.

But just so that i understand your use case properly, would you like to have both snapshots saved (a zoomed in with bounding box drawn, and one "unedited" snapshot), or would it suffice to choose one or the other?

[john-]

For my use case I do not need both. Choosing would be fine: Choice between zoomed with bounding box and full size unedited.

[john-]

I want to put some observations here that may be relevant. It may not make implementation easier so can be ignored.

When browsing EVENTS:

• When hovering over the recording icon, one or more full size images are available (via pop-up) to copy/paste. If recording is triggered by object detection, the images also contain bounding boxes. If they are triggered by motion, there is no bounding box.

For the change proposed in this ticket:

My initial thought is when hovering over the recordings triggered by object there would be a configuration to disable the bounding box on the screenshot. These images are already full size and can by copy/pasted for processing outside of Viseron if they did not have the bounding box. This would allow the top level EVENT listing to be unchanged: It would still show events zoomed in to the bounding box.

However, I see that I have some events indicated but no recording for them. I have trigger_event_recording set for every label in my config. If this is a misconfiguration or defect that could be fixed, then maybe this approach work instead of changing the snapshot image in the EVENT listing.

Here is an additional thing I noticed unrelated to this ticket:

I only now just realized that if I scroll down in the pop-up there is metadata listed as well as download icon (for wither snapshot or recording). No matter how small I make my Firefox display it always requires scrolling down to view this.

I also just noticed that if there are more than 3 recordings that scrolling is needed to see the 3rd recording (and beyond).

I mention this as I did not notice in the documentation here. Alternately, I am not sure what a software change would be to make this clearer.

[john-]

There is another approach I want to explore (before modifying user facing code).

I want to see what is involved in writing a script that will query the events and based on that export the images that correspond those those events. The export could include metadata in the name (e.g. label of object detected) or as a separate file (bounding box).

[roflcoopter]

Right that is actually a good idea. Viseron should store the full images, along with bounding boxes in the database so that they can be shown with/without bboxes.

Users could then toggle the bboxes on/off in the UI or when exporting.

What do you think about that?

[john-]

I like the approach of storing the data like you outlined.

There is already an image cache mechanism in Viseron to help UI performance? Might be needed if rendering is done from the raw data.

[roflcoopter]

No there is no such thing currently, would need to think about how to handle that. For simplicity its probably better to do the drawing and cropping server side, and store them somewhere with a configurable size in case of needing to reuse the images

[john-]

But just so that i understand your use case properly, would you like to have both snapshots saved (a zoomed in with bounding box drawn, and one "unedited" snapshot), or would it suffice to choose one or the other?

So my original answer was that I do not need both. However, I hacked a version of Viseron that does not do the bounding box or zoom/crop. It only saves the raw image. It turns out that it is nice to see the zoomed/boxed version to immediately see that was detected. If the detection is wrong I would then save the raw image for future training.

In addition to the approach of using the metadata to render the zoomed/cropped version for UI and make available the raw image for download, I was thinking if there was any feasibility of saving the zoomed/cropped version as a thumbnail inside the full image. I haven't found anything that does this but thought I would mention it as an alternate solution. In any case it might not be any more straightforward then rendering on demand or saving two versions.

[roflcoopter]

I think the best way is to save two versions, but make it configurable.

Rendering on demand will make the UI slower, and the Events page is already pretty heavy