3.2.0 - Proper live streaming

[roflcoopter]

Proper live streaming is now available in Viseron, utilizing the great go2rtc application.
Previously Viseron resorted to using the MJPEG stream for live streaming, which results in low quality and high latency.
With the new go2rtc component, Viseron can now stream using WebRTC or MSE, which gives a much better experience.
The new Live page is available at /live and can be accessed from the sidebar.
The Live page is very similar to the Events page, using the same responsiveness so that you can view multiple cameras at once.

In order to use it, you will need to update your configuration to include the new go2rtc component.

Breaking changes

• port in webserver is now deprecated. In order to support the new go2rtc streaming server, an NGINX reverse proxy is now included in the container, which listens to the old default port 8888

New features

• A new Live Page is now available, which allows for better live streaming and support for more protocols.
  • The Live Page is available at /live and can be accessed from the main menu. It utilizes the new go2rtc component to support WebRTC/MSE streaming.
  • The MJPEG stream can now be accessed at the Live page by using the toggle in the extra options menu of the video player.
• A new go2rtc streaming server is now included in the container. This allows for better live streaming and support for more protocols.
• NGINX reverse proxy - A reverse proxy is now included in the container to enhance performance and provide live streaming through go2rtc.
• A new YOLOcomponent which uses Ultralytics YOLOv8 for object detection is now available.
• You can now directly go to a specific timestamp on Events Page by using query parameters. More information on this can be found in the documentation.
• The EdgeTPU component can now utilize multiple EdgeTPU devices for better performance. See the updated documentation for more information.
• Fullscreen button is now available on the Events Page (and the new Live Page) to allow for a better viewing experience.
  Does not work in iOS, because Apple.
• New way to calculate which files to move or delete in the tiered storage system.
  The new implementation uses numpy instead of pure SQL queries, which should improve performance at the cost of slightly (temporary) increased memory usage.
  You can limit the CPU usage of this calculation by setting the new tier_check_cpu_limit config option.
  You can further tune the workload using the config options tier_check_batch_size, tier_check_sleep_between_batches and tier_check_workers.

Changes

• HLS.js upgraded to 1.6.2
• If you have a substream with different resolution than the main stream, the video player now gets the correct resolution from the main stream
• EdgeTPU devices will now be restarted in case of multiple errors, helping to recover from temporary issues.
• Segment processing is now done in a separate process, for better performance and stability of the main process.
• Some unusual circumstances may cause the size based cleanup to fail. If thats the case and the first tier is full, a tier check as well as a cleanup job will be scheduled to remedy that.

Fixes

• Camera name overlapping the video stream controls on the Events Page
• PTZ component failing to initialize due to missing devicemgmt.wsdl file
• Fix for DeepStack face recognition not working
• Includes coco.names file in the RPI3 build, which is required for the Darknet object detection model.
• Properly resubscribe to MQTT topics when the connection is lost.

Developer experience

• Fixes a long standing issue with the Devcontaner failing to start due to a missing config folder.
  When cloning using the instructions in the Developer docs, the config folder was not present on the host since cloneInVolume was used.
  The config folder is now dynamically set to the correct path in the Devcontainer using the env variable VISERON_CONFIG_PATH.

Docker images will be on Docker Hub shortly

All Changes

• pin gpac version to 2.4.0 by @roflcoopter in pin gpac version to 2.4.0 #983
• set requested timestamp from query parameter by @roflcoopter in set requested timestamp from query parameter #984
• expose main stream resolution to get correct aspect ratio for streams by @roflcoopter in expose main stream resolution to get correct aspect ratio for streams #988
• ptz: find location of .wsdl file by @roflcoopter in ptz: find location of .wsdl file #989
• Go2rtc and NGINX by @roflcoopter in Go2rtc and NGINX #991
• implement EdgeTPU support for multiple devices by @roflcoopter in implement EdgeTPU support for multiple devices #992
• add title to config.yaml code blocks by @roflcoopter in add title to config.yaml code blocks #993
• edgetpu restart on failure by @roflcoopter in edgetpu restart on failure #994
• read frames in subprocess by @roflcoopter in read frames in subprocess #995
• merge master back into dev by @roflcoopter in merge master back into dev #1002
• Live page by @roflcoopter in Live page #1007
• run fragmenter in subprocess by @roflcoopter in run fragmenter in subprocess #1008
• trigger tier check when fragmenter cant move file to first tier by @roflcoopter in trigger tier check when fragmenter cant move file to first tier #1014
• remove config bind mount and change its location in devcontainer by @roflcoopter in remove config bind mount and change its location in devcontainer #1015
• upgrade to hls.js 1.6.5 by @roflcoopter in upgrade to hls.js 1.6.5 #1017
• camera snapshot auth using cookies by @roflcoopter in camera snapshot auth using cookies #1016
• add ssh to devcontainer by @roflcoopter in add ssh to devcontainer #1022
• properly close queues in ChildProcessWorker by @roflcoopter in properly close queues in ChildProcessWorker #1023
• wrap start of queue manager server in a lock by @roflcoopter in wrap start of queue manager server in a lock #1024
• include logo as png by @roflcoopter in include logo as png #1025
• catch TypeError in duplicate log filter by @roflcoopter in catch TypeError in duplicate log filter #1026
• update throttling mechanism with unique throttle key by @roflcoopter in update throttling mechanism with unique throttle key #1028
• add index to Files table to improve tier check performance by @roflcoopter in add index to Files table to improve tier check performance #1029
• make sure only one cleanup job can be started at a time by @roflcoopter in make sure only one cleanup job can be started at a time #1032
• delete unneeded binaries and docker images in ci by @roflcoopter in delete unneeded binaries and docker images in ci #1037
• fix timing issue with _concatenate_fragements by @john- in fix timing issue with _concatenate_fragements #1039
• Tier check file processing by @roflcoopter in Tier check file processing #1040
• copy coco.names to rpi image by @roflcoopter in copy coco.names to rpi image #1042
• fix: update deepstack face detection coordinates to use correct keys by @roflcoopter in fix: update deepstack face detection coordinates to use correct keys #1041
• yolo component by @john- in yolo component #1033
• extract tag from docker compose file before tagging by @roflcoopter in extract tag from docker compose file before tagging #1043
• resub on mqtt reconnection by @roflcoopter in resub on mqtt reconnection #1046
• make mjpeg stream viewable from Live page by @roflcoopter in make mjpeg stream viewable from Live page #1047
• force pytorch CPU install in CI by @roflcoopter in force pytorch CPU install in CI #1048
• cap HLS error to max 200 chars by @roflcoopter in cap HLS error to max 200 chars #1051
• enable react query retries by @roflcoopter in enable react query retries #1052
• fix cleanup jobs to run on all tier paths by @roflcoopter in fix cleanup jobs to run on all tier paths #1053
• add live view documentation by @roflcoopter in add live view documentation #1054

New Contributors

• @john- made their first contribution in fix timing issue with _concatenate_fragements #1039

Full Changelog: v3.1.2...v3.2.0

---

This discussion was created from the release 3.2.0 - Proper live streaming.

[dodog]

First of all — thanks for the continued work on Viseron and the v3.2.0 release!

I noticed when updating to v3.2.0 that the Docker image size has increased significantly compared to v3.1.2 — it went from about 4,4 GB to about 13,1 GB (roughly 3× larger).

Is the new go2rtc component cause of the growth?
Would it be possible confirm what is contributing to the increased image size?

[john-]

The Ultralytics library and dependencies (for a broad range of YOLO versions) is pretty large. That may be a contributor. I am not in touch with the other changes.

[roflcoopter]

@john- is probably right, will see if it's possible to reduce the size.

Which Docker image are you pulling?

[dodog]

ok, thank you for explanation. I am pulling roflcoopter/viseron:latest (linux/amd64). Compressed is roughly 8GB uncommpressed 13 GB.

[mianos]

On the go2rtc, I have added it and the streaming is suburb, but I was thinking, should I be running the go2rtc stream back into the ffmpeg instead of reading from the cameras twice, at the same time? The feed used a uuid it seems to I am not sure what the source would be?

Thanks in advance, and thanks for fixing the 3.2 for me.

rfo@mega:~$ cat crap.yaml
go2rtc:
  streams:
    door:
      - rtsp://admin:[email protected]:554/Preview_01_main
    car:
      - rtsp://admin:[email protected]:554/Preview_01_main

#  webrtc:
#    candidates:
#      - 192.168.XX.X:8555
#      - stun:8555

  log:
    level: debug


ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel
  camera:
    door:
      name: Front door
      host: door.mianos.com
      port: 554
      path: /Preview_01_main #Reolink path either /Preview_01_main or /Preview_01_sub
      username: admin
      password: nope
      substream:
        port: 554
        path: /Preview_01_sub
      mjpeg_streams:
        my_stream:
          draw_objects: true
        objects:
          draw_objects: true
          draw_motion: true
      recorder:
        idle_timeout: 5
      frame_timeout: 10
    car:
      name: Car
      host: car.mianos.com
      port: 554
      path: /Preview_01_main #Reolink path either /Preview_01_main or /Preview_01_sub
      username: admin
      password: nope
      substream:
        port: 554
        path: /Preview_01_sub
      mjpeg_streams:
        my_stream:
          draw_objects: true
        objects:
          draw_objects: true
          draw_motion: true
      recorder:
        idle_timeout: 5
      frame_timeout: 10

[roflcoopter]

Yes using the go2rtc stream can be beneficial when it comes to reducing the connections to your camera.

This is a basic config for how that wuld look like (also adding this to the docs for future reference)

ffmpeg:
  camera:
    camera_one:
      name: Camera 1
      host: localhost
      port: 8554
      path: /camera_one

    camera_two:
      name: Camera 2
      host: localhost
      port: 8554
      path: /camera_two

go2rtc:
  streams:
    camera_one:
      - rtsp://user:[email protected]:554/Streaming/Channels/101/
    camera_two:
      - rtsp://user:[email protected]:554/Streaming/Channels/101/

[mianos]

thanks, so i have reolink cameras, with the substream, this is is my working config:

go2rtc:
  streams:
    door:
      - rtsp://user:[email protected]:554/Preview_01_main
    car:
      - rtsp://user:[email protected]:554/Preview_01_main
    door_sub:
      - rtsp://user:[email protected]:554/Preview_01_sub
    car_sub:
      - rtsp://user:[email protected]:554/Preview_01_sub
  log:
    level: debug


ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel
  camera:
    door:
      name: Front door
      host: localhost
      port: 8554
      path: /door
      substream:
        port: 8554
        path: /door_sub
      mjpeg_streams:
        objects:
          draw_objects: true
          draw_motion: true
      recorder:
        idle_timeout: 5
      frame_timeout: 10
    car:
      name: Car
      host: localhost
      port: 8554
      path: /car
      substream:
        port: 8554
        path: /car_sub
      mjpeg_streams:
        objects:
          draw_objects: true
          draw_motion: true
      recorder:
        idle_timeout: 5
      frame_timeout: 10