We already delete events that are older than one day to prevent the database from growing indefinitely. This should also be done for container logs. But this is not possible with our current schema, as the logs are a single field with no reference to time frames.
Maybe the following will work, please verify: Instead of always concatenating container logs, we could split them by hours:
- Introduce a new column indicating the start for the logs collected during this period, e.g.
period.
- Calculate period by
$time - $time % 3600. Note that we're working with millisecond timestamps.
- Use
LastUpdate to decide whether to set or concat Logs.
- Since we now have several rows per container in the database, the primary key of the table must be adjusted, as well as the select statement for the warm-up phase, since it must provide the latest timestamp per container.
- Delete container logs that are older than one day. Have a look at how it's done for events.
- Change Web accordingly.
We already delete events that are older than one day to prevent the database from growing indefinitely. This should also be done for container logs. But this is not possible with our current schema, as the logs are a single field with no reference to time frames.
Maybe the following will work, please verify: Instead of always concatenating container logs, we could split them by hours:
period.$time - $time % 3600. Note that we're working with millisecond timestamps.LastUpdateto decide whether to set or concatLogs.