Default NAP security-violation logs to be gzipped individually via custom processor

[kamalchaturvedi]

Proposed changes

We added a custom log processor logsgzip, that performs gzip compression on each log processed by the OpenTelemetry collector log pipeline in Agent V3. This processor is plugged to the pipeline before the batch processor.

Purpose: It compresses individual log records into gzip format to reduce their size, which improves performance when processing large log volumes.

Execution:

• Filters log records to only process those with STRING or BYTES data types.
• Compresses the content of each valid log record using gzip.
• Replaces the original log content body with the compressed version
• Passes the compressed logs to the next consumer in the pipeline

Performance optimization: The processor uses a sync.Pool to efficiently reuse gzip.Writer instances, reducing memory allocations and garbage collection overhead during high-throughput processing.

Error handling: It collects and aggregates errors that occur during processing but continues to process as many logs as possible.

Benchmarking: We performed benchmarks of this processor with and without a sync.Pool for efficient utilization of gzip writer. Results have been summarized in the README.md.

Updated OTLP Logs Pipeline Structure (from processors section)

processors:
  resource:
    attributes:
      - key: resource.id
        action: insert
        value: 95ced039-0aaa-3b06-90ff-c61109e96fc5
  batch:
    send_batch_size: 8192
    timeout: 20s
    send_batch_max_size: 0
  logsgzip: {}

exporters:
  otlp/0:
    endpoint: "connect.saas.crt.nginxlab.net:443"
    timeout: 10s
    retry_on_failure:
      enabled: true
      initial_interval: 10s
      max_interval: 60s
      max_elapsed_time: 10m
    tls:
      insecure: false
      insecure_skip_verify: false
      
      
      
    auth:
      authenticator: headers_setter
extensions:
  headers_setter:
    headers:
      - action: "insert"
        key: "authorization"
        value: "LwL2DzRt2BjTdeeNuyrBhrzC/bxtJnJHBxIyBIfqeyE="
      - action: "insert"
        key: "uuid"
        value: "95ced039-0aaa-3b06-90ff-c61109e96fc5"

service:
  telemetry:
    logs:
      level: INFO
      output_paths: ["/var/log/nginx-agent/opentelemetry-collector-agent.log"]
      error_output_paths: ["/var/log/nginx-agent/opentelemetry-collector-agent.log"]
  extensions:
    - headers_setter
  pipelines:
    metrics:
      receivers:
        - hostmetrics
      processors:
        - resource
        - batch
      exporters:
        - otlp/0
    logs:
      receivers:
        - tcplog/0
        - tcplog/1
      processors:
        - resource
        - logsgzip
        - batch
      exporters:
        - otlp/0

Checklist

Before creating a PR, run through this checklist and mark each as complete.

• I have read the CONTRIBUTING document
• I have run make install-tools and have attached any dependency changes to this pull request
• If applicable, I have added tests that prove my fix is effective or that my feature works
• If applicable, I have checked that any relevant tests pass after adding my changes
• If applicable, I have updated any relevant documentation (README.md)
• If applicable, I have tested my cross-platform changes on Ubuntu 22, Redhat 8, SUSE 15 and FreeBSD 13

[github-actions]

✅ All required contributors have signed the F5 CLA for this PR. Thank you!
_{Posted by the CLA Assistant Lite bot.}

[kamalchaturvedi]

I have hereby read the F5 CLA and agree to its terms

[kamalchaturvedi]

recheck