Added support for RFC5424 structured data (#67)

* Added support for RFC5424 structured data

Signed-off-by: Tero Saarni <[email protected]>

Author: tsaarni

CHANGELOG.md

@@ -1,5 +1,6 @@
 ## 3.0.6
   - Change codec instance comparison [#69](https://github.com/logstash-plugins/logstash-output-syslog/pull/69)
+  - Added support for RFC5424 structured data [#67](https://github.com/logstash-plugins/logstash-output-syslog/pull/67)
 
 ## 3.0.5
   - Docs: Set the default_codec doc attribute.
@@ -40,4 +41,3 @@
  - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully,
    instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
  - Dependency on logstash-core update to 2.0
-

docs/index.asciidoc

@@ -59,6 +59,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
 | <<plugins-{type}s-{plugin}-ssl_key_passphrase>> |<<password,password>>|No
 | <<plugins-{type}s-{plugin}-ssl_verify>> |<<boolean,boolean>>|No
 | <<plugins-{type}s-{plugin}-use_labels>> |<<boolean,boolean>>|No
+| <<plugins-{type}s-{plugin}-structured_data>> |<<string,string>>|No
 |=======================================================================
 
 Also see <<plugins-{type}s-{plugin}-common-options>> for a list of options supported by all
@@ -234,9 +235,21 @@ Verify the identity of the other end of the SSL connection against the CA.
 use label parsing for severity and facility levels
 use priority field if set to false
 
+[id="plugins-{type}s-{plugin}-structured_data"]
+===== `structured_data`
 
+  * Value type is <<string,string>>
+  * There is no default value for this setting.
+
+RFC5424 structured data is a string of one or more structured data elements, including brackets.
+The elements need to be formatted according to link:https://datatracker.ietf.org/doc/html/rfc5424#section-6.3[RFC5424 section 6.3], for example:
+
+["source",subs="attributes"]
+  `[exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][examplePriority@32473 class="high"]`
+
+The new value can include `%{foo}` strings to help you build a new value from other parts of the event.
 
 [id="plugins-{type}s-{plugin}-common-options"]
 include::{include_path}/{type}.asciidoc[]
 
-:default_codec!:
+:default_codec!:

lib/logstash/outputs/syslog.rb

@@ -125,13 +125,16 @@ class LogStash::Outputs::Syslog < LogStash::Outputs::Base
   # syslog message format: you can choose between rfc3164 or rfc5424
   config :rfc, :validate => ["rfc3164", "rfc5424"], :default => "rfc3164"
 
+  # RFC5424 structured data.
+  config :structured_data, :validate => :string, :default => ""
+
   def register
     @client_socket = nil
 
     if ssl?
       @ssl_context = setup_ssl
     end
-    
+
     if @codec.class.name == "LogStash::Codecs::Plain"
       if @codec.config["format"].nil?
         @codec = LogStash::Codecs::Plain.new({"format" => @message})
@@ -141,6 +144,11 @@ def register
 
     # use instance variable to avoid string comparison for each event
     @is_rfc3164 = (@rfc == "rfc3164")
+
+    if @is_rfc3164 && !@structured_data.empty?
+      raise LogStash::ConfigurationError, "Structured data is not supported for RFC3164"
+    end
+
   end
 
   def receive(event)
@@ -169,8 +177,9 @@ def publish(event, payload)
       syslog_msg = "<#{priority.to_s}>#{timestamp} #{sourcehost} #{appname}[#{procid}]: #{message}"
     else
       msgid = event.sprintf(@msgid)
+      sd = @structured_data.empty? ? "-" : event.sprintf(@structured_data)
       timestamp = event.sprintf("%{+YYYY-MM-dd'T'HH:mm:ss.SSSZZ}")
-      syslog_msg = "<#{priority.to_s}>1 #{timestamp} #{sourcehost} #{appname} #{procid} #{msgid} - #{message}"
+      syslog_msg = "<#{priority.to_s}>1 #{timestamp} #{sourcehost} #{appname} #{procid} #{msgid} #{sd} #{message}"
     end
 
     begin

spec/outputs/syslog_spec.rb

@@ -139,4 +139,28 @@
 
     it_behaves_like "syslog output"
   end
+
+  context "structured data is not supported for RFC3164" do
+    let(:options) { {"host" => "foo", "port" => "123", "rfc" => "rfc3164", "structured_data" => "[foo@12345]" } }
+
+    it "should raise exception" do
+      expect { subject.register }.to raise_error(LogStash::ConfigurationError)
+    end
+  end
+
+  context "send with both structured data and message" do
+    let(:options) { {"host" => "foo", "port" => "123", "rfc" => "rfc5424", "structured_data" => '[exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][examplePriority@32473 class="high"]' } }
+    let(:output) { /^<13>1 #{RFC3339_DATE_TIME_REGEX} baz LOGSTASH - - \[exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"\]\[examplePriority@32473 class="high"\] bar\n/m }
+
+    it_behaves_like "syslog output"
+  end
+
+  context "set structured data elements from event" do
+    let(:event) { LogStash::Event.new({"message" => "bar", "host" => "baz", "pod" => "mypod" }) }
+    let(:options) { {"host" => "foo", "port" => "123", "rfc" => "rfc5424", "structured_data" => '[exampleSDID@32473 pod="%{pod}"]' } }
+    let(:output) { /^<13>1 #{RFC3339_DATE_TIME_REGEX} baz LOGSTASH - - \[exampleSDID@32473 pod="mypod"\] bar\n/m }
+
+    it_behaves_like "syslog output"
+  end
+
 end