updating the logstash migration example

Signed-off-by: Anton Rubin <[email protected]>

Author: AntonEliatra

_data-prepper/migrating-from-logstash-data-prepper.md

@@ -29,20 +29,126 @@ As of the Data Prepper 1.2 release, the following plugins from the Logstash conf
 
 ## Running Data Prepper with a Logstash configuration
 
-1. To install Data Prepper's Docker image, see Installing Data Prepper in [Getting Started with OpenSearch Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started#1-installing-data-prepper).
+If you have OpenSearch running on your host and want to run Data Prepper Docker container with Logstash configuration, follow these steps:
 
-2. Run the Docker image installed in Step 1 by supplying your `logstash.conf` configuration.
+1. Update the `elasticsearch` section of `logstash.conf` to point to your OpenSearch instance. The host name has to match the OpenSearch certificate SANs, for example `node-0.example.com` if demo installation is used.
 
-```
-docker run --name data-prepper -p 4900:4900 -v ${PWD}/logstash.conf:/usr/share/data-prepper/pipelines.conf opensearchproject/data-prepper:latest pipelines.conf
-```
+    ```
+    input {
+      http {
+        port => 4910 # Note the port used in this example
+      }
+    }
+    filter {
+      grok {
+        match => { "message" => "%{COMBINEDAPACHELOG}" }
+        tag_on_failure => []
+      }
+    }
+    output {
+      # Point this at your OpenSearch/OSD endpoint
+      elasticsearch {
+        hosts    => ["https://node-0.example.com:9200"]   # change to your host:port
+        index    => "logstash-%{+YYYY.MM.dd}"
+        user     => "admin"                
+        password => "<admin_pass>"
+        ssl => true
+        ssl_certificate_verification => true
+      }
+    }
+    ```
+    {% include copy-curl.html %}
+
+1. Supply your `logstash.conf` configuration to Data Prepper Docker container, using the following command:
+
+    ```bash
+    docker run --rm --name data-prepper \
+      --add-host node-0.example.com:host-gateway \
+      -p 4910:4910 \
+      -v "${PWD}/logstash.conf:/usr/share/data-prepper/logstash.conf" \
+      --entrypoint bin/data-prepper \
+      opensearchproject/data-prepper:latest \
+      /usr/share/data-prepper/logstash.conf \
+      /usr/share/data-prepper/config/data-prepper-config.yaml
+    ```
+    {% include copy-curl.html %}
 
 The `logstash.conf` file is converted to `logstash.yaml` by mapping the plugins and attributes in the Logstash configuration to the corresponding plugins and attributes in Data Prepper.
-You can find the converted `logstash.yaml` file in the same directory where you stored `logstash.conf`.
+You can find the converted `logstash.yaml` file in the same directory where you stored `logstash.conf`. See the converted `logstash.yaml` sample file:
+
+```
+logstash-converted-pipeline:
+  source:
+    http:
+      max_connection_count: 500
+      request_timeout: 10000
+      port: 4910
+  processor:
+    - grok:
+        match:
+          message:
+            - "%{COMBINEDAPACHELOG}"
+  sink:
+    - opensearch:
+        hosts:
+          - "https://node-0.example.com:9200"
+        username: "admin"
+        password: "<admin_pass>"
+        index: "logstash-%{yyyy.MM.dd}"
+```
 
 
 The following output in your terminal indicates that Data Prepper is running correctly:
 
 ```
-INFO  org.opensearch.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
+INFO  org.opensearch.dataprepper.plugins.source.loghttp.HTTPSource - Started http source on port 4910...
 ```
+
+To test this further, run the following command on your host to push sample data to Data Prepper:
+
+```bash
+curl -X POST "http://localhost:4910/log/ingest" \                           
+  -H "Content-Type: application/json" \
+  -d '[{"message":"hello"}]'
+```
+{% include copy-curl.html %}
+
+After a couple of seconds you can query OpenSearch `logstash-*` index for this document:
+
+```bash
+curl -k -uadmin:"<admin_pass>" "https://localhost:9200/logstash-*/_search?pretty"
+```
+{% include copy-curl.html %}
+
+The sample document will be returned:
+
+```json
+{
+  "took" : 2,
+  "timed_out" : false,
+  "_shards" : {
+    "total" : 1,
+    "successful" : 1,
+    "skipped" : 0,
+    "failed" : 0
+  },
+  "hits" : {
+    "total" : {
+      "value" : 1,
+      "relation" : "eq"
+    },
+    "max_score" : 1.0,
+    "hits" : [
+      {
+        "_index" : "logstash-2025.10.10",
+        "_id" : "dHnSzZkBIk7UWjH_Kjxh",
+        "_score" : 1.0,
+        "_source" : {
+          "message" : "hello"
+        }
+      }
+    ]
+  }
+}
+```
+