Start Use Case Guide with EKS templating examples

Signed-off-by: Wesley Pettit <[email protected]>

Author: PettitWesley

README.md

@@ -4,6 +4,7 @@
 
 - [Versioning FAQ](#versioning-faq)
 - [Debugging Guide](troubleshooting/debugging.md)
+- [Use Case Guide](use_cases/)
 - [Public Images](#public-images)
     - [Using the stable tag](#using-the-stable-tag)
     - [Using SSM to find available versions](#using-ssm-to-find-available-versions)
@@ -52,6 +53,10 @@ No. We continue to consume Fluent Bit from its main repository. We are not forki
 
 [Please read the debugging.md](troubleshooting/debugging.md)
 
+### Use Case Guide
+
+[A set of tutorials on use cases that Fluent Bit can solve](use_cases/).
+
 ### Public Images
 
 Each release updates the `latest` tag and adds a tag for the version of the image. The `stable` tag is also available which marks a release as the latest stable version. Deploying `latest` to prod without going through a test stage first is not recommended.

use_cases/README.md

@@ -0,0 +1,33 @@
+# AWS for Fluent Bit Use Case Guide
+
+The goal of this guide is a central location for all tutorials on solving use cases for AWS customers with Fluent Bit.
+
+## Troubleshooting errors and issues
+
+## ECS FireLens Examples
+
+All FireLens (side-car) examples for ECS are housed in their own repo: [aws-samples/amazon-ecs-firelens-examples](https://github.com/aws-samples/amazon-ecs-firelens-examples)
+
+We have guides on everything from JSON parsing to sending Fluent Bit's internal metrics to CloudWatch Metrics, and much much more. 
+
+## ECS Daemon Service Examples
+
+Coming Soon!
+
+## EKS/Kubernetes Examples
+
+### Getting Started with Fluent Bit and Kubernetes
+* [Kubernetes Logging powered by AWS for Fluent Bit](https://aws.amazon.com/blogs/containers/kubernetes-logging-powered-by-aws-for-fluent-bit/)
+* [Set up Fluent Bit as a DaemonSet to send logs to CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html)
+* [AWS for Fluent Bit helm charts](https://github.com/aws/eks-charts/tree/master/stable/aws-for-fluent-bit)
+* [Fluent Community maintained Kubernetes Repo](https://github.com/fluent/fluent-bit-kubernetes-logging)
+
+### Common Use Cases
+* [Customize CloudWatch log group or stream name based on kubernetes metadata](k8s-metadata-customize-cw/)
+* [Customize the tag based on Kubernetes metadata](k8s-metadata-customize-tag/)
+
+### Monitoring Fluent Bit
+* [Fluent Bit Prometheus endpoint](https://docs.fluentbit.io/manual/administration/monitoring)
+
+
+

use_cases/k8s-metadata-customize-cw/README.md

@@ -0,0 +1,47 @@
+# Customizing CloudWatch Log Group or Stream with Kubernetes Metadata
+
+### Option 1: Use the high performance cloudwatch_logs plugin
+
+To use the [recommended AWS CloudWatch Plugin](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch/), please see the main [Customize the tag based on Kubernetes metadata](k8s-metadata-customize-tag/) example, which demonstrates the technique with the cloudwatch_logs plugin. 
+
+### Option 2: Use templating in the older Golang cloudwatch plugin
+
+The [lower performance CloudWatch plugin](https://github.com/aws/amazon-cloudwatch-logs-for-fluent-bit#new-higher-performance-core-fluent-bit-plugin) has a [templating feature](https://github.com/aws/amazon-cloudwatch-logs-for-fluent-bit#templating-log-group-and-stream-names) which can be used to customize the log group and stream names. 
+
+First, you must enable the [kubernetes filter](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes), which can add metadata like this:
+```
+kubernetes: {
+    annotations: {
+        "kubernetes.io/psp": "eks.privileged"
+    },
+    container_hash: "<some hash>",
+    container_name: "myapp",
+    docker_id: "<some id>",
+    host: "ip-10-1-128-166.us-east-2.compute.internal",
+    labels: {
+        app: "myapp",
+        "pod-template-hash": "<some hash>"
+    },
+    namespace_name: "default",
+    pod_id: "198f7dd2-2270-11ea-be47-0a5d932f5920",
+    pod_name: "myapp-5468c5d4d7-n2swr"
+}
+```
+
+The kubernetes metadata can be referenced just like any other keys using the templating feature, for example, the following will result in a log group name which is /eks/{namespace_name}/{pod_name}.
+
+```
+    [OUTPUT]
+      Name              cloudwatch
+      Match             application.*
+      region            us-east-1
+      log_group_name    /eks/$(kubernetes['namespace_name'])/$(kubernetes['pod_name'])
+      log_stream_name   $(kubernetes['host'])/$(kubernetes['namespace_name'])/$(kubernetes['pod_name'])/$(kubernetes['container_name'])
+      auto_create_group on
+```
+
+If you want to deploy this example yourself, included is an altered version of the [Amazon CloudWatch Container Insights Daemonset](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html) that uses this technique the customize the log stream and group names. Follow the steps 1 & 2 to create a namespace and config map. Then, instead of step 3, apply the file in this example with:
+
+```
+kubectl apply -f fluent-bit.yaml
+```