Merge pull request #11 from customink/SimpleHashMap

Refactor to Simple HashMap. Docs. New Interface

Author: metaskills

Diff for: Cargo.lock

@@ -15,12 +15,9 @@ crate_type = ["cdylib"]
 anyhow = "1.0.57"
 tokio = { version = "1.19.2", features = ["full"] }
 futures = { version = "0.3.21" }
-serde = { version = "1.0.137", features = ["derive"] }
-serde_json = "1.0.81"
-aws-config = "0.13.0"
-aws-sdk-ssm = "0.13.0"
-lambda_runtime = "0.5.1"
-lambda_extension = { git = "https://github.com/awslabs/aws-lambda-rust-runtime" }
+aws-config = "0.14.0"
+aws-sdk-ssm = "0.14.0"
+lambda-extension = "0.5.0"
 # lib
 redhook = "2.0"
 libc = "0.2.126"

Diff for: Cargo.toml

@@ -2,38 +2,115 @@
 
 [![Actions Status](https://github.com/customink/crypteia/actions/workflows/test.yml/badge.svg)](https://github.com/customink/crypteia/actions/workflows/test.yml)
 
-# Crypteia
+# 🛡 Crypteia
 
-## Lambda Extension for Secure SSM Parameters as Environment Variables
+## Rust Lambda Extension for any Runtime to preload SSM Parameters as Secure Environment Variables!
 
-To-Do List:
+Super fast and only performaned once during your function's initialization, Crypteia turns your serverless YAML from this:
 
-- Huge thanks to this guy! https://github.com/jakejscott/rust-parameters-lambda-extension/issues/1
-- Ensure lambda lifecycle events correct. Witnessed shutdown errors. Maybe we need some invoke hook to make sure SSM finished work?
-- Make sure dev containers work locally with VS Code Remote Development. Add docs below.
-- Set raw environment variables.
-- Maybe rename to something like cold environments?
+```yaml
+Environment:
+  Variables:
+    SECRET: x-crypteia-ssm:/myapp/SECRET
+```
 
-## Development Environment
+Into real runtime (no matter the lang) environment variables backed by SSM Parameter Store. For example, assuming the SSM Parameter path above returns `1A2B3C4D5E6F` as the value. Your code would return:
 
-Using Codespaces or VS Code Remote Containers...
+```javascript
+process.env.SECRET; // 1A2B3C4D5E6F
+```
 
-- https://github.com/microsoft/vscode-remote-try-rust
-- https://github.com/microsoft/vscode-dev-containers/tree/main/containers/rust/history
+```ruby
+ENV['SECRET'] # 1A2B3C4D5E6F
+```
+
+We do this using our lib via `LD_PRELOAD` with [redhook](https://github.com/geofft/redhook) in coordination with our [Lambda Extension](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-extensions-api.html) binary. See installation & usage sections for more details.
+
+💕 Many thanks to the following projects & people for their work, code, and personal help that made Crypteia possible:
+
+- **[Hunter Madison](https://github.com/hmadison)**: Who taught me about how to use redhook based on Michele Mancioppi's [opentelemetry-injector](https://github.com/mmanciop/opentelemetry-injector) project.
+- **[Jake Scott](https://github.com/jakejscott)**: And his [rust-parameters-lambda-extension](https://github.com/jakejscott/rust-parameters-lambda-extension) project which served as the starting point for this project.
+
+## Installation
+
+🚧 🚧 🚧 TODO: Installation instructions for both `crypteia` binary extensions and `libcrypteia.so` file via `LD_PRELOAD`...
+
+## Usage
+
+First, you will need your secret environment variables setup in [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html). These can be whatever [hierarchy](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-hierarchies.html) you choose. Parameters can be any string type. However, we recommend using `SecureString` to ensure your secrets are encrypted within AWS. For example, let's assume the following paramter paths and values exists.
+
+- `/myapp/SECRET` -> `1A2B3C4D5E6F`
+- `/myapp/access-key` -> `G7H8I9J0K1L2`
+- `/myapp/envs/DB_URL` -> `mysql2://u:p@host:3306`
+- `/myapp/envs/NR_KEY` -> `z6y5x4w3v2u1`
+
+Crypteia supports two methods to fetch SSM parameters:
 
-# Building
+1. `x-crypteia-ssm:` - Single path for a single environment variable.
+2. `x-crypteia-ssm-path:` - Path prefix to fetch many environment variables.
 
-```sh
-./build.sh
+Using whatever serverless framework you prefer, setup your function's environment variables using either of the two SSM interfaces from above. For example, here is a environment variables section for an [AWS SAM](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started.html) template that demonstrates all of Crypteia's features.
+
+```yaml
+Environment:
+  Variables:
+    SECRET: x-crypteia-ssm:/myapp/SECRET
+    ACCESS_KEY: x-crypteia-ssm:/myapp/access-key
+    X_CRYPTEIA_SSM: x-crypteia-ssm-path:/myapp/envs
+    DB_URL: x-crypteia
+    NR_KEY: x-crypteia
 ```
 
-# Usage
+When your function initializes, each of the four environmet variables (`SECRET`, `ACCESS_KEY`, `DB_URL`, and `NR_KEY`) will return values from their respective SSM paths.
 
-```shell
-FOO_PARAM=ssm_parameter:/my/parameter
-FOO_PARAM=my-parameter
 ```
+process.env.SECRET;       // 1A2B3C4D5E6F
+process.env.ACCESS_KEY;   // G7H8I9J0K1L2
+process.env.DB_URL;       // mysql2://u:p@host:3306
+process.env.NR_KEY;       // z6y5x4w3v2u1
+```
+
+Here are a few details about the internal implementation on how Crypteia works:
+
+1. When accessing a single parameter path via `x-crypteia-ssm:` the environment variable name available to your runtime is used as is. No part of the parameter path effects the resulting name.
+2. When using `x-crypteia-ssm-path:` the environment variable name can be anything and the value is left unchanged.
+3. The parameter path hierarchy passed with `x-crypteia-ssm-path:` must be one level deep and end with valid environment variable names. These names must match environement placeholders using `x-crypteia` values.
+
+For security, the usage of `DB_URL: x-crypteia` placeholders ensures that your application's configuration is in full control on which dynamic values can be used with `x-crypteia-ssm-path:`.
+
+#### IAM Permissions
+
+Please use AWS' [Restricting access to Systems Manager parameters using IAM policies](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-access.html) guide for details on what policies your function's IAM Role will need. For an appliction to pull both single parameters as well as bulk paths, I have found the following policy helpful. It assumed the `/myapp` prefix and using AWS default KMS encryption key.
 
-```shell
-FOO_PARAMS=ssm_parameters:/my/path/envs
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Action": [
+        "ssm:GetParameter",
+        "ssm:GetParametersByPath",
+        "ssm:GetParameters",
+        "ssm:GetParameterHistory",
+        "ssm:DescribeParameters"
+      ],
+      "Resource": "arn:aws:ssm:us-east-1:123456789012:parameter/myapp*",
+      "Effect": "Allow"
+    },
+    {
+      "Action": "kms:Decrypt",
+      "Resource": "arn:aws:kms:us-east-1:123456789012:key/4914ec06-e888-4ea5-a371-5b88eEXAMPLE",
+      "Effect": "Allow"
+    }
+  ]
+}
 ```
+
+## Development
+
+🚧 🚧 🚧 TODO: Talk more about Codespaces or VS Code Remote Containers...
+
+- https://github.com/microsoft/vscode-remote-try-rust
+- https://github.com/microsoft/vscode-dev-containers/tree/main/containers/rust/history
+
+🚧 🚧 🚧 TODO: Speak to commands and running tests.

Diff for: README.md

@@ -4,5 +4,4 @@ set -e
 cargo test
 
 ./bin/build
-
 ./test/libcrypteia.sh 

Diff for: bin/test

@@ -1,5 +1,6 @@
 extern crate libc;
 use libc::c_char;
+use std::collections::HashMap;
 use std::ffi::CStr;
 use std::ffi::CString;
 use std::sync::Once;
@@ -19,16 +20,28 @@ redhook::hook! {
                 println!("[crypteia] Initialized libcrypteia using LD_PRELOAD");
             }
         });
-        let original_value = redhook::real!(getenv)(name);
-        if original_value.is_null() {
-            return original_value;
+        let env_value = redhook::real!(getenv)(name);
+        if env_value.is_null() {
+            return env_value;
         }
-        let given_name = CStr::from_ptr(name).to_str().unwrap();
-        if given_name == "HELLO" {
-            // https://doc.rust-lang.org/std/ffi/struct.CStr.html#method.as_ptr
-            let result = CString::new("WORLD").unwrap();
-            return result.into_raw();
+        let name_str = CStr::from_ptr(name).to_str().unwrap();
+        let name_string = name_str.to_string();
+        // TODO: Replace this with a shared data structure from binary.
+        let crypteia_envs: HashMap<String, String> = HashMap::from([
+            ("SECRET".to_string(), "1A2B3C4D5E6F".to_string()),
+        ]);
+        if crypteia_envs.contains_key(&name_string) {
+            let env_value_str = CStr::from_ptr(env_value).to_str().unwrap();
+            let env_value_string = env_value_str.to_string();
+            if env_value_string.starts_with("x-crypteia") {
+                let crypteia_value = crypteia_envs.get(&name_string).unwrap().clone();
+                let crypteia_value_c_string = CString::new(crypteia_value).unwrap();
+                crypteia_value_c_string.into_raw()
+            } else {
+                env_value
+            }
+        } else {
+            env_value
         }
-        original_value
     }
 }

Diff for: src/lib.rs

@@ -1,17 +1,15 @@
 mod ssm;
-use lambda_extension::{extension_fn, Error, LambdaEvent, NextEvent};
-use serde_json::json;
+use lambda_extension::{service_fn, Error, LambdaEvent, NextEvent};
 use std::collections::HashMap;
 
 #[tokio::main]
 async fn main() -> Result<(), Error> {
-    println!("[crypteia] init");
-    let vars: HashMap<String, String> = std::env::vars().collect();
-    let config = aws_config::load_from_env().await;
-    let ssm_client: aws_sdk_ssm::Client = aws_sdk_ssm::Client::new(&config);
-    let parameters = ssm::fetch_parameters(vars, &ssm_client).await.unwrap();
-    println!("[crypteia] fetched: {}", json!(&parameters));
-    let func = extension_fn(parameters_extension);
+    println!("[crypteia] Init");
+    let env_vars: HashMap<String, String> = std::env::vars().collect();
+    // TODO: Pass this data structure to the shared object library somehow.
+    let _parameters = ssm::get_envs(env_vars).await.unwrap();
+    println!("[crypteia] Fetched environment variables");
+    let func = service_fn(parameters_extension);
     lambda_extension::run(func).await
 }