Parameter store as Cloud Config not working in native image on AWS Lambda #1042

kieranjen · 2021-04-29T09:33:52Z

I am trying to use the parameter store feature explained in the documentation here. When building the application as a native image and trying to run on AWS Lambda, I get an error on startup.

Task List

Steps to reproduce provided
Stacktrace (if present) provided
Example that reproduces the problem uploaded to Github
Full description of the issue provided (see below)

Steps to Reproduce

Build the example application with ./gradlew buildNativeLambda
Create an SSM parameter called /config/application/example/awsTest with any value e.g. Test
Upload the built native image zip to an AWS Lambda custom runtime
Run the lambda function using the following JSON as the lambda input:

{
  "body": "eyJ0ZXN0IjoiYm9keSJ9",
  "resource": "/",
  "path": "/",
  "httpMethod": "POST",
  "isBase64Encoded": true,
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "/"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.eu-west-2.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.eu-west-2.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/",
    "resourcePath": "/",
    "httpMethod": "POST",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

Expected Behaviour

When the application starts it should pick up the configuration from the SSM parameter store and print and return it from the API call.

Actual Behaviour

The following exception is raised when the application is starting up:


io.micronaut.context.exceptions.ConfigurationException: Exception thrown instantiating MicronautLambdaRuntimeHandler
--
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:40)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.createHandler(AbstractMicronautLambdaRuntime.java:192)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.startRuntimeApiEventLoop(AbstractMicronautLambdaRuntime.java:313)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.run(AbstractMicronautLambdaRuntime.java:129)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.main(MicronautLambdaRuntime.java:50)
Caused by: com.amazonaws.serverless.exceptions.ContainerInitializationException: Error starting Micronaut container: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:252)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:162)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:115)
at io.micronaut.function.aws.proxy.MicronautLambdaHandler.<init>(MicronautLambdaHandler.java:54)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:38)
... 4 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1565)
at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:219)
at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:2857)
at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:231)
at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:165)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:248)
... 8 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1972)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2724)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2710)
at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2249)
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1559)
... 13 more
Caused by: java.lang.ExceptionInInitializerError
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:315)
at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:143)
at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:137)
at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:100)
at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:80)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
at io.micronaut.aws.sdk.v1.AWSClientConfiguration.<init>(AWSClientConfiguration.java:36)
at io.micronaut.aws.sdk.v1.$AWSClientConfigurationDefinition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1943)
... 17 more
Caused by: java.lang.IllegalArgumentException
at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:249)
at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:263)
at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:336)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
... 26 more
Request loop failed with: Exception thrown instantiating MicronautLambdaRuntimeHandler

Environment Information

Operating System: AWS Lambda (native image)
Micronaut Version: 2.4.2
JDK Version: java11

Example Application

https://github.com/kieranjen/micronaut-graal-ssm-params

The text was updated successfully, but these errors were encountered:

hchang-chwy · 2021-08-05T19:57:40Z

Seems to be related to AWSParameterStoreConfiguration where it extends AWSClientConfiguration from aws sdk v1 lib which is by nature not compatible with native image.

graemerocher · 2021-08-05T21:36:30Z

There is a PR waiting on @sdelamo #1060

hchang-chwy · 2021-08-06T13:34:12Z

Per https://arnoldgalovics.com/tackling-java-cold-startup-times-on-aws-lambda-with-graalvm/, I was finally able to fix the issue by adding below files under src/main/graal

resource-config.json

{
  "resources": [
    {
      "pattern": "com/amazonaws/internal/config/awssdk_config_default.json"
    },
    {
      "pattern": "com/amazonaws/partitions/endpoints.json"
    },
    {
      "pattern": "com/amazonaws/sdk/versionInfo.properties"
    }
  ]
}

reflect.json

[
  {
    "name": "com.amazonaws.internal.config.InternalConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HostRegexToRegionMappingJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.JsonIndex",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  }
]

graemerocher · 2021-08-06T13:52:21Z

@ilopmar maybe we should include the above config until the PR @sdelamo has assigned to him is dealt with?

ilopmar · 2021-08-06T13:58:51Z

We do have support for ParameterStore but using SDK v2 (which includes it out of the box). This is our test application: https://github.com/micronaut-graal-tests/micronaut-aws-sdk2-graal/tree/2.5.x_paramstore

graemerocher · 2021-08-06T14:42:49Z

So what is #1060 for?

ilopmar · 2021-08-06T14:46:00Z

@sdelamo can confirm it but I think that PR adds support for use Parameter Store as a Distributed Config provider in Micronaut.

sdelamo · 2021-08-09T05:58:08Z

#1060 is share logic for both distributed configuration solutions AWS Parameter store and AWS Secret Manager.

graemerocher · 2021-10-20T10:39:56Z

@sdelamo can this be closed since in Micronaut 3 it now uses the v2 SDK?

graemerocher added the relates-to: graal label Apr 29, 2021

dstepanov self-assigned this May 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parameter store as Cloud Config not working in native image on AWS Lambda #1042

Parameter store as Cloud Config not working in native image on AWS Lambda #1042

kieranjen commented Apr 29, 2021

hchang-chwy commented Aug 5, 2021

graemerocher commented Aug 5, 2021

hchang-chwy commented Aug 6, 2021

graemerocher commented Aug 6, 2021

ilopmar commented Aug 6, 2021

graemerocher commented Aug 6, 2021

ilopmar commented Aug 6, 2021

sdelamo commented Aug 9, 2021

graemerocher commented Oct 20, 2021

Parameter store as Cloud Config not working in native image on AWS Lambda #1042

Parameter store as Cloud Config not working in native image on AWS Lambda #1042

Comments

kieranjen commented Apr 29, 2021

Task List

Steps to Reproduce

Expected Behaviour

Actual Behaviour

Environment Information

Example Application

hchang-chwy commented Aug 5, 2021

graemerocher commented Aug 5, 2021

hchang-chwy commented Aug 6, 2021

graemerocher commented Aug 6, 2021

ilopmar commented Aug 6, 2021

graemerocher commented Aug 6, 2021

ilopmar commented Aug 6, 2021

sdelamo commented Aug 9, 2021

graemerocher commented Oct 20, 2021