CfnResource Data retrieval breaks when used from Python CDK but not when used from CloudFormation

[suttonsp0]

This issue has got me scratching my head. It started when I built a custom resource and put the data I'm looking for into the helper like so:

helper.Data["FileServerIp"] = file_server_ip

I accessed the data in the python cdk app like so:

CfnOutput(
     self,
     "FileServerIpOutput",
     value=fsx_file_server_ip_retrieval.get_att_string("FileServerIp")
)

This resulted in this error:

CustomResource attribute error: Vendor response doesn't contain FileSystemId attribute in object

I confirmed in the logs that it is returning the ip address correctly:

[DEBUG]	2024-11-26T22:21:56.075Z	0fa30484-a10c-4f34-bc30-089ba0698aed	
{
    "Status": "SUCCESS",
    "PhysicalResourceId": "FileServerIpRetrieval-fs-06c2bba33ed9f31da",
    "StackId": "arn:aws:cloudformation:us-east-2:redacted:stack/test6/ba055540-ac44-11ef-bfcf-064a3121c883",
    "RequestId": "0eeb6ee4-e11a-4e16-bf14-3caf8bc65190",
    "LogicalResourceId": "FsxInstructionGenerationLambdaConstructFsxFileServerIpRetrievalD63DD733",
    "Reason": "",
    "Data": {
        "FileServerIp": "10.0.42.3"
    },
    "NoEcho": false
}

The weird part is that I configured the lambda to not tear down on failure and then I re-used it from a CloudFormation template like this:

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  MyTest:
    Type: "Custom::MyTest"
    Properties:
      ServiceToken: "arn:aws:lambda:us-east-2:redacted:function:test-9-FsxFileServerIPRetrievalLambdaF104C778-on3uRRR1cQQt"
      FileSystemId: "my-file-system-id"
Outputs:
  FileSystemId:
    Value: !GetAtt MyTest.FileServerIp

And it worked. I don't know why it would matter that it was compiled from python cdk instead of CloudFormation yet these are the results I see.

This was what the CfnOutput compiled to:

"Outputs": {
  "FsxInstructionGenerationLambdaConstructFileserverip1F40FB57": {
   "Value": {
    "Fn::GetAtt": [
     "FsxInstructionGenerationLambdaConstructFsxFileServerIpRetrievalD63DD733",
     "FileServerIp"
    ]
   }
  }
 }

And this is what I had for the custom resource:

"FsxInstructionGenerationLambdaConstructFsxFileServerIpRetrievalD63DD733": {
   "Type": "AWS::CloudFormation::CustomResource",
   "Properties": {
    "ServiceToken": {
     "Fn::GetAtt": [
      "FsxInstructionGenerationLambdaConstructFsxFileServerIpRetrievalProviderframeworkonEventDE938461",
      "Arn"
     ]
    },
    "FileSystemId": "fs-06c2bba33ed9f31da"
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete"
  },

This makes CfnResource unusable for my team and unfortunately we'll have to go back to just handling the events directly until this is fixed.

[zentavr]

Have the same issue

[zentavr]

@suttonsp0 if you check out CDK doc here there is a mention of:

N.B.: if you use the provider framework in this module you will write AWS Lambda Functions that look a lot like,
but aren't exactly the same as the Lambda Functions you would write if you wrote CloudFormation Custom
Resources directly, without this framework.

Specifically, to report success or failure, have your Lambda Function exit in the right way: return data for success,
or throw an exception for failure. Do not post the success or failure of your custom resource to an HTTPS URL
as the CloudFormation documentation tells you to do.

The library takes care to send data back to event['ResponseURL'] but actually you should not do that. You custom resource lambda would be in fact invoked by another lambda which CDK creates silently. That custom lambda expects that your own lambda did a return of the special structure.

In the same document there are examples in Typescript, like:

exports.handler = async (event, context) => {
    return {
      PhysicalResourceId: '1234',
      NoEcho: true,
      Data: {
        mySecret: 'secret-value',
        hello: 'world',
        ghToken: 'gho_xxxxxxx',
      },
    };
  };

The example (skeleton) in Python from the same doc:

def on_event(event, context):
  print(event)
  request_type = event['RequestType']
  if request_type == 'Create': return on_create(event)
  if request_type == 'Update': return on_update(event)
  if request_type == 'Delete': return on_delete(event)
  raise Exception("Invalid request type: %s" % request_type)

def on_create(event):
  props = event["ResourceProperties"]
  print("create new resource with props %s" % props)

  # add your create code here...
  physical_id = ...

  return { 'PhysicalResourceId': physical_id }

def on_update(event):
  physical_id = event["PhysicalResourceId"]
  props = event["ResourceProperties"]
  print("update resource %s with props %s" % (physical_id, props))
  # ...

def on_delete(event):
  physical_id = event["PhysicalResourceId"]
  print("delete resource %s" % physical_id)
  # ...

as for on_update - probably you need to return something as well if the actual resource changes (like PhysicalResourceId).

I'd wasted 3 days for that and hope my experience saves your time and nerves :)