Updating stack with lambda event source mapping replacement allowed before deleting the existing

[jacdel96]

Description:

I have tried this with Kinesis and DDb stream event sources for Lambda. It's true for those, I predict it is the case for stream-based synchronous lambda events.

In a SAM Template, if a Lambda event source mapping is replaced and the template is used to update a CFN stack, an error stating that the first event source mapping must be deleted before a new one is created will cause the stack update to fail.

This will occur if the event source mapping is defined through an "AWS::Lambda::EventSourceMapping" type resource initially.

Something interesting happens if you first define the event mapping as a property (under "Events") in a lambda resource definition (type "AWS::Serverless::Function"), and then decide to replace that event mapping later, such as removing the Events property from the lambda resource and creating an "AWS::Lambda::EventSourceMapping" resource instead, then deploying this template change as a stack update.

The stack update will complete, and the new event source mapping is created, before deleting the old one, seemingly working around the above validation error that CFN would normally display.

Is this intentional? A deployment my team made wherein an event source mapping was replaced in the way described above caused kinesis processing to be delayed, due to a lambda needing to iterate over all records in every shard. The lambda processing eventually caught up. My observation is that the deployment caused the shard pointers to reset, and perhaps that is why the validation error occurs when you first define the mapping as a "AWS::Lambda::EventSourceMapping" resource.

EDIT:
So it is clear what I am referring to -
Example of Event source mapping defined in lambda resource:

MyLambdaFunction
  Type: AWS::Serverless::Function
  Properties:
    **Lambda Properties**
    ...
    Events:
      LambdaKinesisEvent:
        Type: Kinesis
        Properties:
           BatchSize: **Some Number**
           ...More properties

I'll note that it would appear LambdaKinesisEvent post-stack deployment is of type "AWS::Lambda::EventSourceMapping".

Example of Event source mapping resource:

MyKinesisEvent:
   Type: AWS::Lambda::EventSourceMapping
   Properties:
     BatchSize: **Some Number**
     FunctionName: **Function Arn**
     EventSourceArn: **StreamArn**
     ...More properties

If I were to deploy with MyKinesisEvent first, and then deploy the template again deleting that resource and moving the definition to MyLambdaFunction, an error telling me to first delete MyKinesisEvent would appear.

If I were to deploy with the event defined in the lambda resource first, and then deleted that in favor of the strict AWS::Lambda::EventSourceMapping MyKinesisEvent and updated the stack, no complaints. The shard iterator pointer would be reset to reprocess all events in the stream however, which could cause issues.

[sriram-mv]

The stack update will complete, and the new event source mapping is created, before deleting the old one, seemingly working around the above validation error that CFN would normally display.

Is this an ordering issue you were seeing? you can set starting position within a stream, But I'm assuming that did not help.