Feature request: SSM parameter(s) for latest layer compatible with semantic version range

[scratchclaggy]

Use case

Package managers within the Node ecosystem give users the ability to specify different semantic version ranges for the packages used within a project. However, when Powertools users leverage the published lambda layers, they are not able to enjoy the same semver guarantees as they rely on in their package.json.

Solution/User Experience

While users can currently access the latest powertools layer that is available via an SSM parameter, it would be useful for a similar parameter to exist which points at the latest layer compatible with an arbitrary semver range.

e.g.

package.json -> ^2.17.0
/aws/service/powertools/typescript/generic/all/2 -> arn:aws:lambda:{region}:094274105915:layer:AWSLambdaPowertoolsTypeScriptV2:23

Alternative solutions

Perhaps instead of publishing a compatible version via SSM parameters, we could export a utility function that returns the layer ARN given some semver range as input. This would be more flexible but would limit its use to CDK or other programatic IaC tools.

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[dreamorosi]

Hi @scratchclaggy, thank you for opening the issue.

As shared on the Discord thread, the feature request deviates a bit from what we initially discussed - and I don't see us doing what is described above.

AWS Lambda layers versioning is somewhat rigid. When publishing a new layer version you cannot specify an arbitrary version number. Instead, the version number of the layer is an integer number incremented every time you publish a new version. Likewise, when attaching a layer to a Lambda function, you can only do so by setting a specific version number - you can't for example use a range.

We understand that this makes things somewhat complex when the contents of that layer follow a different - arguably more sane - versioning strategy, like semantic versioning.

Because of this, we started publishing SSM parameters that allow you to lookup a Lambda layer ARN. We started by publishing two types:

• /aws/service/powertools/typescript/generic/all/latest - which always points to the latest version available on npm
• /aws/service/powertools/typescript/generic/all/<version> - where <version> is a specific version available on npm

After the discussion we had on Discord, and thanks to your proposal, we discussed adding a third one:

• /aws/service/powertools/typescript/generic/all/2.x - which will point to the latest version available on npm for that major version

This will allow customers to use the SSM parameter without worrying about automatically upgrading to a major version that they might not have tested, and that might contain breaking changes.

We discussed this within the team, and concluded that it's something we consider in scope and can work on in the coming weeks.

I believe that this is as far as we want to go in this area, at least for the time being. We won't be implementing a true semantic versioning, nor the concept of ranges.

As stated also in our tenets, our focus is primarily on enhancing the developer experience within AWS Lambda, and while Lambda layers are definitely adjacent to this - they're not something we have the capacity to focus on at this time.

We provide the Lambda layers primarily for convenience, and while I completely understand that their versioning scheme is not optimized for keeping production and development aligned, solving this problem is not something that fits with our current mission.

To sum up, we are going to add a new SSM parameter that follows the latest version of a major release (i.e. 2.x) - this is something we believe will cover most of the use cases since our versioning strategy is well documented and relatively slow.

For all those other customers who want to have a tighter control over which version of Powertools for AWS goes in their Lambda function, bundling dependencies is probably the preferred option.

[scratchclaggy]

Thank you for the feedback! Sorry for getting a bit carried away, I didn't intend to deviate from what we covered over in Discord, I was just trying to explain the general case without locking things down. In my head, offering a 2.x release parameter is an, admittedly loose, form of semantic version ranges.

I agree that offering a parameter that is tied to the latest for a given major is going to cover 95% of people's requirements, myself included. Those who aren't satisfied always have the option to ship their own dependencies as discussed. Trying to fully cover semantic versioning in a way that mirrors package managers would be pretty unwieldy.

I am happy to put in some work for this one, did you want me to have a crack or have you already got someone in mind for it?

What are your thoughts on my alternative idea? As mentioned, it's only useful when you're using CDK or similar, but it feels like it could be a little more flexible than the SSM parameter.

[am29d]

What are your thoughts on my alternative idea? As mentioned, it's only useful when you're using CDK or similar, but it feels like it could be a little more flexible than the SSM parameter.

This would force to cross the boundary from runtime library to build library. I think CDK can be a good place to add this utility, similar to the layers they package already. But we'd need to propagate the version changes after each release and it be a bit challenging to reduce the replication delay of this information to CDK repo. Not impossible, but I don't really see the practical side.

I like the idea of having 2.x label similar to latest, it'd be probably what most people would expect to pin a major version for a layer.

[dreamorosi]

It's been over 8mo since the last interaction on this thread and besides the initial conversation there hasn't been any significant traction/interest from customers (besides OP).

Given this is a non-trivial effort I'm inclined to close the issue for now and let the team focus on other higher-impact activities.

We might revisit the decision once we publish our next major version or if there's renewed interest.

[powertools-for-aws-oss-automation]

Warning

This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.