fix(layers): do not ship AWS SDK clients in the Lambda layer - #5512
Open
svozza wants to merge 3 commits into
Open
fix(layers): do not ship AWS SDK clients in the Lambda layer#5512svozza wants to merge 3 commits into
svozza wants to merge 3 commits into
Conversation
The layer is mounted at /opt/nodejs/node_modules, which precedes /var/runtime/node_modules on NODE_PATH. Any SDK package shipped in the layer therefore shadows the runtime's copy for customer code as well, not just for Powertools. Because the layer only ever included a subset of the SDK - notably @aws-sdk/client-dynamodb but never @aws-sdk/lib-dynamodb - functions using the DynamoDB DocumentClient resolved a layer-provided client against a runtime-provided lib. When those straddle the AWS SDK 3.928.0 boundary, which moved serializerMiddleware registration from per-command to per-client, the mismatch fails at runtime with: serializerMiddleware is not found when adding DocumentMarshall middleware before serializerMiddleware The Node.js Lambda runtime already provides a complete, self-consistent AWS SDK v3, so rely on that instead of exposing a partial one. zod and @smithy/* are not provided by the runtime and remain in the layer. Closes #5511
The layer no longer ships AWS SDK clients, so the previous wording - that both @aws-lambda-powertools/* and @aws-sdk/* are 'already present the layer' - is no longer accurate. Remove @aws-sdk/* from the esbuild exclusion notes and from the CDK, SAM and Serverless Framework snippets, so customers bundle their own SDK rather than relying on whatever the layer happens to expose. Bundling your own SDK resolves it from /var/task, which takes precedence over both the layer and the runtime and is therefore the only configuration guaranteed to be self-consistent. Also fixes the missing 'in' in 'present the layer'.
Record that layer version 49 is the last to bundle AWS SDK v3 clients, explain that /opt/nodejs/node_modules took precedence so those clients were used by customer code too, and that only a subset ever shipped - which is what allowed mismatched SDK packages at runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Lambda layer is mounted at
/opt/nodejs/node_modules, which precedes/var/runtime/node_modulesonNODE_PATH, so any SDK package we ship shadows the runtime's copy for customer code as well as our own. Because we only ever shipped a subset — notably@aws-sdk/client-dynamodbbut never@aws-sdk/lib-dynamodb— functions using the DynamoDB DocumentClient resolved a layer-provided client against a runtime-provided lib, which fails at runtime when the two straddle the AWS SDK 3.928.0 boundary. This change stops shipping AWS SDK clients in the layer and relies on the complete, self-consistent SDK the Node.js runtime already provides.Changes
modulesToInstallinlayers/src/layer-publisher-stack.tsto['zod'], removing@aws-sdk/client-dynamodb,@aws-sdk/util-dynamodb,@aws-sdk/client-ssm,@aws-sdk/client-secrets-managerand@aws-sdk/client-appconfigdata@aws-sdk/*client may be shipped, so the dependency is not reintroduced@aws-sdk/*from theesbuildexclusion notes and snippets indocs/getting-started/lambda-layers.md(CDK, SAM and Serverless Framework). The previous wording told customers to exclude it "since the packages are already present the layer", which is no longer true — and bundling your own SDK resolves it from/var/task, which takes precedence over both the layer and the runtime and is the only configuration guaranteed to be self-consistentzodstays in the layer, and@smithy/service-error-classificationplus@aws-sdk/typesremain as transitive dependencies ofaws-xray-sdk-core— none of these are provided by the Lambda runtimeVerified with an A/B test on
nodejs22.xusing two layers that are identical except for the presence of the bundled SDK. The test function performs a plain DocumentClient round-trip and does not use Powertools at all:@aws-sdk/client-dynamodbresolves from/opt/nodejs@3.454.0/var/runtime@3.1049.0@aws-sdk/lib-dynamodbresolves from/var/runtime@3.1049.0/var/runtime@3.1049.0Also verified that all utilities (Logger, Metrics, Tracer including
captureAWSv3Client, Parameters, Idempotency, Parser, Batch) work with no@aws-sdk/*in the layer, across CJS and ESM onnodejs22.xandnodejs24.x, and that customers who bundle their own SDK are unaffected because/var/taskcontinues to take precedence. The layer e2e suite passes (8/8).Issue number: closes #5511
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.