Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

23.2 breaks a few packages that use json trough webpack (date-fns) #55826

Open
ThomasAunvik opened this issue Nov 12, 2024 · 13 comments · May be fixed by #55828
Open

23.2 breaks a few packages that use json trough webpack (date-fns) #55826

ThomasAunvik opened this issue Nov 12, 2024 · 13 comments · May be fixed by #55828
Labels
regression Issues related to regressions. v23.x v23.x Issues that can be reproduced on v23.x or PRs targeting the v23.x-staging branch.

Comments

@ThomasAunvik
Copy link

Version

v23.2.0

Platform

Linux arch 6.11.6-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Fri, 01 Nov 2024 03:30:35 +0000 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Use a package that includes for example date-fns on a NextJS project.

How often does it reproduce? Is there a required condition?

100%

What is the expected behavior? Why is that the expected behavior?

To build

What do you see instead?

 npm run build

> next build

   ▲ Next.js 15.0.3
   - Environments: .env

 ✓ Linting and checking validity of types    
   Creating an optimized production build ...
Failed to compile.

./node_modules/react-remove-scroll/dist/es2015/Combination.js + 22 modules
Unexpected end of JSON input

./node_modules/date-fns/format.js + 21 modules
Unexpected end of JSON input

./node_modules/date-fns/locale/fr.js + 5 modules
Unexpected end of JSON input


> Build failed because of webpack errors

Additional information

Seems to be a newly arrived issue only for 23.2.

Tested with 23.1 and 23.0 with no issue.

@RedYetiDev RedYetiDev added the needs more info Issues without a valid reproduction. label Nov 12, 2024
@RedYetiDev
Copy link
Member

RedYetiDev commented Nov 12, 2024

Hi! Can you provide a minimal reproduction?

"next build" runs a lot of different steps, so it can be hard to pinpoint the exact issue

@ThomasAunvik
Copy link
Author

@RedYetiDev

I managed to make a minimal reproduce it with just webpack and date-fns

https://github.com/ThomasAunvik/node232-err-json

npm run build

Error i then recieved:

> [email protected] build
> webpack

assets by status 0 bytes [cached] 1 asset
orphan modules 1.53 MiB [orphan] 826 modules
./src/index.js + 42 modules 107 KiB [built] [code generated]

WARNING in configuration
The 'mode' option has not been set, webpack will fallback to 'production' for this value.
Set 'mode' option to 'development' or 'production' to enable defaults for each environment.
You can also set it to 'none' to disable any default behavior. Learn more: https://webpack.js.org/configuration/mode/

ERROR in ./src/index.js + 42 modules
Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at ConcatenationScope.matchModuleReference (/home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/ConcatenationScope.js:132:13)
    at ConcatenatedModule.codeGeneration (/home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/optimize/ConcatenatedModule.js:1192:41)
    at /home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/Compilation.js:3505:22
    at /home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/Cache.js:97:5
    at Hook.eval [as callAsync] (eval at create (/home/thaun/Documents/Projects/node23err/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Cache.get (/home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/Cache.js:79:18)
    at ItemCacheFacade.get (/home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/CacheFacade.js:115:15)
    at Compilation._codeGenerationModule (/home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/Compilation.js:3498:9)
    at /home/thaun/Documents/Projects/node23err/node_modules/webpack/lib/Compilation.js:3403:11

webpack 5.96.1 compiled with 1 error and 1 warning in 1098 ms

@RedYetiDev
Copy link
Member

Have you reported this to Webpack yet?

@ThomasAunvik
Copy link
Author

ThomasAunvik commented Nov 12, 2024

No, but i can make one. I was unsure if it could be a node issue or a webpack, as I assume it would break compatability.

@targos
Copy link
Member

targos commented Nov 12, 2024

I'm bisecting.

@alexander-akait
Copy link

@targos @ThomasAunvik Some kind of regression with buffers, we have:

JSON.parse(Buffer.from(match[2], "hex").toString("utf-8"))

And we run this very very very many times, and in some cases buffer is just <Buffer > using 5b2267657444656661756c744f7074696f6e73225d as a value, I can't reproduce it in its pure form, but the variable stack is like this

@targos
Copy link
Member

targos commented Nov 12, 2024

45c6a9e1f6e165eb0ab2f7b5635662aa1875c171 is the first bad commit
commit 45c6a9e1f6e165eb0ab2f7b5635662aa1875c171
Author: Aviv Keller <[email protected]>
Date:   Mon Oct 28 20:32:26 2024 -0400

    src: migrate `String::Value` to `String::ValueView`

    Fixes #54417
    Ref: #55452

    PR-URL: https://github.com/nodejs/node/pull/55458
    Refs: https://github.com/nodejs/node/issues/55452
    Reviewed-By: Vladimir Morozov <[email protected]>
    Reviewed-By: Yagiz Nizipli <[email protected]>
    Reviewed-By: James M Snell <[email protected]>

 src/inspector_js_api.cc |  6 +++---
 src/node_buffer.cc      | 27 ++++++++++++---------------
 src/string_bytes.cc     | 20 ++++++++++----------
 3 files changed, 25 insertions(+), 28 deletions(-)

@targos targos added regression Issues related to regressions. v23.x v23.x Issues that can be reproduced on v23.x or PRs targeting the v23.x-staging branch. labels Nov 12, 2024
@RedYetiDev
Copy link
Member

RedYetiDev commented Nov 12, 2024

😅

@alexander-akait
Copy link

alexander-akait commented Nov 12, 2024

I'm trying to reproduce it in its pure form but without success, we have:

static matchModuleReference(name) {
    const match = MODULE_REFERENCE_REGEXP.exec(name);
    if (!match) return null;
    const index = Number(match[1]);
    const asiSafe = match[5];

    try {
        JSON.parse(Buffer.from(match[2], "hex").toString("utf-8"))
    } catch (e) {
        console.log(`"${match[2]}"`);
        console.log(`"${Buffer.from(match[2], "hex")}"`);
        console.log(`"${Buffer.from(match[2], "hex").toString("utf-8")}"`);
    }

    return {
        index,
        ids:
            match[2] === "ns"
                ? []
                : JSON.parse(Buffer.from(match[2], "hex").toString("utf-8")),
        call: Boolean(match[3]),
        directImport: Boolean(match[4]),
        asiSafe: asiSafe ? asiSafe === "1" : undefined
    };
}

and it failed with values :

"5b2267657444656661756c744f7074696f6e73225d"
""
"" 

@ThomasAunvik
Copy link
Author

ThomasAunvik commented Nov 12, 2024

I tried reconstructing the match, and it passed.

@alexander-akait

Buffer.from(match[2].split("").join("").toString(),"hex").toString("utf-8"))

Could it be a regex part that makes the issue? Doing the string directly doesn't do the same issue.

const name = "__WEBPACK_MODULE_REFERENCE__10_5b2267657444656661756c744f7074696f6e73225d_call_directImport_asiSafe1__"
const MODULE_REFERENCE_REGEXP =
	/^__WEBPACK_MODULE_REFERENCE__(\d+)_([\da-f]+|ns)(_call)?(_directImport)?(?:_asiSafe(\d))?__$/;

const match = MODULE_REFERENCE_REGEXP.exec(name);
JSON.parse(Buffer.from(match[2], "hex").toString("utf-8"));

Tried running this, though I did not receive the similar result trough node.

@alexander-akait
Copy link

@ThomasAunvik

Could it be a regex part that makes the issue? Doing the string directly doesn't do the same issue.

Honestly I don't think so, this code works since version 10 of Node.js

@alexander-akait
Copy link

alexander-akait commented Nov 12, 2024

Using match[2].split("").join("").toString(), you recreate string, something is really wrong with the strings/buffers, we can rewrite it like this, but I think that the main problem lies in another aspect and soon there will be other reports, perhaps no less confusing...

We don't really do anything special, you can use the repository - https://github.com/ThomasAunvik/node232-err-json and see that the string being sent does not contain anything unusual

@alexander-akait
Copy link

@ThomasAunvik #55826 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression Issues related to regressions. v23.x v23.x Issues that can be reproduced on v23.x or PRs targeting the v23.x-staging branch.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants