Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't formatting over 4 files with .sh script #25

Open
doox911-opensource opened this issue Oct 12, 2023 · 6 comments
Open

Can't formatting over 4 files with .sh script #25

doox911-opensource opened this issue Oct 12, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@doox911-opensource
Copy link

doox911-opensource commented Oct 12, 2023

OS

Windows 11 Pro. WSL 2.

11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz 2.80 GHz

RAM 16gb

Dprint version

0.41.0

JSON plugin

https://plugins.dprint.dev/json-0.17.4.wasm

dprint.json

{
  "plugins": [
    "https://plugins.dprint.dev/json-0.17.4.wasm"
  ]
}

My script

# /bin/sh

echo "dprint:"

START=$(date +%s%N)
  
npx dprint fmt --incremental=false
  
END=$(date +%s%N)

milliseconds=$((($END - $START)/1000000))

seconds=$(($milliseconds/1000))

minutes=$(($seconds/60))

echo "$milliseconds"
echo "$seconds"
echo "$minutes"

Log

WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_1.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_2.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_3.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_4.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_5.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_6.json
....

I created 100 identical files from https://github.com/Nilstrieb/jsonformat/blob/master/benches/large-file.json.

When I format 4 files everything is fine! But when I try to format more, then I see a WARNING and not one file is formatted.

@doox911-opensource doox911-opensource changed the title Can't formatting over 4 files in .sh script Can't formatting over 4 files with .sh script Oct 12, 2023
@dsherret
Copy link
Member

Can you run with dprint fmt --verbose? For me it formats, but it is very slow for such a large file.

@doox911-opensource
Copy link
Author

Can you run with dprint fmt --verbose? For me it formats, but it is very slow for such a large file.

dprint:
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64-3707102107370057232
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "[my_private_path]/jsonformat/benches" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "[my_private_path]/jsonformat/benches" } }] }
[VERBOSE] File(s) matched: GlobOutput { file_paths: ["[my_private_path]/jsonformat/benches/dprint.json", "[my_private_path]/jsonformat/benches/large-file_1.json", "[my_private_path]/jsonformat/benches/large-file_10.json", "[my_private_path]/jsonformat/benches/large-file_100.json", "[my_private_path]/jsonformat/benches/large-file_11.json", "[my_private_path]/jsonformat/benches/large-file_12.json", "[my_private_path]/jsonformat/benches/large-file_13.json", "[my_private_path]/jsonformat/benches/large-file_14.json", "[my_private_path]/jsonformat/benches/large-file_15.json", "[my_private_path]/jsonformat/benches/large-file_16.json", "[my_private_path]/jsonformat/benches/large-file_17.json", "[my_private_path]/jsonformat/benches/large-file_18.json", "[my_private_path]/jsonformat/benches/large-file_19.json", "[my_private_path]/jsonformat/benches/large-file_2.json", "[my_private_path]/jsonformat/benches/large-file_20.json", "[my_private_path]/jsonformat/benches/large-file_21.json", "[my_private_path]/jsonformat/benches/large-file_22.json", "[my_private_path]/jsonformat/benches/large-file_23.json", "[my_private_path]/jsonformat/benches/large-file_24.json", "[my_private_path]/jsonformat/benches/large-file_25.json", "[my_private_path]/jsonformat/benches/large-file_26.json", "[my_private_path]/jsonformat/benches/large-file_27.json", "[my_private_path]/jsonformat/benches/large-file_28.json", "[my_private_path]/jsonformat/benches/large-file_29.json", "[my_private_path]/jsonformat/benches/large-file_3.json", "[my_private_path]/jsonformat/benches/large-file_30.json", "[my_private_path]/jsonformat/benches/large-file_31.json", "[my_private_path]/jsonformat/benches/large-file_32.json", "[my_private_path]/jsonformat/benches/large-file_33.json", "[my_private_path]/jsonformat/benches/large-file_34.json", "[my_private_path]/jsonformat/benches/large-file_35.json", "[my_private_path]/jsonformat/benches/large-file_36.json", "[my_private_path]/jsonformat/benches/large-file_37.json", "[my_private_path]/jsonformat/benches/large-file_38.json", "[my_private_path]/jsonformat/benches/large-file_39.json", "[my_private_path]/jsonformat/benches/large-file_4.json", "[my_private_path]/jsonformat/benches/large-file_40.json", "[my_private_path]/jsonformat/benches/large-file_41.json", "[my_private_path]/jsonformat/benches/large-file_42.json", "[my_private_path]/jsonformat/benches/large-file_43.json", "[my_private_path]/jsonformat/benches/large-file_44.json", "[my_private_path]/jsonformat/benches/large-file_45.json", "[my_private_path]/jsonformat/benches/large-file_46.json", "[my_private_path]/jsonformat/benches/large-file_47.json", "[my_private_path]/jsonformat/benches/large-file_48.json", "[my_private_path]/jsonformat/benches/large-file_49.json", "[my_private_path]/jsonformat/benches/large-file_5.json", "[my_private_path]/jsonformat/benches/large-file_50.json", "[my_private_path]/jsonformat/benches/large-file_51.json", "[my_private_path]/jsonformat/benches/large-file_52.json", "[my_private_path]/jsonformat/benches/large-file_53.json", "[my_private_path]/jsonformat/benches/large-file_54.json", "[my_private_path]/jsonformat/benches/large-file_55.json", "[my_private_path]/jsonformat/benches/large-file_56.json", "[my_private_path]/jsonformat/benches/large-file_57.json", "[my_private_path]/jsonformat/benches/large-file_58.json", "[my_private_path]/jsonformat/benches/large-file_59.json", "[my_private_path]/jsonformat/benches/large-file_6.json", "[my_private_path]/jsonformat/benches/large-file_60.json", "[my_private_path]/jsonformat/benches/large-file_61.json", "[my_private_path]/jsonformat/benches/large-file_62.json", "[my_private_path]/jsonformat/benches/large-file_63.json", "[my_private_path]/jsonformat/benches/large-file_64.json", "[my_private_path]/jsonformat/benches/large-file_65.json", "[my_private_path]/jsonformat/benches/large-file_66.json", "[my_private_path]/jsonformat/benches/large-file_67.json", "[my_private_path]/jsonformat/benches/large-file_68.json", "[my_private_path]/jsonformat/benches/large-file_69.json", "[my_private_path]/jsonformat/benches/large-file_7.json", "[my_private_path]/jsonformat/benches/large-file_70.json", "[my_private_path]/jsonformat/benches/large-file_71.json", "[my_private_path]/jsonformat/benches/large-file_72.json", "[my_private_path]/jsonformat/benches/large-file_73.json", "[my_private_path]/jsonformat/benches/large-file_74.json", "[my_private_path]/jsonformat/benches/large-file_75.json", "[my_private_path]/jsonformat/benches/large-file_76.json", "[my_private_path]/jsonformat/benches/large-file_77.json", "[my_private_path]/jsonformat/benches/large-file_78.json", "[my_private_path]/jsonformat/benches/large-file_79.json", "[my_private_path]/jsonformat/benches/large-file_8.json", "[my_private_path]/jsonformat/benches/large-file_80.json", "[my_private_path]/jsonformat/benches/large-file_81.json", "[my_private_path]/jsonformat/benches/large-file_82.json", "[my_private_path]/jsonformat/benches/large-file_83.json", "[my_private_path]/jsonformat/benches/large-file_84.json", "[my_private_path]/jsonformat/benches/large-file_85.json", "[my_private_path]/jsonformat/benches/large-file_86.json", "[my_private_path]/jsonformat/benches/large-file_87.json", "[my_private_path]/jsonformat/benches/large-file_88.json", "[my_private_path]/jsonformat/benches/large-file_89.json", "[my_private_path]/jsonformat/benches/large-file_9.json", "[my_private_path]/jsonformat/benches/large-file_90.json", "[my_private_path]/jsonformat/benches/large-file_91.json", "[my_private_path]/jsonformat/benches/large-file_92.json", "[my_private_path]/jsonformat/benches/large-file_93.json", "[my_private_path]/jsonformat/benches/large-file_94.json", "[my_private_path]/jsonformat/benches/large-file_95.json", "[my_private_path]/jsonformat/benches/large-file_96.json", "[my_private_path]/jsonformat/benches/large-file_97.json", "[my_private_path]/jsonformat/benches/large-file_98.json", "[my_private_path]/jsonformat/benches/large-file_99.json"], config_files: [] }
[VERBOSE] Finished globbing in 11ms
[VERBOSE] Running for config: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_1.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_10.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_100.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_11.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_12.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_13.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/dprint.json in 0ms
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 35ms
[VERBOSE] Created instance of dprint-plugin-json in 28ms
[VERBOSE] Created instance of dprint-plugin-json in 21ms
[VERBOSE] Created instance of dprint-plugin-json in 15ms
[VERBOSE] Created instance of dprint-plugin-json in 8ms
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] CPU usage: 88%
[VERBOSE] High CPU. Reducing parallelism.
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_12.json in 5386ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_12.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_14.json in 5414ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_100.json in 5491ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_100.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_13.json in 5542ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_13.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_10.json in 5580ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_10.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_1.json in 5627ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_1.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_11.json in 5669ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_11.json
[VERBOSE] CPU usage: 88%
[VERBOSE] CPU usage: 16%
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_1.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_10.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_100.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_11.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_12.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_13.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] CPU usage: 1%
[VERBOSE] Low CPU. Increasing parallelism.
[VERBOSE] CPU usage: 1%
[VERBOSE] CPU usage: 0%

Then the script script freezes and it is clear about the CPU load that there is no work.

@dsherret
Copy link
Member

Ok, that looks like a dprint bug. Thanks for the output. I'll look into it.

@dsherret dsherret added the bug Something isn't working label Oct 12, 2023
@dsherret
Copy link
Member

@doox911-opensource I haven't been able to reproduce it yet on multiple machines. When you get a chance, would you be able to try setting the CI env var to 1 (ex. CI=1 npx dprint fmt --verbose --incremental=false)? That will disable CPU throttling. I'm curious if it's caused by that.

If it's too much of an ask, I also wonder if this occurs in dprint 0.39.1 and starts occurring in 0.40.0.

@doox911-opensource
Copy link
Author

doox911-opensource commented Oct 13, 2023

@doox911-opensource I haven't been able to reproduce it yet on multiple machines. When you get a chance, would you be able to try setting the CI env var to 1 (ex. CI=1 npx dprint fmt --verbose --incremental=false)? That will disable CPU throttling. I'm curious if it's caused by that.

If it's too much of an ask, I also wonder if this occurs in dprint 0.39.1 and starts occurring in 0.40.0.

Script:

# /bin/sh

echo "dprint:"

START=$(date +%s%N)
  
CI=1 npx dprint fmt --incremental=false --verbose
  
END=$(date +%s%N)

milliseconds=$((($END - $START)/1000000))

seconds=$(($milliseconds/1000))

minutes=$(($seconds/60))

echo "$milliseconds"
echo "$seconds"
echo "$minutes"

or just a command:

time npx dprint fmt --incremental=false large-file_*.json

or

npx dprint fmt --incremental=false large-file_*.json

The problem is observed not only when running dprint in the script

Result:

doox911@doox911:/mnt/c/Users/[my_private_path]$ ./dprint.sh
dprint:
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64-3707102107370057232
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_private_path]" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_private_path]" } }] }
[VERBOSE] File(s) matched: GlobOutput { file_paths: ["/mnt/c/Users/[my_private_path]/.prettierrc.json", "/mnt/c/Users/[my_private_path]/dprint.json", "/mnt/c/Users/[my_private_path]/large-file_1.json", "/mnt/c/Users/[my_private_path]/large-file_10.json", "/mnt/c/Users/[my_private_path]/large-file_100.json", "/mnt/c/Users/[my_private_path]/large-file_11.json", "/mnt/c/Users/[my_private_path]/large-file_12.json", "/mnt/c/Users/[my_private_path]/large-file_13.json", "/mnt/c/Users/[my_private_path]/large-file_14.json", "/mnt/c/Users/[my_private_path]/large-file_15.json", "/mnt/c/Users/[my_private_path]/large-file_16.json", "/mnt/c/Users/[my_private_path]/large-file_17.json", "/mnt/c/Users/[my_private_path]/large-file_18.json", "/mnt/c/Users/[my_private_path]/large-file_19.json", "/mnt/c/Users/[my_private_path]/large-file_2.json", "/mnt/c/Users/[my_private_path]/large-file_20.json", "/mnt/c/Users/[my_private_path]/large-file_21.json", "/mnt/c/Users/[my_private_path]/large-file_22.json", "/mnt/c/Users/[my_private_path]/large-file_23.json", "/mnt/c/Users/[my_private_path]/large-file_24.json", "/mnt/c/Users/[my_private_path]/large-file_25.json", "/mnt/c/Users/[my_private_path]/large-file_26.json", "/mnt/c/Users/[my_private_path]/large-file_27.json", "/mnt/c/Users/[my_private_path]/large-file_28.json", "/mnt/c/Users/[my_private_path]/large-file_29.json", "/mnt/c/Users/[my_private_path]/large-file_3.json", "/mnt/c/Users/[my_private_path]/large-file_30.json", "/mnt/c/Users/[my_private_path]/large-file_31.json", "/mnt/c/Users/[my_private_path]/large-file_32.json", "/mnt/c/Users/[my_private_path]/large-file_33.json", "/mnt/c/Users/[my_private_path]/large-file_34.json", "/mnt/c/Users/[my_private_path]/large-file_35.json", "/mnt/c/Users/[my_private_path]/large-file_36.json", "/mnt/c/Users/[my_private_path]/large-file_37.json", "/mnt/c/Users/[my_private_path]/large-file_38.json", "/mnt/c/Users/[my_private_path]/large-file_39.json", "/mnt/c/Users/[my_private_path]/large-file_4.json", "/mnt/c/Users/[my_private_path]/large-file_40.json", "/mnt/c/Users/[my_private_path]/large-file_41.json", "/mnt/c/Users/[my_private_path]/large-file_42.json", "/mnt/c/Users/[my_private_path]/large-file_43.json", "/mnt/c/Users/[my_private_path]/large-file_44.json", "/mnt/c/Users/[my_private_path]/large-file_45.json", "/mnt/c/Users/[my_private_path]/large-file_46.json", "/mnt/c/Users/[my_private_path]/large-file_47.json", "/mnt/c/Users/[my_private_path]/large-file_48.json", "/mnt/c/Users/[my_private_path]/large-file_49.json", "/mnt/c/Users/[my_private_path]/large-file_5.json", "/mnt/c/Users/[my_private_path]/large-file_50.json", "/mnt/c/Users/[my_private_path]/large-file_51.json", "/mnt/c/Users/[my_private_path]/large-file_52.json", "/mnt/c/Users/[my_private_path]/large-file_53.json", "/mnt/c/Users/[my_private_path]/large-file_54.json", "/mnt/c/Users/[my_private_path]/large-file_55.json", "/mnt/c/Users/[my_private_path]/large-file_56.json", "/mnt/c/Users/[my_private_path]/large-file_57.json", "/mnt/c/Users/[my_private_path]/large-file_58.json", "/mnt/c/Users/[my_private_path]/large-file_59.json", "/mnt/c/Users/[my_private_path]/large-file_6.json", "/mnt/c/Users/[my_private_path]/large-file_60.json", "/mnt/c/Users/[my_private_path]/large-file_61.json", "/mnt/c/Users/[my_private_path]/large-file_62.json", "/mnt/c/Users/[my_private_path]/large-file_63.json", "/mnt/c/Users/[my_private_path]/large-file_64.json", "/mnt/c/Users/[my_private_path]/large-file_65.json", "/mnt/c/Users/[my_private_path]/large-file_66.json", "/mnt/c/Users/[my_private_path]/large-file_67.json", "/mnt/c/Users/[my_private_path]/large-file_68.json", "/mnt/c/Users/[my_private_path]/large-file_69.json", "/mnt/c/Users/[my_private_path]/large-file_7.json", "/mnt/c/Users/[my_private_path]/large-file_70.json", "/mnt/c/Users/[my_private_path]/large-file_71.json", "/mnt/c/Users/[my_private_path]/large-file_72.json", "/mnt/c/Users/[my_private_path]/large-file_73.json", "/mnt/c/Users/[my_private_path]/large-file_74.json", "/mnt/c/Users/[my_private_path]/large-file_75.json", "/mnt/c/Users/[my_private_path]/large-file_76.json", "/mnt/c/Users/[my_private_path]/large-file_77.json", "/mnt/c/Users/[my_private_path]/large-file_78.json", "/mnt/c/Users/[my_private_path]/large-file_79.json", "/mnt/c/Users/[my_private_path]/large-file_8.json", "/mnt/c/Users/[my_private_path]/large-file_80.json", "/mnt/c/Users/[my_private_path]/large-file_81.json", "/mnt/c/Users/[my_private_path]/large-file_82.json", "/mnt/c/Users/[my_private_path]/large-file_83.json", "/mnt/c/Users/[my_private_path]/large-file_84.json", "/mnt/c/Users/[my_private_path]/large-file_85.json", "/mnt/c/Users/[my_private_path]/large-file_86.json", "/mnt/c/Users/[my_private_path]/large-file_87.json", "/mnt/c/Users/[my_private_path]/large-file_88.json", "/mnt/c/Users/[my_private_path]/large-file_89.json", "/mnt/c/Users/[my_private_path]/large-file_9.json", "/mnt/c/Users/[my_private_path]/large-file_90.json", "/mnt/c/Users/[my_private_path]/large-file_91.json", "/mnt/c/Users/[my_private_path]/large-file_92.json", "/mnt/c/Users/[my_private_path]/large-file_93.json", "/mnt/c/Users/[my_private_path]/large-file_94.json", "/mnt/c/Users/[my_private_path]/large-file_95.json", "/mnt/c/Users/[my_private_path]/large-file_96.json", "/mnt/c/Users/[my_private_path]/large-file_97.json", "/mnt/c/Users/[my_private_path]/large-file_98.json", "/mnt/c/Users/[my_private_path]/large-file_99.json", "/mnt/c/Users/[my_private_path]/package.json"], config_files: [] }
[VERBOSE] Finished globbing in 13ms
[VERBOSE] Running for config: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/.prettierrc.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_10.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_1.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_100.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_11.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/dprint.json in 0ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/.prettierrc.json in 0ms
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_13.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_14.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 37ms
[VERBOSE] Created instance of dprint-plugin-json in 29ms
[VERBOSE] Created instance of dprint-plugin-json in 23ms
[VERBOSE] Created instance of dprint-plugin-json in 14ms
[VERBOSE] Created instance of dprint-plugin-json in 8ms
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_10.json in 6465ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_10.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_12.json in 6471ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_100.json in 6546ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_100.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_14.json in 6578ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_14.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_11.json in 6729ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_11.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_1.json in 6841ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_1.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_13.json in 6958ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_13.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_1.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_10.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_100.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_11.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_12.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_13.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_14.json
23096
23
0

For dprint 0.39.1:

doox911@doox911:/mnt/c/Users/[my_path]$ ./dprint.sh
dprint:
dprint 0.39.1
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Busting plugins cache due to deserialization error: missing field `fileExtensions` at line 1 column 425
[VERBOSE] Deleting directory: /home/doox911/.cache/dprint/cache/plugins
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Acquiring file lock at /home/doox911/.cache/dprint/cache/locks/.3615629463447869408.lock
[VERBOSE] Acquired file lock at /home/doox911/.cache/dprint/cache/locks/.3615629463447869408.lock
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Busting plugins cache due to deserialization error: missing field `fileExtensions` at line 1 column 425
[VERBOSE] Deleting directory: /home/doox911/.cache/dprint/cache/plugins
[VERBOSE] Downloading url: https://plugins.dprint.dev/json-0.17.4.wasm
[VERBOSE] Creating instance of compiling
[VERBOSE] Created instance of compiling in 5ms
[VERBOSE] Dropped compiling (1 instances) in 0ms
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json
[VERBOSE] Writing file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.c2973d0f.tmp
[VERBOSE] Renaming /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.c2973d0f.tmp -> /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64
[VERBOSE] Writing file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.296480ae.tmp
[VERBOSE] Renaming /home/doox911/.cache/dprint/cache/plugin-cache-manifest.296480ae.tmp -> /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_path]" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_path]" } }] }
[VERBOSE] File(s) matched: ["/mnt/c/Users/[my_path]/.prettierrc.json", "/mnt/c/Users/[my_path]/dprint.json", "/mnt/c/Users/[my_path]/large-file_1.json", "/mnt/c/Users/[my_path]/large-file_10.json", "/mnt/c/Users/[my_path]/large-file_100.json", "/mnt/c/Users/[my_path]/large-file_11.json", "/mnt/c/Users/[my_path]/large-file_12.json", "/mnt/c/Users/[my_path]/large-file_13.json", "/mnt/c/Users/[my_path]/large-file_14.json", "/mnt/c/Users/[my_path]/large-file_15.json", "/mnt/c/Users/[my_path]/large-file_16.json", "/mnt/c/Users/[my_path]/large-file_17.json", "/mnt/c/Users/[my_path]/large-file_18.json", "/mnt/c/Users/[my_path]/large-file_19.json", "/mnt/c/Users/[my_path]/large-file_2.json", "/mnt/c/Users/[my_path]/large-file_20.json", "/mnt/c/Users/[my_path]/large-file_21.json", "/mnt/c/Users/[my_path]/large-file_22.json", "/mnt/c/Users/[my_path]/large-file_23.json", "/mnt/c/Users/[my_path]/large-file_24.json", "/mnt/c/Users/[my_path]/large-file_25.json", "/mnt/c/Users/[my_path]/large-file_26.json", "/mnt/c/Users/[my_path]/large-file_27.json", "/mnt/c/Users/[my_path]/large-file_28.json", "/mnt/c/Users/[my_path]/large-file_29.json", "/mnt/c/Users/[my_path]/large-file_3.json", "/mnt/c/Users/[my_path]/large-file_30.json", "/mnt/c/Users/[my_path]/large-file_31.json", "/mnt/c/Users/[my_path]/large-file_32.json", "/mnt/c/Users/[my_path]/large-file_33.json", "/mnt/c/Users/[my_path]/large-file_34.json", "/mnt/c/Users/[my_path]/large-file_35.json", "/mnt/c/Users/[my_path]/large-file_36.json", "/mnt/c/Users/[my_path]/large-file_37.json", "/mnt/c/Users/[my_path]/large-file_38.json", "/mnt/c/Users/[my_path]/large-file_39.json", "/mnt/c/Users/[my_path]/large-file_4.json", "/mnt/c/Users/[my_path]/large-file_40.json", "/mnt/c/Users/[my_path]/large-file_41.json", "/mnt/c/Users/[my_path]/large-file_42.json", "/mnt/c/Users/[my_path]/large-file_43.json", "/mnt/c/Users/[my_path]/large-file_44.json", "/mnt/c/Users/[my_path]/large-file_45.json", "/mnt/c/Users/[my_path]/large-file_46.json", "/mnt/c/Users/[my_path]/large-file_47.json", "/mnt/c/Users/[my_path]/large-file_48.json", "/mnt/c/Users/[my_path]/large-file_49.json", "/mnt/c/Users/[my_path]/large-file_5.json", "/mnt/c/Users/[my_path]/large-file_50.json", "/mnt/c/Users/[my_path]/large-file_51.json", "/mnt/c/Users/[my_path]/large-file_52.json", "/mnt/c/Users/[my_path]/large-file_53.json", "/mnt/c/Users/[my_path]/large-file_54.json", "/mnt/c/Users/[my_path]/large-file_55.json", "/mnt/c/Users/[my_path]/large-file_56.json", "/mnt/c/Users/[my_path]/large-file_57.json", "/mnt/c/Users/[my_path]/large-file_58.json", "/mnt/c/Users/[my_path]/large-file_59.json", "/mnt/c/Users/[my_path]/large-file_6.json", "/mnt/c/Users/[my_path]/large-file_60.json", "/mnt/c/Users/[my_path]/large-file_61.json", "/mnt/c/Users/[my_path]/large-file_62.json", "/mnt/c/Users/[my_path]/large-file_63.json", "/mnt/c/Users/[my_path]/large-file_64.json", "/mnt/c/Users/[my_path]/large-file_65.json", "/mnt/c/Users/[my_path]/large-file_66.json", "/mnt/c/Users/[my_path]/large-file_67.json", "/mnt/c/Users/[my_path]/large-file_68.json", "/mnt/c/Users/[my_path]/large-file_69.json", "/mnt/c/Users/[my_path]/large-file_7.json", "/mnt/c/Users/[my_path]/large-file_70.json", "/mnt/c/Users/[my_path]/large-file_71.json", "/mnt/c/Users/[my_path]/large-file_72.json", "/mnt/c/Users/[my_path]/large-file_73.json", "/mnt/c/Users/[my_path]/large-file_74.json", "/mnt/c/Users/[my_path]/large-file_75.json", "/mnt/c/Users/[my_path]/large-file_76.json", "/mnt/c/Users/[my_path]/large-file_77.json", "/mnt/c/Users/[my_path]/large-file_78.json", "/mnt/c/Users/[my_path]/large-file_79.json", "/mnt/c/Users/[my_path]/large-file_8.json", "/mnt/c/Users/[my_path]/large-file_80.json", "/mnt/c/Users/[my_path]/large-file_81.json", "/mnt/c/Users/[my_path]/large-file_82.json", "/mnt/c/Users/[my_path]/large-file_83.json", "/mnt/c/Users/[my_path]/large-file_84.json", "/mnt/c/Users/[my_path]/large-file_85.json", "/mnt/c/Users/[my_path]/large-file_86.json", "/mnt/c/Users/[my_path]/large-file_87.json", "/mnt/c/Users/[my_path]/large-file_88.json", "/mnt/c/Users/[my_path]/large-file_89.json", "/mnt/c/Users/[my_path]/large-file_9.json", "/mnt/c/Users/[my_path]/large-file_90.json", "/mnt/c/Users/[my_path]/large-file_91.json", "/mnt/c/Users/[my_path]/large-file_92.json", "/mnt/c/Users/[my_path]/large-file_93.json", "/mnt/c/Users/[my_path]/large-file_94.json", "/mnt/c/Users/[my_path]/large-file_95.json", "/mnt/c/Users/[my_path]/large-file_96.json", "/mnt/c/Users/[my_path]/large-file_97.json", "/mnt/c/Users/[my_path]/large-file_98.json", "/mnt/c/Users/[my_path]/large-file_99.json", "/mnt/c/Users/[my_path]/package.json"]
[VERBOSE] Finished globbing in 16ms
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_12.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_1.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/.prettierrc.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_11.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_100.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_10.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/dprint.json in 3ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/.prettierrc.json in 2ms
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_14.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_13.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_13.json in 7134ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_13.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_100.json in 7286ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_100.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_1.json in 7363ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_1.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_10.json in 7369ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_10.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_11.json in 7383ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_11.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_12.json in 7593ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_14.json in 7626ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_14.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_1.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_11.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_12.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_100.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_10.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_14.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_13.json
17109
17
0

@dsherret
Copy link
Member

There were some memory improvements made in 0.19.2. Not sure if that fixes it, but maybe.

I still haven't been able to reproduce this one. Going to dump the test script I've been using that sets this up so I don't lose it again:

import $ from "jsr:@david/[email protected]";

const data = await $.request("https://raw.githubusercontent.com/Nilstrieb/jsonformat/master/benches/large-file.json")
  .text();
const tempDir = $.path("tempDir");
tempDir.ensureDirSync();

for (let i = 0; i < 100; i++) {
  tempDir.join("file" + i + ".json").writeTextSync(data);
}

await $`dprint fmt --incremental=false`;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants