Get core `wasp` CLI functionality working on Windows

[Martinsos]

🎯 Goal

• Be able to run wasp CLI on Windows (native, not WSL) with all the core functionality working. I mean wasp CLI version built during wasp development, not installed one (binary).
• This also includes tests running in CI to ensure no regressions happen. This might be challenging due to having to run test infrastructure on Win? Depends on the tests I guess, let's see.

Core functionality is wasp start, wasp build, wasp db migrate-dev, wasp waspls, wasp clean, ... .

Non-core functionality are commands that one can do without: wasp deploy (because you can deploy manually), wasp start db (because you can start the db manually), ... . These we will handle separately, in another issue.

🗒️ Currently known core features that are not working:

• Calling npm or npx from the wasp CLI: can't find npm.
  Latest ongoing attempt at fixing that: Better Win support: correct resolving of node/npm/npx & prisma #3258 (inspired by attempts Add support for Windows NPM #658 and Fix #48: Missing npm on native Windows #3020).
• Encoding issues (potentially not just Win issue though).
  Solved in Add support for Windows NPM #658 ? If so, we should extract that part and create PR.
• Some wrong file paths.
  Solved in Add support for Windows NPM #658 ? If so, we should extract that part and create PR.
• Prisma migrate not working on Win (due to script).
  Solved in Add support for Windows NPM #658 but also got solved on its own in the meantime? If so, confirm and mark as already done.

What works

After #3258 gets merged, we will be able to run local dev version of wasp using cabal run wasp-cli and execute some basic commands.
Notice however that there is more to be "built" in order to run fully working dev version of wasp, like setting up the TS packages/libs that come with it. So just running cabal run wasp-cli will crash again at points where those are needed, if they are not set up.
Since we don't have the equivalent of ./run script for Windows, right now that has to be done manually.

• install_packages_to_data_dir.ps1 for Window  #3318 adds script for taking care of the TS packages, which is a crucial component for making this easier.

[Martinsos]

There is an old PR where a big part of the issues with Windows is fixed: #658 . Specifically:

1. Calling of npm and npx and similar works (logic was added to invoke them differently when on Win, taking into account they are .ps1 scripts and .cmd and so on).
2. Wasp CLI was made more robust regarding encoding (this is a general thing, actually).
3. Some wrong paths were fixed.
4. We made prisma migrate work on Win (script was problem). I think this got solved on main also in the meantime because script is not needed anymore.
5. Possibly some other small fixes.

On the other hand, there is another PR #3020 which also fixes calling of npm, npx and similar but in a different way: instead of trying to invoke processes directly, it calls npm and npx as shell commands, so then the shell (e.g. on Win) takes care to figure the correct name of the executable (aliases).

We certainly want to extract the improvements from the first PR and contribute them to the main branch, while for correct calling of npm/npx we should pick one of the solutions from these two PRs and also merge that into main.

[Martinsos]

I put this one into product backlog because it seems to me we are potentially close to having this working, and if we can get it "locked in", in the sense that we have it working + being tested in CI, we have a good stepping stone for the future, to add full support for Win. If we don't do it soon, then more degradation will creep in, and we will again have to at some point later have a bigger effort (instead of doing small fixes on the way in reaction to CI failing).

[Martinsos]

Note for later: I made some progress and now have npm/npx/node being called correctly, but still Wasp fails on wasp db dev-migrate with

wasp-cli: C:\Program Files\nodejs\npm.cmd: readCreateProcessWithExitCode: invalid argument (Invalid argument)

From what I understood, that seems to be happening in ensurePackageDependenciesAreInstalled function. I am not sure why, but I am a bit suspect of our usage of P.cwd in packageCreateProcess (I read in the docs it is implementation specific and it is advised to not use it to ensure portability), so might be worth checking that out.

[nodtem66]

Hi Martin,

I checked out your commit and reproduced the error. It turns out that packageDir in packageCreateProcess is invalid.
The problem was solved after I copied waspc/packages/ to waspc/data/packages/ and built them manually.

Then we should see a new error that about npx:

Internal Wasp error (bug in the compiler):
  Failed while calling prisma format via wasp's prisma ts package: Error: spawn npx ENOENT
      at ChildProcess._handle.onexit (node:internal/child_process:285:19)
      at onErrorNT (node:internal/child_process:483:16)
      at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
    errno: -4058,
    code: 'ENOENT',
    syscall: 'spawn npx',
    path: 'npx',
    spawnargs: [
      'prisma',
      'format',
      '--schema',
      'C:\\Users\\....\\AppData\\Local\\Temp\\prisma--26004-j4zlwOhxaaNz-.schema'
    ]
  }

  CallStack (from HasCallStack):
    error, called at src\Wasp\Psl\Format.hs:35:21 in waspc-0.18.1-inplace:Wasp.Psl.Format

It is because the Node's spawn could not find the localtion of npx which is npx.ps1 on Window. One way to patch this new error is adding the option shell: true to Node's spawn in

wasp/waspc/packages/prisma/src/format.ts

Lines 25 to 28 in 0b1cf61

	const process = spawn(
	"npx",
	["prisma", "format", "--schema", tmpPrismaSchemaFile.path],
	{},

[Martinsos]

Hi Martin,

I checked out your commit and reproduced the error. It turns out that packageDir in packageCreateProcess is invalid. The problem was solved after I copied waspc/packages/ to waspc/data/packages/ and built them manually.

Then we should see a new error that about npx:

Internal Wasp error (bug in the compiler):
  Failed while calling prisma format via wasp's prisma ts package: Error: spawn npx ENOENT
      at ChildProcess._handle.onexit (node:internal/child_process:285:19)
      at onErrorNT (node:internal/child_process:483:16)
      at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
    errno: -4058,
    code: 'ENOENT',
    syscall: 'spawn npx',
    path: 'npx',
    spawnargs: [
      'prisma',
      'format',
      '--schema',
      'C:\\Users\\....\\AppData\\Local\\Temp\\prisma--26004-j4zlwOhxaaNz-.schema'
    ]
  }

  CallStack (from HasCallStack):
    error, called at src\Wasp\Psl\Format.hs:35:21 in waspc-0.18.1-inplace:Wasp.Psl.Format

It is because the Node's spawn could not find the localtion of npx which is npx.ps1 on Window. One way to patch this new error is adding the option shell: true to Node's spawn in

wasp/waspc/packages/prisma/src/format.ts

Lines 25 to 28 in 0b1cf61

const process = spawn(
"npx",
["prisma", "format", "--schema", tmpPrismaSchemaFile.path],
{},

Oh this is awesome, thanks a lot! I commented on the PR, I think we can merge that one quickly.
Would you like to also create a PR for the npx issue in format.ts?

Btw you might also be interested to throw a look at #3258, specifically comment #3258 (comment) -> do you have any thoughts on that? If not sure, don't bother too much, I will try to run it again and see what the exact error is, it might make more sense to look at it then with more info.