Moving logic from templates to libs

[infomiho]

High-level overview

This PR:

1. Introduces the waspc/libs directory
   • This directory contains the source code of libraries
2. Introduces the data/libs directory
   • This directory contains the built libraries
   • The built libraries will be copied to .wasp/out alongside the SDK directory so the SDK can import the libs
3. Introduces auth library in the libs directory (named @wasp.sh/libs-auth)
   • This library contains some auth code that was extracted from the Wasp SDK
   • It demonstrates how we can build the source code with tsdown
     
   • It demonstrates how to pack the library using npm pack and copy it into the data dir
   • It demonstrates how to define deps e.g. oslo
   • It demonstrates how to have multiple entry points e.g. ./sdk and ./sdk/browser to be able to separate server and browser exports
4. Introduces the concept of a WaspLib which is how we refer to the libraries in Haskell
   • We define the libs by name and this is enough to know where to find the packaged libs (.tgz files) and where to copy them
   • We also generate the dependencies needed for SDK from the WaspLib (e.g. file:../../lib-1.0.0.tgz)
   • We defined the version to be fixed to 0.0.0 since it will be replaced with the lib checksum when copied
5. Uses the auth library in the SDK and the server
   • It uses some jwt, cookies and password helpers, these helpers depend on external deps e.g. oslo and @node-rs/argon2 (native deps)
   • It defined some React helpers in the ./sdk/browser entry to show we can split the code

Note

Note for reviewers: there are three sections:

1. the auth library
2. the way the auth library is used in the SDK
3. and the machinery to package the libs and copy them to the data dir.

I'd suggest reviewing the auth library last since it's just a JS project, nothing new to see there.

How to test it locally

1. Run ./tools/install_packages_to_data_dir.sh
2. Start the waspc/examples/todoApp with the cabal run or your alias of choice e.g. wrun
3. Try using Auth UI to verify the browser code works
4. Try to login with email to verify password hashing works, reset a password to verify JWTs still work, try OAuth to verify cookies still work

I've tested it in development and production build modes and it works as expected ✅

Known issues in development

You have to delete the node_modules and the @wasp.sh/libs-auth entry from package-lock.json to install the new development lib version that you built locally.

EDIT: fixed by using a checksum of the library's .tgz checksum as the version which prevents npm from using a stale version. Also, I've excluded the libs from being optimized (and then becoming stale) by Vite in development mode.

Left to do

• We need to update the snapshot tests runner not to care about the .tgz files. I'll do this in a separate PR to keep it easier to review this one. Ignore tgz files in e2e tests #3078
• Update development workflow in the waspc/README.md and in run script

Closes #2936
Closes #3069

[cloudflare-workers-and-pages]

Deploying wasp-docs-on-main with    Cloudflare Pages

Latest commit:	7c08e2d
Status:	✅  Deploy successful!
Preview URL:	https://461bd450.wasp-docs-on-main.pages.dev
Branch Preview URL:	https://templates-to-libs.wasp-docs-on-main.pages.dev

View logs

[infomiho]

I made this refactor by mistake not realizing that I'm touching the server and not the SDK. It works because the @wasp.sh/libs-auth is installed in the root node_modules... but it breaks our philosophy that we introduced with 0.12.0: user imports from the SDK -> framework imports user code.

If the server imports the auth lib directly, I consider that importing the SDK since the auth lib is just an implementation detail of the SDK.

So, how do we tackle this? The server doesn't have oslo listed as one of its deps (meaning this worked just because SDK had oslo as a dep).

Do we add oslo as the dep of the server to be precise?

Can we avoid the server using oslo directly? That would mean that the server uses the SDK?

Do we consider server using the auth lib directly a violation of our 0.12.0 philosophy?

cc: @sodic

[cprecioso]

I think a straightforward solution would be to have libs-auth-server and libs-auth-sdk etc
Or the same library with multiple exports should be fine, no?

[infomiho]

I've encoded this worry in this issue as we'll need to figure out the naming: #3069

[infomiho]

Latest status: I've implemented the proposal from the linked issue in this PR #3069

[sodic]

I like the idea of not "dragging" the server's import chain through the SDK if it isn't necessary. Since SDK will become heavily templated, we want to decouple it from the rest of our codebase as best we can, which means minimizing SDK exports into the framework code.

the SDK since the auth lib is just an implementation detail of the SDK.

This makes sense too because framework code importing stuff from multiple locations seems messy, and I'm 90% sure it will have to import templated stuff at some point.

I'll have to think about this some more, we can discuss it on a call.

[Martinsos]

@infomiho I went through all the comments and commented again to those that needed it!

I think we are are tehre 99% on my side. Most of the comments have RAW in them, so please check them out and then resolve them so they don't clutter the convo.
There are only a few that might need more communication, but I thikn we can get those done quickly.

[cprecioso]

Approving this PR as in:

• Did a mid-depth review, I think we're covered on deep ones
• Downloaded it and checked that it works
• Happy with the solution design
• Green light to merge once you address the outstanding comments

I saw that you agreed with all my comments, so feel free to resolve them once they're fixed. I will unsubscribe myself from this PR now but you can @ tag me if you need my input anywhere.

[infomiho]

@sodic ready for another look!

@cprecioso I have only one of your comments to resolve (using WaspLibs in the deps command) - I maybe tackle it in a separate PR even to make it easier to review? #2989 (comment)

@Martinsos responded to all of your comment and resolved the ones marked with RAW if it was all clear

[cprecioso]

@infomiho fine by me. I take it this comment will also be part of that PR?

[Martinsos]

ONly one comment left on my side but it is RAW, so approving!