Publish new release?

[figsoda]

No description provided.

[lovesegfault]

cc. @maxbrunsfeld

[skovhus]

There has been 20 commits to master since the last release from Mar 4, 2021.

Any plans to publish more often?

For https://github.com/bash-lsp/bash-language-server I'm considering building our wasm version of tree-sitter-bash based on the source code instead of on the published version.

@verhovsky

[figsoda]

For https://github.com/nix-community/patsh I am not able to use the latest versions of tree-sitter and tree-sitter-bash together as well due to its dependency on tree-sitter 0.19

I think it's generally a good idea to publish new versions of the grammar bindings even if no changes were made, just so it is compatible with the latest version of tree-sitter

[verhovsky]

@skovhus I can't publish releases. I've been asking for one myself lol #100 (comment) so I just ended up publishing a fork under my own namespace https://www.npmjs.com/package/@curlconverter/tree-sitter-bash

[figsoda]

cc @dcreager as a co-owner of the rust crate

[erikarvstedt]

Please publish a new release.
The latest version fails on this simple snippet while the playground succeeds:

(a a#a)
(a a#a)

[webpro]

Sorry to bother you @maxbrunsfeld, but it would be very helpful if you could provide a status update for this repository. If there are no plans for further releases or development, then that is fine too. Folks would be able to make better decisions such as using @verhovsky's fork or look for alternative solutions. Thanks!

[amaanq]

I can look into this shortly.

[webpro]

That would be great, @amaanq! And now that I see your profile: hats off 🙏

[amaanq]

I released 0.20.0 since #187

[verhovsky]

@amaanq there's no release on GitHub so there's no prebuilt binaries so it's causing #188

[amaanq]

Not sure I understand - I've never seen those binaries in a tree-sitter release in any other repo but this, what's with those 60 binary assets..?

I can create a GH release, but I have no clue what the assets are/how to create them. I also am not a huge fan of distributing binaries, not every machine is the same, what's compiled/built on mine might not work universally

[verhovsky]

@amaanq yea, tree-sitter-bash seems to be one of the only languages we were/are pre-building binaries for. Ideally we should do it for all languages...

We have node-tree-sitter (which runs native code) and web-tree-sitter (which runs through WASM either from the terminal or from a web browser). If you're using web-tree-sitter to parse Bash, you have to npm install tree-sitter-bash and then run a command to generate the wasm file. With node-tree-sitter, you need to compile the C code (or C++) for the bash parser into a native program for your operating system and computer architecture before you can require() it

tree-sitter-bash/bindings/node/index.js

Line 2 in 9ed0720

module.exports = require("../../build/Release/tree_sitter_bash_binding");

tree-sitter-bash/binding.gyp

Line 4 in 9ed0720

"target_name": "tree_sitter_bash_binding",

This requires Python and a C compiler. One way to work around this is

https://github.com/prebuild/prebuild

https://napi.inspiredware.com/build-tools/prebuild.html

It works like this, you configure you packages so that when users run npm install <your package>, instead of installing it the usual way, that command will instead try to download a binary file from GitHub Releases and if it can't download one, fallback to compiling the code with node-gyp (which is what requires Python, and a C/C++ compiler). We do this for node-tree-sitter here:

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/package.json#L19

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/package.json#L35

And in tree-sitter-bash repo here

tree-sitter-bash/package.json

Line 30 in 9ed0720

"install": "prebuild-install || node-gyp rebuild",

The way you do releases is you first make an empty GitHub release, then make a release on npm and then you run prebuild -t 16.0.0 -t 18.0.0 -t 20.0.0 --strip -u ${{ secrets.GH_TOKEN }} which compiles the code for the given Node versions and uploads a file for each Node version to GitHub Releases.

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L15-L16

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L92-L93

https://github.com/tree-sitter/node-tree-sitter/releases/tag/v0.20.5

We run this "compile and upload to github" command under each of the 3 operating systems.

[amaanq]

Gotcha, this was good to know.

So, since I've published 0.20.0 and 0.20.1 (and maybe 0.20.2, fixing some more stuff atm and if it's sizable enough I'll release another ver ;) ), will I still be able to run prebuild myself after it's been published? Still a bit lost on that process

Anyways, this is a bandaid solution to a bigger problem, I want to add a pack and publish command to the core cli itself to build the parser.so, wasm binary, and now these binaries as well after learning about this :)

Pack would create them, and publish upload them to GH releases/artifacts.

[verhovsky]

will I still be able to run prebuild myself after it's been published

You would first have to create a tag and a GitHub Release for the corresponding versions 0.20.0 and 0.20.1

https://github.com/tree-sitter/tree-sitter-bash/releases/new

After that, yes, if you have can make a GitHub API key with the appropriate permissions (which I'm guessing you can't), you can run

prebuild -t 16.0.0 -t 18.0.0 -t 20.0.0 --strip -u $GITHUB_API_KEY

locally from your tree-sitter-bash/ directory and it should prebuild and then upload the files. I don't know how it works but I assume that it works by checking version in package.json and uploading to that GitHub Release. It will only compile for whichever operating system you have, if you have a Windows, a Linux and a macOS computer and you run that command on each one then it'll upload 3 sets of files for each OS and you'd have a complete release. For node-tree-sitter we run the command three times from each OS and for macOS we prebuild for both Intel and ARM architectures. We also compile all of them for Electron, which is different from Node for whatever reason.

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L72-L74

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L88-L91

[amaanq]

I see, I'll ask Max about this, I don't think I can create a GH API Key

Thanks for the explanation!

Hm wait, Max made the node token available org-wide recently so I could auto publish npm packages as well as the rust crates, is it possible to auto-run prebuild & uploading to the release in the CI then?

[verhovsky]

Yea, exactly, it's all possible you just need to copy the code from node-tree-sitter to here. The only changes are related to making it work with that if

tree-sitter-bash/.github/workflows/publish.yml

Line 70 in 9ed0720

if: env.version_changed == 'true'

the way node-tree-sitter works is that release is triggered when you create a new release on GitHub

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L10-L12

Which I guess is one more manual step than your way of scanning for an increased version number on each commit.

I don't think I can create a GH API Key

I don't think you should bother manually pre-building binaries for those releases. I would just make the tag and github release for the relevant commits just so that we're not missing versions on GitHub Releases, but keep them empty until you make a release that prebuilds the binaries through github actions.

[amaanq]

Since I've yet to create a release for 0.20.0/1, is it possible for me to copy the build CI workflow here, then create the releases, and let that CI action then auto-upload the binaries?

I'm assuming the release type "released" is related to the GH release version, not the npmjs version, right? (Just want to double check 😅)

If so, I can just add that then cut releases right after

[verhovsky]

I don't have experience doing that, but that should work, yea. When you create a GitHub Release you have to choose a tag as the released commit. I see you already have git tags for those versions, so when you release one of those tags I assume it would run for that commit and upload to the relevant release for that tagged commit.

I'm assuming the release type "released" is related to the GH release version, not the npmjs version, right?

I don't understand the question because I think it's a bit misguided.

In here https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L3-L12

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master
  release:
    types:
      - released

Says the entire workflow (which is in one file) runs on pushes to master, pull requests to master and when a new github release is created. Then we only make npm releases and prebuild binaries when it's a new github release being created

https://github.com/tree-sitter/node-tree-sitter/blob/d47af7824ab4cfd1c7195c4f9d8225104636a5db/.github/workflows/build.yml#L50

publish:
    if: ${{ github.event_name == 'release' }}

Like I said, it runs on the state of the code at the commit (since it runs a git checkout). I don't know but it probably either gets the version it'll upload to GitHub releases from the package.json, or maybe it just uploads to the latest github release, either should work fine if you create the releases in order.