Skip to content

Find and build all packages individually #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# CHANGELOG

## 0.10.2

- Bump Rust to 1.45.2
- Build all packages found in the source tree individually. The search root is
the first command line argument, defaulting to `.`.

## 0.10.1

## 0.10.0

## 0.9.0

- Bump Rust to 1.45.0
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
FROM trzeci/emscripten:1.39.8-fastcomp

# Note: I tried slim and had issues compiling wasm-pack, even with --features vendored-openssl
FROM rust:1.45.0
FROM rust:1.45.2

# setup rust with Wasm support
RUN rustup target add wasm32-unknown-unknown
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.PHONY: build publish run debug

DOCKER_NAME := "cosmwasm/rust-optimizer"
DOCKER_TAG := 0.10.1
DOCKER_TAG := 0.10.2
CODE ?= "/path/to/contract"
USER_ID := $(shell id -u)
USER_GROUP = $(shell id -g)
Expand Down
39 changes: 23 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,12 @@ optimization on the build size, using binary stripping and `wasm-opt`.

## Usage

*This works for most cases, for cosmwasm builds see advanced*
*This works for one or more independent crates. When working with workspaces, see below.*

The easiest way is to simply use the [published docker image](https://hub.docker.com/r/cosmwasm/rust-optimizer).
You must set the local path to the smart contract you wish to compile and
it will produce an `artifacts` directory with `<crate_name>.wasm`
It searches for all `Cargo.toml`s in the source tree and creates
a smart contract build for every result.
It will produce an `artifacts` directory with `<crate_name>.wasm`
and `contracts.txt` containing the hashes. This is just one file.

Run it a few times on different computers
Expand All @@ -24,7 +25,7 @@ you to produce a smaller build that works with the cosmwasm integration tests
docker run --rm -v "$(pwd)":/code \
--mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
cosmwasm/rust-optimizer:0.10.1
cosmwasm/rust-optimizer:0.10.2
```

Demo this with `cosmwasm-examples` (going into eg. `erc20` subdir before running),
Expand All @@ -34,27 +35,33 @@ Note that we use one registry cache (to avoid excessive downloads), but the targ
contract that we compile. This means no interference between contracts, but very fast recompile times when making
minor adjustments to a contract you had previously created an optimized build for.

## Advanced usage
## Workspace usage

*This is designed for cosmwasm samples. You cannot provide automatic verification for these*
Sometimes the basic usage above finds and builds too many `Cargo.toml`s,
leading to undesired results, especially in workspaces. In such cases
you can pass an extra argument that serves as a path to the search root.
This way you can avoid running the build command on a workspace root or dependencies.

If you have a more complex build environment, you need to pass a few more
arguments to define how to run the build process.
**Build all packages somewhere in ./contracts**

[`cosmwasm`](https://github.com/CosmWasm/cosmwasm) has a root workspace
and many contracts under `./contracts/*`, which are **excluded** in the
top-level `Cargo.toml`. In this case, we compile each contract separately
with it's own cache. However, since they may refer to packages via path
(`../../packages/std`), we need to run the script in the repo root. In this
case, we can use the optimize.sh command:
```sh
docker run --rm -v "$(pwd)":/code \
--mount type=volume,source="devcontract_cache_burner",target=/code/contracts/burner/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
cosmwasm/rust-optimizer:0.10.2 ./contracts
```

**Build a single cotract in a sub-folder**

```shell
```sh
docker run --rm -v "$(pwd)":/code \
--mount type=volume,source="devcontract_cache_burner",target=/code/contracts/burner/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
cosmwasm/rust-optimizer:0.10.1 ./contracts/burner
cosmwasm/rust-optimizer:0.10.2 ./contracts/burner
```

Note: the path argument must be relative since it is evaluated in the docker guest environment.

## Development

Take a look at the [Makefile](https://github.com/CosmWasm/rust-optimizer/blob/master/Makefile)
Expand Down
60 changes: 33 additions & 27 deletions optimize.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,43 @@ command -v shellcheck > /dev/null && shellcheck "$0"

export PATH=$PATH:/root/.cargo/bin

mkdir -p artifacts
contractdir="$1"

# There are two cases here
# 1. All contracts (or one) are included in the root workspace (eg. `cosmwasm-template`, `cosmwasm-examples`, `cosmwasm-plus`)
# In this case, we pass no argument, just mount the proper directory.
# 2. Contacts are excluded from the root workspace, but import relative paths from other packages (only `cosmwasm`).
# In this case, we mount root workspace and pass in a path `docker run <repo> ./contracts/hackatom`

# This parameter allows us to mount a folder into docker container's "/code"
# and build "/code/contracts/mycontract".
# Note: if contractdir is "." (default in Docker), this ends up as a noop
echo "Building contract in $(realpath -m "$contractdir")"
(
cd "$contractdir"

# Linker flag "-s" for stripping (https://github.com/rust-lang/cargo/issues/3483#issuecomment-431209957)
# Note that shortcuts from .cargo/config are not available in source code packages from crates.io
RUSTFLAGS='-C link-arg=-s' cargo build --release --target wasm32-unknown-unknown --locked
)

# wasm-optimize on all results
for wasm in "$contractdir"/target/wasm32-unknown-unknown/release/*.wasm; do
name=$(basename "$wasm")
echo "Optimizing $name"
wasm-opt -Os "$wasm" -o "artifacts/$name"
contracts_root="$1"

# See https://stackoverflow.com/a/23357277/2013738 to understand this find-to-array monster
contract_dirs=()
while IFS= read -r -d $'\0'; do
contract_dir=$(dirname "$REPLY")
contract_dirs+=("$contract_dir")
done < <(find "$contracts_root" \( -name target -o -name package -o -name .git \) -type d -prune -false -o -name Cargo.toml -type f -print0)
echo "Found the following contract dirs: ${contract_dirs[*]}"

for contract_dir in "${contract_dirs[@]}"; do
echo "Building contract in $(realpath -m "$contract_dir")"
(
cd "$contract_dir"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue of the . directory being called in any random order and overriding others can be fixed here if we just filter for actual crates (eg. if there is the members field in Cargo.toml skip it)

Otherwise, I like the find


# Linker flag "-s" for stripping (https://github.com/rust-lang/cargo/issues/3483#issuecomment-431209957)
# Note that shortcuts from .cargo/config are not available in source code packages from crates.io
RUSTFLAGS='-C link-arg=-s' cargo build --release --target wasm32-unknown-unknown --locked
)
done

mkdir -p artifacts
artifacts_dir=$(realpath artifacts)

# find the target directories and build results, optimize and write to artifacts
while IFS= read -r -d '' target_dir; do
for wasm in "$target_dir"/release/*.wasm; do
echo "Found build result $wasm"
name=$(basename "$wasm")
wasm-opt -Os "$wasm" -o "$artifacts_dir/$name"
echo "Created artifact $name"
done
done < <(find . -name wasm32-unknown-unknown -type d -print0)

# create hash
(
cd artifacts
cd "$artifacts_dir"
sha256sum -- *.wasm > checksums.txt
)

Expand Down