-
Notifications
You must be signed in to change notification settings - Fork 826
Open
Labels
Description
Hello,
I am encountering an error when trying to mount an S3 bucket using the MOUNT_CACHED option:
Configuration file
name: test
resources:
instance_type: g6e.2xlarge
disk_size: 50
disk_tier: medium
autostop:
idle_minutes: 10
down: true
file_mounts:
/outputs:
source: s3://my-bucket/
mode: MOUNT_CACHED
setup: |
sudo apt-get update && sudo apt-get install -y mdadm
# ...
# Env variable for rcache
export RCLONE_CACHE_DIR=/opt/dlami/nvme/rclone_cache
# Login to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ****Error message
Installing rclone...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 21.9M 0 192k 0 0 236k 0 0:01:34 --:--:-- 0:01:34 236k
100 21.9M 100 21.9M 0 0 12.1M 0 0:00:01 0:00:01 --:--:-- 12.1M
dpkg: error: dpkg frontend lock was locked by another process with pid 2746
Note: removing the lock file is always wrong, and can end up damaging the
locked area and the entire system. See <https://wiki.debian.org/Teams/Dpkg/FAQ>.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
1 22.7M 1 416k 0 0 414k 0 0:00:56 0:00:01 0:00:55 414k
100 22.7M 100 22.7M 0 0 12.5M 0 0:00:01 0:00:01 --:--:-- 12.5M
sudo: yum: command not found
sky.exceptions.CommandError: Command to mount failed with return code 1.
----- CMD -----
echo '
#!/usr/bin/env bash
set -e
{ [ "$(whoami)" == "root" ] && function sudo() { "$@"; } || true; }
MOUNT_PATH=/outputs
MOUNT_BINARY=rclone
# Check if path is already mounted
if grep -q $MOUNT_PATH /proc/mounts ; then
echo "Path already mounted - unmounting..."
fusermount -uz "$MOUNT_PATH"
echo "Successfully unmounted $MOUNT_PATH."
fi
# Install MOUNT_BINARY if not already installed
if [ -x "$(command -v rclone)" ]; then
echo "$MOUNT_BINARY already installed. Proceeding..."
else
echo "Installing $MOUNT_BINARY..."
ARCH=$(uname -m) && if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then ARCH_SUFFIX="arm"; else ARCH_SUFFIX="amd64"; fi && (which dpkg > /dev/null 2>&1 && (which rclone > /dev/null || (cd ~ > /dev/null && curl -O https://downloads.rclone.org/v1.68.2/rclone-v1.68.2-linux-${ARCH_SUFFIX}.deb && sudo dpkg -i rclone-v1.68.2-linux-${ARCH_SUFFIX}.deb && rm -f rclone-v1.68.2-linux-${ARCH_SUFFIX}.deb))) || (which rclone > /dev/null || (cd ~ > /dev/null && curl -O https://downloads.rclone.org/v1.68.2/rclone-v1.68.2-linux-${ARCH_SUFFIX}.rpm && sudo yum --nogpgcheck install rclone-v1.68.2-linux-${ARCH_SUFFIX}.rpm -y && rm -f rclone-v1.68.2-linux-${ARCH_SUFFIX}.rpm))
fi
# Check if mount path exists
if [ ! -d "$MOUNT_PATH" ]; then
echo "Mount path $MOUNT_PATH does not exist. Creating..."
sudo mkdir -p $MOUNT_PATH
sudo chmod 777 $MOUNT_PATH
else
# Check if mount path contains files
if [ "$(ls -A $MOUNT_PATH)" ]; then
echo "Mount path $MOUNT_PATH is not empty. Please mount to another path or remove it first."
exit 42
fi
fi
echo "Mounting $SOURCE_BUCKET to $MOUNT_PATH with $MOUNT_BINARY..."
mkdir -p ~/.sky/rclone_log && touch ~/.sky/rclone_log/729cf69f1171639df275d67931d4dbdf.log && [ ! -f /bin/fusermount3 ] && sudo ln -s /bin/fusermount /bin/fusermount3 || true; mkdir -p ~/.config/rclone && echo '"'"'[sky-s3-my-bucket]
type = s3
provider = AWS
access_key_id = my_access_key
secret_access_key = my_secret_key
acl = private
SkyPilot version:
skypilot, version 1.0.0.dev20250807
Is there something I'm doing wrong? Thank you.