Adjusting paths and back-filling from another directory tree

[grifferz]

Hi,

I have a quite large rsnapshot setup which I'm looking at decomissioning and have been evaluating restic and rustic as the replacement.

While it would be feasible to just leave the rsnapshot directory tree where it is and refer to it as the place to go for data from before the cutover, it would be nice to import that data in to the new repo for convenience. My tests with restic have shown that the compression and better deduplication provide a decent space saving if nothing else.

The question is how best to do it.

Just in case anyone is unfamiliar with rsnapshot, it has left me with a directory structure like this:

/srv/ranspshot/daily.0
.
├── foo.example.com
│   ├── etc
│   │   ├── blah
│   │   .
│   │   .
│   ├── home
│   │   ├── andy
│   │   .
├── bar.example.com
│   ├── etc
│   │   ├── blah
│   │   .
│   │   .
│   ├── home
│   │   ├── andy
.   .   .

That is, a top level directory called daily.0 with the most recent daily snapshot, with one directory tree per host below that.

I took a naive approach with restic of changing to the root of this tree (daily.0) and kicking off a scripted set of backup commands like this:

cd foo.example.com
restic backup \
    etc/ \
    home/ \ 
    srv/ \
    --one-file-system \
    --tag from_rsnapshot \
    --host foo.example.com

After that completed I used restic rewrite --new-time ... to set the time of that snapshot to the mtime on the parent directory, so it appears like this data got into restic at the time that rsnapshot actually ran.

My thought with this was that later on for the regular snapshots on the real host foo.example.com I'd just be doing:

cd / && restic backup etc/ home/ srv/ --one-file-system

However, what happened is that restic made all the paths absolute so for example, one of the files backed up is /srv/rsnapshot/daily.0/foo.example.com/home/andy/.bash_profile and also there is a later snapshot from the real client with a file /home/andy/.bash_profile. restic doesn't know that the first snapshot is a parent of the second or that these two files are the same file.

Now, after reading some rustic documentation I understand that rustic has an --as-path option so possibly what I could have done is:

rustic backup \
    etc/ \
    home/ \
    srv/ \
    --as-path=. \
    --one-file-system \
    --tag from_rsnapshot \
    --host foo.example.com

and this would have stored these paths as relative from the start?

Then later, on the real host:

cd / && rustic backup \
    etc/ \
    home/ \
    srv/ \
    --as-path=. \
    --one-file-system

Is that correct?

I do see however that the documentation for --as-path says

Specifies the path for the backup when the source contains a single path.

so maybe that wouldn't work after all?

On the other hand, my reading of #294 suggests that rustic does in fact handle relative paths differently so is --as-path the wrong thing to be looking at and if I had just used rustic with relative paths to begin with this would in fact just work for me?

I apologise that my testing hasn't got as far as that yet — I need support for --cacert since I use rest-server so that has complicated my efforts to quickly try this out from my old rsnapshot host.

Any other advice or thoughts gladly received.

Thanks,
Andy

[grifferz]

I arranged a real TLS cert so I was no longer blocked on --cacert support, and was able to do some testing.

As I thought, --as-path is not right for me since I want multiple paths in each snapshot. Just continuing to use relative paths every time appears to do what I want.

[aawsome]

@grifferz
Can you please describe what problem you would like to solve? If you backup your rsnapshot data in any way, the data and metadata will be stored in a restic/rustic repo. So, the only difference is the tree-structure within. What do you want to solve here? Parent-detection? Diffing old and new snapshots? Or do you just want to have the "old" snapshots look exactly the same as the new ones?

--as-path currently works only for a singe path, correct.
Also if you backup a relative path (without ..), rustic will also just store a relative path.

[aawsome]

sorry, didn't see your last comment when writing this..