If server's disk is full, Obnam seems to give no useful error message:
2015-08-29 16:31:42 ERROR Can't back up
/home/liw/ick/liw.state/extrautils/builds/106/build.log: RD5FA4X:
System error: chunks/1052/1065/3340/212da18852839223: None:
Failure
2015-08-29 16:31:42 ERROR OSError(None, 'Failure')
It looks like Obnam is getting too little info from paramiko, but maybe that can be fixed.
Reported by Bazyli Brzóska, http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org/2015-April/000144.html
Backing up two hardlinks to the same file, then mounting the backup repository with the FUSE plugin, results in the hardlinks being shown as having different inode numbers. They do thus not expose the hardlinks correctly.
Reported by Ilya Zonov (http://listmaster.pepperfish.net/pipermail/obnam-support-obnam.org/2015-April/003516.html, the list archive shows the message wrong, but can be decoded with base64).
Only the last file ends up in the repository.
See http://listmaster.pepperfish.net/pipermail/obnam-support-obnam.org/2014-June/003103.html for details.
Valery Yundin reports:
It is a problem not only when backup root is a symbolic link, but also when any parent directory of backup root is a symbolic link. Unless you explicitly ask to restore a directory which is already below symlink.
http://listmaster.pepperfish.net/pipermail/obnam-support-obnam.org/2014-April/002976.html
--liw
Alexander Sitnik reported that he'd noticed that "obnam fsck" checks all chunks in all generations, resulting in the same chunks being checked over and over. It should speed up "obnam fsck" if each chunk was checked only once. I note that this may need to be done with some care so that memory use doesn't increase too much. --liw
The following suggestion was sent to me by private e-mail (not sure if I can show the sender's name):
Date: Mon, 14 Oct 2013 11:48:41 +1100
Subject: Re: Obnam repo size with .sql dump files seem too big
Lars,
We implemented a strategy for identifying repeated chunks, even in
gzip-compressed files that have changes between versions. It might
work for obnam. The downside is it creates variable-sized chunks,
though you can set upper and lower limits.
Firstly, we identify chunk boundaries by using a rolling checksum
like Adler32, rolling over say a 128 byte contiguous region. Any
other efficient FIR (box-car) checksum algorithm would work. When
the bottom N bits of the checksum are zero, declare a block
boundary. This gives blocks of average size 2**N. If you want more
certainty, you can enforce a lower limit of 2**(N-3), and upper
limit of 2**N or 2**(N+1), for example (thereby either creating,
or ignoring, the boundaries that are defined by the checksum.
These variable-sized chunks will re-synchronise after differences
in two streams. To make it work with deflate, we flush the
compression context (the dictionary, we maintain the history,
losing 1-2% of compression) on each block boundary… but I'm not
sure that compression is necessary for obnam.
By choosing N appropriately, you get the block size you want. We
used N=12, for internet delivery (using HTTP subrange requests) of
updated files. For each file, we publish a catalog of Adler32 and
SHA-1 block checksums. Clients download that using HTTP, then
analyse their files before making requests for blocks they lack.
The invention of doing this in a deflate stream is due to Tim
Adam. When we discover we already have a given block, we can prime
the decompressor with that history before decompressing a received
update block. Really it should be implemented using 7zip instead
of deflate; a 32KB quotation history is too small.
I haven't tried this yet. It will require a new repository format version, so I've been working on adding that support instead. --liw
Obnam needs tests for using every filesystem type available as live data or repository.
Not sure how to arrange that without root access, but there's a need to do that.
It seems obnam mount
(the FUSE plugin) can't handle a client without
non-checkpoint generations. This is unfortunate, even if it is fairly
unlikely to happen. Should be easy enough to fix.
--liw
Obnam has no test for what happens when the filesystem fills up.