DUMP: You can't update the dumpdates file when dumping a subdirectory

[Neil Marjoram]

Hello all,

I'm a little confused, I've been running the same config for years, last
change a couple of years ago, and now I am starting to get some errors
on two systems.

Both the systems are RHEL 5.x (latest), running a compiled 2.5.2p1 to
get over a redhat omission, interestingly one system is a clone of the
other (nearly!), both the system fail backing up /usr/local. Other
directories backup with no issues. I've got no clues, the debug spits
out exactly the same error as the log, so no extra info there,

/--  host1 /usr/local lev 0 FAILED [dump (20830) /sbin/dump returned 1]
sendbackup: start [host1:/usr/local level 0]
sendbackup: info BACKUP=/sbin/dump
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/sbin/restore -xpGf - ...
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
|   DUMP: You can't update the dumpdates file when dumping a subdirectory
|   DUMP: The ENTIRE dump is aborted.
sendbackup: error [dump (20830) /sbin/dump returned 1]
\--------


Does anyone know what this is?

Many thanks,

Neil.

--
Neil Marjoram
Systems Manager
Adastral Park Campus
University College London
Ross Building
Adastral Park
Martlesham Heath
Ipswich - Suffolk
IP5 3RE




Reclaim Your Inbox!
http://www.mozilla.org/products/thunderbird

[Jean-Louis Martineau-2]

Like the message says: You can't use DUMP to backup a subdirectory, DUMP
work only on partition.
You must use GNUTAR to backup a directory.

Jean-Louis

Neil Marjoram wrote:

> Hello all,
>
> I'm a little confused, I've been running the same config for years,
> last change a couple of years ago, and now I am starting to get some
> errors on two systems.
>
> Both the systems are RHEL 5.x (latest), running a compiled 2.5.2p1 to
> get over a redhat omission, interestingly one system is a clone of the
> other (nearly!), both the system fail backing up /usr/local. Other
> directories backup with no issues. I've got no clues, the debug spits
> out exactly the same error as the log, so no extra info there,
>
> /--  host1 /usr/local lev 0 FAILED [dump (20830) /sbin/dump returned 1]
> sendbackup: start [host1:/usr/local level 0]
> sendbackup: info BACKUP=/sbin/dump
> sendbackup: info RECOVER_CMD=/bin/gzip -dc |/sbin/restore -xpGf - ...
> sendbackup: info COMPRESS_SUFFIX=.gz
> sendbackup: info end
> |   DUMP: You can't update the dumpdates file when dumping a subdirectory
> |   DUMP: The ENTIRE dump is aborted.
> sendbackup: error [dump (20830) /sbin/dump returned 1]
> \--------
>
>
> Does anyone know what this is?
>
> Many thanks,
>
> Neil.
>

[Aaron J. Grier-2]

On Thu, Oct 02, 2008 at 10:15:52AM -0400, Jean-Louis Martineau wrote:
> >sendbackup: info end
> >|   DUMP: You can't update the dumpdates file when dumping a subdirectory
> >|   DUMP: The ENTIRE dump is aborted.
> >sendbackup: error [dump (20830) /sbin/dump returned 1]
>
> Like the message says: You can't use DUMP to backup a subdirectory, DUMP
> work only on partition.
> You must use GNUTAR to backup a directory.

I beg your pardon, but /sbin/dump is perfectly capable of dumping
subdirectories on most unixes.  it just won't record (or read) the date
of the dump in /etc/dumpdates.

a lot of /sbin/dump will accept a -T parameter to specify a date for
incrementals.  I assume tar requires a similar parameter, perhaps it
could be used with /sbin/dump as well?

--
  Aaron J. Grier | "Not your ordinary poofy goof." | agrier@...

[Jon LaBadie]

On Thu, Oct 02, 2008 at 06:03:23PM -0700, Aaron J. Grier wrote:

> On Thu, Oct 02, 2008 at 10:15:52AM -0400, Jean-Louis Martineau wrote:
> > >sendbackup: info end
> > >|   DUMP: You can't update the dumpdates file when dumping a subdirectory
> > >|   DUMP: The ENTIRE dump is aborted.
> > >sendbackup: error [dump (20830) /sbin/dump returned 1]
> >
> > Like the message says: You can't use DUMP to backup a subdirectory, DUMP
> > work only on partition.
> > You must use GNUTAR to backup a directory.
>
> I beg your pardon, but /sbin/dump is perfectly capable of dumping
> subdirectories on most unixes.  it just won't record (or read) the date
> of the dump in /etc/dumpdates.

I took JLM's comments as "can't be used by amanda".

If dump won't record/read /etc/dumpdates when dumping
a subdirectory, incrementals will be a real problem.

>
> a lot of /sbin/dump will accept a -T parameter to specify a date for
> incrementals.  I assume tar requires a similar parameter, perhaps it
> could be used with /sbin/dump as well?

Not sure if it is used, but gnutar has a similar -N (--newer-mtime)
option.  But if it is based on "mtime" as the option name suggests,
then changes to the inode would be missed.

Ufsdump doesn't have a time-based option.

--
Jon H. LaBadie                  jon@...
 JG Computing
 12027 Creekbend Drive [phone]
 Reston, VA  20194 [phone] (fax)

[Dustin J. Mitchell-3]

On Thu, Oct 2, 2008 at 9:03 PM, Aaron J. Grier <agrier@...> wrote:
> I beg your pardon, but /sbin/dump is perfectly capable of dumping
> subdirectories on most unixes.  it just won't record (or read) the date
> of the dump in /etc/dumpdates.

I'm happy to be proven wrong (I've not used dump myself), but it was
my understanding that dump, in general, worked at the filesystem
level, against a block device.  For example, the OSX manpage (which is
just inherited from the BSD manpages) says:

     dump [-0123456789cnu] [-B records] [-b blocksize] [-d density]
          [-f file] [-h level] [-s feet] [-T date] filesystem

where the use of the term "filesystem" means, to my understanding, a
filesystem and not an arbitrary subdirectory.  Now, you may have a
filesystem mounted at /usr/local, in which case you can use dump to
back up /usr/local, but I don't think that's what you meant.

Can you point to some documentation to support your assertion?

Dustin

--
Storage Software Engineer
http://www.zmanda.com

[Jean-Louis Martineau-2]

Aaron,

You are right some dump works on directory but amanda doesn't use the -T
parameter.
It is a dangerous option, all backup program working with a timestamps
(all dump flavour and star) have the same problem. If a directory is
moved across the directories you backup, it is not include in the dump
until the next full.
GNUTAR doesn't have this problem because it keep a complete list of
files, not just a timestamps.

A small example of the problem:
/home is the root of the partition, you have two files:
  /home/foo/a/b/c
  /home/bar/c/b/a
You backup the two directory separately.
You have the following backup sequence and one user operation:

        /home/foo   /home/bar     TAPE
day 1       0          1         tape-01
day 2       1          1         tape-02
day 3       1          1         tape-03
day 4       1          1         tape-04
day 5       1          0         tape-05
a user do: mv /home/foo/a /home/bar
day 6       1          1         tape-06
day 7       1          1         tape-07
day 8       0          1         tape-08
day 9       1          1         tape-01
day 10      1          1         tape-02
day 11      1          1         tape-03
day 12      1          0         tape-04


/home/bar/a/b/c will not be in the backup of /home/bar until day 12 on
tape-04
If you restore the backup for day 6 or 7
    /home/foo level 0 on tape tape-01, level 1 on tape tape-06 or tape-07
    /home/bar level 0 on tape tape-05, level 1 on tape tape-06 or tape-07
  then you end up with no /home/bar/a/b/c file, and no /home/foo/a/b/c
file, you can recover /home/foo/a/b/c from level 0 of /home/foo on
tape-01, but how do you know the user moved the files.

That's worse, after day 9 and overwrite of tape-01, the a/b/c file is
not on tape, there is no way to recover it
On day 12, it will be on the level 0 backup of /home/bar.

Jean-Louis

Aaron J. Grier wrote:

> On Thu, Oct 02, 2008 at 10:15:52AM -0400, Jean-Louis Martineau wrote:
>  
>>> sendbackup: info end
>>> |   DUMP: You can't update the dumpdates file when dumping a subdirectory
>>> |   DUMP: The ENTIRE dump is aborted.
>>> sendbackup: error [dump (20830) /sbin/dump returned 1]
>>>      
>> Like the message says: You can't use DUMP to backup a subdirectory, DUMP
>> work only on partition.
>> You must use GNUTAR to backup a directory.
>>    
>
> I beg your pardon, but /sbin/dump is perfectly capable of dumping
> subdirectories on most unixes.  it just won't record (or read) the date
> of the dump in /etc/dumpdates.
>
> a lot of /sbin/dump will accept a -T parameter to specify a date for
> incrementals.  I assume tar requires a similar parameter, perhaps it
> could be used with /sbin/dump as well?
>
>  

[John Hein]

Dustin J. Mitchell wrote at 00:41 -0400 on Oct  3, 2008:
 > On Thu, Oct 2, 2008 at 9:03 PM, Aaron J. Grier <agrier@...> wrote:
 > > I beg your pardon, but /sbin/dump is perfectly capable of dumping
 > > subdirectories on most unixes.  it just won't record (or read) the date
 > > of the dump in /etc/dumpdates.
 >
 > I'm happy to be proven wrong (I've not used dump myself), but it was
 > my understanding that dump, in general, worked at the filesystem
 > level, against a block device.  For example, the OSX manpage (which is
 > just inherited from the BSD manpages) says:
 >
 >      dump [-0123456789cnu] [-B records] [-b blocksize] [-d density]
 >           [-f file] [-h level] [-s feet] [-T date] filesystem
 >
 > where the use of the term "filesystem" means, to my understanding, a
 > filesystem and not an arbitrary subdirectory.  Now, you may have a
 > filesystem mounted at /usr/local, in which case you can use dump to
 > back up /usr/local, but I don't think that's what you meant.
 >
 > Can you point to some documentation to support your assertion?

This was touched on as part of a larger thread in April.

http://www.mail-archive.com/amanda-users@.../msg40187.html

In short, some OS's support a dump with files, but it's not clear how
well that is supported (or will work at all) in amanda, particularly
with incremental dumps.

I think it'd be great if we could use the native filesystem's dump (or
dump-like) tool rather than gtar.  Then you can backup filesystem
specific attributes that gtar may not handle.  (and you don't touch
atime, which has always been an annoying quibble with having to use
tar).  Having to dump the whole filesystem, of course, is really the
big hurdle with using a dump or dump-like tool.

[Chris Hoogendyk]

As has been said before: on Linux, gnu-tar is preferred over dump (Linus
famously once said that dump is broken); but, on Solaris, ufsdump is
preferred.

I have not tested the specific instance that Jean-Louis points out
below. I have tested instances where directories or files within a
partition were moved around and/or deleted (this is just testing ufsdump
by itself). With ufsdump, if you restore a partition from a full and a
subsequent incremental, the incremental recovery takes into account
moves and deletions. Unlike the situation some complain about with
implementation on other platforms and other backup software, ufsrestore
from an incremental ufsdump will move and or delete the files appropriately.

The man pages for ufsdump on Solaris 9 say

          files_to_dump

           Specifies the files to dump. Usually it  identifies  a
           whole file system by its raw device name (for example,
           /dev/rdsk/c0t3d0s6). Incremental dumps (levels 1 to 9)
           of files changed after a certain date only apply  to a
           whole file system.  Alternatively,  files_to_dump  can
           identify  individual  files or directories.  All named
           directories that may be examined by the  user  running
           ufsdump,  as  well  as any explicitly-named files, are
           dumped. This dump is equivalent to a level 0  dump  of
           the  indicated portions of the filesystem, except that
           /etc/dumpdates is not updated even if  the  -u  option
           has  been  specified.  In all cases, the files must be
           contained in the same file system, and the file system
           must  be  local  to  the system where ufsdump is being
           run.


Which basically means that ufsdump/restore should always work, although
users who use it for some portion of a file system will end up always
getting full backups even if they try to tell ufsdump to do an
incremental (it won't).

Parameters for ufsdump are different in some cases than Linux or BSD
dump. T is time to wait for an autoload. Again, ufsdump won't do what
the T option is used for on some other systems.


---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst

<hoogendyk@...>

---------------

Erdös 4




Jean-Louis Martineau wrote:

> Aaron,
>
> You are right some dump works on directory but amanda doesn't use the
> -T parameter.
> It is a dangerous option, all backup program working with a timestamps
> (all dump flavour and star) have the same problem. If a directory is
> moved across the directories you backup, it is not include in the dump
> until the next full.
> GNUTAR doesn't have this problem because it keep a complete list of
> files, not just a timestamps.
>
> A small example of the problem:
> /home is the root of the partition, you have two files:
>  /home/foo/a/b/c
>  /home/bar/c/b/a
> You backup the two directory separately.
> You have the following backup sequence and one user operation:
>
>        /home/foo   /home/bar     TAPE
> day 1       0          1         tape-01
> day 2       1          1         tape-02
> day 3       1          1         tape-03
> day 4       1          1         tape-04
> day 5       1          0         tape-05
> a user do: mv /home/foo/a /home/bar
> day 6       1          1         tape-06
> day 7       1          1         tape-07
> day 8       0          1         tape-08
> day 9       1          1         tape-01
> day 10      1          1         tape-02
> day 11      1          1         tape-03
> day 12      1          0         tape-04
>
>
> /home/bar/a/b/c will not be in the backup of /home/bar until day 12 on
> tape-04
> If you restore the backup for day 6 or 7
>    /home/foo level 0 on tape tape-01, level 1 on tape tape-06 or tape-07
>    /home/bar level 0 on tape tape-05, level 1 on tape tape-06 or tape-07
>  then you end up with no /home/bar/a/b/c file, and no /home/foo/a/b/c
> file, you can recover /home/foo/a/b/c from level 0 of /home/foo on
> tape-01, but how do you know the user moved the files.
>
> That's worse, after day 9 and overwrite of tape-01, the a/b/c file is
> not on tape, there is no way to recover it
> On day 12, it will be on the level 0 backup of /home/bar.
>
> Jean-Louis
>
> Aaron J. Grier wrote:
>> On Thu, Oct 02, 2008 at 10:15:52AM -0400, Jean-Louis Martineau wrote:
>>  
>>>> sendbackup: info end
>>>> |   DUMP: You can't update the dumpdates file when dumping a
>>>> subdirectory
>>>> |   DUMP: The ENTIRE dump is aborted.
>>>> sendbackup: error [dump (20830) /sbin/dump returned 1]
>>>>      
>>> Like the message says: You can't use DUMP to backup a subdirectory,
>>> DUMP work only on partition.
>>> You must use GNUTAR to backup a directory.
>>>    
>>
>> I beg your pardon, but /sbin/dump is perfectly capable of dumping
>> subdirectories on most unixes.  it just won't record (or read) the date
>> of the dump in /etc/dumpdates.
>>
>> a lot of /sbin/dump will accept a -T parameter to specify a date for
>> incrementals.  I assume tar requires a similar parameter, perhaps it
>> could be used with /sbin/dump as well?