Patch for knoppix-mkimage

[Ronny Standtke]

Hi all,

Today I used Knoppix-5.3.1 for a while and then decided to create a persistent
KNOPPIX disk image.

My first attempt failed with an error message from rsync that there is no
space left on the device. I tried increasing the disk image size again and
again but it did not help. Then I noticed that the path names in the rsync
error message became longer and longer and looked like an endless recursion.
At this point in time I started to investigate this issue a little more...

knoppix-mkimage mounts the disk image to /tmp/knxhome and uses this directory
as the destination when rsync'ing /ramdisk. There is one central issue
here: /tmp/knxhome is a subdirectory of /ramdisk. So basically, you are
rsyncing a directory to one of its own subdirectories. In previous versions
of Knoppix this was probably not really a problem, because older versions of
rsync first get a complete file-list and then start to copy files. But, since
version 3.0 rsync uses a feature called "incremental file-list recursion". It
just looks up some files, transfers them, looks up the next bunch of files,
etc. Now you probably see the problem: When /ramdisk contains much more files
than rsync catches with its first file-list creation round and rsync finally
reaches /ramdisk/tmp/knxhome it considers the already copied files as new
files to be copied. Eternal recursion starts and the universe collapses. :-)

You can reproduce this bug by booting Knoppix, creating a lot of files in your
home directory (i.e. surf the web for a while, install some software packages
or unpack some source packages) and try creating the persistent disk image.

I fixed this bug by using /tmp_knxhome as mount point for the disk image (see
attached patch). I am not a l33t shell script h4xor, not at all. I don't know
if I did everything right. Please take a look.

Greetings

Ronny

[knoppix-mkimage.diff]

---

--- knoppix-mkimage.original 2008-04-26 08:13:58.498639618 +0200
+++ knoppix-mkimage 2008-04-26 08:17:03.212588966 +0200
@@ -346,14 +346,16 @@
 # Start progress display and copy data
 gauge "$MESSAGE9" &
 mke2fs -m0 "$LOOPDEV" 2>"$TMP.err" || { killgauge; $DIALOG --cr-wrap --title "$TITLE1" --msgbox "$ERROR `cat $TMP.err`" 10 75; bailout; }
-mkdir -p /tmp/knxhome
-mount -t ext2 "$LOOPDEV" /tmp/knxhome 2>"$TMP.err" && rsync --exclude /ramdisk/var/tmp/ --exclude /ramdisk/etc/mtab --exclude /etc/mtab -Ha /ramdisk/ /tmp/knxhome 2>"$TMP.err" || { umount /tmp/knxhome 2>/dev/null; killgauge; $DIALOG --cr-wrap --title "$TITLE1" --msgbox "$ERROR `cat $TMP.err`" 10 75; bailout; }
+MOUNT_POINT=/tmp_knxhome
+mkdir -p $MOUNT_POINT
+mount -t ext2 "$LOOPDEV" $MOUNT_POINT 2>"$TMP.err" && rsync --exclude /ramdisk/var/tmp/ --exclude /ramdisk/etc/mtab --exclude /etc/mtab -Ha /ramdisk/ $MOUNT_POINT 2>"$TMP.err" || { umount $MOUNT_POINT 2>/dev/null; killgauge; $DIALOG --cr-wrap --title "$TITLE1" --msgbox "$ERROR `cat $TMP.err`" 10 75; bailout; }
 
 # Save knoppix.sh script ?
 # Rather add some links in the network/printer configuration scripts that go to the pesistent homedir.
 
 # Finish
-umount /tmp/knxhome
+umount $MOUNT_POINT
+rmdir $MOUNT_POINT
 [ -n "$LOOPDEV" ] && losetup -d "$LOOPDEV"
 umount "$DIRECTORY" 2>/dev/null
 killgauge

---

[Klaus Knopper-2]

On Sun, Apr 27, 2008 at 12:32:44AM +0200, Ronny Standtke wrote:

> Hi all,
>
> Today I used Knoppix-5.3.1 for a while and then decided to create a persistent
> KNOPPIX disk image.
>
> My first attempt failed with an error message from rsync that there is no
> space left on the device. I tried increasing the disk image size again and
> again but it did not help. Then I noticed that the path names in the rsync
> error message became longer and longer and looked like an endless recursion.
> At this point in time I started to investigate this issue a little more...
>
> knoppix-mkimage mounts the disk image to /tmp/knxhome and uses this directory
> as the destination when rsync'ing /ramdisk. There is one central issue
> here: /tmp/knxhome is a subdirectory of /ramdisk. So basically, you are
> rsyncing a directory to one of its own subdirectories. In previous versions
> of Knoppix this was probably not really a problem, because older versions of
> rsync first get a complete file-list and then start to copy files. But, since
> version 3.0 rsync uses a feature called "incremental file-list recursion". It
> just looks up some files, transfers them, looks up the next bunch of files,
> etc. Now you probably see the problem: When /ramdisk contains much more files
> than rsync catches with its first file-list creation round and rsync finally
> reaches /ramdisk/tmp/knxhome it considers the already copied files as new
> files to be copied. Eternal recursion starts and the universe collapses. :-)
>
> You can reproduce this bug by booting Knoppix, creating a lot of files in your
> home directory (i.e. surf the web for a while, install some software packages
> or unpack some source packages) and try creating the persistent disk image.

Curiously, I did not catch this bug before, probably because of what you
suggested, that previous versions of rsync handled overlapping src/dst
copies differently. And it just was wrong to mount the persistent home
inside the directory to copy in the first place. Actually, /tmp should
probably be a different tmpfs than /ramdisk.

> I fixed this bug by using /tmp_knxhome as mount point for the disk image (see
> attached patch). I am not a l33t shell script h4xor, not at all. I don't know
> if I did everything right. Please take a look.

Your patch looks OK, thanks a lot. Scheduled for next release.

Regards
-Klaus Knopper


--
To UNSUBSCRIBE, email to debian-knoppix-request@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Count Of Dracula]

> Your patch looks OK, thanks a lot. Scheduled for next release.
>
> Regards
> -Klaus Knopper
>

May I humbly ask when you are planning to release next version of Knoppix?

Thanks


--
To UNSUBSCRIBE, email to debian-knoppix-request@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Klaus Knopper-2]

On Sun, Apr 27, 2008 at 09:49:19PM +0530, Count Of Dracula wrote:
> > Your patch looks OK, thanks a lot. Scheduled for next release.
> >
> > Regards
> > -Klaus Knopper
> >
>
> May I humbly ask when you are planning to release next version of Knoppix?

I would like to get the CD version ready, together with the DVD. At the
moment, not enough time for both. :-(

If your next question would be "How can I help?", you could make a list
(or start a wiki/poll for this) of software that can be removed from the
DVD version in order to fit on a CD (we need to get down to below 2GB
uncompressed size). Finding out what to keep probably the most difficult
task.

Regards
-Klaus


--
To UNSUBSCRIBE, email to debian-knoppix-request@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Daniel Baumann-4]

Klaus Knopper wrote:
> If your next question would be "How can I help?", you could make a list
> (or start a wiki/poll for this) of software that can be removed from the
> DVD version in order to fit on a CD (we need to get down to below 2GB
> uncompressed size). Finding out what to keep probably the most difficult
> task.

it would be helpfull if you would publish the *actual*[0] list of
packages you're installing in knoppix; the complete package list (i.e.
the output of dpkg -l) is unmaintainable as it includes depends and
recommends of the actual packages as well.

[0] http://lists.debian.org/debian-knoppix/2006/09/msg00000.html

--
Address:        Daniel Baumann, Burgunderstrasse 3, CH-4562 Biberist
Email:          daniel.baumann@...
Internet:       http://people.panthera-systems.net/~daniel-baumann/


--
To UNSUBSCRIBE, email to debian-knoppix-request@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...