[Grml] Using grml to build a stacked Debian Live system

[T o n g]

Hi, 

These are what grml users had been asking/longing for:

  On Sun, 08 Mar 2009 15:32:01 +0100, Lars-Erik Helander wrote:

  > Regarding the persistence solution, I think I would rather like to have
  > a solution where the installed packages are not forced to be loaded into
  > main memory - as I understand that the live-snapshot would do, or ...?
  > Do grml have some toolchain that will allow me to either add stuff to an
  > existing squashfs or have the system "union mount" additional squashfs
  > files?

  * Michael Schierl <schierlm-public <at> gmx.de> [20061105 19:15]:

  > . . .  I want to carry around as few disks
  > as possible. So I thought if it is possible to splitting the squashfs
  > image into two images that are merged by unionfs, where the smaller
  > one contains a basic system that includes all stuff from grml-small
  > (which may be larger than 50MB, but as small as possible) and have a
  > cheatcode to load only the small part into ram (and not use the large
  > part at all). I do not know if this is feasible . . . 

but the answer isn't available up until now -- now, you can build
a stacked Debian Live system with the help of aufs + grml
tools.

But why "stacked" you may ask first. 

If grml is build on top of grml-medium, which is build on top of
grml-small, then putting all 3 flavor on the same USB would not
cost you more than one CD space, whereas currently it almost need
two.

Another great advantage is that, I never need to remaster my live
system any more just for my own customization -- putting my extra
stuff in is just as simple as putting in several modules. My
live-usb always contains the latest tools/docs of my own, without
going through the live system remastering process. I had about 7
layered modules that I build for slax and I just put them on top
of grml. It works perfectly.

Further, with stacked live system, you can, for the first time,
load grml-small into ram (or grml-medium depending on your ram
size), and leave the rest on USB/CD.

Most importantly, it'll be much faster. Suppose your stacked live
system is layered like the following:

1. All console tools
2. Xorg + lightweight window manager + essential X tools like gparted
3. Heavy weight desktop system (Gnome/Kde) 
4. Occasionally used applications (KOffice/Latex)

If you limit your self to only certain tasks (say console-only
hacks), then you only need a limited number of modules to be
loaded; all other modules are not loaded on start up. hence it
will start faster, use less ram, and cause applications to run
faster.  Moreover your CD drive will only seek in a small area
(instead of all across the whole CD). You don't even need to load
it to ram if the loaded modules can be completed cached by the
kernel. This significantly improves the speed. On the other hand,
you can enjoy the full fledged desktop applications if you want
to, from the very same CD.

OK, enough "selling". Let's see how it can be achieved using aufs
+ grml tools. I'll use the scenario that grml-full is build on top
of grml-medium, which is build on top of grml-small as the
example.


1. Use aufs to create the stacked framework. The first command
creates necessary directories, and the 2nd creates a stacked aufs
file system, of which /grml/grml-layers/grml-work is the RW
working branch. In this aufs file system, if a file is not
available in grml-full, aufs looks in grml-medium, else in
grml-small.

  mkdir -p /grml/grml-live/grml_chroot/ /grml/grml-layers/{grml-full,grml-medium,grml-small,grml-work}
  mount -t aufs -o br=/grml/grml-layers/grml-work=rw:/grml/grml-layers/grml-full:/grml/grml-layers/grml-medium:/grml/grml-layers/grml-small none /grml/grml-live/grml_chroot/

2. Use grml-live to build the first layer. 

  CLASSES='GRMLBASE,GRML_SMALL,RELEASE,AMD64'
  ARCH=amd64
  grml-live -a $ARCH -c $CLASSES . . .

3. Now the first layer, grml-small, is available under
  /grml/grml-live/grml_chroot/, which is the mount point of the
  aufs file system. We need to move the content from RW layer to
  the lower /grml/grml-layers/grml-small layer. Note that the final
  umount and mount is recommended after the move, because the
  inode number may be changed.

  aubrsync move /grml/grml-live/grml_chroot /grml/grml-layers/grml-work /grml/grml-layers/grml-small
  umount /grml/grml-live/grml_chroot/
  !mount

4. Install the next layer. This time grml-debootstrap is used instead, to
install all grml-medium packages on top of the current chroot system.

  grml-debootstrap --verbose --arch amd64 --target /grml/grml-live/grml_chroot/ . . . 

5. Now grml-medium is available under /grml/grml-live/grml_chroot/. Again,
we need to move and reflect all modifications from upper to lower
layer. This time is to grml-medium:

  aubrsync move /grml/grml-live/grml_chroot /grml/grml-layers/grml-work /grml/grml-layers/grml-medium
  umount /grml/grml-live/grml_chroot/
  !mount

6. Repeat step 4 and 5 to finish grml-full.

7. Create squashfs filesystem

  ln -s grml.squashfs /grml/grml-live/grml_cd/live/grml_small.squashfs
  mksquashfs-lzma /grml/grml-layers/grml-medium /grml/grml-live/grml_cd/live/grml_medium.squashfs -b 256k -lzma
  mksquashfs-lzma /grml/grml-layers/grml-full /grml/grml-live/grml_cd/live/grml_full.squashfs -b 256k -lzma

8. Create stacked squashfs filesystem. I'm using grml-full as an
example. Under /grml_cd/live/, create a file, say
grml-full.module, in which contains:

  grml_small.squashfs
  grml_medium.squashfs
  grml_full.squashfs

8. Make it bootable. I'm using grub as an example. In the kernel
line, use the following. Note the grml-full is the file you just
named without the .module extension.

  kernel ... live-media-path=/live module=grml-full

9. Put them on USB and test it. That's it.

Note, the above is for illustration purpose only. Substitute with
your own CLASS or path please.

PS. mika, if you want, please send me the wiki access info to my 
bts email address, I can put the above into wiki. 

Thanks