Setting the Locale on Linux

A locale is a system setting that allows users to configure the operating system for their own language, date, currency, etc. on Linux. There are slightly different ways to set up locales on different Linux distributions. Here I write how I would normally set up locales on Linux.

To set the system-wide default locale, I would append the LANG= variable to the Linux kernel. The kernel will happily accept the LANG= variable and pass it to the operating system. With syslinux, I would have an entry in syslinux.cfg as follows.

LABEL debian
KERNEL vmlinuz-3.6.0
INITRD /boot/initramfs.lzma
APPEND LANG=en_US.UTF-8 TERM=vt100 root=/dev/sda9

On Debian, Ubuntu, Mint and MEPIS, the system-wide locale may be specified in /etc/environment or /etc/default/locale.

LANG=C.UTF-8

To set your own locale different from the system-wide locale, just add a line to your .bashrc or .xsession:

export LANG=zh_CN.UTF-8

Generating locale-archive Again

The number of locales available on the system may be incomplete in order to save disk space. To see which locales are available, run the following command:

locale -a

If you want to add or remove locales, edit /etc/locale.gen. The following is a sample locale.gen.

ar_SA.UTF-8 UTF-8
bn_IN UTF-8
de_DE.UTF-8 UTF-8
en_US ISO-8859-1
en_US.UTF-8 UTF-8
es_MX.UTF-8 UTF-8
fa_IR UTF-8
fr_FR.UTF-8 UTF-8
gu_IN UTF-8
hi_IN UTF-8
id_ID.UTF-8 UTF-8
it_IT.UTF-8 UTF-8
ja_JP.UTF-8 UTF-8
kn_IN UTF-8
ko_KR.UTF-8 UTF-8
ml_IN UTF-8
mr_IN UTF-8
or_IN UTF-8
pa_PK UTF-8
pt_BR.UTF-8 UTF-8
ru_RU.UTF-8 UTF-8
ta_IN UTF-8
te_IN UTF-8
th_TH.UTF-8 UTF-8
tr_TR.UTF-8 UTF-8
vi_VN UTF-8
zh_CN.UTF-8 UTF-8
zh_TW.UTF-8 UTF-8

After changing the file locale.gen, you need to run locale-gen to regenerate the file /usr/lib/locale/locale-archive.

locale-gen

Enabling All Possible Locales

If you have plenty disk space, then you can enable all locales supported by the current glibc implementation. The list of supported locales is at /usr/share/i18n/SUPPORTED. Just copy it to /etc/locale.gen and rebuild the locales. It will regenerate the file /usr/lib/locale/locale-archive to about 100MB.

cp /usr/share/i18n/SUPPORTED /etc/locale.gen
locale-gen

Running Applications In Different Locales

Use the env command to run an application in a different locale. For example, to run pidgin in Chinese on an English desktop, I would issue:

env LANG=zh_CN.UTF-8 pidgin

Disk Cloning / Imaging over Network with SSH, Netcat, DD and XZ

Today we have affordable, ample storage and faster bandwidth to facilitate partition imaging and disk cloning over network. Nowadays, it's common and feasible to take the image of a whole partition for various reasons. Compared to file-based backups using tar, disk imaging provides the following advantages.

• The boot sector is preserved so that it's easy to make it bootable after the restore.
  
• Information such as UUID and LABEL is presered, which helps identify the partition in booting and mounting.
  
• Information such as ACL and XATTR is preserved, which helps restrict file access and secure the system.
  
• Every bit in the unused sectors is preserved, which may assist in digital forensics to uncover deleted or hidden information.
  

There are commercial programs for disk imaging and backup (Norton Ghost, Acronis True Image). However, Linux users can use readily available tools to get things done. For disk cloning/imaging, we can use ssh, netcat, dd and xz. Note that dd will fail on physically damaged disks. For such disks, use ddrescue instead.

For security and compression, we are going to use ssh and xz in this tutorial. If you don't like xz, feel free to substitute xz with gzip, bzip2 or lzop. Also, netcat is used to stream the dd output over the network. On Debian and Ubuntu derivatives, you need the following packages.

• bzip2, gzip, lzop, lzma OR xz-utils
  
• dd
  
• netcat
  
• ssh
  

We are making these assumptions in the following scenarios.

• Sending computer S
  
  This computer has IP address 192.168.1.1 and needs to back up partition /dev/sda1.
  
• Sending Port
  
  We'll send using port 5525.
  
• Receiving computer T
  
  This computer has IP address 192.168.1.2 and needs to restore partition /dev/sda2.
  
• Receiving Port
  
  We'll receive at port 7749.
  

Disk Cloning using dd, xz, netcat and ssh

In this scenario, we will clone a disk partition, simultaneously sending an image of the source partition /dev/sda1 from computer S (192.168.1.1) and restoring it at /dev/sda2 on computer T (192.168.1.2). Make sure that the source partition is not mounted or is mounted read-only. Also, make sure that the target partition size is greater than or equal to the source partition size.

1. At the sending computer, compress the source partition /dev/sda1 with xz and set up netcat to send it at port 5525:
   
   

   dd if=/dev/sda1 bs=16M | xz | nc -l 5525

   
   
2. At the receiving computer, set up a SSH tunnel to the sending computer (192.168.1.1):
   
   

   ssh -f -N -L 7749:127.0.0.1:5525 username@192.168.1.1

   
   
3. At the receiving computer, type the following command to receive the partition image and restore it at /dev/sda2:
   
   

   nc 127.0.0.1 7749 | xz -d | dd of=/dev/sda2 bs=16M

   
   

Alternatively, we could take the following steps to achieve the same thing. However, we start at the receiving computer.

1. At the receiving computer with the target partition /dev/sda2, type the following command to receive the partition image:
   
   

   nc -l 7749 | xz -d | dd of=/dev/sda2 bs=16M

   
   
2. At the sending computer with the source partition /dev/sda1, set up a SSH tunnel to the receiving computer (192.168.1.2):
   
   

   ssh -f -N -L 5525:127.0.0.1:7749 username@192.168.1.2

   
   
3. At the sending computer, type the following command to compress the source partition /dev/sda1 and transmit it over the SSH tunnel:
   
   

   dd if=/dev/sda1 bs=16M | xz | nc 127.0.0.1 5525

   
   

   Note that the transfer may take many hours for a large partition.

   
   

Disk Imaging using dd, xz, netcat and ssh

In this scenario, we will just send an image of the source partition /dev/sda1 to the receiving computer T (192.168.1.2) without restoring it. Make sure that the source partition is not mounted or is mounted read-only. A question remains whether to compress the image at the sending or receiving computer. The answer depends on which computer is more powerful. For this example, we'll compress at the sending computer (for network bandwidth reason).

1. At the sending computer, compress the source partition /dev/sda1 with xz and stream it using netcat:
   
   

   dd if=/dev/sda1 bs=16M | xz | nc -l 5525

   
   
2. At the receiving computer, set up a SSH tunnel to the sending computer (192.168.1.1):
   
   

   ssh -f -N -L 7749:127.0.0.1:5525 username@192.168.1.1

   
   
3. At the receiving computer, type the following command to receive the file:
   
   

   nc 127.0.0.1 7749 > partimg.xz

   
   

Alternatively, we could take the following steps to achieve the same thing.

1. At the receiving computer, set up netcat to listen at port 7749 and save the incoming data to a file partimg.xz.
   
   

   nc -l 7749 | dd of=partimg.xz bs=16M

   
   
2. At the sending computer, establish a SSH tunnel to the receiving computer (192.168.1.2) first:
   
   

   ssh -f -N -L 5525:192.168.1.2:7749 username@192.168.1.2

   
   
3. At the sending computer, type the following command to compress the source partition /dev/sda1 and transmit it over the SSH tunnel:
   
   

   dd if=/dev/sda1 bs=16M | xz | nc 127.0.0.1 5525

   
   

   Note that the transfer may take many hours for a large partiiton.

   
   

Alternative Simple Commands for Disk Cloning / Imaging

I don't like these methods for some reason, but here I show the simpler methods where netcat is not needed. For disk cloning, type something like this:

dd if=/dev/sda1 bs=16M | xz | ssh username@192.168.1.2 "xz -d | dd of=/dev/sda2 bs=16M"

Just to send an image file, run a command as follows:

dd if=/dev/sda1 bs=16M | xz | ssh username@192.168.1.2 "dd of=partimg.xz bs=16M"

Also Read:

• Linux: Using dd To Back Up Hard Drive Partitions.
  
• Backing up Hard Disk Partitions in Linux
  

Download Linux Kernel 3.5.4 / 3.6.0

I am sharing my kernel builds for Linux 3.5.4 and 3.6.0. Hope it works for everyone.

• linux360-586mmx.zip
  
• linux354-586mmx.zip
  

To Compile ndiswrapper 1.58rc1 for Linux 3.5.4

ndiswrapper allows Linux users to use Windows drivers for PCI/USB network adapters on Linux. If you own a network device for which Linux support is still absent or premature, then you need to compile ndiswrapper yourself and install Windows driver for your network device. Fortunately, today's Linux supports most network devices out-of-box, so most distributions don't bother to include ndiswrapper. However, I am compiling ndiswrapper believing that Windows NDIS drivers would outperform native Linux drivers.

To my surprise, building the latest version of ndiswrapper (1.58rc1) did not require any patch. After compiling Linux 3.5.4, I downloaded the ndiswrapper source from the sourceforge site. I chose the testing version because I thought it would work better with Linux 3.5.x. I unpacked the source.

tar xzvf ndiswrapper-1.58rc1.tar.gz
cd ndiswrapper-1.58rc1

I typed the following commands to compile and install ndiswrapper.

KVERS=3.5.4 make uninstall
KVERS=3.5.4 make
KVERS=3.5.4 make install

Then, I generated modules.* files again.

depmod -e -m -F /boot/System.map-3.5.4 3.5.4

The following files were installed by ndiswrapper.

/lib/modules/3.5.4/misc/ndiswrapper.ko
/sbin/loadndisdriver
/usr/sbin/ndiswrapper
/usr/sbin/ndiswrapper-buginfo
/usr/share/man/man8/loadndisdriver.8
/usr/share/man/man8/ndiswrapper.8

Also read:

• Linux: Installing Windows Vista/7 Wireless Driver for use with ndiswrapper
  
• Linux: To Compile wpa_supplicant on Debian 3.1 Sarge
  

My All-Purpose Init Script for initrd / initramfs Boot Images

Here I post my init script. It is used in my initrd/initramfs boot images for the following purposes.

• Boot from a live CD / DVD
  
• Boot from a read-only filesystem image, compressed in SquashFS
  
• Copy the filesystem image to a RAM disk and run Linux entirely on memory
  
• Boot Linux from a USB flash drive
  
• Boot Linux from a local disk partition
  
• Run a rescue shell without booting Linux
  

I have yet to implement network booting. Right now it is good enough for my current needs. It takes the following boot parameters.

• root=
  
  Specifies the root partition to boot Linux from. For example, /dev/sda2.
  
• label=
  
  Specifies the label of the device to boot from. For example, label=DEBIAN.
  
• uuid=
  
  Specifies the UUID of the device to boot from. For example, uuid=dcfd6a0a-2a0f-4b3d-8a1a-5e7d642ebfbd
  
• boot=
  
  Can be cdrom, loop, ram, usb or ata.
  
• vmode=
  
  Specifies the screen resolution of the framebuffer video. For example, vmode=640x480
  
• single
  
  Boot into the single-user mode.
  
• nox
  
  Boot into the console mode in runlevel 2
  

#!/bin/dash

# Define a function to parse kernel command line options.
get_opt() {
  echo $@ | cut -d "=" -f 2
}

# Define a function to load drivers.
loadmod() {
  for i in $@ ; do
    for j in $(grep $i /tmp/pcimodules.txt); do
      modprobe $j
    done
  done
}

# Define a function to guess the partition type.
gpart() {
  for i in $(blkid | grep $1); do
    case $i in
      *\=*)
        eval $i
        ;;
      *)
        true
        ;;
    esac
  done
}

# Define a function for mounting the root partition.
mountr() {
  if [ $uuid ]; then
    if [ $# = 2 ]; then
      mount -r -U $uuid $2
    elif [ $# = 1 ]; then
      mount -r -U $uuid $1
    else mount -r -U $uuid /mnt
    fi
  elif [ $label ]; then
    if [ $# = 2 ]; then
      mount -r -L $label $2
    elif [ $# = 1 ]; then
      mount -r -L $label $1
    else mount -r -L $label /mnt
    fi
  else
    gpart $1
    case $TYPE in
      ext*)
        e2fsck -p $1
        [ $# = 2 ] && mount $1 $2 || mount $1 /mnt
        ;;
      jfs)
        jfs_fsck $1
        if [ $# = 2 ]; then
          mount -t jfs -o ro,iocharset=utf8 $1 $2
        else mount -t jfs -o ro,iocharset=utf8 $1 /mnt
        fi
        ;;
      vfat)
        if [ $# = 2 ]; then
          mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 $2
        else mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 /mnt
        fi
        ;;
      *)
        [ $# = 2 ] && mount -r $1 $2 || mount -r $1 /mnt
        ;;
    esac
  fi
}

# Create a union filesystem
union() {
  mount -t tmpfs none /opt/tmp
  modinfo unionfs > /dev/null 2>&1 &&
  mount -t unionfs -o dirs=/opt/tmp=rw:/opt=ro none /mnt ||
  ( mkdir /opt/tmp/.change
    modprobe fuse
    unionfs-fuse -o allow_other,use_ino,suid,dev,nonempty,kernel_cache \
     -o cow,chroot=/opt,max_files=32768 /tmp/.change=RW:/=RO /mnt )
}

# Mount proc and sysfs.
mount -t proc none /proc
mount -t sysfs none /sys

# Find the available PCI hardware
mount -t tmpfs none /tmp
pcimodules > /tmp/pcimodules.txt

# Populate /dev (Needs kernel >= 2.6.32)
mount -t devtmpfs none /dev
mkdir -m 755 /dev/pts
mount -t devpts -o gid=5,mode=620 none /dev/pts

# Set default values
boot=ata
root=/dev/sda6

# Find the root=, label=, uuid= and boot= values on kernel command line.
for i in $(cat /proc/cmdline); do
  case $i in
    root\=*)
        root=$(get_opt $i)
        case $root in
          /dev/cdr* | /dev/dvd* | /dev/sr* | /dev/scd*)
            boot=cdrom
            ;;
          0x200)
            root=/dev/fd0
            ;;
        esac
        ;;
    label\=* | uuid\=* | boot\=* | vmode\=* )
        eval $i
        ;;
    single)
        RUNLEVEL=single
        ;;
    nox)
        RUNLEVEL=2
        ;;
  esac
done

# Activate framebuffer display devices.
if [ $vmode ]; then
  if [ $boot = cdrom ]; then
    modprobe uvesafb scroll=ywrap mode_option=$vmode-16
  else for i in $(grep fb /tmp/pcimodules.txt); do
    case $i in
      atyfb)
        modprobe $i mode=$vmode-16
        ;;
      nvidiafb | rivafb)
        modprobe nvidiafb mode_option=$vmode bpp=16 hwcur=1
        ;;
      radeonfb | savagefb)
        modprobe $i mode_option=$vmode-16
        ;;
      sisfb)
        modprobe $i mode=$vmodex16 mem=12288 font=SUN12x22
        ;;
      viafb | vt8623fb)
        modprobe viafb viafb_mode=$vmode viafb_bpp=16
        ;;
      *)
        modprobe $i
        ;;
    esac
  done
    if grep -q i915 /tmp/pcimodules.txt; then true
    else [ -c /dev/fb0 ] || modprobe uvesafb scroll=ywrap mode_option=$vmode-16
    fi
  fi
fi

case $boot in
    cdrom)
        # Boot Linux from a live CD.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sr_mod &&
        sleep 7
        modprobe isofs
        mount -t iso9660 /dev/sr0 /media
        [ -d /media/isolinux -o -d /media/boot/isolinux ] || 
          mount -t iso9660 /dev/sr1 /media
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          if [ $root = /dev/ram ]; then 
               echo "Please wait until the RAM disk is ready."
               dd if=$SQF of=/dev/ram1 bs=2048 &&
               mount -t squashfs /dev/ram1 /opt
          else modprobe loop
               mount -t squashfs -o loop $SQF /opt
          fi
        else
          mount --move /media /opt
        fi
        union
        ;;
    loop)
        # Boot Linux from an image file.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root /media
        modprobe loop
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          mount -t squashfs -o loop $SQF /opt
        elif [ -f /media/*.[Ii][Ss][Oo] ]; then
          ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
          modprobe isofs
          mount -t iso9660 -o loop $ISO /opt
        fi
        union
        ;;
    ram)
        # Boot Linux from ramdisk.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root /media
        echo "Please wait until the RAM disk is ready."
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          dd if=$SQF of=/dev/ram1 &&
          mount -t squashfs /dev/ram1 /opt
        elif [ -f /media/*.[Ii][Ss][Oo] ]; then
          ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
          dd if=$ISO of=/dev/ram1 bs=2048 &&
          modprobe isofs
          mount -t iso9660 /dev/ram1 /opt
        fi
        union
        ;;
    usb*)
        # Boot Linux from a USB drive.
        loadmod ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root
        ;;
    ata*)
        loadmod ata_ ahci pdc_adma &&
        modprobe sd_mod &&
        mountr $root
        ;;
esac

# Make sure that init exists and is executable.
if [ -x /mnt/sbin/init ]; then
  mount --move /dev /mnt/dev
  mount --move /proc /mnt/proc
  mount --move /sys /mnt/sys
  umount /tmp

# Start init from the root filesystem.
  cd /mnt
  [ -f /media/updates.zip ] && unzip -o /media/updates.zip
  case $boot in
    cdrom)
      [ $root = /dev/ram ] && umount /media
      [ $RUNLEVEL ] || RUNLEVEL=3
      ;;
    loop | ram)
      umount /media
      [ $RUNLEVEL ] || RUNLEVEL=3
      ;;
    *)
      [ $RUNLEVEL ] || RUNLEVEL=5
      ;;
  esac
  [ -d initrd ] && pivot_root . initrd
  exec chroot . /sbin/init $RUNLEVEL
fi

# Start a shell as a last resort.
echo "Error booting from the root filesystem. Starting a shell."
exec /bin/dash

The following are examples of boot parameters that can be used with my init script.

• Boot Linux from the local hard drive partition /dev/sda8
  
  

  boot=ata root=/dev/sda8

  
  
• Boot Linux from the latest squashfs file (*.sq*) on /dev/sda1
  
  

  boot=loop root=/dev/sda1

  
  
• Boot Linux from the CD-ROM with 1024x768 video resolution
  
  

  boot=cdrom vmode=1024x768

  
  
• Copy the squashfs image from CD-ROM into memory and run Linux on memory
  
  

  boot=cdrom root=/dev/ram ramdisk_size=573440 vmode=800x600

  
  
• Boot Linux from the second partition of a USB drive
  
  

  boot=usb root=/dev/sda2

  
  

Also read:

• Manually Creating initrd / initramfs to Boot Linux
  
• To Create a Live Linux CD/DVD
  
• Using UnionfsFuse on Debian/Ubuntu Linux
  

Using UnionfsFuse on Debian/Ubuntu Linux

Unionfs-fuse is a user-space implementation of unionfs that makes it possible to run Linux over a read-only medium, such as CD-ROM. Unionfs-fuse is a convenient alternative to kernel implementations (unionfs, aufs and overlayfs), so users don't have to patch the kernel in order to try unionfs. There are many possible uses of unionfs:

• Create a live CD
  
• Run Linux from a read-only filesystem image, such as a squashfs file
  
• Freeze an existing Linux system and save all changes in the memory
  

Unionfs achieves all this by merging a read-only filesystem and a writeable filesystem into a single virtual filesystem and mount it at a certain mount point. Let's try and apply unionfs-fuse to the third situation above. Basically, we need to create an initramfs file containing unionfs-fuse and a custom init script. The basic procedure is like this:

1. Load kernel drivers necessary to access the underlying storage device
   
2. Mount the read-only filesystem image at /opt
   
3. Mount tmpfs at /opt/tmp and create a directory /opt/tmp/.change
   
4. Use unionfs-fuse to merge the two and mount it at /mnt
   
5. Use chroot and start /sbin/init to boot the new unionfs filesystem
   

To carry about the above steps, an init script might look like this:

#!/bin/dash
# Use the small but functional dash to process this script

# Mount /proc, /sys, /dev and /dev/pts just in case
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev
mkdir -m 755 /dev/pts
mount -t devpts -o gid=5,mode=620 none /dev/pts

# Write commands here to load modules necessary to access a hard drive
modprobe pata_via
modprobe sd_mod

# Mount the Linux filesystem read-only at /opt
mount -r /dev/sda6 /opt

# Mount tmpfs at /opt/tmp
mount -t tmpfs none /opt/tmp
mkdir /opt/tmp/.change

# Create a unionfs mount at /mnt
modprobe fuse
unionfs-fuse -o allow_other,use_ino,suid,dev,nonempty,kernel_cache \
   -o cow,chroot=/opt,max_files=32768 /tmp/.change=RW:/=RO /mnt

# Make sure that init exists and is executable
if [ -x /mnt/sbin/init ]; then
  mount --move /dev /mnt/dev
  mount --move /proc /mnt/proc
  mount --move /sys /mnt/sys

# Start init from the root filesystem with runlevel 5.
  exec chroot /mnt /sbin/init 5
fi

The computer will boot into a virtual unionfs, consisting of writable tmpfs on top of read-only root filesystem. Make sure to modify /etc/rc.local so that it contains the following snippet of code.

for i in `ps ax | grep unionfs | grep -v grep | awk '{print $1}'`; do
  echo $i > /var/run/sendsigs.omit.d/unionfs.$i;
done

This prevents Linux from killing unionfs-fuse during shutdown so the system will properly shut down.

Related Posts

• Manually Creating initrd / initramfs to Boot Linux

To Compile UnionFS-fuse 0.26 on Debian Linux

I am trying to switch to unionfs-fuse for my live CD, but so far I haven't much success yet. Debian's unionfs-fuse package in Sid is outdated (version 0.24), so I compiled version 0.26. I had to install libfuse-dev in order to be able to build UnionFS-fuse. I probably needed cmake too, but I didn't choose to install cmake.

• cmake
  
• gcc
  
• libfuse-dev
  
• make
  

I edited Makefile to change PREFIX.

PREFIX=/usr
BINDIR=/bin
SBINDIR=/sbin

I just typed make to begin compilation.

make
make install

The following files are installed.

/usr/bin/unionfs
/usr/sbin/mount.unionfs
/usr/share/man/man8/unionfs-fuse.8

To make it compatible with Debian and derivatives, I renamed unionfs.

mv /usr/bin/unionfs /usr/bin/unionfs-fuse

To Compile Linux Kernel 3.5.4

It's been a long time since I compiled a Linux kernel last time. The last one I built was version 3.1.4 on December 3, 2011. I need to build a new kernel so that my Linux system is kept up-to-date. In order to compile a kernel, I made sure that all the necessary packages were installed.

• binutils
  
• gcc & g++
  
• libglade2-dev, libqt3-mt-dev or libqt4-dev
  
• make
  
• module-init-tools
  
• patch
  
• pkg-config
  

I downloaded the kernel source from kernel.org and unpacked the kernel source package.

tar xjf linux-3.5.4.tar.bz2
cd linux-3.5.4

I copied the config-* file of my current kernel to the kernel source tree and renamed it to .config. This makes kernel configuration easier because I only need to set the values for new kernel settings.

cp /boot/config-3.1.4 .config

If you can't find your current kernel config at /boot, you'll find a copy at /proc.

zcat /proc/config.gz > .config

Type one of the following commands to start configuring the kernel.

• If libglade2-dev is installed:
  
  

  make gconfig

  
  
• If libqt3-mt-dev or libqt4-dev is installed:
  
  

  make xconfig

  
  

When you finish configuring the kernel, save the changes by selecting File, Save. To see what changes you made, try the following command:

diff -ub .config.old .config

Optionally, open Makefile with an editor and define EXTRAVERSION if necessary.

VERSION = 3
PATCHLEVEL = 5
SUBLEVEL = 4
EXTRAVERSION = -586mmx

It is useful to set EXTRAVERSION when you are building different kernels for several CPU types. Now let's compile the kernel.

make bzImage
make modules
make install
make modules_install
depmod -e -m -F /boot/System.map-3.5.4 3.5.4

Set the default root device and ramdisk size for the new kernel.

cd /boot
rdev vmlinuz-3.5.4 1,0
ramsize vmlinuz-3.5.4 4096

Here's my config for kernel 3.5.4 if you need it. Make sure to change the processor family to suit your computer before building the kernel.

Also read:

• To Compile ndiswrapper 1.58rc1 for Linux 3.5.4
  

To Create a Live Linux CD/DVD

A live CD is a bootable CD that contains the whole operating system and any applications you want to run from the CD. With the increasing popularity of DVD and Blu-ray drives, there is even a live DVD. A live CD is useful when you want to copy your existing Linux system to a CD and run it on multiple computers. In this post, I show how I normally create a live CD.

Rebuild a kernel with SquashFS and UnionFS patch

A new kernel should be built with SquashFS and UnionFS patches. The live CD will have a huge file that is actually a SquashFS image containing a copy of your existing Linux system. We will use UnionFS so that any changes to the read-only SquashFS will be saved to the volatile memory. Download the patches from the following locations.

• squashfs.sourceforge.net
  
• unionfs.filesystems.org
  

After patching the kernel and rebuilding it, install the new kernel.

Make a SquashFS image of an existing Linux system

A SquashFS image needs to be created from an existing Linux system. Copy the contents of the Linux filesystem to an empty directory, assuming the filesystem is mounted at /mnt.

mkdir /tmp/livecd
cd /mnt
tar cf - . | (cd /tmp/livecd; tar xf -)

Empty the contents of fstab so that the live CD will not mount any hard disk partition, including the original root partition.

cp /dev/null /tmp/livecd/etc/fstab

Create a squashfs image.

mksquashfs /tmp/livecd /tmp/linuxfs.sqz -comp xz -b 262144 -Xdict-size 196608 -no-xattrs -e boot lost+found selinux srv

Create a Custom InitRamFS image

This is a tricky part of the job. Basically, the init script of the initramfs image will mount the CD-ROM, then mount the SquashFS image, and then mount a UnionFS. Refer to this post on building an initramfs image.

Create a Live CD structure.

Create an empty directory which will be the contents of live CD. The live CD will contain:

• SquashFS image (named linuxfs.sqz, for example)
  
• isolinux.bin and isolinux.cfg
  
• kernel
  
• initramfs image
  

Finally, create an ISO image for the live CD.

genisoimage -V LIVECD -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table /tmp/isofs > ~/livecd.iso

Burn the ISO image.

cdrskin -v dev=/dev/sr0 speed=8 driveropts=burnfree ~/livecd.iso

Manually Creating initrd / initramfs to Boot Linux

Initrd and initramfs files are used together with a Linux kernel and a bootloader (such as GRUB and syslinux) to start up Linux and boot it in many different ways. Many Linux distributions provide prepackaged kernels and convenient tools for creating initramfs, but I like to build my own kernel and initramfs because it allows me to customize the boot process. For instance, one can choose to boot from the local disk or network-mounted disk with the help of initramfs and some user-defined boot parameters. Here's a list of things that I'd like to achieve with my custom initramfs.

1. Boot from a live CD
   
2. Boot from a read-only filesystem image, compressed with SquashFS
   
3. Copy the filesystem image to a RAM disk and run Linux entirely on memory
   
4. Boot Linux from a USB flash drive or a Firewire disk
   
5. Boot Linux from a local disk partition
   
6. Boot Linux from a network drive
   
7. Run a rescue shell without booting Linux
   
8. Start a kdrive X-server and run gparted, partclone or partimage
   

In this post, I show how I normally create initrd / initramfs files. For this tutorial, the following Debian/Ubuntu packages are needed:

• busybox
  
  provides small essential utilities for booting and rescue shell.
  
• cpio
  
  is used to create the actual initramfs format.
  
• dash
  
  provides a minimalist shell for running the init script and the rescue shell.
  
• e2fsprogs, jfsutils, etc.
  
  provides fsck to check Linux filesystems and replay any stale journal
  
• lzma or xz-utils
  
  compresses the initramfs
  
• module-init-tools
  
  is used to load kernel modules necessary to activate hardware and mount the filesystem
  
• pciutils
  
  is used to detect PCI hardware
  
• unionfs-fuse
  
  makes it possible to use read-only Linux systems, such as live CD and filesystem images
  
• unzip
  
  is used to apply customized settings and changes to the unionfs boot mode
  
• v86d
  
  is used to set the screen resolution for the framebuffer screen
  

First, create a text file with a list of files to put in initramfs. The following is an example of such file:

bin/busybox
bin/dash
bin/mount
etc/filesystems
etc/fuse.conf
etc/group
etc/modprobe.d
lib/ld-linux.so.2
lib/libacl.so.1
lib/libattr.so.1
lib/libblkid.so.1
lib/libbz2.so.1.0
lib/libcom_err.so.2
lib/libc.so.6
lib/libdl.so.2
lib/libe2p.so.2
lib/libext2fs.so.2
lib/libfuse.so.2
lib/libkmod.so.2
lib/libmount.so.1
lib/libm.so.6
lib/libpci.so.3
lib/libpthread.so.0
lib/libresolv.so.2
lib/librt.so.1
lib/libselinux.so.1
lib/libsepol.so.1
lib/libuuid.so.1
lib/libx86.so.1
lib/libz.so.1
lib/modules/3.5.3/kernel/drivers/ata
lib/modules/3.5.3/kernel/drivers/block/loop.ko
lib/modules/3.5.3/kernel/drivers/cdrom/cdrom.ko
lib/modules/3.5.3/kernel/drivers/connector/cn.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-core.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-ohci.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-sbp2.ko
lib/modules/3.5.3/kernel/drivers/i2c/algos/i2c-algo-bit.ko
lib/modules/3.5.3/kernel/drivers/scsi/sd_mod.ko
lib/modules/3.5.3/kernel/drivers/scsi/sr_mod.ko
lib/modules/3.5.3/kernel/drivers/usb/host/ehci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/host/ohci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/host/uhci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/storage/usb-storage.ko
lib/modules/3.5.3/kernel/drivers/video
lib/modules/3.5.3/kernel/fs/ext3/ext3.ko
lib/modules/3.5.3/kernel/fs/ext4/ext4.ko
lib/modules/3.5.3/kernel/fs/fat/fat.ko
lib/modules/3.5.3/kernel/fs/fat/vfat.ko
lib/modules/3.5.3/kernel/fs/fuse/fuse.ko
lib/modules/3.5.3/kernel/fs/isofs/isofs.ko
lib/modules/3.5.3/kernel/fs/jbd2/jbd2.ko
lib/modules/3.5.3/kernel/fs/jbd/jbd.ko
lib/modules/3.5.3/kernel/fs/jfs/jfs.ko
lib/modules/3.5.3/kernel/fs/nls/nls_cp437.ko
lib/modules/3.5.3/kernel/fs/nls/nls_iso8859-1.ko
lib/modules/3.5.3/kernel/fs/nls/nls_utf8.ko
lib/modules/3.5.3/kernel/fs/reiserfs/reiserfs.ko
lib/modules/3.5.3/kernel/fs/squashfs/squashfs.ko
lib/modules/3.5.3/kernel/fs/xfs/xfs.ko
lib/modules/3.5.3/kernel/lib/crc16.ko
lib/modules/3.5.3/kernel/lib/crc-t10dif.ko
lib/modules/3.5.3/modules.dep
lib/modules/3.5.3/modules.dep.bin
lib/modules/3.5.3/modules.pcimap
sbin/blkid
sbin/e2fsck
sbin/jfs_fsck
sbin/modprobe
sbin/v86d
usr/bin/pcimodules
usr/bin/unionfs-fuse
usr/bin/unzip

Then, create an empty directory, for example, /tmp/initrd and copy the files listed in the above-mentioned text file (called rd354.txt) to the new directory.

mkdir /tmp/initrd
cd /
tar cvhf - -T rd354.txt | (cd /tmp/initrd; tar xf -)

Then, create the necessary directory structure under /tmp/initrd:

cd /tmp/initrd
mkdir -p dev media mnt proc root sys tmp

Since I chose to use busybox, I need to create symbolic links to busybox in /bin. Busybox provides incomplete functionality for modprobe, mount, sh and unzip, so I removed their symbolic links.

cd /tmp/initrd/bin
for f in $(./busybox --list); do [ -e $f ] || ln -s busybox $f ; done
rm modprobe unzip

Then, create essential device nodes in /tmp/initrd/dev:

cd /tmp/initrd/dev
MAKEDEV std fb0 fd0 sda sdb scd
mknod console c 5 1
mknod fuse c 10 229
chmod 666 fuse

Now, create an init script. The contents of this init script is crucial for customization of the boot process. The following is the script I use. This script is capable of booting a live CD and running Linux within memory in addition to booting Linux from hard drives and USB flash.

#!/bin/dash

# Define a function to parse kernel command line options.
get_opt() {
  echo $@ | cut -d "=" -f 2
}

# Define a function to load drivers.
loadmod() {
  for i in $@ ; do
    for j in $(grep $i /tmp/pcimodules.txt); do
      modprobe $j
    done
  done
}

# Define a function to guess the partition type.
gpart() {
  for i in $(blkid | grep $1); do
    case $i in
      *\=*)
        eval $i
        ;;
      *)
        true
        ;;
    esac
  done
}

# Define a function for mounting the root partition.
mountr() {
  if [ $uuid ]; then
    if [ $# = 2 ]; then
      mount -r -U $uuid $2
    elif [ $# = 1 ]; then
      mount -r -U $uuid $1
    else mount -r -U $uuid /mnt
    fi
  elif [ $label ]; then
    if [ $# = 2 ]; then
      mount -r -L $label $2
    elif [ $# = 1 ]; then
      mount -r -L $label $1
    else mount -r -L $label /mnt
    fi
  else
    gpart $1
    case $TYPE in
      ext*)
        e2fsck -p $1
        [ $# = 2 ] && mount $1 $2 || mount $1 /mnt
        ;;
      jfs)
        jfs_fsck $1
        if [ $# = 2 ]; then
          mount -t jfs -o ro,iocharset=utf8 $1 $2
        else mount -t jfs -o ro,iocharset=utf8 $1 /mnt
        fi
        ;;
      vfat)
        if [ $# = 2 ]; then
          mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 $2
        else mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 /mnt
        fi
        ;;
      *)
        [ $# = 2 ] && mount -r $1 $2 || mount -r $1 /mnt
        ;;
    esac
  fi
}

# Create a union filesystem
union() {
  mount -t tmpfs none /opt/tmp
  modinfo unionfs > /dev/null 2>&1 &&
  mount -t unionfs -o dirs=/opt/tmp=rw:/opt=ro none /mnt ||
  ( mkdir /opt/tmp/.change
    modprobe fuse
    unionfs-fuse -o allow_other,use_ino,suid,dev,nonempty,kernel_cache \
     -o cow,chroot=/opt,max_files=32768 /tmp/.change=RW:/=RO /mnt )
}

# Mount proc and sysfs.
mount -t proc none /proc
mount -t sysfs none /sys

# Find the available PCI hardware
mount -t tmpfs none /tmp
pcimodules > /tmp/pcimodules.txt

# Populate /dev (Needs kernel >= 2.6.32)
mount -t devtmpfs none /dev
mkdir -m 755 /dev/pts
mount -t devpts -o gid=5,mode=620 none /dev/pts

# Set default values
boot=ata
root=/dev/sda6

# Find the root=, label=, uuid= and boot= values on kernel command line.
for i in $(cat /proc/cmdline); do
  case $i in
    root\=*)
        root=$(get_opt $i)
        case $root in
          /dev/cdr* | /dev/dvd* | /dev/sr* | /dev/scd*)
            boot=cdrom
            ;;
          0x200)
            root=/dev/fd0
            ;;
        esac
        ;;
    label\=* | uuid\=* | boot\=* | vmode\=* )
        eval $i
        ;;
    single)
        RUNLEVEL=single
        ;;
    nox)
        RUNLEVEL=2
        ;;
  esac
done

# Activate framebuffer display devices.
if [ $vmode ]; then
  if [ $boot = cdrom ]; then
    modprobe uvesafb scroll=ywrap mode_option=$vmode-16
  else for i in $(grep fb /tmp/pcimodules.txt); do
    case $i in
      atyfb)
        modprobe $i mode=$vmode-16
        ;;
      nvidiafb | rivafb)
        modprobe nvidiafb mode_option=$vmode bpp=16 hwcur=1
        ;;
      radeonfb | savagefb)
        modprobe $i mode_option=$vmode-16
        ;;
      sisfb)
        modprobe $i mode=$vmodex16 mem=12288 font=SUN12x22
        ;;
      viafb | vt8623fb)
        modprobe viafb viafb_mode=$vmode viafb_bpp=16
        ;;
      *)
        modprobe $i
        ;;
    esac
  done
    if grep -q i915 /tmp/pcimodules.txt; then true
    else [ -c /dev/fb0 ] || modprobe uvesafb scroll=ywrap mode_option=$vmode-16
    fi
  fi
fi

case $boot in
    cdrom)
        # Boot Linux from a live CD.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sr_mod &&
        sleep 7
        modprobe isofs
        mount -t iso9660 /dev/sr0 /media
        [ -d /media/isolinux -o -d /media/boot/isolinux ] || 
          mount -t iso9660 /dev/sr1 /media
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          if [ $root = /dev/ram ]; then 
               echo "Please wait until the RAM disk is ready."
               dd if=$SQF of=/dev/ram1 bs=2048 &&
               mount -t squashfs /dev/ram1 /opt
          else modprobe loop
               mount -t squashfs -o loop $SQF /opt
          fi
        else
          mount --move /media /opt
        fi
        union
        ;;
    loop)
        # Boot Linux from an image file.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root /media
        modprobe loop
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          mount -t squashfs -o loop $SQF /opt
        elif [ -f /media/*.[Ii][Ss][Oo] ]; then
          ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
          modprobe isofs
          mount -t iso9660 -o loop $ISO /opt
        fi
        union
        ;;
    ram)
        # Boot Linux from ramdisk.
        loadmod ata_ ahci pdc_adma ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root /media
        echo "Please wait until the RAM disk is ready."
        if [ -f /media/*.[Ss][Qq]* ]; then
          SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
          dd if=$SQF of=/dev/ram1 &&
          mount -t squashfs /dev/ram1 /opt
        elif [ -f /media/*.[Ii][Ss][Oo] ]; then
          ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
          dd if=$ISO of=/dev/ram1 bs=2048 &&
          modprobe isofs
          mount -t iso9660 /dev/ram1 /opt
        fi
        union
        ;;
    usb*)
        # Boot Linux from a USB drive.
        loadmod ^.hci-hcd
        modprobe usb-storage &&
        modprobe sd_mod &&
        sleep 7
        mountr $root
        ;;
    ata*)
        loadmod ata_ ahci pdc_adma &&
        modprobe sd_mod &&
        mountr $root
        ;;
esac

# Make sure that init exists and is executable.
if [ -x /mnt/sbin/init ]; then
  mount --move /dev /mnt/dev
  mount --move /proc /mnt/proc
  mount --move /sys /mnt/sys
  umount /tmp

# Start init from the root filesystem.
  cd /mnt
  [ -f /media/updates.zip ] && unzip -o /media/updates.zip
  case $boot in
    cdrom)
      [ $root = /dev/ram ] && umount /media
      [ $RUNLEVEL ] || RUNLEVEL=3
      ;;
    loop | ram)
      umount /media
      [ $RUNLEVEL ] || RUNLEVEL=3
      ;;
    *)
      [ $RUNLEVEL ] || RUNLEVEL=5
      ;;
  esac
  [ -d initrd ] && pivot_root . initrd
  exec chroot . /sbin/init $RUNLEVEL
fi

# Start a shell as a last resort.
echo "Error booting from the root filesystem. Starting a shell."
exec /bin/dash

Copy the init script to the initrd folder (Assuming its filename is init.sh).

cp init.sh /tmp/initrd/init
cp init.sh /tmp/initrd/etc
chmod 775 /tmp/initrd/init

Now, use cpio and lzma to create an initramfs file. This assumes that the kernel was compiled with CONFIG_RD_LZMA option enabled.

cd /tmp/initrd
find . | cpio -H newc -o | lzma -c > ../initram.lzm

A file named initram.lzm will be created. If you want to create an old-style initrd file instead, use the mkcramfs command. This assumes your kernel has built-in cramfs support (CONFIG_CRAMFS):

mkcramfs /tmp/initrd /boot/initrd.bin

Copy the kernel and the initramfs file to your boot directory (whereever it is). I use SYSLINUX which I installed at /dev/sda1 partition. Finally, edit the boot configuration file. The following is the contents of a sample syslinux.cfg:

LABEL debian
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND root=/dev/sda7

LABEL sid
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=usb label=SID edd=off vmode=800x600

LABEL bfile
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=loop root=/dev/sda1

LABEL ram
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=ram ramdisk_size=524288 root=/dev/sda1

Cloud Storages for Linux

Today the trend is to put your valuable data somewhere on the Internet and access it anywhere. This is a dramatic departure from the past as our trust grows towards Internet companies. More companies are offering cloud storage services, for example, Box.net and Zumodrive. Here I show how to set up Linux to access cloud storages.

Box.net offers 5GB of free online storage for personal use, but each upload must be smaller than 25MB. Zumodrive offers 2GB free space initially, but unlike box.net you can upload any size of files using the ZumoDrive desktop application.

Adding a Box.net folder in KDE Dolphin

1. Select Network from the Places panel. Then, double-click on Add Network Folder.
   
   
   


2. Choose the WebFolder (webdav) and click Next.
   
   
   


3. Enter information for Box.net as follows.
   
   

Setting up Zumodrive on Mint KDE edition

1. Download the Zumodrive package for Ubuntu 8 or later (zumodrive-ubuntu8-i386-0.989.deb).
   
2. The Zumodrive package depends on openjdk-6-jre, but Sun's JRE is installed in Mint, so you don't have to install openjdk-6-jre.
   
   However, you need install libinotifytools0, libgnet2.0-0 and libnautilus-extention1. I used Synaptics to install the missing packages.
   
3. Unpack the zumodrive package into the system.
   
   

   cd /
   dpkg --extract zumodrive-ubuntu8-i386-0.989.deb .

   
   
4. Copy /usr/share/applications/zumodrive.desktop to your Desktop folder, so you can start Zumodrive by double-clicking the icon on desktop. Optionally, right-click on the icon, choose Properties and make it executable.



5. Now double-click on the Zumodrive icon. Once you type in your email and password, you can access Zumodrive through the ~/Zumodrive folder.
   

Compiling Linux Kernel 2.6.38

Let's compile a Linux kernel. To build a kernel, I installed the following packages:

• binutils
  
• gcc-4.4
  
• libglade2-dev for "make gconfig"
  
• libqt3-mt-dev or libqt4-dev for "make xconfig"
  
• make
  
• module-init-tools
  
• patch
  

libglade2-dev is needed to configure the kernel by running “make gconfig”. In addition, I am going to include the following features.

• unionfs is a union filesystem that allows me to merge several filesystems into a single logical one. It has several uses, such as Live CD's and modifiable read-only filesystem.
  
• ndiswrapper allows me to use many network devices by installing native Windows drivers.
  

I downloaded the kernel source and unpacked it.

tar xjvf linux-2.6.38.tar.bz2
cd linux-2.6.38

Then, I applied the unionfs patch.

gzip -dc ../unionfs-2.5.3_for_2.6.18.8.diff.gz | patch -p1 -l

Then, I configured the kernel with a GTK configuration tool.

cp /boot/config-2.6.31.5 .config
make gconfig

Open Makefile with an editor and define EXTRAVERSION if necessary.

Now let's compile the kernel.

make bzImage
make install
make modules
make modules_install
depmod -e -F /boot/System.map-2.6.38 -m 2.6.38

Set the default root device and ramdisk size for the new kernel.

cd /boot
rdev vmlinuz-2.6.38 1,0
ramsize vmlinuz-2.6.38 4096

Finally, compile ndiswrapper (Patches available here).

tar xzf ndiswrapper-1.55.tar.gz
cd ndiswrapper-1.55
KVERS=2.6.31.6 make uninstall
KVERS=2.6.31.6 make
KVERS=2.6.31.6 make install

Generate modules.* files.

depmod -e -m -F /boot/System.map-2.6.31.6 2.6.31.6

Building mplayer on Debian Linux

MPlayer is one of my favorite media players. It is used with such frontend as smplayer, gnome-mplayer or MPlayerGUI to provide a good alternative to Windows Media Player and VLC. First, I installed the necessary compiler toolchain.

• autoconf
  
• automake
  
• binutils
  
• bison
  
• cpp
  
• g77 or fort77
  
• gcc
  
• gettext
  
• g++
  
• libtool
  
• make

Next, I installed the necessary development libraries:

• libasound2-dev
  
• libbz2-dev
  
• libesd0-dev
  
• libfontconfig1-dev
  
• libfribidi-dev
  
• libjpeg62-dev
  
• libncurses5-dev
  
• libpng12-dev
  
• libxv-dev

Some useful libraries are not included in the Debian distribution, so I had to compile them on my own. They are faac, lame, libdca, live, x264 and xvidcore. First, I built liblzo.

tar xzvf /usr/src/lzo-2.04.tar.gz
cd lzo-2.04/
./configure --prefix /usr
make
make install

Then, I built the giflib.

tar xjvf /usr/src/giflib-4.1.6.tar.bz2
cd giflib-4.1.6/
ls
./configure --build=i586-pc-linux-gnu --prefix=/usr --disable-shared
make
make install

I built cdparanoia.

tar xzvf /usr/src/cdparanoia-III-10.2.src.tgz
cd cdparanoia-III-10.2/
./configure --build=i586-pc-linux-gnu --prefix=/usr --disable-shared
make
make install

I compiled libdca in the following way:

tar xjvf libdca-0.0.5.tar.bz2
cd libdca-0.0.5/
./configure --build=i586-pc-linux-gnu --prefix=/usr --disable-shared
make
make install

I built ogg, vorbis, theora and speex.

./configure --build=i586-pc-linux-gnu --prefix=/usr --disable-shared
make
make install

To compile lame, run:

tar xzvf /usr/src/lame-3.98.4.tar.gz

cd lame-3.98.4/

./configure --build=i586-pc-linux-gnu --prefix=/usr --disable-shared --mandir=/usr/share/man --enable-expopt=full --enable-nasm

make

make install

Unpack live555 source in /usr/lib and compile live555 as follows:

cd /usr/lib/live

./genMakefiles linux

make

To compile x264, do:

tar xjvf /usr/src/x264-snapshot-20110102-2245.tar.bz2

cd x264-snapshot-20110102-2245/

sh configure --host=i586-pc-linux-gnu --prefix=/usr --enable-pic --disable-asm

make

make install

Compile the xvid library as follows:

cd xvidcore/build/generic

./configure --build=i586-pc-linux-gnu --prefix=/usr

make

make install

Finally, you're ready to compile MPlayer. Get the latest snapshot tarball and extract it:

wget ftp://ftp.mplayerhq.hu/MPlayer/releases/mplayer-export-snapshot.tar.bz2

tar xjvf mplayer-export-snapshot.tar.bz2

cd mplayer-export-2009-05-29

CPPFLAGS='-DLIBTWOLAME_STATIC' ./configure --prefix=/usr --enable-runtime-cpudetection --yasm=/usr/bin/yasm --language=all --confdir=/etc/mplayer

make

make install

chmod 4755 /usr/bin/mplayer

ssh + netcat + tar + xz = Secure Network Transfer Link

I found a way to use simple command-line tools to transfer files between computers far apart. Using this method, I was able to duplicate the contents of an entire filesystem securely over a SSH tunnel between two computers. In this method, no NFS server or scp command is needed. However, netcat plays an important role in this method. Let's first make sure we have everything ready.

• OpenSSH
  
• netcat
  
• tar or cpio
  
• xz, lzma, lzo, bzip2 or gzip

I'll be really brief.

1. At the computer where you will receive files (say, 192.168.1.2), type the following commands to start netcat in listening mode and use tar + xz to unpack the incoming stream of data.

   
   
   

   cd /my/downdoad/folder
   nc -l 7749 | xz -dc | tar xvf -

   
   
   

2. At the computer where you will send files (say, 192.168.1.1), create a ssh tunnel to the computer receiving files (192.168.1.2).

   
   
   

   ssh -l username -L 5525:192.168.1.2:7749 192.168.1.2

   
   
   
3. Open another terminal window and type the following command to start sending files.
   
   
   

   tar cvf - . | xz -c | nc 127.0.0.1 5525

Partition Imaging with Netcat

Backing up hard drive partitions over the network can be accomplished with just a few simple tools like netcat. This is another handy usage of netcat. At the receiving computer where you'll store the backup, type the command to receive the partition image:

nc -l 7749 | lzma -dc | dd of=/dev/sda8 bs=640K

At the sending computer from which you'll transmit the partition, establish an SSH link first:

ssh -l username -L 5525:192.168.1.2:7749 192.168.1.2

Then compress the partition data and transmit over the secure SSH channel:

dd if=/dev/sda11 bs=640K | lzma -9c | nc 127.0.0.1 5525

Note that the transfer may take many hours for a large partition.

Debian Linux: Simple steps to recover files from /lost+found

The following steps illustrate how one can use simple shell commands to recover files from the /lost+found directory in Linux. This assumes that you actually have some files in /lost+found.

1. First, concatenate all the *.md5sums files in /var/lib/dpkg/info into a single file:

   
   

   cd /var/lib/dpkg/info
   cat *.md5sums | sort -k 2 > /tmp/all.md5

   
   
   

2. Go to the /lost+found directory. fsck may have placed some files here after fixing the Linux partition. If you don't find any file here, you can relax and skip the following steps. Run md5sum on files in /lost+found.

   
   

   cd /lost+found
   md5sum * | sort > /tmp/lost.md5

   
   
   

3. Put only the md5sum values into a temporary file, called 0.txt.

   
   

   awk '{print $1}' /tmp/lost.md5 > /tmp/0.txt

   
   
   

4. Search all.md5 for the md5sum values in 0.txt and save the results in 1.txt.

   
   

   for f in $(cat /tmp/0.txt); do grep $f all.md5 >> /tmp/1.txt; done

   
   
   

5. Put the names of files in /lost+found into a temporary fie 2.txt.

   
   

   awk '{print $2}' /tmp/lost.md5 > /tmp/2.txt

   
   
   

6. Move the lost+found files to their original locations.

   
   

   cd /
   for $f in $(cat /tmp/2.txt); do MD5=$(grep $f /dev/shm/lost.md5 | awk '{print $1}'); ORIGIN=$(grep $MD5 /tmp/1.txt | awk '{print $2}'); mv /lost+found/$f /$ORIGIN; done

   
   

Checking Integrity of A Debian/Ubuntu System

Sometimes, a Linux filesystem becomes corrupted, system files are damaged, or some crucial files get lost. This often happens, regardless of which filesystem (ext2, ext3, ext4, jfs, reiserfs, reiser4, or xfs) is used. There are many possible reasons, such as:

• Unstable hardware, for example, memory or hard drive problem
  
• Overheat, power surge, quake or another environmental disaster
  
• Buggy software, such as a bug in the kernel or the filesystem driver
  
• Compromised security, for example, network intrusion or attack
  
• Worm or virus infection

Files in Linux systems can be categorized into the following three:

1. Verifiable System Files
   
   In Linux systems that are managed by packages (such as Debian and Ubuntu), these files are installed by packages and make up the bulk of the filesystem. These files reside in such directories as /bin, /lib, /sbin and /usr. They are usually static, which means they don't normally change except when the system is updated, or locally compiled binaries are installed.
   
2. Changeable System Files
   
   These files are auxiliary system files for system configuration, initialization or customization, and system data (such as logs and cache). They reside in /boot, /etc, /opt, /srv and /var.
   
3. User Data
   
   These files are created and used by superuser (a.k.a root) and normal users, or software-generated during casual user activities. Typically, they are in /home, /media, /mnt and /root.
   

This post focuses on verifiable system files (installed by packages). When the filesystem becomes corrupted (but not completely unreadable), it is possible to verify and restore the system integrity by using package checksums. Before you continue, make sure to fsck the filesystem.

e2fsck -r -v /dev/sda7

In this example, /dev/sda7 points to an ext2 partition we're going to check. Be aware that you cannot fsck a mounted filesystem. Therefore, boot with a Debian Live CD (or a Ubuntu CD) and run fsck. After you've performed fsck, there may be some files created in the /lost+found directory. We'll deal with them later. First, mount the filesystem.

mount -t ext2 /dev/sda7 /mnt

Go to /var/lib/dpkg/info. Then, concatenate all the md5sums files. Most, if not all, Debian and Ubuntu packages come with a md5sum file that we can use to check the integrity of the package and the files installed by the package.

cd /var/lib/dpkg/info
cat *.md5sums | sort > /dev/shm/all.md5

all.md5 has md5 checksums of all the files installed on the system. Now, check the files on the Debian/Ubuntu system against the concatenated md5sums file.

cd /
md5sum -c /dev/shm/all.md5 > /dev/shm/check.txt 2>&1

/dev/shm/check.txt now contains the results of the integrity check. It looks like this:

bin/bash: OK
bin/bunzip2: OK
bin/bzcat: FAILED

In this example, /bin/bzcat is damaged. To find all the missing or damaged files, use a command like this one:

grep -v ': OK$' /dev/shm/check.txt

Let's reinstall this file. First, find out which package this file belongs to.

dpkg -S /bin/bzcat

We'll see the following result.

bzip2: /bin/bzcat

Now we know that we need to reinstall bzip2. Let's download the package.

dpkg -p bzip2 | grep 'Filename: '

This command will let us know the name of the package to download. Use wget to download it.

wget ftp://ftp.us.debian.org/debian/pool/main/b/bzip2/bzip2_1.0.5-4_i386.deb

You can just reinstall the package.

dpkg -i bzip2_1.0.5-4_i386.deb

Or, you can just extract one file:

dpkg --fsys-tarfile bzip2_1.0.5-4_i386.deb | tar xf - ./bin/bzcat

Alternatively,

dpkg --fsys-tarfile bzip2_1.0.5-4_i386.deb | tar xOf - ./bin/bzcat > /mnt/bin/bzcat

To restore a file from the /lost+found directory, you can also use the MD5SUMS file. First, run md5sum on files in /lost+found.

cd /lost+found
md5sum *

You may get an output like this.

9aaa2176d20c1b1203e3abbac55a2513  #124531

To find out what #124531 file is originally, find its md5 checksum from the all.md5 file above.

grep 9aaa /dev/shm/all.md5

You'll get a result like this.

9aaa2176d20c1b1203e3abbac55a2513  bin/bzip2

Now you can just move it to its place.

mv \#124531 /mnt/bin/bzip2

After you restore all damaged files and restore files from /lost+found, you can find missing files in the system. Go to /var/lib/dpkg/info again and concatenate all the list files.

cd /var/lib/dpkg/info
cat *.list | sort | uniq > /dev/shm/all.txt

The .list files in the /var/lib/dpkg/info directore show the list of files installed by packages. Let's find what's missing from the system.

cd /
for f in $(cat /dev/shm/all.txt ); do test -e "$f" || echo "$f" >> /dev/shm/nonexist.txt ; done

The file /dev/shm/nonexist.txt will show which files are missing from the system. You can then replace the missing files as done previously.

Linux: Using dd To Back Up Hard Drive Partitions

I am going to use the omnipresent and omnipotent tool called dd to back up a hard drive partition. I am working with the drive /dev/sdb. First, I save a text file that has information on the partition table layout.

fdisk -l /dev/sdb > hdpt.txt
fdisk -l -u /dev/sdb >> hdpt.txt

Then, I choose the compression format to use for the backup archive.

• gzip
  
• bzip2
  
• lzma
  
• xz

My choice for the compression format is lzma which provides superior compression and faster decompression. The following command backs up a partition at /dev/sdb1 with dd and lzma.

dd if=/dev/sdb1 | lzma -9c > backup01.bin.lzma

To restore this backup later, use the following command:

lzcat backup01.bin.lzma | dd of=/dev/sdb1

Linux Commands To Partition and Format a Drive

Partitioning

In addition to the wonderful gparted, we can also use fdisk to partition a disk:

fdisk /dev/sdb

The device names for hard disks and USB drives are typically /dev/sd?, for example, /dev/sda, /dev/sdb, /dev/sdc, etc. For Linux kernels 2.6.18 or older, IDE hard drives may be called /dev/hda, /dev/hdb, etc.

Formatting

To format a FAT16 partition:

mkdosfs -F 16 -n LABEL -r 512 -v /dev/sdb1

To format EXT2 partition:

mke2fs -L SID -v /dev/sdb2

To format a JFS partition:

jfs_mkfs -c -L Debian_Sid /dev/sdb2

Installing MBR

MBR is a boot code necessary for booting from the hard drive or USB flash.

install-mbr /dev/sdb -v --drive 0x80 --enable +12

Installing bootloaders

For SYSLINUX:

syslinux /dev/sdb1

Checking Filesystem Integrity

For FAT16/FAT32:

dosfsck -r -v -V /dev/sdb1

Use cdebootstrap to install Debian/Ubuntu Linux

Using the official CD from Debian or Ubuntu to install a fresh new system can be tedious if you just want a minimal or customized system. An alternative way to install Debian or Ubuntu is to use cdebootstrap. cdebootstrap is a simple command-line program that downloads packages from a Debian/Ubuntu archive, unpacks them into a mounted filesystem, and set 'em up. But it's not a full-fledged installer, so you need to take care of partitioning and setting up bootloader in order to complete the installation.

In order to use cdebootstrap, you need to have an already-running Linux system. However, any Linux live CD will suffice. Fortunately, I have created my own Debian Live CD with included cdebootstrap. I booted my Debian Live and opened mlterm.

If you don't have cdebootstrap, there are many ways to install it. Download a cdebootstrap-static package from http://packages.debian.org/sid/cdebootstrap-static that matches your architecture. If you are running an rpm-based system, such as Fedora, use ar, tar and gzip to unpack the package. For example:

wget http://ftp.us.debian.org/debian/pool/main/c/cdebootstrap/cdebootstrap-static_0.5.5_i386.deb

ar xv cdebootstrap-static_0.5.5_i386.deb

gzip -dc data.tar.gz > /tmp/data.tar

cd /; tar xvf /tmp/data.tar

Before running cdebootstrap, make sure that you have an empty partition and have mounted it. GParted can be used to create an empty partition. In the following example, I used cdebootstrap-static to install Ubuntu 6.06 Dapper into a filesystem mounted on /mnt.

cdebootstrap-static --allow-unauthenticated --flavour=minimal dapper /mnt http://archive.ubuntu.com/ubuntu

--allow-unauthenticated option is used in case ubuntu-archive-keyring or debian-archive-keyring is not installed. There are three flavours to choose from: minimal, standard and build. For suites other than dapper, you can choose one among etch, lenny, squeeze, sid, breezy, dapper, edgy, feisty, gutsy, hardy, intrepid, oldstable, stable, testing, unstable.. The URL at the end of command line is optional. For Debian, you can type http://archive.debian.org/debian.

cdebootstrap downloaded and installed the following set of packages into the new filesystem mounted at /mnt. This list is for a minimal flavour.

apt
base-files
base-passwd
bash
belocs-locales-bin
bsdutils
coreutils
debconf
debconf-i18n
debianutils
diff
dpkg
e2fslibs
e2fsprogs
findutils
gcc-4.0-base
gnupg
grep
gzip
hostname
initscripts
libacl1
libattr1
libblkid1
libbz2-1.0
libc6
libcap1
libcomerr2
libdb4.3
libgcc1
libgcrypt11
libgnutls12
libgpg-error0
libldap2
liblocale-gettext-perl
liblzo1
libncurses5
libopencdk8
libpam-foreground
libpam-modules
libpam-runtime
libpam0g
libreadline5
libsasl2
libsasl2-modules
libselinux1
libsepol1
libslang2
libss2
libssl0.9.8
libstdc++6
libtasn1-2
libtext-charwidth-perl
libtext-iconv-perl
libtext-wrapi18n-perl
libusb-0.1-4
libuuid1
locales
login
lsb-base
makedev
mawk
mount
ncurses-base
ncurses-bin
perl-base
python-minimal
python2.4-minimal
readline-common
sed
sysv-rc
sysvinit
tar
ubuntu-keyring
util-linux
zlib1g

In order to fine-tune the newly installed system, chroot into it and install additional packages.

chroot /mnt /bin/bash

To Build Gnome-MPlayer on Debian 5.0 Lenny

After compiling MPlayer on Debian 3.1 Sarge, I looked for a decent frontend to MPlayer. I found two frontends, SMplayer and Gnome-Mplayer. I chose to try Gnome-MPlayer because I am a GNOME person. However, Gnome-mplayer is not yet packaged for Debian 5.0 Lenny and previous releases. So I had to build Gnome-mplayer on my own. I downloaded the following packages in preparation for building Gnome-mplayer.

• gcc
  
• libasound2-dev
  
• libcurl4-gnutls-dev
  
• libdbus-glib-1-dev
  
• libdiscid0-dev
  
• libgpod-dev
  
• libgtk2.0-dev
  
• libmusicbrainz3-dev
  
• libneon26-gnutls-dev
  
• make

I downloaded Gnome-MPlayer source from the following location:

• http://code.google.com/p/gnome-mplayer/downloads/list

For Debian Lenny, download version 0.9.6. Versions higher than that requires latest GTK releases. I unpacked the Gnome-MPlayer source.

tar xzvf gnome-mplayer-0.9.6.tar.gz
cd gnome-mplayer-0.9.6

Then, I configured gnome-mplayer compilation:

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --with-libgpod --with-libmusicbrainz3

Then, I initiated compilation:

make
make install

The following files are installed on the system after compiling Gnome-mplayer:

/usr/bin/gnome-mplayer
/usr/share/locale/bg/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/de/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/el/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/en/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/es/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/fr/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/it/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/ja/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/ko/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/pl/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/pt_BR/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/ro/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/ru/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/sr/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/sr@latin/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/tr/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/zh_CN/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/zh_HK/LC_MESSAGES/gnome-mplayer.mo
/usr/share/locale/zh_TW/LC_MESSAGES/gnome-mplayer.mo
/usr/share/applications/gnome-mplayer.desktop
/usr/share/doc/gnome-mplayer/README
/usr/share/doc/gnome-mplayer/COPYING
/usr/share/doc/gnome-mplayer/AUTHORS
/usr/share/doc/gnome-mplayer/ChangeLog
/usr/share/doc/gnome-mplayer/INSTALL
/usr/share/doc/gnome-mplayer/NEWS
/usr/share/doc/gnome-mplayer/keyboard_shortcuts.txt
/usr/share/doc/gnome-mplayer/dbus.txt
/usr/share/doc/gnome-mplayer/plugin-interaction.txt
/usr/share/pixmaps/gnome-mplayer.png
/etc/gconf/schemas/gnome-mplayer.schemas