17 March 2013

361. Installing Debian on a USB stick (from a running Debian system)

Post 70 (Installing Debian on a USB stick -- live usb vs a true and full installation) is receiving a lot of traffic. While the method of installing debian used there -- using virtualbox -- works, it is slow and unnecessarily complex.

A better, faster and easier way is to use a chroot. Sure, it's a bit more command line oriented, but that doesn't necessarily make it harder.
You'll need an empty USB stick. Some sticks are faster than others, but they are all slower than spinning disks, and a lot slower than SSDs (never tried a USB3 stick though).

1. Prepare the USB stick
Attach the USB stick to a computer with Debian. Find out what device it is (do ls /dev/sd* before and after attaching the stick, and if it automounts, you can also check the output of df -h). In this case we'll assume that it's /dev/sdb.


sudo apt-get install util-linux e2fsprogs
sudo umount /dev/sdb*
sudo fdisk /dev/sdb
Command (m for help): o
Building a new DOS disklabel with disk identifier 0x209d6329.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdb: 4051 MB, 4051697664 bytes
125 heads, 62 sectors/track, 1021 cylinders, total 7913472 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x209d6329

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): 
Using default response p
Partition number (1-4, default 1): 
Using default value 1
First sector (2048-7913471, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-7913471, default 7913471): 
Using default value 7913471

Command (m for help): a                                                                               
Partition number (1-4): 1

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

It really is that simple to set up the partition table. o wipes the previous table, n creates a new partition, and a makes it bootable. w writes the changes. Now prepare the file system:

sudo mkfs.ext4 /dev/sdb1
mke2fs 1.42.5 (29-Jul-2012) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 247504 inodes, 988928 blocks 49446 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1015021568 31 block groups 32768 blocks per group, 32768 fragments per group 7984 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done
2. Mount and bootstrap the USB stick

Note that you can change between i386 and amd64, testing and stable and different mirrors, by editing the debootstrap line below. I'd recommend i386 since it will run on 64 bit machines as well, which is useful if you're traveling.

sudo apt-get install debootstrap coreutils
mkdir ~/tmp/usbstick -p
sudo mount /dev/sdb1 ~/tmp/usbstick
sudo debootstrap --arch i386 testing $HOME/tmp/usbstick http://ftp.au.debian.org/debian/
I: Retrieving InRelease I: Checking Release signature I: Valid Release signature (key id 9FED2BCBDCD29CDF762678CBAED4B06F473041FA) I: Retrieving Packages [..] I: Configuring tasksel... I: Configuring tasksel-data... I: Base system installed successfully.

sudo mount -t proc none $HOME/tmp/usbstick/proc
sudo mount --bind /sys $HOME/tmp/usbstick/sys
sudo mount --bind /dev $HOME/tmp/usbstick/dev
sudo cp /etc/resolv.conf $HOME/tmp/usbstick/resolv.conf
sudo chroot $HOME/tmp/usbstick/

3. Basic setup

Edit the sources.list line as needed. The key is to enable non-free so you can install all the non-free firmware so that youre prepared for most types of hardware.

rm /etc/apt/sources.list
echo 'deb  http://ftp.au.debian.org/debian/ testing main contrib non-free' >> /etc/apt/sources.list
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 usbstick >> /etc/hosts'
source /etc/bash.bashrc
useradd -m traveller
passwd traveller
usermod -G sudo -a traveller

If you're going to travel with this, you may want to install AIDE to make sure that you're executables haven't been changed (you can also use tripwire). Of course, if anyone has the ability to regenerate the database, then this won't help you.
apt-get install aide

You will have to wait with generating the database until you're properly booted from the USB stick.

4. Make the usbstick bootable
Make sure to install grub to /dev/sdb so that you don't mess up your desktop.

grep -v rootfs /proc/mounts > /etc/mtab
sudo apt-get install grub-pc linux-base linux-image-3.2.0-4-686-pae


You need to set up your /etc/fstab:
blkid /dev/sdb1 -o export|head -n 1 > /etc/fstab

Edit /etc/fstab (your UUID will vary):
UUID=24056f3d-d0c5-4297-adc4-40b2d2007412       /       ext4    errors=remount-ro,user_xattr    0 1

Run
update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-3.2.0-4-686-pae Found initrd image: /boot/initrd.img-3.2.0-4-686-pae done

At this point you've used ca 559 Mb.

5. Installing command line tools
The following installs a curses-based network connection manager, a mail program, a chat program and a browser. All of them work in the terminal:
sudo apt-get install wicd-curses mutt mcabber elinks rcconf

You've now used ca 670 Mb.

To set up Mutt and Mcabber, see here: http://verahill.blogspot.com.au/2011/12/configuring-mutt-for-two-imap-accounts.html
http://verahill.blogspot.com.au/2011/12/linux-basics-google-talk-using-gajim-or.html

6. Installing a desktop
This is luckily very easy. The bad thing is that USB sticks are very slow, so starting a heavy desktop from a USB stick is going to lead to long boot times.

To install gnome (2340 Mb) do
sudo apt-get install gnome rcconf

To install lxde (523 Mb) do
sudo apt-get install lxde rcconf

and to install xfce (457 Mb) do
sudo apt-get install xfce4 rcconf

We installed rcconf to prevent network-manager to start if it has been installed:
Run
rcconf
And unstar whatever you don't want to launch at boot.
To reduce boot times, edit /etc/default/grub and change to
GRUB_CMDLINE_LINUX_DEFAULT="quiet text"
Run
update-grub

You will now have to start whatever desktop you installed manually by running
startx

from the command line i.e. you only start the desktop if you really need it.

Clean your cache to free up some space:

apt-get clean

With xfce4 and cleaning, you've now used a total of 951 Mb.

7. Pull in some extra firmware
In case you'll be using this usb stick on random computers you might want to pull in as many different firmware as you can stomach:
apt-get install firmware-atheros firmware-iwlwifi firmware-ralink firmware-realtek

And so on.
You might also want to install programs such as clamav , but that's your business.

8. Unmounting your usbstick
Once you've installed everything you need:
exit
sudo umount $HOME/tmp/usbstick/sys
sudo umount $HOME/tmp/usbstick/dev
sudo umount $HOME/tmp/usbstick/proc
sudo umount $HOME/tmp/usbstick/

Now you should be able to boot from the USB stick on any computer that supports boot from USB -- and that would be most boxes made in the past five years or so.

Done.


Note that you can generate initramfs manually in debian using
sudo mkinitramfs -o /boot/initrd.img-VER -v VER

where VER e.g. 3.8.0 or 3.2.0-4-amd64 -- the name should be consistent with the vmlinuz, config and system.map suffixes.

Links to this post:
http://atomowki.net/dokuwiki/doku.php
http://andrewgudgel.com/cl-debian-usb.htm

16 March 2013

360. Screenshots from a desktop based Scientific Linux installation

This is a pretty basic post, but linux users span a wide range of skill levels.

I was originally planning to do post 359 via a chroot in Scientific Linux, but decided that doing it via Debian was..well..more convenient (the approach can easily be translated to any distro which has yum, and that includes e.g. Arch, in addition to the obvious suspects)

Anyway, as part of all that I decided to grab some screenshots of the installation process, and since I have them I might as well share them. I did this in virtualbox since I was originally planning on testing out the whole thing there before going to physical hardware.

So, here's Scientific Linux. The easiest approach is just to get whole dvd:
wget http://ftp1.scientificlinux.org/linux/scientific/6.3/x86_64/iso/SL-63-x86_64-2012-08-02-Install-DVD.iso

Launch the installation:






















 



Once the installation finishes, restart/










 
Click on the network icon in the top right corner, and then click on the desired network interface.
Open a terminal and do yum upgrade.






15 March 2013

359. Installing Scientific Linux (CentOS, RHEL) via a chroot in a running debian installation

This is a fun post. I've got a small server and I'd like to put Scientific Linux on it. I figure that between using Arch on my laptop, and Debian on my beowulf cluster, my work desktop, my home desktop and on another two laptops what I really need is to become a bit more proficient in the use of RHEL clones.

I've settled on Scientific Linux since I encounter that more often than CentOS.

My mini server lacks any form of video interface though, so the easiest solution is to take out the harddrive and install Scientific Linux via a USB tether and a chroot, similar to how things are done when making standard installs of Arch or Gentoo (it's all coming together...)

In a future post I'll write up how to install debian via a running debian system and a chroot (hint: debootstrap).

Anyway.

1. Install yum on your debian system
sudo apt-get install yum

2. Set up the Scientific Linux repos for yum on your debian system
Because it's easier, we'll disable gpg check.

If you're target is a 32 bit box, change x86_64 to i386 (my little server runs a 32 bit atom cpu and I screwed this up once)

Create /etc/yum/repo.d/sl.repo
[sl] name=Scientific Linux 6.3 - x86_64 baseurl=http://ftp.scientificlinux.org/linux/scientific/6.3/x86_64/os/ http://ftp1.scientificlinux.org/linux/scientific/6.3/x86_64/os/ http://ftp2.scientificlinux.org/linux/scientific/6.3/x86_64/os/ ftp://ftp.scientificlinux.org/linux/scientific/6.3/x86_64/os/ enabled=1 gpgcheck=0 [sl-security] name=Scientific Linux 6.3 - x86_64 - security updates baseurl=http://ftp.scientificlinux.org/linux/scientific/6.3/x86_64/updates/security/ http://ftp1.scientificlinux.org/linux/scientific/6.3/x86_64/updates/security/ http://ftp2.scientificlinux.org/linux/scientific/6.3/x86_64/updates/security/ ftp://ftp.scientificlinux.org/linux/scientific/6.3/x86_64/updates/security/ enabled=1 gpgcheck=0
Test that everything is working by running

yum repolist
sl | 3.2 kB 00:00 sl-security | 1.9 kB 00:00 repo id repo name status sl Scientific Linux 6.3 - x86_64 enabled: 6,399 sl-security Scientific Linux 6.3 - x86_64 - security updates enabled: 1,584 repolist: 7,983
It might take 60 s before you get any feedback, so don't interrupt the command.

3. Mount your external harddrive
mkdir $HOME/tmp/jail -p
sudo mount /dev/sdb1 $HOME/tmp/jail

assuming of course that /dev/sdb1 is your intended root partition (see the arch and gentoo install guides for examples of how to use fdisk)

4. Install a basic system on your mounted harddrive
sudo yum --installroot=$HOME/tmp/jail install bash sl-release coreutils yum iputils vim

5. Chroot your mounted harddrive
sudo cp /etc/resolv.conf $HOME/tmp/jail/etc/resolv.conf
sudo cp /etc/yum/repos.d/* /$HOME/tmp/jail/etc/yum.repos.d/
sudo mount -t proc none $HOME/jail/proc
sudo mount --bind /sys $HOME/tmp/jail/sys
sudo mount --bind /dev $HOME/tmp/jail/dev
sudo chroot $HOME/tmp/jail
bash-4.1# head -n 1 /etc/issue.net Scientific Linux release 6.3 (Carbon) bash-4.1# ping -c 1 google.com PING google.com (74.125.237.98) 56(84) bytes of data. 64 bytes from syd01s12-in-f2.1e100.net (74.125.237.98): icmp_seq=1 ttl=51 time=16.0 ms
We are now working in Scientific Linux. Sure, we're still using the debian kernel, but everything else is from the SL repos:

First set up /etc/fstab. I normally do
blkid > /etc/fstab

and then edit it. In this case I ended up with:
UUID=45495a62-a141-4033-a739-8063f0e80a56 / ext4 errors=remount-ro,user_xattr 0 1 UUID=6dc86d8d-0ad6-4574-8dea-9d28b0151362 none swap sw 0 0
Install everything we need to boot our new system and make it run:
yum install kernel openssh-clients openssh-server passwd dhclient


Important:
passwd

If you don't set a password you can't log in.


A. Grub legacy -- I couldn't make this work; see B
yum install grub 

Make the system bootable (the first command is so that df works)
grep -v rootfs /proc/mounts > /etc/mtab
grub-install --no-floppy /dev/sdb
grub-install --no-floppy /dev/sdb Installation finished. No error reported. This is the contents of the device map /boot/grub/device.map. Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'. (fd0) /dev/fd0 (hd0) /dev/sda (hd1) /dev/sdb
The last command took a LONG time without any visual feedback until the very end.

Create a /boot/grub/grub.conf file:
default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz title Scientific Linux (2.6.32-358.2.1.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-358.2.1.el6.x86_64 ro root=UUID=45495a62-a141-4033-a739-8063f0e80a56 initrd /initramfs-2.6.32-358.2.1.el6.x86_64.img

B. Grub2 -- this worked perfectly for me
Grub and I don't play well together. I'm more at home with grub2 which..well..allows for a bit of stupidity. After spending a fair amount of time trying to figure out grub (legacy) I gave up and did the following:

yum install wget diffutils bison gcc gcc-c++ kernel-devel flex freetype-devel make fuse-libs fuse-devel 
cd /tmp
wget ftp://ftp.gnu.org/gnu/grub/grub-2.00.tar.gz
tar xvf grub-2.00.tar.gz
cd grub-2.00/
./configure
make
make install
grub-mkconfig -o /boot/grub/grub.cfg
yum erase grub
grub-install /dev/sdb


Set up the following files:

/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=science!
/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
If you want static inet instead, do
DEVICE=eth0
HWADDR=D4:AE:52:71:79:14
IPADDR=10.1.1.1
NETMASK=255.255.0.0
BOOTPROTO=none
ONBOOT=yes
MTU=1500
TYPE=Ethernet
You're now ready to exit the chroot, unmount everything and reboot. Things to do after the first reboot include setting up a user (useradd -m verahill) and install sudo.
Anyway, there are plenty of guides online for that...

I tried this on physical hardware, and it works  Since the box is headless I only had ssh access, setting up proper rules in /etc/udev/rules.d/ was important so that I could configure the network via /etc/sysconfig/network-scripts files before putting the drive back in the server.

/etc/udev/rules.d/70-persistent-net.rules
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0e:b6:2a:a9:30", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0e:b6:2a:a9:31", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

This is a headless box (no graphics card, no vga output -- only a serial port), but I wanted to be able to use for a student to use gnuplot etc. in a pinch.

sudo yum install xorg-x11-server-Xvfb xorg-x11-server-Xorg

To set up keyword-less key-based ssh log in, edit /etc/ssh/sshd_config and uncomment
AuthorizedKeysFile .ssh/authorized_keys
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 644 ~/.ssh/authorized_keys

Done.