Showing posts with label installing linux on a usb stick. Show all posts
Showing posts with label installing linux on a usb stick. Show all posts

03 February 2013

331. A full linux install on a USB stick: Arch and LXDE on a thumbdrive

Update 9 June 2013: I've updated the torrent URL for the current image. Note that you will likely have to do this every month or so. See for the latest image.

Original post:
Post 330 (How to install Arch linux) got me thinking: one of my most popular posts is post 70 ("Installing Debian on a USB stick -- live usb vs a true and full installation").

I'm new to Arch and LXDE and Openbox -- I normally run Debian with Gnome 3 -- so I'm probably not doing this the best way. But if you're also new to Arch and struggling with the beginner's wiki you may find some ideas here.

I admit: this post and post 330 aren't that different. Here I basically refine what I did in post 330, and adapt it for desktop use. I also don't have to worry about networking here since we're not talking about a headless box anymore, but an interactive system. There'll most likely be a third post at some point which will be a streamlined version of this post. Anyway.

Whatever I do here will obviously also apply to a desktop installation of Arch -- i.e. if you're struggling with LXDE/openbox on Arch, see here.

 Debian is fine and dandy, but the approach I describe there involves using a virtualbox to install debian on a thumbdrive.

In post 330 I learned how to use a chroot to set things up, which speeds things up significantly (you could of course just boot from a live CD etc., but that's too easy).

 Also, while I'm very happy with debian and it gives pretty small installs if done correctly, Arch might just give even leaner installs.[citation needed]

So here's how to install Arch linux and a graphical DE (LXDE -- I've never warmed to XCE for some reason).

We'll do 32 bit Arch here -- a reason to make a bootable linux drive is so that you can travel with your own safe desktop, and 32 bit linux will work on almost all hardware you're likely to encounter, while 64 bit linux is limited to more modern hardware. Note that if you really want to be secure you will need to encrypt the entire USB stick with e.g. truecrypt -- otherwise in particular your gpg/pgp keys are at risk. Ultimately, security is a matter of vigilance and risk reduction.

You do need a running linux machine  (or at least a live disk) -- I don't know how to do this from Windows.

1. Formatting the thumbdrive
2. Chrooting the drive and installing stuff with pacstrap
3. Arch-chroot to install the boot loader, setting up internet, and installing a desktop

Formatting the thumbdrive
Plug in your drive to your running linux machine
df -h
rootfs 28G 21G 5.7G 79% / udev 10M 0 10M 0% /dev tmpfs 397M 852K 396M 1% /run /dev/disk/by-uuid/d7479df8-0d84-4236-a92c-b05327b8fa3d 28G 21G 5.7G 79% / tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 2.3G 728K 2.3G 1% /run/shm /dev/sda5 112G 82G 25G 78% /home /dev/sdb2 937M 18M 872M 2% /media/ext /dev/sdb1 1014M 4.5M 1009M 1% /media/win32
sudo umount /dev/sdb1 /dev/sdb2

Time to start fdisk. First we delete any existing partitions (and data...), then we make one partition, and then we'll make it bootable. We won't bother with a swap disk -- we don't have the space and we want to keep disk i/o to a minimum.

sudo fdisk /dev/sdb
Command (m for help): p Disk /dev/sdb: 2063 MB, 2063597056 bytes 64 heads, 62 sectors/track, 1015 cylinders, total 4030463 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000cf539 Device Boot Start End Blocks Id System /dev/sdb1 62 2079231 1039585 e W95 FAT16 (LBA) /dev/sdb2 2079232 4027519 974144 83 Linux Command (m for help): d Partition number (1-4): 1 Command (m for help): d Selected partition 2 Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-4030462, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-4030462, default 4030462): Using default value 4030462 Command (m for help): a Partition number (1-4): 1 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
Create the filesystem:
sudo mkfs.ext4 /dev/sdb1
mke2fs 1.42.5 (29-Jul-2012) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 125952 inodes, 503551 blocks 25177 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=515899392 16 block groups 32768 blocks per group, 32768 fragments per group 7872 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Allocating group tables: done Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done
sudo fsck /dev/sdb1
fsck from util-linux 2.20.1 e2fsck 1.42.5 (29-Jul-2012) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/sdb1: 11/125952 files (0.0% non-contiguous), 16846/503551 blocks
Make sure to verify you filesystem.

Get the arch iso and set up your chroot:
Note that you will most likely have to update the torrent URL every month or so. Obviously you'll need to change the mount command as well according to the iso name.
sudo apt-get install bittorrent squashfs-tools
cd ~/Downloads/
sudo mount -o loop archlinux-2013.06.01-dual.iso /mnt
unsquashfs -d arch /mnt/arch/i686/root-image.fs.sfs

If you want x86_64 do unsquashfs -d arch/ /mnt/arch/x86_64/root-image.fs.sfs instead

Time to get busy:
sudo umount /mnt
sudo mount -o loop arch/root-image.fs /mnt
sudo cp /etc/resolv.conf /mnt/etc/resolv.conf
sudo mount -o bind /proc /mnt/proc
sudo mount -o bind /sys /mnt/sys
sudo mount -o bind /dev /mnt/dev
sudo mount -o bind /dev/pts /mnt/dev/pts
sudo chroot /mnt
[root@lithium /]# 

The first chroot:
mount /dev/sdb1 /mnt
mkdir /run/shm
haveged -w 1024
pacman-key --init
pacman-key --populate archlinux

If you're installing i686 (i.e. 32 bit) linux from an x86_64 (i.e. 64 bit) linux, edit /etc/pacman.conf and set Architecture:
Architecture = i686
pacstrap /mnt base vim grub-bios openssh wicd lxde-common lxsession desktop-file-utils openbox upower
(196/196) installing openbox [#############] 100% Place menu.xml, rc.xml and autostart in ~/.config/openbox They can be found in /etc/xdg/openbox Optional dependencies for openbox pyxdg: for the xdg-autostart script
This will download a large number of packages (196 packages; ca 178 M download, ca 750 Mb installed), and will take a while. Make sure that all packages end with the architecture you want (e.g. -i686 or -x86_64) or '-any'. I've omitted base-devel above, since you might not need it.

Generate a proto-fstab:
genfstab -p /mnt >> /mnt/etc/fstab

and edit it (/mnt/etc/fstab) to read something like (the uuid will differ):
UUID=c540eff0-0e10-4e7a-9fb6-190a46f38203 / ext4 rw,relatime,data=ordered 0 1 /dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0 /dev/sdb1 /media/usb0 auto rw,user,noauto 0 0
The last two lines will automount CDs and usb sticks. It's not necessarily something you do want to allow -- so you may comment them out.

The second chroot:
arch-chroot /mnt

edit /etc/pacman.conf and change Architecture=auto to i686.

pacman -Syy
ln -s /usr/share/zoneinfo/Australia/Melbourne /etc/localtime
echo "kookaburra" > /etc/hostname

Edit /etc/locale.gen and uncomment the locales you want e.g.
en_GB.UTF-8 UTF-8
echo 'LANG="en_GB.UTF-8"'>/etc/locale
echo 'KEYMAP=us'> /etc/vconsole.conf
mkinitcpio -p linux
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sdb

Don't screw up that last step! If you put the wrong device the arch install won't boot, and most likely not the system you chrooted it from either. i.e. don't accidentally type /dev/sda (or sdb is you have two disks -- it's not difficult to repair, but you'll need a live CD already prepared)

Continuing to set up LXDE/Openbox and network:

pacman -S ifplugd xorg-xinit xorg-server xorg-utils xorg-server-utils

I've only got nvidia boxes at home, but if you're travelling you're likely to run into intel and ati as well. You can probably skip the last package since we're pulling in nouveau

pacman -S xf86-video-nouveau xf86-video-ati xf86-video-intel xf86-video-nv nvidia

At this point startx would get you a black screen with a mouse cursor. Right-clicking on the background yields a menu, but you don't have anything installed.

There are a couple of services we want to run on boot:
systemctl enable wicd
systemctl enable sshd

Edit /etc/wicd/manager-settings.conf;change wired_interface to whatever you're using in udev.

The pc beep on tab is annoying me:
echo "blacklist pcspkr" > /etc/modprobe.d/nobeeep.conf

Create a user:
pacman -S sudo
useradd verahill -m
echo "verahill ALL=(ALL) ALL">> /etc/sudoers
su verahill
cd ~

mkdir ~/.config/openbox -p
cp /etc/xdg/openbox/*.xml ~/.config/openbox
cp /etc/xdg/openbox/* ~/.config/openbox
echo "exec startlxde" > ~/.xinitrc
echo "export DESKTOP_SESSION=LXDE">> ~/.bashrc

At this point we have a setup which supports both wired and wireless internet (depending on drivers) and which has a very rudimentary desktop. We're now using 1.4 Gb, out of which 231 M is the package cache.

The range of programs that you may want to install is probably fairly personal, and will also depend on the size of your USB stick. Here's a basic selection which fits me (install as root, or try sudo, although the latter option works best if you actually boot from your USB stick rather than chroot it):

pacman -S truecrypt firefox chromium thunderbird flashplugin

(when asked, ttf-bitstream-vera is fine)

Miscellaneous packages:
pacman -S conky guake leafpad gparted elinks mcabber mutt gajim screen mlocate xterm lxpanel menumaker obmenu obconf alsa-utils volwheel pulseaudio-alsa

To improve the functionality of screen and vim, see here and here. Setting up Mutt and Mcabber is described here and here. Conky is here (item 4).

To sort out sound you might need to create ~/.asoundrc and fill it with:

pcm.!default.type pulse
ctl.!default.type pulse

(Not everyone is a fan of pulseaudio though)

And see this post for a bit more on encrypting communication and files (I travel in China so these things do matter)):

Some basic science packages that let's me do basic work on the road:
pacman -S octave gnuplot maxima gummi texlive-science gimp

If you pull in all those packages you'll need a pretty big USB drive though (albeit you actually have a small build environment installed) -- it comes in at about 3.4 Gb. Skip the science stuff to save space.

And I think that's that. Time to safely umount:
umount /mnt
sudo umount /mnt/proc
sudo umount /mnt/sys
sudo umount /mnt/dev/pts
sudo umount /mnt/dev
sudo umount /mnt

To boot:
Plug in your USB stick and hit whatever key or key-combination during BIOS start up is necessary to interrupt boot and allow you to select your boot device. Sometimes it's F9, F2, F12 or del.

On boot:
Your first task  will be to get your internet up and running. Assuming that you haven't already played with /etc/udev/rules.d you need to find out what the interfaces are called. Do ifconfig -a, note the names, and then start wicd-curses. Select Preferences and enter the name of e.g. the wired interface.

Then start your desktop by running startx. There'll be plenty of menu options, but most programs will be missing, but that's what your ~/.config/openbox/menu.xml is for. And that's how far I've gotten. Time to explore.

Potential issues:
It doesn't always work on first try -- for some reason. I had to start from scratch twice before everything worked without a hitch. Often it's easier starting from scratch than trying to fix a basic installation.

You might not be able to run the systemctl commands until you've actually booted from the usb stick.

If you're on a laptop and the mouse/touchpad isn't working, install xf86-input-synaptics.

Consider installing wicd-gtk if you'll be working a desktop environment a lot.

20 February 2012

70. Installing Debian on a USB stick -- live usb vs a true and full installation

Update 17 March 2013: See this post for a faster, better way of creating a full install if you're already running Debian.

Original post:
Every now and again I get posts like this one, or this one, via google news. While those posts --describing the use of unetbootin to create a 'live' USB analogue to a live CD -- may be technically correct, there's just so much more you can do.

In 1  we do the old boring bog-standard 'write live cd to a usb' thingy that you see pasted all over the web, but using cat instead of unetbootin.
In 2  we make a real, bootable installation on a usb drive.

1.  Creating a USB version of a live CD -- the boring option
With the current 'hybrid' Debian iso's it couldn't be easier.

 --- START HERE ---

a. Download the iso
Using jigdo is a good option. Downloading a business-card/netinstall iso is another, if you'll have a working internet connection available.

b. Plug in and mount your USB device and  find out the device name of your USB drive.
If it's mounted you can use
df -h
rootfs                                                   93G   36G   54G  40% /
udev                                                    3.9G     0  3.9G   0% /dev
tmpfs                                                   801M  980K  800M   1% /run
/dev/disk/by-uuid/..-10a350f85687   93G   36G   54G  40% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   1.6G   52K  1.6G   1% /tmp
tmpfs                                                   1.6G  816K  1.6G   1% /run/shm
/dev/sda6                                               745G  183G  525G  26% /home
/dev/sdc1                                               2.0G  434M  1.5G  23% /media/XP-KOMKU


../dev/sdc1 on /media/XP-KOMKU type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0077,codepage=cp437,iocharset=utf8,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks)

to list mounted devices. In my case it's an old 2 G usb stick I used to create a Windows XP installation USB device on.

If you prefer a gui tool, start palimpsest (called Disk Utility in gnome)
It will be something akin to sdb1 or sdc1 etc. That means the device name is /dev/sdb or /dev/sdc, respectively. In our case, it's sdc.

c. Unmount but don't detach the device
You don't want anything else writing to it.

me@beryllium:~/Documents/screengrabs$ umount /dev/sdc1
me@beryllium:~/Documents/screengrabs$ df -h
Filesystem                                              Size  Used Avail Use% Mounted on
rootfs                                                   93G   36G   54G  40% /
udev                                                    3.9G     0  3.9G   0% /dev
tmpfs                                                   801M  976K  800M   1% /run
/dev/disk/by-uuid/..-10a350f85687   93G   36G   54G  40% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   1.6G   60K  1.6G   1% /tmp
tmpfs                                                   1.6G  820K  1.6G   1% /run/shm
/dev/sda6                                               745G  183G  525G  26% /home
d. Become root and use cat to write the iso to the USB device

sudo su

And write (assuming that the device name is sdc and you are using the businesscard iso)

cat debian-6.0.3-amd64-businesscard.iso > /dev/sdc


e. boot from your USB drive and go through the same steps as for a CD.  
Plug it in, then start your computer. Hit F12 (or F10 or del or F2) during the bios start-up to select boot medium. Choose to boot from usb. Older BIOSes can't boot from USB.

2. Really INSTALLING Debian on a USB
You can use either virtualbox or quemu for this. Here I show how to use virtualbox.

[NOTE: there's a leaner, faster, better way described here: ]

  •  Note that a tiny 2 Gb USB stick isn't suitable for a full Debian + Gnome installation. It may, however, work well with e.g. openbox and xmonad. Or just work in the terminal -- use mutt for email, mcabber for chat, elinks for http/www etc.
  • Here I show how to install using the businesscard iso, but I've done this with the regular Debian DVD version in the past.
  • Also, few USB devices operate at 480 Mbps -- so be prepared that installation via virtualbox onto a USB device will take considerably longer than a regular installation.
  • Solid state devices degrade with I/O, so use your USB stick for back-up use during e.g. travel etc. Don't trust the integrity of it. 

There are a few good reasons for installing a full linux version on a thumb drive or a regular but external harddrive:
*  a thumb drive will allow you too carry an independent OS with you, allowing you to e.g. temporarily borrow a computer from a friend but running your own system. An installation, as opposed to a 'live cd', allows you to customise the setup and install programs. USB 2 is quite slow, and many thumb drives are slower than that, so you're probably better off running a light system than a full on gnome-shell desktop. That's not so say that you can't -- I have.
* installing onto a usb-connected harddrive will allow you to setup and prepare headless boxes or systems without a direct internet connection.

First you need to install virtualbox, which is available in the debian repos (sudo apt-get install virtualbox virtualbox-guest-additions), and -- in order to enable USB 2 support --  the "VirtualBox 4.1.8 Oracle VM VirtualBox Extension Pack" from
Start virtualbox and go to  File/Preferences/Extensions to install the extension pack you downloaded.

 --- START HERE ---

Installing debian (or any os) on an external USB stick/drive using virtualbox
a. Start virtualbox and click on New.

b. Click through the Name, Operating System and Version questions, Select a memory size which is reasonable for the intended use and target machine. Give it at least 256 Mb. Don't fret about this step -- on each boot your system will autodetect the amount of available RAM. The next question is the key to the whole undertaking: DO NOT SET A START-UP DISK (or any virtual harddisk)

In the next dialogue box you'll be asked if you want to continue or go back - yes, you want to continue.

c. Your new machine is now ready to be set up. Select it and click on settings; Got to storage, IDE controller and choose a virtual CD/DVD disk file. Select the debian installation iso.

d. Next click on USB, enable USB 2 controller, and add a filter for your device by e.g. hitting alt+insert and selecting it from the list.

e. You're ready to install! Start the virtual machine, and the installation should start without you having to select boot device. Most of the setting are the same as in any normal install. Pay attention when you come to the Partition Disk dialogue though. Make sure that you're not doing something silly and accidentally deleting something you shouldn't be a-deleting, even though the risks of that are very slim.

Delete the existing partitions and create a new one. I added a tiny swap space too, hoping that this will make the difference between a freeze and a slow crawl on underpowered systems.
f. Continue with the installation as normal.

g. Postscript:
* /dev/sda1 vs UUID. Once you're done with the installation  and have shut down your virtualbox, mount the USB drive as a normal thumb drive and have a look at fstab in particular -- if you're using relative paths (e.g. sda) instead of absolute paths using UUID, you may run into problems at some point.

In the screengrab you see that

# / was on /dev/sda1 during installation
UUID=8c0c9c98-e73f-4f5a-b7d7-f91f95c9d8ca /               ext4    errors=remount-ro 0       1

Which means we're good.

You may want to comment out
/dev/sda1       /media/usb0     auto    rw,user,noauto  0       0
/dev/sda2       /media/usb1     auto    rw,user,noauto  0       0
/dev/sda5       /media/usb2     auto    rw,user,noauto  0       0

But I haven't in the past and all has been good.

* network interfaces -- you may find that no network interface is configured for you and that no interface shows up when you use ifconfig. Use ip addr to get a list over interfaces. If the interface is e.g. eth2, you may  want to edit /etc/network/interfaces and add

auto eth2
iface eth2 inet dhcp

or (the ip's are just examples)

auto eth2
iface eth2 inet static

And then run sudo service networking restart

That's it. You now have a full installation on either a 'removable' drive or on an HDD which you can install in a desktop/laptop.

Links to this page: