199. NeCTAR -Virtualisation of Australian compute resources -- first steps

So they are seeing whether they can make more efficient use of the compute resources at different institutions in Australia by creating a cloud to pool their resources. One of the potential solutions is called NeCTAR.

---

Getting started
Go to http://dashboard.rc.nectar.org.au/auth/login/?next=/dash/
Log in using your institutions username and password

You now have two options to deal with the key issue:

Method 1 -- generate online
Once you're in, create a keypair under Manage Compute/Access & Security and give it an easy-to-remember name

This is your private key, so protect it: don't lose it and don't expose it. You can't download it again. You delete it, it's gone.

On your computer
mv ~/Downloads/nectar.pem ~/.ssh
chmod og-rwx ~/.ssh/nectar.pem
cp nectar.pem nectar
ssh-keygen -e -f nectar >nectar.pub

 ls nectar* -lah

-rw------- 1 me me 887 Jun 22 11:31 nectar
-rw------- 1 me me 887 Jun 22 11:28 nectar.pem
-rw-r--r-- 1 me me 335 Jun 22 11:31 nectar.pub

To use the key do

ssh -i nectar user@server

Method 2 -- BYOK
You're using linux -- you probably have your own key already.
Go to the Manage Compute/Access & Security, Import Keypair

Paste your ~/.ssh/id_rsa.pub (or id_dsa.pub) key.

And that's the extent of the setup.

---

Test run
Go to Manage Compute/Images & Snapshot and select a Real Linux image (i.e. Debian)

Select image

Hit Launch.

Set up -- don't forget to check SSH to be able to log on. If you want to be able to ping, check icmp as well.

Set up the image -- the defaults are ok, but make sure to check icmp (to be able to ping) and ssh (to be able to log in).

Loading

Generating and loading the image takes about 10-20 seconds -- about the duration of a Victorian earthquake.

Running

Now your image is up an running. To check that all is well

ping -c 3 115.146.92.154

PING 115.146.92.154 (115.146.92.154) 56(84) bytes of data.
64 bytes from 115.146.92.154: icmp_req=1 ttl=54 time=1.70 ms
64 bytes from 115.146.92.154: icmp_req=2 ttl=54 time=1.66 ms
64 bytes from 115.146.92.154: icmp_req=3 ttl=54 time=1.73 ms

--- 115.146.92.154 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.660/1.698/1.733/0.029 ms

To be able to log in via ssh you need to know what username to use -- it's (probably) image specific.

To find out, click on the image name (here: testing4)

Click me

Hit the 'Log' tab

Select 'log'

 And look for the username which is created in addition to root

Look for the username -- here it's debian

ssh -v -i ~/.ssh/tmp/nectar debian@115.146.92.154
The authenticity of host '115.146.92.154 (115.146.92.154)' can't be established.
RSA key fingerprint is 81:a8:a7:0f:a9:68:a0:08:f1:60:45:e3:57:2e:4c:4c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '115.146.92.154' (RSA) to the list of known hosts.

Once you're in you'll be greeted by:

Linux unnamed-virtual-machine 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian@i-00001637:~$ 

debian@i-00001637:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/vda1             9.9G  768M  8.6G   9% /
tmpfs                 2.0G     0  2.0G   0% /lib/init/rw
udev                  2.0G  112K  2.0G   1% /dev
tmpfs                 2.0G     0  2.0G   0% /dev/shm
debian@i-00001637:~$ uname -a
Linux i-00001637 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux
debian@i-00001637:~$ groups
debian cdrom floppy audio dip video plugdev
debian@i-00001637:~$ cat /etc/group|grep debian
cdrom:x:24:debian
floppy:x:25:debian
audio:x:29:debian
dip:x:30:debian
video:x:44:debian
plugdev:x:46:debian
debian:x:1000:

When you're done, don't forget to log out and terminate your image. If you leave it running it will count towards your resource allocations.

Terminating

---

Notes:

1. you'll run into trouble with the key fingerprints eventually as the IP addresses and key fingerprints won't be matching. Either you'll be doing a lot of editing of you ~/.ssh/known_hosts file or you have to relax your security setttings.

2. Yes, you can log in as root as well. The default user does not have sudo powers. 

3. It takes about 60 seconds after the launch of the image before the openssh server is up and accepting connections. Think more desktop speeds than laptop+SSD speeds.

4. For actual production stuff you can crank up the image requirements:

16 cores and 65 GB? Why, thank you!

5. Also, I think the real value of using virtual machine is that you can load a vanilla setup and customize it, then saving it by making a snapshot:

Snapshot saves

A first couple of actions might be to add a new user, and edit /etc/sudoers.

---

Troubleshooting ssh:
If you're having problems logging in using your key, use the ssh -v switch as shown above and parse the output



Unsuccessful:

debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/me/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/me/.ssh/id_dsa
debug1: Trying private key: /home/me/.ssh/id_ecdsa
debug1: Next authentication method: password

A successful authentication should contain

debug1: Offering RSA public key: /home/me/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).

If you are sure that you're using the right key (e.g. using -i), then make sure that you're using the right username -- to find out how to find it, look above.