In
http://www.theage.com.au/technology/technology-news/taxtime-mac-attack-gains-momentum-20120703-21enx.html
there's this little gem:
"..it isn't as if we're asking for Linux or anything like that, Mac OS is a major operating system."
Lindqvist - a blog about Linux and Science. Mostly.
04 July 2012
201. ...not like linux or anything...
The linux community is even more marginalised in Australia than in the US
In
http://www.theage.com.au/technology/technology-news/taxtime-mac-attack-gains-momentum-20120703-21enx.html
there's this little gem:
In
http://www.theage.com.au/technology/technology-news/taxtime-mac-attack-gains-momentum-20120703-21enx.html
there's this little gem:
25 June 2012
200. How long will your nwchem frequency calc take?
Update 19/12/12: Having done a lot more frequency calculations since I posted this I sincerely doubt that this approach works.
Original post
Because I'm stuck waiting for the results of frequency calcs on some large transition metal clusters, I've become interested in understanding the output of frequency calculations in progress. After all, why wait 15 days for a results if there are early signs that the calculation has gone haywire?
Also, it might just be me, but frequency calculations are not that easy to restart, so you want to make sure that you give them enough wall time to finish if you use a queue manager.
I'm sure most of this could be appreciated by RTFM, but who has time for that?
So this is what the calc does:
After the usual boredom of reading in the geometry and doing an energy calculation, followed by an MO analysis, the computational fun starts.
Each cycle contains the following reports:
You now have an idea of how long each step takes. But how many steps in total? I think it's 3N steps, where N is the number of atoms.
For my 50 atoms POM using the values above it'd be roughly 8000 s * 150 = 13 days 22 hours.
Which seems about right...
cat nwch.nwout|grep 'Total DFT'|gawk 'END {print NR}'
Time to start preparing lectures...
Original post
Because I'm stuck waiting for the results of frequency calcs on some large transition metal clusters, I've become interested in understanding the output of frequency calculations in progress. After all, why wait 15 days for a results if there are early signs that the calculation has gone haywire?
Also, it might just be me, but frequency calculations are not that easy to restart, so you want to make sure that you give them enough wall time to finish if you use a queue manager.
I'm sure most of this could be appreciated by RTFM, but who has time for that?
So this is what the calc does:
After the usual boredom of reading in the geometry and doing an energy calculation, followed by an MO analysis, the computational fun starts.
Each cycle contains the following reports:
- Total Density - Mulliken Population Analysis
- Spin Density - Mulliken Population Analysis
- Total Density - Lowdin Population Analysis
- Spin Density - Lowdin Population Analysis
- Expectation value of S2:
- NWChem DFT Module
- Caching 1-el integrals
- Total Density - Mulliken Population Analysis
- Spin Density - Mulliken Population Analysis
with the exception of the first cycle, which also look at the alpha-beta orbital overlaps, the centre of mass, moments of inertia and does a multipole analysis of density and save an initial hessian.
Each cycle ends with a report of the energy for that vibration:
Total DFT energy = -3297.032399945703
One electron energy = -26618.764098759657
Coulomb energy = 12938.745973154924
Exchange-Corr. energy = -382.742230868704
Nuclear repulsion energy = 10765.727956527733
Numeric. integr. density = 441.999974968347
Total iterative time = 7947.4s
If you do cat nwch.nwout|egrep "Total iterative time|Total DFT energy" you can see the progress:
Total DFT energy = -3297.032416366805
Total iterative time = 12146.0s
Total DFT energy = -3297.032399945703
Total iterative time = 7947.4s
Total DFT energy = -3297.032399544749
Total iterative time = 7946.0s
Total DFT energy = -3297.032406934719
Total iterative time = 7945.8s
Total DFT energy = -3297.032405026814
You now have an idea of how long each step takes. But how many steps in total? I think it's 3N steps, where N is the number of atoms.
For my 50 atoms POM using the values above it'd be roughly 8000 s * 150 = 13 days 22 hours.
Which seems about right...
cat nwch.nwout|grep 'Total DFT'|gawk 'END {print NR}'
66so I've got another 8 days before I can get my hand on some juicy thermochemical data...
Time to start preparing lectures...
22 June 2012
199. NeCTAR -Virtualisation of Australian compute resources -- first steps
So they are seeing whether they can make more efficient use of the compute resources at different institutions in Australia by creating a cloud to pool their resources. One of the potential solutions is called NeCTAR.
Getting started
Go to http://dashboard.rc.nectar.org.au/auth/login/?next=/dash/
Log in using your institutions username and password
You now have two options to deal with the key issue:
Method 1 -- generate online
Once you're in, create a keypair under Manage Compute/Access & Security and give it an easy-to-remember name
This is your private key, so protect it: don't lose it and don't expose it. You can't download it again. You delete it, it's gone.
On your computer
mv ~/Downloads/nectar.pem ~/.ssh
chmod og-rwx ~/.ssh/nectar.pem
cp nectar.pem nectar
ssh-keygen -e -f nectar >nectar.pub
ls nectar* -lah
Method 2 -- BYOK
You're using linux -- you probably have your own key already.
Go to the Manage Compute/Access & Security, Import Keypair
Paste your ~/.ssh/id_rsa.pub (or id_dsa.pub) key.
And that's the extent of the setup.
Test run
Go to Manage Compute/Images & Snapshot and select a Real Linux image (i.e. Debian)
Hit Launch.
Set up the image -- the defaults are ok, but make sure to check icmp (to be able to ping) and ssh (to be able to log in).
Generating and loading the image takes about 10-20 seconds -- about the duration of a Victorian earthquake.
Now your image is up an running. To check that all is well
To be able to log in via ssh you need to know what username to use -- it's (probably) image specific.
To find out, click on the image name (here: testing4)
Hit the 'Log' tab
And look for the username which is created in addition to root
Once you're in you'll be greeted by:
When you're done, don't forget to log out and terminate your image. If you leave it running it will count towards your resource allocations.
Notes:
1. you'll run into trouble with the key fingerprints eventually as the IP addresses and key fingerprints won't be matching. Either you'll be doing a lot of editing of you ~/.ssh/known_hosts file or you have to relax your security setttings.
2. Yes, you can log in as root as well. The default user does not have sudo powers.
3. It takes about 60 seconds after the launch of the image before the openssh server is up and accepting connections. Think more desktop speeds than laptop+SSD speeds.
4. For actual production stuff you can crank up the image requirements:
5. Also, I think the real value of using virtual machine is that you can load a vanilla setup and customize it, then saving it by making a snapshot:
A first couple of actions might be to add a new user, and edit /etc/sudoers.
Troubleshooting ssh:
If you're having problems logging in using your key, use the ssh -v switch as shown above and parse the output
Unsuccessful:
A successful authentication should contain
If you are sure that you're using the right key (e.g. using -i), then make sure that you're using the right username -- to find out how to find it, look above.
Getting started
Go to http://dashboard.rc.nectar.org.au/auth/login/?next=/dash/
Log in using your institutions username and password
You now have two options to deal with the key issue:
Method 1 -- generate online
Once you're in, create a keypair under Manage Compute/Access & Security and give it an easy-to-remember name
This is your private key, so protect it: don't lose it and don't expose it. You can't download it again. You delete it, it's gone.
On your computer
mv ~/Downloads/nectar.pem ~/.ssh
chmod og-rwx ~/.ssh/nectar.pem
cp nectar.pem nectar
ssh-keygen -e -f nectar >nectar.pub
ls nectar* -lah
-rw------- 1 me me 887 Jun 22 11:31 nectar
-rw------- 1 me me 887 Jun 22 11:28 nectar.pem
-rw-r--r-- 1 me me 335 Jun 22 11:31 nectar.pub
To use the key do
ssh -i nectar user@server
Method 2 -- BYOK
You're using linux -- you probably have your own key already.
Go to the Manage Compute/Access & Security, Import Keypair
Paste your ~/.ssh/id_rsa.pub (or id_dsa.pub) key.
And that's the extent of the setup.
Test run
Go to Manage Compute/Images & Snapshot and select a Real Linux image (i.e. Debian)
 |
| Select image |
 |
| Set up -- don't forget to check SSH to be able to log on. If you want to be able to ping, check icmp as well. |
 |
| Loading |
 |
| Running |
ping -c 3 115.146.92.154
PING 115.146.92.154 (115.146.92.154) 56(84) bytes of data. 64 bytes from 115.146.92.154: icmp_req=1 ttl=54 time=1.70 ms 64 bytes from 115.146.92.154: icmp_req=2 ttl=54 time=1.66 ms 64 bytes from 115.146.92.154: icmp_req=3 ttl=54 time=1.73 ms --- 115.146.92.154 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 1.660/1.698/1.733/0.029 ms
To be able to log in via ssh you need to know what username to use -- it's (probably) image specific.
To find out, click on the image name (here: testing4)
 |
| Click me |
 |
| Select 'log' |
 |
| Look for the username -- here it's debian |
ssh -v -i ~/.ssh/tmp/nectar debian@115.146.92.154 The authenticity of host '115.146.92.154 (115.146.92.154)' can't be established. RSA key fingerprint is 81:a8:a7:0f:a9:68:a0:08:f1:60:45:e3:57:2e:4c:4c. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '115.146.92.154' (RSA) to the list of known hosts.
Once you're in you'll be greeted by:
Linux unnamed-virtual-machine 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. debian@i-00001637:~$
debian@i-00001637:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 9.9G 768M 8.6G 9% / tmpfs 2.0G 0 2.0G 0% /lib/init/rw udev 2.0G 112K 2.0G 1% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm debian@i-00001637:~$ uname -a Linux i-00001637 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux debian@i-00001637:~$ groups debian cdrom floppy audio dip video plugdev debian@i-00001637:~$ cat /etc/group|grep debian cdrom:x:24:debian floppy:x:25:debian audio:x:29:debian dip:x:30:debian video:x:44:debian plugdev:x:46:debian debian:x:1000:
When you're done, don't forget to log out and terminate your image. If you leave it running it will count towards your resource allocations.
 |
| Terminating |
Notes:
1. you'll run into trouble with the key fingerprints eventually as the IP addresses and key fingerprints won't be matching. Either you'll be doing a lot of editing of you ~/.ssh/known_hosts file or you have to relax your security setttings.
2. Yes, you can log in as root as well. The default user does not have sudo powers.
3. It takes about 60 seconds after the launch of the image before the openssh server is up and accepting connections. Think more desktop speeds than laptop+SSD speeds.
4. For actual production stuff you can crank up the image requirements:
 |
| 16 cores and 65 GB? Why, thank you! |
 |
| Snapshot saves |
Troubleshooting ssh:
If you're having problems logging in using your key, use the ssh -v switch as shown above and parse the output
Unsuccessful:
debug1: Authentications that can continue: publickey,password debug1: Next authentication method: publickey debug1: Offering RSA public key: /home/me/.ssh/id_rsa debug1: Authentications that can continue: publickey,password debug1: Trying private key: /home/me/.ssh/id_dsa debug1: Trying private key: /home/me/.ssh/id_ecdsa debug1: Next authentication method: password
A successful authentication should contain
debug1: Offering RSA public key: /home/me/.ssh/id_rsa debug1: Server accepts key: pkalg ssh-rsa blen 279 debug1: read PEM private key done: type RSA debug1: Authentication succeeded (publickey).
Subscribe to:
Posts (Atom)