15 March 2014

566. Briefly: Annoying warnings when plotting using gnuplot and octave on wheezy March 2014.

Note: I'm not going to give a proper fix for this, but rather a work-around -- and one which isn't very good at that.

When using gnuplot or plotting in octave on wheezy I keep getting the following warnings.
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-ukai.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-ukai.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-ukai.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-ukai.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-ukai.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 16: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 28: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 28: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 28: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 28: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/41-arphic-uming.conf", line 28: Having multiple  in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/64-arphic-uming.conf", line 8: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/64-arphic-uming.conf", line 21: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/64-arphic-uming.conf", line 34: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/64-arphic-uming.conf", line 47: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/65-droid-sans-fonts.conf", line 103: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/65-droid-sans-fonts.conf", line 138: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/90-fonts-baekmuk.conf", line 10: Having multiple values in  isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/90-fonts-baekmuk.conf", line 23: Having multiple values in  isn't supported and may not work as expected

My 'solution' was a bit radical -- I had already set up a system with apt-pinning (http://verahill.blogspot.com.au/2014/03/562-pulling-in-glibc-214-from-testing.html) so I figured that pulling in the fonts from testing couldn't hurt, assuming there were no dependencies to worry about.

So I did:
sudo apt-get install -t testing fonts-arphic-uming
and this worked fine.

The old 41-arphic-uming.conf:
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>

<!--
  Serif faces
 -->
        <alias>
                <family>AR PL ShanHeiSun Uni</family>
                <family>AR PL ShanHeiSun Uni MBE</family>
                <family>AR PL UMing CN</family>
                <family>AR PL UMing HK</family>
                <family>AR PL UMing TW</family>
                <family>AR PL UMing TW MBE</family>
                <default><family>serif</family></default>
        </alias>
<!--
  Monospace faces
 -->
        <alias>
                <family>AR PL ShanHeiSun Uni</family>
                <family>AR PL ShanHeiSun Uni MBE</family>
                <family>AR PL UMing CN</family>
                <family>AR PL UMing HK</family>
                <family>AR PL UMing TW</family>
                <family>AR PL UMing TW MBE</family>
                <default><family>monospace</family></default>
        </alias>
</fontconfig>
The new 41-arphic-uming.conf:
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>

<!--
  Serif faces
 -->
        <alias>
                <family>AR PL ShanHeiSun Uni</family>
                <default><family>serif</family></default>
        </alias>
        <alias>
                <family>AR PL ShanHeiSun Uni MBE</family>
                <default><family>serif</family></default>
        </alias>
        <alias>
                <family>AR PL UMing CN</family>
                <default><family>serif</family></default>
        </alias>
        <alias>
                <family>AR PL UMing HK</family>
                <default><family>serif</family></default>
        </alias>
        <alias>
                <family>AR PL UMing TW</family>
                <default><family>serif</family></default>
        </alias>
        <alias>
                <family>AR PL UMing TW MBE</family>
                <default><family>serif</family></default>
        </alias>
<!--
  Monospace faces
 -->
        <alias>
                <family>AR PL ShanHeiSun Uni</family>
                <default><family>monospace</family></default>
        </alias>
        <alias>
                <family>AR PL ShanHeiSun Uni MBE</family>
                <default><family>monospace</family></default>
        </alias>
        <alias>
                <family>AR PL UMing CN</family>
                <default><family>monospace</family></default>
        </alias>
        <alias>
                <family>AR PL UMing HK</family>
                <default><family>monospace</family></default>
        </alias>
        <alias>
                <family>AR PL UMing TW</family>
                <default><family>monospace</family></default>
        </alias>
        <alias>
                <family>AR PL UMing TW MBE</family>
                <default><family>monospace</family></default>
        </alias>
</fontconfig>
It just remained to pull in the rest of the offending fonts:
sudo apt-get install -t testing fonts-arphic-ukai fonts-droid fonts-baekmuk

14 March 2014

565. Setting up slurm on debian wheezy (very basic)

I have a problem: I've got access to stampede.tacc in Texas which is using slurm as the queue manager. And while I've got SGE figured out (use it on my own cluster, my collaborator's cluster and it's used on the university cluster) I'm having some conceptual issues with SLURM.

I don't have any problems writing slurm scripts -- it's similar enough to SGE. But nowhere do I see anyone use -cwd or any equivalent in their slurm scripts. Either that is because you don't have to, or it's just an oversight in all of the examples that I've seen.

Learning by doing has also been an issue -- whenever I submit a test job it takes many, many hours before it's run. That's no way to learn.

Either way, it's time for me to become more familiar with slurm, so I've decided to set it up on a dedicated box.

I look at this post while setting it up: http://paolobertasi.wordpress.com/2011/05/24/how-to-install-slurm-on-debian/

NOTE: I set up a single node. This won't deal with getting nodes to communicate, configuring master and submit nodes, or anything lik that.

NOTE: the package slurm is a completely different program (network monitor). You need slurm-llnl

I also wonder whether the name has got anything to with this Slurm...


Installation

sudo apt-get install slurm-llnl
Setting up munge (0.5.10-1) ... Not starting munge (no keys found). Please run /usr/sbin/create-munge-key Setting up slurm-llnl-basic-plugins (2.3.4-2+b1) ... Setting up slurm-llnl (2.3.4-2+b1) ... Not starting slurm-llnl slurm.conf was not found in /etc/slurm-llnl Please follow the instructions in /usr/share/doc/slurm-llnl/README.Debian.gz

Open the local file file:///usr/share/doc/slurm-llnl/slurm-llnl-configurator.html in a web browser and fill out the form. I got the following slurm.conf, which I put in /etc/slurm-llnl/ 
slurm.conf
# slurm.conf file generated by configurator.html. # Put this file on all nodes of your cluster. # See the slurm.conf man page for more information. # ControlMachine=ecce64bit #ControlAddr= #BackupController= #BackupAddr= # AuthType=auth/munge CacheGroups=0 #CheckpointType=checkpoint/none CryptoType=crypto/munge #DisableRootJobs=NO #EnforcePartLimits=NO #Epilog= #PrologSlurmctld= #FirstJobId=1 JobCheckpointDir=/var/lib/slurm-llnl/checkpoint #JobCredentialPrivateKey= #JobCredentialPublicCertificate= #JobFileAppend=0 #JobRequeue=1 #KillOnBadExit=0 #Licenses=foo*4,bar #MailProg=/usr/bin/mail #MaxJobCount=5000 #MaxTasksPerNode=128 MpiDefault=none #MpiParams=ports=#-# #PluginDir= #PlugStackConfig= #PrivateData=jobs ProctrackType=proctrack/pgid #Prolog= #PrologSlurmctld= #PropagatePrioProcess=0 #PropagateResourceLimits= #PropagateResourceLimitsExcept= ReturnToService=1 #SallocDefaultCommand= SlurmctldPidFile=/var/run/slurm-llnl/slurmctld.pid SlurmctldPort=6817 SlurmdPidFile=/var/run/slurm-llnl/slurmd.pid SlurmdPort=6818 SlurmdSpoolDir=/var/lib/slurm-llnl/slurmd SlurmUser=verahill #SrunEpilog= #SrunProlog= StateSaveLocation=/var/lib/slurm-llnl/slurmctld SwitchType=switch/none #TaskEpilog= TaskPlugin=task/none #TaskPluginParam= #TaskProlog= #TopologyPlugin=topology/tree #TmpFs=/tmp #TrackWCKey=no #TreeWidth= #UnkillableStepProgram= #UnkillableStepTimeout= #UsePAM=0 # # # TIMERS #BatchStartTimeout=10 #CompleteWait=0 #EpilogMsgTime=2000 #GetEnvTimeout=2 #HealthCheckInterval=0 #HealthCheckProgram= InactiveLimit=0 KillWait=30 #MessageTimeout=10 #ResvOverRun=0 MinJobAge=300 #OverTimeLimit=0 SlurmctldTimeout=300 SlurmdTimeout=300 #UnkillableStepProgram= #UnkillableStepTimeout=60 Waittime=0 # # # SCHEDULING #DefMemPerCPU=0 #EnablePreemption=no FastSchedule=1 #MaxMemPerCPU=0 #SchedulerRootFilter=1 #SchedulerTimeSlice=30 SchedulerType=sched/backfill SchedulerPort=7321 SelectType=select/linear #SelectTypeParameters= # # # JOB PRIORITY #PriorityType=priority/basic #PriorityDecayHalfLife= #PriorityCalcPeriod= #PriorityFavorSmall= #PriorityMaxAge= #PriorityUsageResetPeriod= #PriorityWeightAge= #PriorityWeightFairshare= #PriorityWeightJobSize= #PriorityWeightPartition= #PriorityWeightQOS= # # # LOGGING AND ACCOUNTING #AccountingStorageEnforce=0 #AccountingStorageHost= #AccountingStorageLoc= #AccountingStoragePass= #AccountingStoragePort= AccountingStorageType=accounting_storage/none #AccountingStorageUser= ClusterName=cluster #DebugFlags= #JobCompHost= #JobCompLoc= #JobCompPass= #JobCompPort= JobCompType=jobcomp/none #JobCompUser= JobAcctGatherFrequency=30 JobAcctGatherType=jobacct_gather/none SlurmctldDebug=3 SlurmctldLogFile=/var/log/slurm-llnl/slurmctld.log SlurmdDebug=3 SlurmdLogFile=/var/log/slurm-llnl/slurmd.log # # # POWER SAVE SUPPORT FOR IDLE NODES (optional) #SuspendProgram= #ResumeProgram= #SuspendTimeout= #ResumeTimeout= #ResumeRate= #SuspendExcNodes= #SuspendExcParts= #SuspendRate= #SuspendTime= # # # COMPUTE NODES NodeName=ecce64bit Procs=1 State=UNKNOWN PartitionName=debug Nodes=ecce64bit Default=YES MaxTime=INFINITE State=UP

sudo /usr/sbin/create-munge-key
sudo service slurm-llnl start
[ ok ] Starting slurm central management daemon: slurmctld.
[ ok ] Starting slurm compute node daemon: slurmd.
sudo service munge start
[ ok ] Starting MUNGE: munged.

At that point I tried sinfo, squeue etc., none of which returned anything other than a connection error:
squeue
slurm_load_jobs error: Unable to contact slurm controller (connect failure)
sinfo
slurm_load_partitions: Unable to contact slurm controller (connect failure)
So I rebooted. Which had no effect.The log file /var/log/slurm-llnl/slurmctld.log contains
fatal: Incorrect permissions on state save loc: /var/lib/slurm-llnl/slurmctld
verahill@ecce64bit:~$ sudo chown verahill /var/lib/slurm-llnl/slurmctld verahill@ecce64bit:~$ sudo service slurm-llnl restart
[ ok ] Stopping slurm central management daemon: slurmctld. No /usr/sbin/slurmctld found running; none killed. [ ok ] Stopping slurm compute node daemon: slurmd. No /usr/sbin/slurmd found running; none killed. slurmd dead but pid file exists [ ok ] Starting slurm central management daemon: slurmctld. [ ok ] Starting slurm compute node daemon: slurmd.
verahill@ecce64bit:~$ ps aux|grep slurm
verahill 3790 0.0 0.2 116164 2292 ? Sl 21:12 0:00 /usr/sbin/slurmctld root 3829 0.0 0.1 95064 1380 ? S 21:12 0:00 /usr/sbin/slurmd
verahill@ecce64bit:~$ squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
verahill@ecce64bit:~$ sinfo
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST debug* up infinite 1 idle ecce64bit

Testing
 
verahill@ecce64bit:~$ srun --ntasks=1  --label /bin/hostname && pwd && whoami
0: ecce64bit /home/verahill verahill

Time to write a simple queue script:
job.slurm
#!/bin/bash #SBATCH -J pbe_delta # Job name #SBATCH -o pbe_delta.o%j # Name of stdout output file(%j expands to jobId) #SBATCH -e pbe_delta.o%j # Name of stderr output file(%j expands to jobId) #SBATCH -N 1 # Total number of nodes requested (16 cores/node) #SBATCH -n 1 #SBATCH -t 48:00:00 # Run time (hh:mm:ss) date> output.out pwd >> output.out hostname >> output.out ls -lah
I submitted it using
sbatch job.slurm

and on running it gives two output files:
output.out
Fri Mar 14 17:16:10 EST 2014
/home/verahill/slurm/test
Ecce64bit
and pbe_delta.o4
total 16K
drwxr-xr-x 2 verahill verahill 4.0K Mar 14 17:16 .
drwxr-xr-x 3 verahill verahill 4.0K Mar 14 17:14 ..
-rw-r--r-- 1 verahill verahill  491 Mar 14 17:16 job.slurm
-rw-r--r-- 1 verahill verahill   59 Mar 14 17:16 output.out
-rw-r--r-- 1 verahill verahill    0 Mar 14 17:15 pbe_delta.o3
-rw-r--r-- 1 verahill verahill    0 Mar 14 17:16 pbe_delta.o4

564.Very, very briefly: ps2pdf not working on jessie 12 March 2014

Update: see second comment below post.

ps2pdf example.ps produces

Error: /typecheck in /findfont
Operand stack:
   -66.7   --nostringval--   0   --nostringval--   --nostringval--   {}
Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1910   1   3   %oparray_pop   1909   1   3   %oparray_pop   1893   1   3   %oparray_pop   1787   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   --nostringval--   %array_continue   --nostringval--   1868   6   6   %oparray_pop
Dictionary stack:
   --dict:1164/1684(ro)(G)--   --dict:0/20(G)--   --dict:78/200(L)--   --dict:196/300(L)--   --dict:44/200(L)--   --dict:194/256(L)--
Current allocation mode is local
Current file position is 393007
GPL Ghostscript 9.05: Unrecoverable error, exit code 1
The workaround is to remove the package  fonts-font-awesome, which will remove texlive-fonts-extra. As far as I understand a fix is included upstreams and will work its way down to Jessie eventually.