20 April 2014

573. PCSensors K-type USB thermocouple adapter TEMPer1K4 (0c45:7403) on debian

I just received this usb thermocouple reader (TEMPer1k4): http://www.pcsensor.com/index.php?_a=product&product_id=104

About the product:
lsusb shows 0c45:7403 (Microdia Foot Switch)

dmesg shows
[13448.536120] usb 6-1: new low-speed USB device number 3 using uhci_hcd [13448.709126] usb 6-1: New USB device found, idVendor=0c45, idProduct=7403 [13448.709139] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [13448.709146] usb 6-1: Product: TH1000isoV1.5 [13448.709152] usb 6-1: Manufacturer: RDing


Getting it to work:
You can use the same program as in this post: http://verahill.blogspot.com.au/2013/12/532-temper-temperature-monitoring-usb.html with some minor modifications. Before running
sudo python setup.py install

make some changes in temperusb/temper.py.

Firstly, you need to allow for the detection of devices with the 0c45:7403 ID
 
15 VIDPIDs = [(0x0c45L,0x7401L),(0x0c45L,0x7403L)]
Secondly, you need to make sure that the program knows which device is which and treats them differently. Finally, you need to collect the right data-- the TEMPer1k4 returns a data_s with a length of 8 (caveat -- I haven't checked what the output from 0c45:7401 looks like. Maybe it too yields 8 fields), and the [2:4] data block contains the internal temperature of the USB device, while the [4:6] data block holds the thermocouple junction temperature. Of a sort -- you get a number which you need to translate into a temperature. See the bottom of this post for how I worked it all out. Please note that you should calibrate the thermocouple -- the uncorrected output seems to be at least 2-3 degrees too high.
 
120             data = self._interrupt_read(self._handle)
121             data_s = "".join([chr(byte) for byte in data])
122             if len(data_s)==8:
123 #               print '24 ',struct.unpack('>h', data_s[2:4])[0]/16.0
124 #               print '46 ',struct.unpack('>h', data_s[4:6])[0]
125                 temp_thermocouple = struct.unpack('>h', data_s[4:6])[0]
126                 temp_internal = 0.0625*(struct.unpack('>h', data_s[2:4])[0]/16.0)+0.0025
127                 temp_c=(temp_thermocouple*0.25+1.50)
128                 if calibrate==True:
129                         temp_c=temp_thermocouple  #uncomment for calibration
130             else:
131                 temp_c = 125.0/32000.0*(struct.unpack('>h', data_s[2:4])[0])
132             temp_c = temp_c * self._scale + self._offset
133             if format == 'celsius':
Beyond this, follow the instructions in http://verahill.blogspot.com.au/2013/12/532-temper-temperature-monitoring-usb.html and make sure to set up a rules file with the correct USB ID for this device.

Done!

If you're curious about the details, have a look below.

Analysing USB traffic:
Since I wasn't quite sure where to start I figured the easiest approach would be to sniff the traffic between the usb device and the offically supported programme from PC Sensors. So I installed it Win XP in virtualbox, read a couple of temperatures and compared them with the captured data.

To capture data I did:
sudo apt-get install tshark
sudo modprobe usbmon
lsusb
Bus 008 Device 003: ID 17ef:4815 Lenovo Integrated Webcam [R5U877] Bus 008 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 043: ID 0c45:7403 Microdia Foot Switch Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
sudo tshark -D
1. eth0 2. wlan0 3. nflog 4. nfqueue 5. usbmon1 6. usbmon2 7. usbmon3 8. usbmon4 9. usbmon5 10. usbmon6 11. usbmon7 12. usbmon8 13. any 14. lo (Loopback)
touch 1.pcap chmod ugo+w 1.pcap sudo tshark -i usbmon4 -w $HOME/1.pcap

I opened the 1.pcap file in wireshark and looked for fields called leftover data.

Here are two examples of such 'leftover data' fields:
Host to USB: 01 80 00 00 00 00 00 00
USB to Host: 80 06 17 f0 00 5c 0f ff

Subjecting the system to different temperatures helped me figure out what fields where changing -- I've marked them in bold above (blue for the thermocouple, red for the internal temperature). Interestingly, I got overflow when freezing the thermocouple between two ice cubes, which indicated that there was a fairly high lower limit (1.5 degrees)
The following are some of the data I harvested. The temperature is in the first column, the hex code is in the second one and the decimal value is in the third one:
Internal:
23.63 17a 378
23.69 17b 379
23.94 17f 383
23.75 17c 380
21.88 15e 350
22.31 165 357
22.88 16e 366

Thermocouple: 23.75 05f 95 23.50 05e 9 38.25 099 153 24.00 066 102 87.50 15e 350 79.50 13e 318
In other words, T(thermo)=output(thermo)*0.25+1.50 and T(int.)=(output(int.)-0.0025)/0.0625. Because the internal data is in fields 2:4 it's seen as e.g. 17b0 instead of 17b, so the decimal version needs to be divided by 16 (line 126 above).
I also set up an /etc/temper.conf file and did chown $USER /etc/temper.conf in order to be able to read it (must be a better way). I tried to calibrate the thermocouple in my kitchen with iced water and boiling water. The boiling water gave a reading of 102 degrees, while the iced water gave me 7.5 degrees Celsius even after what should've been long enough to achieve equilibrium. I'll calibrate it in the lab later. So for now a simple offset might be enough to give a working temperature.
temper-poll -p
Device #0 (bus 2 - port 2)
so temper.conf became
2-2: scale = 1.00, offset = -4.0

16 April 2014

572. autorotate/superimpose python script

If you want to calculate reaction coordinates between two structures you need to make sure that the structures haven't been rotated or translated, something which easily happens if you allow symmetry in gaussian and (it seems) z-matrix in nwchem.

I've written a script that lets you take two structures and align and superimpose them so that only the atoms that take part in the reaction move.

It works by you defining a minimum of four atoms that aren't supposed to move /relative to each other/ (i.e. they can be translated/rotated --- just not relative to each other) between the two structures. Four atoms far from each other are ideal. You need to make sure that they also don't lie in the same plane, but form a three-dimensional space.

I've tried this on real molecules too and it works better than I'd ever dared to hope for. The more 'stationary' atoms that you can use to make up the transformation matrix, the better.


Example:
In this example atom F is in a different location in structures a and b. The structures have also been rotated relative to each other.

a.xyz
6 A A 0.00000 0.00000 1.00000 B 0.00000 1.00000 0.00000 C 1.00000 0.00000 0.00000 D -1.00000 0.00000 0.00000 E 0.00000 0.00000 -1.00000 F 0.00000 0.500000 0.00000
b.xyz
6 B A 1 0 0 B 0 1 0 C 0 0 -1 D 0 0 1 E -1 0 0 F 0 -1 0

./autorotate.py a.xyz b.xyz '1,2,3,4'
Selected atoms in molecules 1 and 2 ['A', 0.0, 0.0, 1.0] ['A', 1.0, 0.0, 0.0] ['B', 0.0, 1.0, 0.0] ['B', 0.0, 1.0, 0.0] ['C', 1.0, 0.0, 0.0] ['C', 0.0, 0.0, -1.0] ['D', -1.0, 0.0, 0.0] ['D', 0.0, 0.0, 1.0] Transformation max error: 3.33066907388e-16 Writing to a.rot.xyz

a.rot.xyz
6 A A 1.00000 0.00000 0.00000 B -0.00000 1.00000 -0.00000 C -0.00000 0.00000 -1.00000 D -0.00000 0.00000 1.00000 E -1.00000 -0.00000 -0.00000 F -0.00000 0.50000 -0.00000
This is how it looks (note that the axis aren't aligned with (1,0,0; 0,1,0; 0,0,1) but seem to go through the centre of the molecule):
a.xyz
a.rot.xyz


b.xyz





Code:
#!/usr/bin/python
import sys
import numpy as np

#autorotate input_1.xyz input_2.xyz '1,2,3,4'
# need to pick at least four atoms that are not in the same plane
# input_1.xyz will be rotated to align with input_2.xyz
# you pick at least four atoms that should have the same positions
# relative to one another (i.e. distance and relative geometry). These 
# are then used to calculate an affine transform matrix which is used 
# to rotate and translate structure input_1.xyz to overlap with 
# structure 2

def formatinput(argument):
 infile1=sys.argv[1]
 atoms=sys.argv[3]
 atoms=atoms.split(',')
 coord_sys=[]

 for n in atoms:
  coord_sys+=[int(n)-1]
 try:
  infile2=sys.argv[2]
 except:
  infile2=''
 infile=[infile1,infile2]
 return infile,coord_sys
 
def getrawdata(infile):
 f=open(infile,'r')
 
 n=0
 preamble=[]
 
 struct=[]
 
 for line in f:
  if n<2: data-blogger-escaped-if="" data-blogger-escaped-line.rstrip="" data-blogger-escaped-n="" data-blogger-escaped-preamble="">1:
   line=line.rstrip()
   struct+=[line]
  n+=1
 xyz=[struct]
 
 return xyz, preamble

def getcoords(rawdata,preamble,atoms):
 
 n=0
 cartesian=[]
 
 for structure in rawdata:
  n=n+1
  num="%03d" % (n,)
  for item in structure:
   
   coordx=filter(None,item.split(' '))
   coordy=filter(None,item.split('\t'))
   if len(coordx)>len(coordy):
    coords=coordx
   else:
    coords=coordy
      
   coordinates=[float(coords[1]),float(coords[2]),float(coords[3])]
   element=coords[0]
   cartesian+=[[element,float(coords[1]),float(coords[2]),float(coords[3])]]
     
 return cartesian

def getstructures(rawdata,preamble):
 
 n=0
 cartesian=[]
 
 for structure in rawdata:
  n=n+1
  num="%03d" % (n,)
  for item in structure:
   
   coordx=filter(None,item.split(' '))
   coordy=filter(None,item.split('\t'))
   if len(coordx)>len(coordy):
    coords=coordx
   else:
    coords=coordy
      
   coordinates=[float(coords[1]),float(coords[2]),float(coords[3])]
   element=coords[0]
   cartesian+=[coordinates]
     
 return cartesian

def affine_transform(atoms,structures):
# from http://stackoverflow.com/questions/20546182/how-to-perform-coordinates-affine-transformation-using-python-part-2
 primaryatomcoords=[]
 for n in atoms:
  primaryatomcoords+=[structures[0][n]]

 secondaryatomcoords=[]
 for n in atoms:
  secondaryatomcoords+=[structures[1][n]]

 primary = np.array(primaryatomcoords)
 secondary = np.array(secondaryatomcoords)
 primaryfull = np.array(structures[0])

 # Pad the data with ones, so that our transformation can do translations too
 n = primary.shape[0]
 pad = lambda x: np.hstack([x, np.ones((x.shape[0], 1))])
 unpad = lambda x: x[:,:-1]
 X = pad(primary)
 Y = pad(secondary)
 Xp= pad(primaryfull)

 # Solve the least squares problem X * A = Y
 # to find our transformation matrix A
 A, res, rank, s = np.linalg.lstsq(X, Y)

 transform = lambda x: unpad(np.dot(pad(x), A))

# print "Max error should be as small as possible if the rotation is successful"
# print "If max error is large you may have selected a bad set of atoms"
 print "Transformation max error:", np.abs(secondary - transform(primary)).max()
 secondaryfull=transform(primaryfull)
 return secondaryfull

def transform_xyz(tmatrix,newxyz):
 final_xyz=[]
 for n in newxyz:
  coord=np.mat(str(n[0])+';'+str(n[1])+';'+str(n[2]))
  newcoord=np.dot(tmatrix,coord)
  newcoord=np.matrix.tolist(newcoord)
  final_xyz+=[[ newcoord[0][0],newcoord[1][0],newcoord[2][0]]]
 return final_xyz

def genxyzstring(coords,elementnumber):
 x_str='%10.5f'% coords[0]
 y_str='%10.5f'% coords[1]
 z_str='%10.5f'% coords[2]
 element=elementnumber
 xyz_string=element+(3-len(element))*' '+10*' '+\
 (8-len(x_str))*' '+x_str+10*' '+(8-len(y_str))*' '+y_str+10*' '+(8-len(z_str))*' '+z_str+'\n'
 
 return xyz_string

def write_aligned(aligned_structure,atom_coords,preamble,outfile):
 outfile=outfile.replace('.xyz','.rot.xyz')
 print "Writing to ",outfile
 g=open(outfile,'w')
 g.write(str(preamble[0])+'\n'+str(preamble[1])+'\n')
 
 for n in range(0,len(aligned_structure)):
  xyzstring=genxyzstring(aligned_structure[n],atom_coords[n][0])
  g.write(xyzstring)
 g.close()
 return 0
 
if __name__ == "__main__":
 infile,atoms=formatinput(sys.argv)
 
 xyz=['','']
 preamble=['','']
 
 #get raw data
 xyz[0],preamble[0]=getrawdata(infile[0])
 xyz[1],preamble[1]=getrawdata(infile[1])

 atom_coords=[getcoords(xyz[0],preamble[0],atoms)]
 atom_coords+=[getcoords(xyz[1],preamble[1],atoms)]
 
 #collect structures from raw data
 structures=[getstructures(xyz[0],preamble[0])]
 structures+=[getstructures(xyz[1],preamble[1])]
 
 print "Selected atoms in molecules 1 and 2"
 for n in atoms:
  print atom_coords[0][n],atom_coords[1][n]
  
 #transform structure
 aligned_structure=affine_transform(atoms,structures)
 
 write_aligned(aligned_structure,atom_coords[0],preamble[0],str(infile[0]))
 

09 April 2014

571. Briefly: Dodgy/underpowered UPS?

I've built quite a few computers in the past, and in general I haven't had any issues beyond the odd dodgy RAM stick.

However, a while back I became careless and built a box ('Oxygen') where the motherboard didn't officially support the CPU. Swapping CPUs with another box seemed to solve the issues I had.

See e.g.
http://verahill.blogspot.com.au/2013/10/520-new-node-amd-fx-835032-gb-ram990-fx.html
http://verahill.blogspot.com.au/2013/10/523-random-reboots-troubleshooting-in.html

In the past couple of weeks I've begun to see some worrying signs that all isn't right. In particular I noticed the following in the dmesg output:
[693166.514897] [Hardware Error]: MC2 Error: VB Data ECC or parity error. [693166.514926] [Hardware Error]: Error Status: Corrected error, no action required. [693166.514934] [Hardware Error]: CPU:6 (15:1:2) MC2_STATUS[-|CE|MiscV|-|-|-|-|CECC]: 0x98414000010c0176 [693166.514955] [Hardware Error]: cache level: L2, tx: DATA, mem-tx: EV

A few days after that, the computer turned itself off without returning any additional error messages. It did cause me to look at the sensor output though (I've been logging it every two minutes for months), and I compared it with another computer ('Neon') which is completely stable. Note that both computers have been running the same types of jobs recently (large memory frequency jobs).

Hardware specs:
Oxygen: AMD FX8150, 32 gb ram, Corsair GS700, asrock 990 fx extreme3
Neon: AMD FX8350, 32 gb ram, Corsair GS800, gigabyte 990 fxa

Anyway, this is what I found:
On Neon the power output is very stable, while on Oxygen it jumps up and down between ca 45 W and 130 W.

Has it been a crappy UPS that has been causing the issues all along? Or do these plot mean nothing?