Thursday, May 7, 2009

A trick for the pbs scripts on usc high performance computing cluster (HPC)

I'm trying to run LDA for different numbers of topics. I wrote a
bunch of m-files with all different number of topics hard coded but
then I realized it was being dumb. Actually, it might have saved time
this time, but in the long run I wanted to figure out the hpc pbs
scripts work. Basically its possible to pass variables from the shell
into the pbs script (you can't use the standard ENV vars b/c the job
is forked to different machines). The good idea I had was to pass the
qsub script the command I want to run (distribute) in the variable
CMD. This takes the responsibility out of the pbs script and puts it
back into a normal shell command, which I think is easier for my
purpos. All that's in the run.pbs script is:


#!/bin/bash

source source /usr/usc/matlab/default/setup.sh #puts matlab on path
cd /auto/rcf-proj3/sn/kazemzad/machineLearningTest #goes to the dir that I want


echo $CMD #assumes $CMD is passed using the -v switch
$CMD; #runs #CMD

(END)

Here's an example of how to use it for a test matlab script:


qsub -v CMD="matlab -nosplash -nodesktop -r \"2+2,exit\"" run.pbs


Here's an example of trying to put it all together for a number of topics


for x in 10 50 100 200 300 400 500 600 700 800 900 1000; do echo $x; qsub -l walltime=23:59:59,nodes=1:ppn=2 -v CMD="matlab -nosplash -nodesktop -r \"numTopicsExperiment($x),exit\"" run.pbs; done


This just forks all the different experiments with $x as a parameter.
These commands take 7+ hours to run, so I'm not actually sure it
works. At least it hasn't barfed yet. If I don't post back, assume
it worked and try it yourself if it suits your needs.

Thursday, March 19, 2009

prtools svm on linux

the last few days I've been trying to get svm working with prtools. Prtools seems convenient, but I got an error that qld wasn't found and that it was using the matlab quadratic programming function instead. I found a forum where they have a download for the qld.c function. It was easy to compile with mex and it seems to be running much faster now. I don't know why they didn't include the .c file to begin with, but I figured it out anyway.... here's the link to the forum http://prsysdesign.net/index.php/forums/viewthread/22/

Tuesday, March 3, 2009

Granule for Leither cardfile methodology on linux

I started trying granule for keeping a flashcards so I can remember stuff better. I have a lot of reading for my two classes and for my research. So far I like it, though I haven't tried other programs. Granule http://granule.sourceforge.net/ is not too fancy. Figuring out the user interface was a tiny bit non-intuitive, but not too hard either. One thing that I had some trouble with was figuring out how to name a particular deck of cards. It turns out you have to do a "save as" to actually rename the deck.

I looked a little for other linux flash card programs but I couldn't find to many that looked robust (my criterion was for it to be in portage). I found some linux flashcards for reviewing linux commands : http://www.proprofs.com/flashcards/search.php?search=Linux

in the future, I'll post some of the flash card files I make.