DATA SLURPING COOKBOOK
version 18 MAY 2009 by Jen Winters and John Subasavage
This is used to retrieve data from the 0.9m at CTIO and to do basic bias subtraction and flat-fielding.
**********************************************************************
***** YOU BEGIN BY SLURPING DATA FROM CTIO *****
**********************************************************************
Step 1: (in one terminal window)
A) "cd /nfs/recons3/SMARTS"
B) "mkdir" (call the directory by a eight character string of numbers
that corresponds to the observation date, e.g. 2006.1110)
Step 2: (in a second terminal window)
A) type in 'clark' to ensure that you are on the machine clark
B) type "ssh v12@ctio36.ctio.noao.edu".
use CTIOPI password to get in
if the data can't be found there, try these places:
ssh -l v12 ctio36.ctio.noao.edu
/tmp_mnt/usr/remote/u363/v12/MIRANDA
CTIO36
/usr/remote/u362/iraf/v12/MIRANDA
/u362/iraf/MIRANDA
/ua45/v12/YYYYMMDD.09
***** INSERT SMARTS STEPS NOTED AT END OF FILE IF YOU ARE DOING SMARTS *****
C) v12% will be displayed on the screen; you are now using unix.
D) type "ls" to see where you are and verify that the data you are
looking for is there.
E) change directories into the one that you want by typing "cd " and
the directory name
F) type 'ls' to see what's in that directory; there should be files
ending in ".imh" and ".fits".
G) to see how many entries you will slurp, do a word count by typing
"ls *.fits |wc"
H) type 'rsync -avz *.fits "username"@ftp.chara.gsu.edu:/nfs/recons3/SMARTS/
YYYYMMDD
i.e., rsync -avz 2009* subasavage@ftp.chara.gsu.edu:/nfs/recons3/SMARTS/20090401/)
I) it will ask for log in name, use your chara account and password;
that will bring you into your home directory.
J) logout to come back to GSU and check data
K) when it finishes, if you "ls", it will give you a directory
listing.
L) check the file sizes to make sure they are all the same by typing
"ls -l *.fits".
M) also "ls *.fits |wc" to make sure that all the files were
successfully transferred.
**********************************************************************
***** PROCESS THE DATA AT GSU *****
**********************************************************************
A) cd into the file that you created for the slurping process
(i.e. /nfs/recons3/SMARTS/20030327) type in "xgterm -sb &". In
xgterm, and type "cl".
A1) Sometimes the observer will name the .fits files with *.CTIOPI.*
instead of *.09.*. If this is the case, you will need to rename
the image files.
A2) To do this, type 'ls *.fits >> old'. This will create a file
called 'old' that contains all the slurped image files. copy the
'old' file into another file called 'new'. Then edit the 'new'
file by replacing CTIOPI with 09.
A3) Use 'imrename' in iraf to change all image files names: 'imrename
@old @new'.
B) type "fixname"; it will ask you for the root name; the root name is
the string of eight characters that is the date you are working on
(ie, 20070303). This step will generate a list; wait until it's
done to go to next step.
C) type "gof"
D) type "proc". You will then be asked to enter the bias section; trim
section, ccd readout noise, and ccd gain in turn; the numbers
should be provided. (The numbers in each column are identical all
the way down).
Make sure that the numbers given for you to enter match the ones in
the column. If they do, hit enter; otherwise, enter the numbers
manually in the form specified (including the brackets), and then
hit enter.
Then it will say "now reducing V band star images", "now reducing R
band star images", "now reducing I band star images". Once the
whole process is finished, you will see a "cl>" appear; this lets
you know that you're done with this set of data.
E) "ccdlist *.fits" to make sure that the filter is correct and
included.
**********************************************************************
***** SORT DATA INTO DIRECTORIES *****
**********************************************************************
PHOTOMETRY DIRECTORIES (/nfs/recons3/phot.0.9m/)
A) If there is a photometry night, please make sure that you
create a directory for that night and then COPY all frames
into that directory BEFORE you sort the frames into the parallax
directories.
A) If there are new objects and the frames are photometry
frames,and you don't know what the parallax filter is,
just copy them to the photometry directory (no need to
create a new parallax directory for them. A parallax
filter probably has not yet been selected for them.).
PARALLAX DIRECTORIES (/nfs/recons4/CTIOPI/regions/)
A) hselect *.o.fits 'ra, $I, object' yes >> sort.file
B) If there are objects that are not ours (they are not on
the observing lists): EXO planet data go into
exoplanet.dir under special targets; BIN data go into the
binary directory under special targets.
C) Then, after you have sorted all the non-parallax frames
where they are supposed to go, type
'astrometry.processor sort.file'. This will sort all
the frames into the parallax directories.
D) If the directory does not exist, the code will ask you if
you want to create the directory and then will sort it into
that newly created directory if you reply 'yes'. If you reply
'no', it will ask you for an alternative directory destination.
If you simply hit enter, it will do nothing.
E) The code might also complain if the name of the directory
contains capital letters. If this is the case, simply rename
the directory so that all letters are lower case.
Once the frames have been sorted, we move the entire directory (with
the raw frames only -- no .o.fits files should be left in it, unless a
photometry night was begun and then scrapped) to
/nfs/whiting1/CTIOPI/SMARTS.raw.2007
**********************************************************************
***** UPDATE THE REDUCTION LOG *****
**********************************************************************
(/nfs/morgan3/thenry/public_html/CTIOPI/protected.dir/master.astr.status)
**********************************************************************
***** SMARTS SPECIFIC STEPS *****
**********************************************************************
3) Check to see that the data are in the expected directory. This
step can usually be omitted as the service observer always stores
the data here but if there is ever a problem with the next step,
come back to this one.
cd /ua45/v12/"YYYYMMDD"-"PI NAME"/ (i.e., 20090402-Ciardi)
4) Return to original directory and run smartsproc script.
cd
./smartsproc (the './' is not explicitly necessary)
5) Answer questions using all lower case.
PI name: ciardi
Date (YYYYMMDD): 20090402
6) If it doesn't barf, all is well. Log into IRAF from the current directory.
cl
7) Change directory to where the input and output file names were
written (and where the fits data will be stored).
cd /u363/v12/SMARTS/"PI NAME"/"YYYYMMDD"/
(i.e., /u363.etc/ciardi/20090402/)
wfits @infile @outfile
8) Once all data are written to fits, generate an info file
hselect *.fits $I,object,filters yes > "YYYYMMDD"."PI NAME".inf
(i.e., 20090402.ciardi.inf).
9) Exit IRAF, go into directory data are stored, and ftp back to GSU server
logout
cd /u363/v12/SMARTS/"PI NAME"/"YYYYMMDD"/
rsync -avz 2009* "your
username"@ftp.chara.gsu.edu:/ftp/pub/SMARTS/"PI
NAME"/"YYYYMMDD"/
(i.e., rsync -avz 2009*
subasavage@ftp.chara.gsu.edu:/ftp/pub/SMARTS/ciardi/20090402/)
(NOTE: if the PI name does not exist on the GSU server, this will
barf. You'll need to log onto joy and change directory to
/ftp/pub/SMARTS/ and add the PI name in that directory).
Check the slurping process every few minutes as you will undoubtedly
lose the link and have to kill the process and up arrow to restart the
rsync command.
That's it!