Overview of Reduction Steps

For MIRC-X 6T data obtained after the MIRC-X commissioning in June 2017, please use the MIRC-X python reduction pipeline and follow the instructions in the MIRC-X Pipeline User Manual.

The pipeline to reduce older MIRC data was written by John Monnier in IDL.  The pipeline and the archive of coadded MIRC data are available on the Remote Data Reduction Machine in Alanta. The source code is maintained at the University of Michigan through the Subversion control system.

There are three versions of the MIRC pipeline installed on the Remote Data Reduction Machine.  They are called using the following IDL startup scripts:

  • mirc6b_idl: Starts pipeline_mirc6b that reduces MIRC 6T data and accounts for cross-talk.
  • mirc6T_idl: Starts pipeline_mirc6 that reduces MIRC 6T data.  It does not account for cross-talk.
  • mirc4T_idl: Starts pipeline_v2 that reduces MIRC 4T data.

Below is a listing of the steps to run when reducing 6T or 4T MIRC data.  Each step is described in more detail in the sections below.

MIRC 6T Data (with photometric channels: 2011 July - 2017 May)

To reduce MIRC 6T data, use the mirc6b_idl or mirc6T_idl startup procedures.  The 6b version of the pipeline is recommended, especially for bright resolved targets, because it accounts for cross-talk between baselines.  The 6b version takes longer to run.

  • $ mirc6b_idl
  • IDL> mirclog, date='2011Sep29', coadd_path='/dbstorage/mircs/MIRC_COADDp/', /verbose
  • IDL> .r mirc_process2.script
  • IDL> .r mirc_process3.script
  • IDL> .r mirc_process4.script
  • IDL> .r mirc_process5.script
  • IDL> .r mirc_average_sigclip.script  ---OR--- IDL> .r mirc_average.script
  • IDL> .r mirc_cal.script

MIRC 4T Data (No photometric channels: 2006 June - 2009Aug; With photometric channels: 2009Aug23 - 2011Jan19)

To reduce MIRC 4T data, use the mirc4T_idl startup procedure.

  • $ mirc4T_idl
  • IDL> mirclog, date='2010Nov05', coadd_path='/dbstorage/mircs/MIRC_COADDp/', /verbose
  • IDL> .r mirc_process2.script
  • If NO XCHANS then run IDL> .r mirc_process3.script
  • If XCHANS then run IDL> .r mirc_process3xchan.script
  • IDL> .r mirc_process4.script
  • IDL> .r mirc_process5.script
  • IDL> exit
  • The last two steps need be run using the pipeline_mirc6 (not 6b) startup file:
  • $ mirc6T_idl
  • IDL> .r mirc_average_sigclip.script  ---OR--- IDL> .r mirc_average.script
  • IDL> .r mirc_cal.script

Locating the Coadded MIRC Data

Coadded MIRC data are available on the Remote Data Reduction Machine in Atlanta in the following location:

/dbstorage/mircs/MIRC_COADDp 

These data have been already run through mirc_process1.script which does the following steps:

  1. Remove unused quadrants (keeps two quadrants for fringe data and photometric channels).
  2. Coadd nreads.
  3. Sigma filter the difference frames, but restack so the data look like raw data.
  4. Look for data dropouts and add keyword BADFLAG to header (=0 for OK or 1 for bad).
  5. Output files to MIRC_COADD directory.
  6. Add line to header regarding saturation estimate.

For data obtained between UT 2013 Apr 19 and 2013 Aug 01, use coadded data processed using mirc_process1rf.script which suppresses RF noise by subtracting signal from the neighboring quadrant.  These coadded data can be found in the following location:

dbstorage/mircs/MIRC_COADDrf 

Starting the MIRC IDL Pipeline

Create a working directory on the Remote Data Reduction Machine.  Change directories to the working directory and start the MIRC IDL pipeline using the appropriate startup script (mirc6b_idl, mirc6T_idl, mirc4T_idl):

$ mirc6b_idl 

Create MIRC Log File

Run mirclog.pro to create the .mirclog file:

IDL> mirclog, date='2011Sep29', coadd_path='/dbstorage/mircs/MIRC_COADDp/', /verbose 

The .mirclog groups each target into DATA sequences and SHUTTER sequences (BG = Background; B1-B6 = single beam shutters; FG = foregrounds, light from all telescopes but no fringes).

Edit the .mirclog file by comparing to the hand-written observing logs.  Copies of the scanned observing logs are available in the shared MIRC google drive.  Check to make sure the beam order is correct and that the correct shutter blocks are identified for each data block.  The associated shutter block is identified in the last column of each DATA line.  The #Fiber Explorer notes are sometimes located at the wrong positions in the .mirclog file, so it is important to make sure that the DATA and SHUTTER sequences are identified correctly, particularly when repeated data sets are obtained on the same target.  Remove any files where a beam was lost during the shutter sequences.  If possible, there should not be a fiber explorer map between the data and the associated shutters.

Prior to the installation of the photometric channels (in August 2009), in addition to the DIR line for the data directory, the mirclog file should also contain a line for the DAQDIR which contains the directory path for the DAQ synchronization files for synchronized chopping.

 Here is an example of an edited MIRC log file:

$ more 2011Sep29.mirclog
## MIRC Log for 2011Sep29
## Automatically created by mirclog.pro on Wed Jun 24 15:29:34 2020
## Hand-Edited by ___GHS___ on ___2020Jul06___
##
DIR   /dbstorage/mircs/MIRC_COADDp/2011Sep29/
##Source             MODE     Block  Type  Start  End    SHUTBLOCKS
## Include only one BEAM ORDER per MIRCLOG FILE !!!
BEAM_ORDER W1 S2 S1 E1 E2 W2 
## Fiber Explorer
HD_25490             H_PRISM  12     DATA  1274   1333   13
HD_25490             H_PRISM  13       BG  1334   1343
HD_25490             H_PRISM  13       B1  1344   1353
HD_25490             H_PRISM  13       B2  1354   1363
HD_25490             H_PRISM  13       B3  1364   1373
HD_25490             H_PRISM  13       B4  1374   1383
HD_25490             H_PRISM  13       B5  1384   1393
HD_25490             H_PRISM  13       B6  1394   1403
HD_25490             H_PRISM  13       FG  1404   1413
##
## Fiber Explorer
HD_33256             H_PRISM  14     DATA  1414   1473   15
HD_33256             H_PRISM  15       BG  1474   1483
HD_33256             H_PRISM  15       B1  1484   1493
HD_33256             H_PRISM  15       B2  1494   1503
HD_33256             H_PRISM  15       B3  1504   1513
HD_33256             H_PRISM  15       B4  1514   1523
HD_33256             H_PRISM  15       B5  1524   1533
HD_33256             H_PRISM  15       B6  1534   1543
HD_33256             H_PRISM  15       FG  1544   1553
##
## Fiber Explorer
HD_37468             H_PRISM  16     DATA  1554   1613   17
HD_37468             H_PRISM  17       BG  1614   1623
HD_37468             H_PRISM  17       B1  1624   1633
HD_37468             H_PRISM  17       B2  1634   1643
HD_37468             H_PRISM  17       B3  1644   1653
HD_37468             H_PRISM  17       B4  1654   1663
HD_37468             H_PRISM  17       B5  1664   1673
HD_37468             H_PRISM  17       B6  1674   1683
HD_37468             H_PRISM  17       FG  1684   1703
##
## Fiber Explorer
HD_33256             H_PRISM  18     DATA  1704   1763   19
HD_33256             H_PRISM  19       BG  1764   1773
HD_33256             H_PRISM  19       B1  1774   1783
HD_33256             H_PRISM  19       B2  1784   1793
HD_33256             H_PRISM  19       B3  1794   1803
HD_33256             H_PRISM  19       B4  1804   1813
HD_33256             H_PRISM  19       B5  1814   1823
HD_33256             H_PRISM  19       B6  1824   1833
HD_33256             H_PRISM  19       FG  1834   1853
##
## Fiber Explorer
HD_37468             H_PRISM  20     DATA  1854   1913   21
HD_37468             H_PRISM  21       BG  1914   1923
HD_37468             H_PRISM  21       B1  1924   1933
HD_37468             H_PRISM  21       B2  1934   1943
HD_37468             H_PRISM  21       B3  1944   1953
HD_37468             H_PRISM  21       B4  1954   1963
HD_37468             H_PRISM  21       B5  1964   1973
HD_37468             H_PRISM  21       B6  1974   1983
HD_37468             H_PRISM  21       FG  1984   2003 

Note: Data obtained earlier on this night was collected in K-band mode.  Those data were edited out of the .mirclog file to process only the H_PRISM data.

Create Alias and Calibrator Files

The MIRC IDL pipeline uses the file mirc_aliasfile.txt to identify the preferred name of each target observed and mirc_calibrators.txt to specify the calibrator diameters.  The format of each of these files are shown below.  If a target does not appear in either of these files, then the pipeline will query the user for the necessary information.

$ more mirc_aliasfile.txt 
Alternative ID, MIRC ALIAS
* nu. Tau,HD 25490
HD 25490,HD 25490
* 68 Eri,HD 33256
HD 33256,HD 33256
* sig Ori,sig Ori
sig Ori,sig Ori 
$ more mirc_calibrators.txt
HD_25490   0.599   0.020   SED_Fit_HD25490
nu_tau     0.599   0.020   SED_Fit_HD25490_nuTau
HD_33256   0.655   0.018   SED_Fit_HD33256 

To reduce MIRC 4T data, you also need to create a .mircarray file that specifies which telescope is on which beam and an initial comboinfo.idlvar file.  For the comboinfo file, you can download the sample idlvar file available in the link above and change the file name to the correct date. An example mircarray file for the S1E1W1W2 configuration is shown below (change the telescope configuration as needed):

$ more 2010Nov_S1E1W1W2.mircarray
## Enter Array Information  NEED TO CHECK AND EDIT, NOT SURE WHETHER IT IS THE RIGHT INFO!!!
MIRC    4
B1      S1
B2      E1
B3      W1
B4      W2 

Calculate uv coverage and time information

The routine mirc_process2.script calculates the uv coverage and time information.  It creates a YYYYMMMDD.idlvar file that is used during the few next steps of the pipeline. 

IDL> .r mirc_process2.script 

The script will ask the user to select the mirclog file:

Choose mirclog file.

 

If a target is not already in mirc_aliasfile.txt, the script will as the user to choose an alias for each target:

Choose alias.

 

Compute Photometry

The next step is to run mirc_process3.script to compute photometry information.

IDL> .r mirc_process3.script 
Interactive (1,default) or automatic (0):

The script will ask the user to select the idlvar file created during mirc_process2:

Choose idlvar file.

 

The script then displays the profiles for data and xchan regions of the detector during the data and shutter sequences:

Profiles for data and shutter sequences.

 

Check to see if light is on all of the beams and to make sure there are no fringes in the foreground frames. The script will prompt the user to flag missing beams or identify bad foreground frames. If all is good, then hit enter to continue.

OK? hit enter to continue
hit n to remove FG files due to fringes in the FG
hit numbers 1,2,3,4,5,6 to mark one beam as missing
HD_25490      12
:  

Next the script will plot the flux over time for each beam as measured from the photometric channels (xchan). The vertical dashed lines separate the data, shutter, and foreground files. The beam ratios are printed to the screen.

Photometry in each beam.

 

 Ratio Check: 
Channel: 00   Ratio: 0.986 +/- 0.001
Channel: 01   Ratio: 0.987 +/- 0.001
Channel: 02   Ratio: 0.985 +/- 0.001
Channel: 03   Ratio: 0.989 +/- 0.001
Channel: 04   Ratio: 0.991 +/- 0.001
Channel: 05   Ratio: 0.992 +/- 0.001
Channel: 06   Ratio: 0.992 +/- 0.001
Channel: 07   Ratio: 0.983 +/- 0.001
 End of block loop. hit enter to continue
:  

The script will loop through these profile and flux plots for each target.

Optimize fringe information for  given combiner setup

Run mirc_process4.script to optimize fringe information for a given combiner setup.  This step produces the date_comboinfo.ps file.

IDL> .r mirc_process4.script 

The script will ask the user to select the idlvar file and will continue on from there:

Choose idlvar file.

 

Coherent Integration

Run mirc_process5.script to compute coherent integration.  It is recommended to run this step twice, once with the default integration time of 17 ms and another time with a longer coherent integration time of 75 ms.  The longer coherent integration time will improve signal-to-noise for faint targets.  However, the shorter integration time might be necessary in bad seeing conditions to avoid biasing the visibility calibration.  A comparison of the results between the two coherent integration times can help decide the optimal value. This step creates the date_tcoh0017ms_diagnostics.ps summary file.

IDL> .r mirc_process5.script
Enter coherent integration time
 (default: 17 ms; for faintest targets try 75 ms)
: 17  

The script will ask the user to select the idlvar file:

Choose idlvar file.

 

The script will then show the waterfall fringe plots for each target.  The top panel shows the fringes for each baseline separately.  The lower panel shows fringes summed over all pairs for each telescope.  Dropouts from a particular telescope can be identified more easily in the bottom panel.

Plots of fringe waterfall displays.

 

Data Editing and Averaging

Run mirc_average.script or mirc_average_sigclip.script to average data and remove bad data points.  Both scripts allow the user to interactively remove bad data points.  The sigclip version applies sigma clipping to automatically remove data points that are more than 3 standard deviations discrepant from the median.  After sigma clipping, the user can still interactively reject additional data points.  The sigma clipping option provides a less subjective way to remove discrepant data points.  If you want to run a quick reduction without scrolling through all of the sigma clipping plots, then use mirc_average.script instead.

The data are averaged over a default time interval of 2.5 min.  The routine will save raw level one MIRC_L1*.oifits and summary MIRC_LI*.ps files for each target in the OIFITS output directory. 

IDL> .r mirc_average_sigclip.script

--- or ---

IDL> .r mirc_average.script

Choose the level of data selection (Normal Data Selection is recommended):

 

Choose the idlvar file:

 

Choose normalization method to analyze.  XCHAN is preferred, but can be compared to other flux methods if necessary:

 

Choose which reduction to process (e.g., 17 ms or 75 ms coherent integration time from the previous step):

 

Choose display type (default):

 

Select the Maximum Averaging time (2.5 min is the default):

 

Next you can edit data based on the fringe waterfall plots.  The top panel shows the fringes for each baseline separately.  The lower panel shows fringes summed over all pairs for each telescope.  Dropouts from a particular telescope can be identified more easily in the bottom panel.  The target name is written for each panel.  Follow the instructions on the terminal window to edit fringes.

Fringe editing.

 

 Double click in a square to flag whole block as bad (<0.5 sec)
 Click twice over a range to flag a specific RANGE 
 Click to the right of plot to REPLOT (allows to resize window)
 Click below all plots to reset bad flags!!
 Click to the left of plots to continue..  

Next you can edit data based on the flux from each telescope.  The pipeline loops through the flux rejection for each telescope.  Follow the instructions on the terminal window to edit any dropouts in flux.

Flux editing.

 

 Double click in a square to flag whole block as bad (<0.5 sec)
 Click once to mark a single point as bad
 Click to the right of plot to REPLOT (allows to resize window)
 Click below all plots to reset bad flags!!
 Click to the left of plots to continue..  

Next the script will ask if you want to apply sigma clipping to the visibilities.  It will loop through each baseline and show a plot for each target.  The panels show the data in each of the 8 spectral channels.  Follow the instructions on the screen to change the sigma clipping parameters and press enter to reject the clipped measurements after each plot.

Would you like to apply sigma clipping?
(applies to visibilities, closure phases, and closure amplitudes)
<Y>  or N
: Y
Sigma clipping rejection threshold (N*stdev): 
      3.00000
Would you like to change rejection threshold?
Y of <N>
: N
------------------------------
Sigma clipping of visibilities
Rejection threshold (N*sigma):       3.00000
NOTE: Edge channels will not be used to apply sigma clipping flags
Hit enter to continue
: 
Block:       12  Source: HD_25490
Diamonds - good data; X - sigma clipped
Number of measurements on target (per spectral channel): 
          57
Maximum number of sigma clipped measurements per spectral channel: 
      7.00000
Number of uniquely rejected measurements (not including edge channels): 
       7
Do you want to reject these measurements?
<Y> or N:
:  

 

Sigma clipping.  The diamonds are good data and the x's are rejected data.

 

 Calculating Calibration Table for All Spectral Channels...
----------------------------------------
Finished sigma clipping the visibilities.
Begin visual inspection and flagging of visibilities.
Hit enter to continue
:  

The next step is the interactive data editing.  The script will loop through each baseline and show all targets in only one spectral channel.  The sigma clipping routine will remove most of the outliers, but its good to double check and edit where needed.  Follow instructions on screen to remove data points.

Interactive Visibility Rejection

 

 Click left of axis to continue (or to goback to unzoomed view)
 Click right of axis to change smoothing length
 Click near datapoint to remove a time sample.
 Click below axis twice to specify a range to remove.
 Click above plot twice to specify a range to ZOOM up 

The same procedure of sigma clipping and interactive editing is then repeated for the closure phases and triple amplitudes on all closure triangles.

Data Calibration

The last step runs mirc_cal.script to calibrate the science data using the calibrator observations and estimated calibrator diameters.

IDL> .r mirc_cal.script 

The script will ask the user to choose the directory to process:

 

A plot of the target elevation vs time is shown:

Plot of target elevation vs. time.

 

Next the user will select science targets and then calibrator stars from the list of stars observed during the night.  The user can choose to process science targets and calibrators located in the same region of the sky by selecting only those stars from the list.

 

 

The routine will search the mirc_calibrator.txt file to find the calibrator diameters.  If a calibrator is not included in the file the routine will query the user for the diameter.  After the diameters are read from the file, the user will confirm the adopted diameters in the terminal window.

 
 CALIBRATOR DIAMETER FOUND: 
 target  udsize (mas), err: HD 25490     0.599000    0.0200000 (SED_Fit_HD25490)
 Would you rather use an alternative calibration model?
Y or <N> :  
 CALIBRATOR DIAMETER FOUND: 
 target  udsize (mas), err: HD 33256     0.655000    0.0180000 (SED_Fit_HD33256)
 Would you rather use an alternative calibration model?
Y or <N> :  

Next the user will choose the maximum averaging time for an observing block (default is 15 min).  The user might consider increasing the maximum averaging time if a single set of observations on a target took longer than 15 min.  The length of an observing block is typically between 10-30 minutes.

 

Enter the file extension for saving the calibrated oifits files. If you need to re-run the calibration, you can change the extension to avoid copying over existing files.

 

The program will make plots of the calibrator and target visibilities vs. time for each baseline.  The transfer function is overplotted.  The options printed to the terminal window allow the user to cycle through the wavelength channels and to flag bad data points.

Visibility vs. time for calibrators and science targets.

 

 Click left of axis to continue to next baseline
 Click near point to remove it (be careful: there is no UNDO)
 Click above graph to cycle through wavelengths
 Click to the right to change the xfer function parameters (timescale, sky_scale)
 Click time range below axis to see an alternative view of 
     multi-wavelength calibrated vis2 for data selection / bad flagging convenience 

It is useful to zoom in on a particular target or calibrator to view the visibilities over all wavelength channels. The first and last wavelength channels sometimes show low signal-to-noise, particularly for faint targets, and might need to be removed.

Zoomed in visibility plot showing all spectral channels for the selected target.

 

Click next point: 
WINDOW 1 OPTIONS:
  Click near point to remove it (be careful: no UNDO)
  Click left of axis to return
  Click below axis to mark ALL POINTS AS BAD (NUCLEAR OPTION) 

The same process is repeated for the closure phases (T3phi) and triple amplitudes (T3amp) for each closure triangle.

Triple amplitudes and phases vs. time.

 

When completed, mirc_cal.script will save calibrated MIRC_L2*oifits and summary MIRC_L2*ps files for each science target in the selected OIFITS directory. For each target, the program produces two calibrated OIFITS files: the AVG15m.oifits are averaged over the 15 min observing blocks set in mirc_cal.script and SPLIT.oifits splits each observing block into 2.5 minute chunks of time defined by the averaging time set in mirc_average.script.  The summary ps files contain plots of the uv coverage, vis2 vs baseline, vis2 vs wavelength for each baseline, CP vs wavelength, and T3amp vs wavelength for each closure triangle.  The script also produces a file called XFER.XCHAN.cal_*.ps that plots the transfer fucntion for each baseline and closure triangle during the night.

Applying Standard MIRC Calibration Errors

After the reduction process is complete, run oifits_prep.pro to apply the standard MIRC calibration errors to the visibilities, closure phases, and triple amplitudes.  The file created during this step should be used in subsequent analysis.

IDL> oifits_prep 

The program will ask the user to select the calibrated oifits file.  Multiple files can be selected if you want to merge separate observing blocks on the same target.  Hit OK to add another file or Cancel to continue.

 

Recommended calibration errors for 6T and 4T data are listed below:

Vis2 Error multiplier: 1.5 (default, use 1 to apply uniformly to all measurements)
Min Vis2 Additive Error: 0.001 (default)
Min Vis2 RELATIVE Error: 0.05 (use 0.05 default for MIRC-6T, use 0.10 for MIRC-4T no xchan)
t3amp Error multiplier: 1.5 (defaut, use 1 to apply uniformly to all measurements)
Min T3AMP Additive Error: 0.0002 (default)
Min T3AMP RELATIVE Error: 0.10 (default)
Min T3PHI ADDITIVE Error (degs): 0.30 (default) 

The user then enters the output file name.

 

The file created during this step can then be used in subsequent analysis. 

The reduction process is finished!  Good luck with the model fitting, imaging, and analysis!