High Frequency Data Reduction Hints

Table of Contents:

1. Data reduction path
2. Important notes on FILLM
3. Data weighting
4. Opacity corrections
5. Gain curve application
6. Using clean component models for flux calibration
7. Combining data sets

Data reduction path:

Due to increased resolution in addition to phase instabilities introduced by the ionosphere, K and Q band data reduction (especially when VLA is in A- or B-array configuration) should be done carefully. A detailed description can be found in the VLA Calibration Manual, under Hints for Reducing High Frequency VLA Data in AIPS, here are a few guidelines:

• Load data with FILLM (see the notes on FILLM), applying opacity and gain curve corrections (applied by default in FILLM in the AIPS 31DEC01 version and later). Specify DOWEIGHT = 1 to calculate UV-weights using the nominal sensitivity on each baseline, IF, and polarisation (use DOUVCOMP = -1 to not compress your data). Hereafter, always use DOCAL = 2 to apply the weights. Also set CPARM(8) = 0.05 to create a CL-table that can be interpolated over very short intervals, hopefully short enough to cover the atmospheric phase fluctuations accurately.
• Run SETJY on your absolute flux density calibrator 3C286 or 3C48.
• Run CALIB, at this stage to correct for phase only, with a small solution interval (depending on your signal to noise, e.g. 20 seconds) on all your calibrator sources. You might want to obtain a model for 3C286 or 3C48 from the FITS files in the table below and run CALIB on these sources with the model separately.
• Once you are satisfied with the phases in your SN-table, apply the phase corrections to minimize decorrelation in your calibrator scans before you determine the absolute flux density scale. Hence, create a CL table with CLCAL.
• Re-run CALIB with the corrected phases to obtain the flux density scale, using SOLMODE 'A&P' (note that you preferably want to use models for 3C286 or 3C48). Here the CALSOUR should be your calibrators other than 3C286/3C48!
• And run CALIB again for the absolute flux density calibrator 3C286, or 3C48, using a model and the previous/above values used.
• Run GETJY to obtain the secondary calibrator flux densities, check those values carefully.
• The final flux density calibration table is obtained by running CLCAL again.
• From here you are probably set to continue as usual according to the 'standard recipe'.

Important notes on FILLM:

A few changes has been done to FILLM, appearing in the AIPS 31DEC01 version and later. The BPARM adverb specifies corrections made to the initial CL table attached to the data. These corrections currently include that for opacity and antenna gain curve, both as a function of elevation and time. The defaults (BPARM(n) = 0) will get you our current best estimate for these two corrections. To turn these corrections off completely, set BPARM(1) = -1; BPARM(2) = -1. This will get you behaviour similar to FILLM pre-31DEC01. Note that if you forgot to turn these corrections off, and don't really want them, there is no need to run FILLM again - simply use EXTDEST to get rid of CL table version 1 (that created by FILLM), and run INDXR on the dataset.

Please note that FILLM puts the gain and opacity information into the initial CL table. This means that in later calibration steps users will need to set DOCALIB = 2 even when GAINUSE = 1.

If you are not sure those corrections have been made, check your CL1 table by plotting the amplitude using SNPLT. If those values are different from 1.0, corrections have been made.

Data weighting:

Within AIPS, uv-data needs to be weighted based on measured VLA "nominal sensitivities" (which are related to the system temperatures). Data weighting is especially important at Q and K bands where the receivers on each antenna can be quite different, so that weights will vary between IFs, polarisations, and baselines by more than a factor of 2. Because the weights are derived from system parameters measured during the observations and are applied to the visibilities, this method produces good estimates of the true weights irrespective of the source strength, frequency, or observing technique. Thus, it can be used for essentially any kind of observing.

In the AIPS 31DEC01 versions and later, the following is considered as standard data reduction methods for all observing bands.

1. Fill data with FILLM using DOWEIGHT > 0 (or equivalently cparm(2) = 8).

   UV-weights will be calculated using the nominal sensitivity on each baseline, IF, and polarisation. Note that weights are not fully retained if the data are stored in compressed form (in compressed format spectral channels, polarisations, and IFs are merged in weight at each time/baseline sample). If you want the absolute best sensitivity you can possibly achieve, in continuum or spectral line data, and there is more than one IF or polarisation, then specify uncompressed data (DOUVCOMP = -1) when it is filled into AIPS.

   If you are asking the VLA analysts to fill your data and put it up for ftp, you must tell them to run FILLM with DOWEIGHT > 0 and specify if you want the data compressed or uncompressed.

2. Always use DOCAL = 2 in CALIB and SPLIT.

   Once you bring in the weights with FILLM, you must calibrate and apply them. If you try to split out the data without applying the calibrated weights, you will end up applying uncalibrated weights which will really mess things up. If you decide you don't want to use the weights, you must go back and re-run FILLM. Weights must be calibrated along with the visibilities. To calibrate the weights, DOCAL = 2 is an option in all calibration tasks. CALIB solutions will depend on the weights: If the weights on a baseline show that the data is very noisy, CALIB allows the solution on that baseline to vary more. This way, a bad antenna won't "drag around" other antennas. If you don't have the weights in CALIB, then you will tend to get more failed solutions on bad data (rather than having relatively poor solutions on down-weighted data). Weights are not used in the bandpass calibration; all spectral line channels have the same weight. If you don't split out the calibrated data before imaging, set DOCAL = 2 in IMAGR to apply the calibration to the uv data and the weights.

3. Check to make sure everything is OK.

   There are several routines that allow you to examine the weights:

   • PRTUV/UVPRT to look at the weights directly. In UVPRT you can apply calibration.
   • UVPLT to plot the weights versus elevation, time, etc., with and without DOCAL = 2 to examine the effect.
   • VPLOT to plot the weights for each baseline on separate pages.

4. NOTE on imaging weighted data:

   Exercise caution when imaging weighted data. If a uv point from a relatively poor baseline is isolated in a cell in the uv plane, then Uniform weighting will increase the weight in this cell and negate all the hard work done to down-weight this uv point in the first place. Thus, Uniform weighting is generally not recommended for data that has been weighted using VLA nominal sensitivities. This is especially true for VLA A or B array data where the uv plane is relatively sparsely sampled. Thus, Natural or Robust weighting during the imaging process is generally better.

Opacity corrections:

One effect at high frequencies is the variation of total opacity as a function of elevation. Under good conditions a typical K-band zenith opacity could be 0.05, but varies a lot depending on the weather. The sky opacity at Q-band is dominated by the atmospheric oxygen line at 60 GHz, and so unlike the phase fluctuations which are determined by water, the atmospheric attenuation does not vary greatly under different weather conditions. Opacity corrections are thus more important at K-band.

In the AIPS 31DEC01 version and later, FILLM automatically loads weather data recorded at the VLA. From this surface weather data FILLM determines the zenith opacity as a function of time, and puts information into the initial CL table. This means that in later calibration steps users will need to set DOCALIB = 2 even when GAINUSE = 1. The opacity correction can be turned off by setting BPARM(1) = -1.

If you prefer to enter opacity corrections derived from a tipping scan, you should define BPARM(1) = -1 when loading the data using FILLM. Using the task CLCOR (opcode 'OPAC') you can apply your derived opacity value.

Gain curve application:

In order to correct the antennae gain for the function of elevation, the AIPS 31DEC01 version (and later) of FILLM applies gain curve correction by default (BPARM(2) = 0). Manual gain curve corrections should then not be applied, unless you suppress those corrections (by specifying BPARM(2) = -1). Check your CL1 table to make sure if gain curve corrections have been made or not.

If you wish to apply gain curve corrections manually, you can use the AIPS task CLCOR (setting opcode = 'gain'). RUNFILE scripts to do this are available at Steve Myers Gain Curve Archive. Select the appropriate RUNFILE and copy that to the directory where you start AIPS from, and set the AIPS VERSION = 'PWD'. To run the script, type RUN FILENAME. The sequence of procedures to run is explained in the header to the RUNFILE.

Using clean component models for flux calibration:

Models for the primary flux calibrators 3C48, 3C138, 3C147 and 3C286 in A and A+B array have been produced (see Claire Chandler's calibration page), and can be used as models in CALIB. Currently the following FITS files are available:

These models are calibrated to the new (1999.2) flux density scale. When using clean component models for flux calibration, be aware that the flux from the clan components are used instead of those set by SETJY. However, the fluxes have to be non-zero in the SU table for CALIB to believe that the flux is known, hence SETJY must be run anyway.

Combining data sets:

When combining two datasets, pay attention to the weights. Earlier version of AIPS did not have the DOWEIGHT = 1 option when running FILLM (see above), so if you want to combine an older data set with a newer you must either re-reduce your old data using weights, or fill your new data with DOWEIGHT < 0.

After splitting each of your two data sets, you can combine those sets using DBCON. At this point it is very important that you check the relative weighting between the two datasets. Ideally, if the datasets were produced in an identical way (observationally and calibrationally), the resulting weights would be comparable. Therefore you would expect observations during bad weather to have lower weights in general. However, real data probably differ from this ideal situation, so it is important that you check that the weights make sense between the two datasets (using PRTUV, UVPRT or UVPLT). If the 'bad ' data has lower weights it is probably ok to DBCON your data with no re-weighting. If you however need to down-weight the bad data you must re-weight in DBCON - by some factor you consider appropriate.

At other occasions you might want some datasets to contribute more to the final image (e.g. a resolution issue), in which case you can also adjust the relative weighting using 'reweight' in DBCON. To determine the relative weighting, you may want to run IMAGR on your two sets before DBCON. Using the exact same parameters (UVWTFN, CELLSIZE, IMSIZE etc), IMAGR will report the 'sum of gridding weights' for the data sets respectively.