L-Band data analysis hints

The AIPS Cookbook has detailed discussions on how to analyze the observations from a synthesis telescope like the VLA. The hints given below are guidelines and are expected to supplement what is given in this book. An understanding of the standard synthesis data analysis is a pre-requisite to follow these guidelines. The hints given below are particularly useful for producing deep continuum images from L-band multichannel observations, although the tasks mentioned are specific to AIPS.

• Load the data with FILLM (see the notes on FILLM). Specify DOWEIGHT = 1 to calculate UV-weights using the nominal sensitivity on each baseline, IF, and polarization (use DOUVCOMP = -1 to uncompress your data). Hereafter, always use DOCAL = 2 to apply the weights (especially in SPLIT).



• The 'channel 0' data which invariably accompanies the multichannel data is not useful for making wide-field, deep continuum images for reasons explained under Observing Strategy. It is recommended that the multichannel data be used for this purpose.



• Edit the data using a variety of tasks like QUACK, TVFLG, SPFLG, UVFLG, CLIP, and FLGIT.



• It is possible to carry out the first step of normal calibration of the data using the procedures given in the AIPS Cookbook. However, the following procedure is recommended. SPLIT the bandpass calibrator without any calibration (DOCALIB = -1). Note that at this stage the purpose is to estimate the bandpass corrections and it is not relevant to apply the weights (DOCALIB = 2) yet. Phase self-calibrate the bandpass calibrator. Calculate and check the bandpass corrections. Copy the bandpass table into the multisource data base. Apply the bandpass corrections to the original database including the bandpass calibrator since it may be used for phase calibration. This process ensures phase coherence while averaging across the band. Calculate and apply the amplitude and phase calibrations (DOCALIB = 2).



• For wide-field, 3D imaging, one needs an inner dense grid of facets which cover the primary beam out to almost the first null calculated so that 3D effects are small enough to be ignored. In addition one needs a number of outlying fields to cover the bright confusing sources which lie in the first sidelobe or beyond. One can find the outlying sources by making a tapered image of a very large field and/or using external catalogs like the NVSS.
  In the former method, once one has an image one needs an easy way to pick out and record the confusing sources. This can be done by displaying a box on the TV which covers the area sampled by the dense inner grid, so one can see what parts of the image to ignore. Then use TVMAX in AIPS to point at the outer confusing sources and measure their positions. AIPS uses a simple ASCII file as the input source for IMAGR to tell it where to put the outlying fields as well as the dense grid. Thus one can simply use the mouse and an editor to move the TVMAX fitted positions into the list of coordinates for AIPS.
  In the latter method, one can use tasks like FACES, SETFC, and FILEBOX along with the NVSS/WENSS catalogs to create an ascii BOXFILE which can be used in IMAGR for 3D imaging. All sources above a certain specified flux density limit will be included in this process. Of course, this uses a primary beam model even beyond the first nulls with which to multiply the catalogued source flux densities to decide to include them in the imaging process.



• Besides the external weights one also needs to pick the weighting schemes to use on the gridded uv data to get the best compromise between beam size and noise characteristics. The basic idea here is to obtain a dirty beam whose main beam is as close to the Gaussian (with which the CLEAN components will be convolved) as possible with rapidly decreasing sidelobes. Since the CLEAN components are convolved with the corresponding Gaussian and added to the residuals which have been convolved with the dirty beam, the above procedure helps to keep the difference in the treatment of CLEAN components and that of noise to a minimum. This requires trying several different sets of parameters before one starts. The best solution depends on the detailed uv sampling, so it is not possible in general to decide this in advance. The weighting scheme is usually a combination of UVBOX, ROBUST, and UVTAPER. One needs to do this to be sure one is getting the best combination before going through the very long reduction process to come.



• Once one has the full grid set up, one can make the initial set of images. One needs to clean this image to a moderate level so one can see all the sources. For deep imaging one normally will have multiple days with the same or similar uv coverage. For this initial image, one uses just one day's data to minimize the processing time.



• Without boxes to limit the areas CLEAN can consider, CLEAN will scatter power outside the regions with real sources to optimize its fit to the data. This will bias the noise low, bias the source flux densities low, and take longer to run than if one restricts the algorithm. It will also produce a non-optimum model for the selfcal process. There are other ways to solve this problem besides boxes such as rejecting isolated clean components by some method before restoring the image but boxes seems to be the most straight-forward, even if it is expensive in real time for the user.
  As a first try, one can use the task BOXES which will create CLEAN boxes containing all NVSS (or, any other user supplied catalog) sources above a certain user-specified threshold flux density that fall in the images specified by the data and the fields specified by the adverb BOXFILE. Subsequently, these boxes can be modified suitably.
  It is also possible to produce CLEAN boxes interactively. One displays the image, or part of it, on the TV and uses the graphics overlays to mark where the BOXES should go. The interactive program, FILEBOX, writes the marked positions in the same ASCII file as it stores the field centers and sizes. This makes editing this file very flexible. This process is very time consuming, however.



• After the BOXES, one reruns IMAGR, staring from scratch with the clean components and clean down to the current noise level. A typical setup for the task IMAGR is given here. Read the help file of IMAGR for explanations.



• One then selfcals the first days uv data using all the clean components and re-images the field. Usually, one does this the first time with a phase-only selfcal, re-images, and then does it again with a phase and amplitude selfcal and re-images again. After each re-imaging one checks the clean boxes, usually adding quite a few after the first selfcal. The amplitude and phase selfcal is usually done with a longer solution interval to increase the S/N since one is solving for more variables the second time. Besides fixing poor calibration the amplitude and phase step also forces the two IFs to be on approximately the same flux density scale, which minimizes error for bright sources.



• Once one has a good model one can subtract out the sources and do a uv editing step on the uv residuals. This subtraction can be done using UVSUB, use UVPLT to plot the data, and CLIP to remove remaining high points. Sometimes a pass through TVFLG is used here. Then one adds the MODEL back into the remaining data with UVSUB.



• Once one has a good image for the first data set, one can use this model to calibrate the rest of the data sets. Often just one amplitude and phase selfcal will be good enough. However, one needs to go through the previous step on each of the data sets separately using the first day's image as input.



• At this stage one wants to average the multiple days together to minimize the size of this full dataset. In AIPS one converts the times into HA's (TI2HA), edits the headers to make the individual datasets look like they all were observed on the same day in the same array (DOARRAY = 1), DBCONs all the datasets together (see the notes on combining), sorts them in baseline-time order, averages them optimally versus time using UBAVG, and sorts them back in time-baseline order for IMAGR. The task UBAVG runs through the data on each baseline and averages versus time in such a way as to produce less than an N percent error inside some radius from the phase center. This can make a significant difference in imaging since the uv-based CLEAN process is dominated by the number of uv points it must subtract a model from. This effort and book-keeping is well worth it because one can reduce a big database by a factor of 10 or more and thus speed up the imaging by almost the same factor.



• At this point one normally makes an image with the full dataset, adds and resets the boxes again, does a selfcal using the optimally averaged dataset, and then makes an "almost" final image.



• At this stage the residual pointing errors usually limit the dynamic range of the image. The beam squint produces pointing differences between the right and left circular feeds which are understood to first order. For sources near the field center this is not too big a problem; however, for bright sources far from the pointing center the effect can be large and can be a big problem, scattering sidelobes over large regions of the image and increasing significantly the average noise level. This can be helped by a good choice of weighting in IMAGR. The pointing error also varies with parallactic angle as the position of a bright source relative to the pointing error vector changes. One can take the following path to overcome these limitations.
  One can divide the data into small ranges of parallactic angle, by polarization and by IF. Then one images and selfcals each subset separately, forcing the same clean beam for each one. Then one measures the noise on each resulting image and stacks them together weighting them optimally. If one has, say 6 hours for each uv-track (not more than six hours to minimize the system temperature increase at low elevations), one can divide the data into two time intervals, two IFs and two polarizations, producing 8 images in the end which must be combined.



• Depending on the complexity of the sources being imaged, it is possible to use multi-resolution CLEAN in IMAGR. Alternatively, one can use VTESS which deconvolves using the principles of Maximum Entropy Method. Both multi-resolution CLEAN and VTESS are particularly useful if one is dealing with complicated, and extended sources. Both these tasks require quite a bit of experimentation to produce the best images.

A few changes have been made in FILLM, appearing in the AIPS 31DEC01 version and later. The BPARM adverb specifies corrections made to the initial CL table attached to the data. These corrections currently include that for opacity and antenna gain curve, both as a function of elevation and time. The defaults (BPARM(n) = 0) will get you our current best estimate for these two corrections. To turn these corrections off completely, set BPARM(1) = -1; BPARM(2) = -1. This will get you behavior similar to FILLM pre-31DEC01. Note that if you forgot to turn these corrections off, and don't really want them, there is no need to run FILLM again - simply use EXTDEST to get rid of CL table version 1 (that created by FILLM), and run INDXR on the dataset.

Please note that FILLM puts the gain and opacity information into the initial CL table. This means that in later calibration steps users will need to set DOCALIB = 2 even when GAINUSE = 1.

If you are not sure those corrections have been made, check your CL1 table by plotting the amplitude using SNPLT. If those values are different from 1.0, corrections have been made.

Within AIPS, uv-data needs to be weighted based on the measured VLA "nominal sensitivities" which are related to the system temperatures. Data weighting is important at L band where the system temperatures of different antennas are different and vary as a function of elevation. Because the weights are derived from system parameters measured during the observations and are applied to the visibilities, this method produces good estimates of the true weights irrespective of the source strength, frequency, or observing technique. Thus, it can be used for essentially any kind of observing.

In the AIPS 31DEC01 versions and later, the following is considered as standard data reduction methods for all observing bands.

1. Fill data with FILLM using DOWEIGHT > 0 (or equivalently CPARM(2) = 8).
   
   UV-weights will be calculated using the nominal sensitivity on each baseline, IF, and polarization. Note that weights are not fully retained if the data are stored in compressed form. In compressed format spectral channels, polarizations, and IFs are merged in weight at each time/baseline sample. If you want the absolute best sensitivity you can possibly achieve, in continuum or spectral line data, and there is more than one IF or polarization, then specify uncompressed data (DOUVCOMP = -1) when it is filled into AIPS.
   
   If you are asking the VLA analysts to fill your data and put it up for ftp, you must tell them to run FILLM with DOWEIGHT > 0 and specify if you want the data compressed or uncompressed.
   
   
2. Always use DOCAL = 2 in CALIB and SPLIT.
   
   Once you bring in the weights with FILLM, you must calibrate and apply them. If you try to split out the data without applying the calibrated weights, you will end up applying uncalibrated weights which will really mess things up. If you decide you don't want to use the weights, you must go back and re-run FILLM. Weights must be calibrated along with the visibilities. To calibrate the weights, DOCAL = 2 is an option in all calibration tasks. CALIB solutions will depend on the weights: If the weights on a baseline show that the data is very noisy, CALIB allows the solution on that baseline to vary more. This way, a bad antenna won't "drag around" other antennas. If you don't have the weights in CALIB, then you will tend to get more failed solutions on bad data (rather than having relatively poor solutions on down-weighted data). Weights are not used in the bandpass calibration; all spectral line channels have the same weight. If you don't split out the calibrated data before imaging, set DOCAL = 2 in IMAGR to apply the calibration to the uv data and the weights.



3. Check to make sure everything is OK.
   
   There are several routines that allow you to examine the weights:
   • PRTUV/UVPRT to look at the weights directly. In UVPRT you can apply calibration.
   • UVPLT to plot the weights versus elevation, time, etc., with and without DOCAL = 2 to examine the effect.
   • VPLOT to plot the weights for each baseline on separate pages.
   
   
   
4. Imaging the weighted data: Exercise caution when imaging weighted data. If a uv point from a relatively poor baseline is isolated in a cell in the uv plane, then Uniform weighting will increase the weight in this cell and negate all the hard work done to down-weight this uv point in the first place. Thus, Uniform weighting is generally not recommended for data that has been weighted using VLA nominal sensitivities. This is especially true for VLA A or B array data where the uv plane is relatively sparsely sampled. Thus, Natural or Robust weighting during the imaging process is generally better.

When combining two datasets, pay attention to the weights. Earlier version of AIPS did not have the DOWEIGHT = 1 option when running FILLM, so if you want to combine an older data set with a newer you must either re-reduce your old data using weights, or fill your new data with DOWEIGHT < 0.

After splitting each of your two data sets, you can combine those sets using DBCON. At this point it is very important that you check the relative weighting between the two datasets. Ideally, if the datasets were produced in an identical way (observationally and calibrationally), the resulting weights would be comparable. Therefore you would expect observations during bad weather to have lower weights in general. However, real data probably differ from this ideal situation, so it is important that you check that the weights make sense between the two datasets (using PRTUV, UVPRT or UVPLT). If the 'bad ' data has lower weights it is probably OK to DBCON your data with no re-weighting. If you however need to down-weight the bad data you must re-weight in DBCON - by some factor you consider appropriate.

At other occasions you might want some datasets to contribute more to the final image (e.g. a resolution issue), in which case you can also adjust the relative weighting using 'reweight' in DBCON. To determine the relative weighting, you may want to run IMAGR on your two sets before DBCON. Using the exact same parameters (UVWTFN, CELLSIZE, IMSIZE etc), IMAGR will report the 'sum of gridding weights' for the data sets respectively.

The National Radio Astronomy Observatory is a facility of the National Science Foundation operated under cooperative agreement by Associated Universities, Inc.