Sky Pattern Correction

Page author and contact: Rongpu Zhou

Changes in the DECam Community Pipeline (henceforth CP) between DR8 and DR9 of the Legacy Surveys (in particular the switch from dark sky flats to DES star flats) led to residual sky patterns with a sufficiently high amplitude to, e.g., significantly impact DESI target selection. The sky residual patterns exist in all three bands (\(grz\)), with the \(z\hbox{-}\mathrm{band}\) having the worst residuals. So, starting with DR9, we additionally correct DECaLS images to account for the residual sky pattern from the CP. Below, we include some representative \(z\hbox{-}\mathrm{band}\) exposures from DR8 imaging, and from DR9 imaging before and after applying the sky correction described on this page:

../../files/z_1076_image_603114_dr8.png../../files/z_1076_image_603114.png../../files/z_1076_image_603114_medianscale.png

Sky templates

The residual sky patterns are usually very stable for many nights. Therefore we correct for the sky patterns using sky templates that are generated for each night or multiple consecutive nights depending on the number of exposures available. All the DR9 exposures are grouped into "runs" of adjacent exposures of the same filter, and a sky template is generated for each run. We require that each run has at least 25 exposures that meet our quality cuts (described below), and if a night has insufficient exposures, the next (or a later) night is included until there are at least 25 exposures.

We also apply quality cuts for exposures to be used in stacking. We require that an exposure is not too close to another exposure in the run, that it meets some depth cut, and that it doesn't have too many bad CCDs. Images with PLVER=4.1 are excluded from template creation and from sky pattern correction, because such images used the older version of the CP sky-subtraction code and so are not affected by the residual patterns. Before stacking, the images are normalized by the median of the ccdskycounts value from all CCDs that cover the exposure of interest, and all DR8 sources (blobs) are masked. Note that the CP ood mask is not applied.

Finally, we obtain the sky templates by median-stacking the images and smoothing them with a Gaussian filter of \(\sigma=120\,\mathrm{pixels}\). Up to 50 exposures are used in the stack due to memory limitations. All the templates are visually inspected, and some of the problematic templates are regenerated after rejecting bad exposures, such as those that are contaminated by bright stars or satellite trails. Some example final sky templates are below for (from left-to-right) \(g\hbox{-}\mathrm{band}\), \(r\hbox{-}\mathrm{band}\) and \(z\hbox{-}\mathrm{band}\):

../../files/g_264_sky_template.png../../files/r_578_sky_template.png../../files/z_1076_sky_template.png

Each blob-masked CCD image is fit with its corresponding template, and the median scale over the exposure (rather than, i.e. the per-CCD scale) is used to apply the sky correction. For exposures outside the DR8 footprint or that cover fewer than 10 CCDs that have per-CCD fits, we use the median ccdskycounts value over the exposure as the sky scale.

The sky-corrected image can be calculated as:

\(\mathrm{image\_corrected} = \mathrm{image\_original} - \mathrm{skyscale} \times \mathrm{template}\)

Special steps were taken for certain \(r\hbox{-}\mathrm{band}\) images that were affected by edge glow in early DECam exposures. In addition, the following DECam CCDs also undergo special handling:

  • N10 (because for certain nights, only half of the CCD is stable)
  • N15 (because it is tarnished by a large hot spot on some nights)
  • S7 (because only half of the CCD is usable)
  • S30 (this CCD was disabled, and produced bad images around the time when it came back online)

Code

The code used to correct for sky patterns for DR9 of the Legacy Surveys is available on GitHub.

Additional Information and Products

Some additional plots characterizing the DECaLS DR9 sky corrections are available for DESI collaborators on the wiki. The various files generated to make the sky corrections are also available at NERSC:

  • Sky runs: /global/cfs/cdirs/desi/users/rongpu/dr9/sky_pattern/skyrunsgoodcountexpnumv48dr8
  • Sky templates: /global/cfs/cdirs/cosmo/work/legacysurvey/dr9m/calib/sky_pattern/sky_templates
  • Per-exposure sky scales: /global/cfs/cdirs/cosmo/work/legacysurvey/dr9m/calib/sky_pattern/sky-scales.fits