APP-x Data Set Description

FTP Site CASPR User's Guide

Data Set Overview

Retrieval results are available as monthly mean images and monthly mean area-average values covering the entire Arctic or Antarctica. Daily images and means are not currently online due to the huge data volume. The pixel size is 25 x 25 km and the period of coverage is 1982 - 2004. Some days may be missing because of missing or invalid AVHRR data. The local solar times are 1400 (Arctic and Antarctic), 0400 (Arctic), and 0200 (Antarctic).

The ftp site contains the monthly mean image result files and monthly mean area-average text files, one latitude image, one longitude image, and monthly mean cloud and Sun glint mask images. All files are stored in netCDF format and compressed by gzip command. The file naming conventions will be described in the Data Set Naming Conventions section below. Images are 361 x 361 pixels for the Arctic and 321 x 321 pixels for the Antarctica in size, with contents and data types as described in the Image Parameter Files section below. The means files are described in the Mean Parameter Files section.

The land mask infomation is contained in image parameter files as indicated by parameter sequential number 9 in Table 1, where a value of 254 indicates land and a value of 0 indicates non-land (ocean or lake). The latitude and longitude files are 361 x 361 (Arctic) and 321 x 321 (Antarctica) pixels in size, short integer (two byte) values in degrees times 100. Latitude are positive north of the equator and negative to the south. Longitudes are positive east of the Prime Meridian and negative to the west (0 to +-180 degrees times 100). These files are also stored in netCDF format and compressed by gzip command too.

Instrument Description

Data from the Advanced Very High Resolution Radiometer (AVHRR), on-board NOAA polar-orbiting satellites, are used in this study.  The specific data set is a product of the AVHRR Polar Pathfinder (APP) project. The APP data are twice-daily composites available at 5 km pixel size for June 1981- December 2004. The current series of NOAA Polar Orbiting Environmental Satellites (POES) has been operational since mid-1978. NOAA-7,-9,-11, -14, and -16 satellite data were used for the AVHRR Polar Pathfinder Twice-Daily 5 km EASE-Grid Composites. The primary instruments aboard this third generation of satellites (TIROS-N, and NOAA-6 through NOAA-16) are the AVHRR sensor and the TIROS Operational Vertical Sounder (TOVS).

The AVHRR sensor is a five-channel system. The scan mirror collects earth observation data during a discrete part of the scan cycle. The scan mirror observes the scene below the spacecraft in a continuous line from horizon to horizon as it rotates. Energy from the scene is collected by a telescope and separated according to wavelength by beam splitters. Signals are amplified, filtered, and applied to the 10-bit analog/digital converter, which samples all five channels simultaneously. The five channels, numbered sequentially 1...5, cover the following spectral ranges: 0.58 - 0.68 um (visible), 0.725 - 1.05 um (reflected infrared), 1.58-1.64 um (near-infrared, daytime only for NOAA 16 before May 1,2003)/3.55 - 3.92 um (reflected/thermal infared), 10.3 - 11.3 um (thermal infrared), 11.5 - 12.5 um (thermal infrared).

The 10-bit resolution digital data is processed to create direct readout of High Resolution Picture Transmission (HRPT) data, Automatic Picture Transmission (APT) data, 4 km Global Area Coverage (GAC) data, and 1 km Local Area Coverage (LAC) data, to ground stations throughout the world. The AVHRR instrument scans in the across-track direction with a continuously rotating scan mirror, viewing a swath of over 100 degrees and up to 55 degrees off-nadir. Spatial resolution is approximately 1.1 km when the view is at nadir. Scanning to 55 degrees (68 degrees satellite zenith angle relative to the earth's surface) off nadir results in a ground resolution of over 2.4 km by 6.5 km at the maximum off-nadir position. The all of five channel signals received will be calibrated by calibration methods (Cracknell, 1997).

Data Processing

The 5 km APP data cover the period June 1981- December 2004, but these data are subsampled to 25 km (every 5th pixel) and the time series is shortened slightly, covering January 1982 - December 2004. The APP standard products are clear sky surface temperature and broadband albedo, a cloud mask, sea ice motion, and the calibrated, geolocated channel data and viewing/illumination geometry (Maslanik et al., 2000;  Maslanik et al., 1998; Meier 1997).  We have extended this product set to include the all sky surface skin temperature and broadband albedo, cloud information (mask, type, particle phase, effective radius, optical depth, temperature, and pressure), Sun glint mask and radiative fluxes, sea ice properties (mask, concentration/extent, thickness/age),  and cloud forcing using algorithms in the Cloud and Surface Parameter Retrieval (CASPR) system (Key, 2009). The calculation of cloudy sky surface skin temperature is based on an empirical relationship between the clear sky surface skin temperature, wind speed, and solar zenith angle (daytime). The cloudy sky broadband surface albedo is determined using the clear sky broadband albedo (interpolated from nearby pixels) adjusted by the cloud optical depth and the solar zenith angle.  Radiative fluxes are computed in CASPR using FluxNet (Key and Schweiger, 1998).  Sea ice concentration/extent is determined using tie-point algorithm, and ice thikcness/age is derived using One-dimensional Thermaldynamic Ice Model (OTIM) (Wang, et al., 2009) and WMO ice type classification. See Key (2009) and references therein for more information on the algorithms and their validation.

Image Parameter Files

One netCDF output file is created corresponding to each input image. The format for daily and montly mean image files is the same. First is a list of the parameters that the file contains. The values are: 1=the parameter is present, 0=not present, as a 29-element short integer array. The parameters are given in Table 1 below. In this data set, all parameters are included in the file. Next are two short integer values that specify the size (columns by rows) of the data arrays that follow. The image size in this data set is 361 x 361 (columns by rows) for the Arctic and 321 x 321 for the Antarctica, with 25 km pixels. Then come the 30 parameters as short integer (2-byte) arrays, except for the cloud phase, cloud type, and the refined surface type mask which are byte for daily images, and for monthly mean images of the cloud phase (byte type) indicating the relative frequency of cloud phase occurrence of water or ice phase with the value ranged from 0~100, and cloud type (byte type) indicating the relative frequency of cirrus cloud occurrence in all cloud types with the value ranged from 0~100. After the image data comes the number of profile levels (short integer), followed by the temperature, humidity, and pressure profiles (float array) that are nearest the center of the image. Each profile has the specified number of levels.

Clear pixels in cloud parameter arrays and cloudy pixels in clear arrays are generally MISSING, but that is not guaranteed. MISSING is also used for any invalid pixel. MISSING values should, of course, be ignored. If no clear sky pixels are found for one or more surface types in the image, all results are set to MISSING. No cloud or flux calculations are done for solar zenith angles between DARKZEN (88 degrees) and 90 degrees, so the corresponding pixels values are MISSING. The value for MISSING is 9999.0.

Note: There is no cloud and Sun glint masks in the *.params.cdf files. However, cloud mask can be created directly from the cloud phase array or access the cloud mask data set to get them. Values of 0 and 1 in the cloud mask data set indicate clear and cloudy, respectively; a value of 199 indicates invalid data for daily images. The monthly cloud mask images are also the frequency of the cloud occurrence during that month, which can be obtained in this data set too.
 

Table 1. Output parameters, their integer reference numbers, and output factor which is a multiplier. For example, surface temperature is stored in degree        Kelvin times 10, as an integer (bytarr = 1 byte, intarr = 2 bytes, fltarr = 4 bytes). Note: parameters 23-30 are  currently not available, missing value assigned.

Reference 

Number

Description
Internal Units/Range
Output Factor 

and Type

1 Surface temperature, all-sky K x10, intarr
2 Broadband albedo, all-sky Unitless, [0,1] x1000, intarr
3 Cloud particle effective radius microns [water: 2.5-20, ice: 20-120] x10, intarr
4 Cloud visible, vertical optical depth Unitless [water: 0-150, ice: 0-50] x10, intarr
5 Cloud particle phase 0=liquid, 1=ice Bytarr
6 Cloud top temperature K x10, intarr
7 Cloud top pressure mb x10, intarr
8 Precipitable water (from radiosonde data) cm x10, intarr
9 Surface type mask (crude) open ocean=0, snow-free land=254, snow=4, ice=3, Bytarr
10 Downwelling shortwave flux at the surface W m-2 x10, intarr
11 Downwelling longwave flux at the surface W m-2 x10, intarr
12 Upwelling shortwave flux at the surface W m-2 x10, intarr
13 Upwelling longwave flux at the surface W m-2 x10, intarr
14 Downwelling shortwave flux at the TOA W m-2 x10, intarr
15 Upwelling shortwave flux at the TOA W m-2 x10, intarr
16 Upwelling longwave flux at the TOA W m-2 x10, intarr
17 Shortwave cloud forcing at the surface W m-2 x10, intarr
18 Longwave cloud forcing at the surface W m-2 x10, intarr
19
Cloud type
00 - 99, two cloud type classifications from CASPR and CLAVR Bytarr
20
Ice concentration
Unitless, [0,100]
x10, intarr
21
Ice thickness
m
x100, intarr
22
Ice age
0=-fee, 1=new, 2=grey, 3=grey-white, 4=thin FY, 5=Median FY, 6=thick FY, 7=Old
Bytarr
23
Ice motion
m s-2
x100, intarr
24
Ice density
kg m-3
x10, intarr
25
Ice salinity
ppm
x10, intarr
26
Snow fraction
Unitless, [0,100]
x10, intarr
27
Snow depth
m
x100, intarr
28
Snow type
TBD
Bytarr
29
Snow water equivalent amount
kg m-3 x10, intarr
30
Snow density
kg m-3 x10, intarr

Mean Parameter Files

Another file type contains the entire area-average mean value of each parameter for each image. This is a text file with the date, parameter mean values, and parameter standard deviations for each of the standard parameters (Table 1) plus cloud amount and Sun glint amount. If more than one image was processed then data from all images is present. The file is structured as follows. The first record is the list of binary values corresponding to each parameter except for ice thickness and age that share one binary value, 1 if computed, 0 if not. The second record is the number of cases (equals the number of images processed) that the file contains. The third record contains the date, which are year, month, day and hour. The fourth record contains the mean values of the parameters (32 total). The fifth record contains the standard deviations for all parameters. The records for the means and standard deviations will have values for every parameter, but those that were not computed will have values of MISSING. This includes the situation where no clear sky pixels were found for one or more of the surface types present. An example is:

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
355
1982 1 1 4.00
255.080 9999.00 24.7873 16.4422 0.529512 248.905
659.184 0.492330 14.7740 0.00000 220.012 0.00000
246.725 0.00000 0.00000 184.294 0.00000 42.2505
0.520151 0.766329 0.00000 47.1034 0.448512 1.86857
9999.00 9999.00 9999.00 9999.00 9999.00 9999.00
9999.00 9999.00
17.7031 9999.00 23.2415 16.7399 0.499188 13.1449
185.106 0.466179 54.3855 0.00000 66.9068 0.00000
66.3277 0.00000 0.00000 28.6767 0.00000 27.8601
1.34608e-05 0.423207 0.00000 47.2468 0.731315 2.69890
9999.00 9999.00 9999.00 9999.00 9999.00 9999.00
9999.00 9999.00
 ...

Data Set Naming Conventions

The monthly mean parameter image naming convention is mean_yyyymmdd_tttt.params.cdf.gz, where monthly means it is monthly mean data set, yyyy is 4-digit year, mm is 2-digit month, dd is 2 digit day which is always 99 for monthly mean data set, and tttt is the time of day (1400, 0400 or 0200). The params means parameter data file, cdf indicates netCDF format file, gz denotes it was gzipped. For example, mean_19970999_1400.params.cdf.gz is a monthly mean parameter image for September 1997 at local solar time 1400. It was saved in netCDF file format, and compressed by gzip. The monthly mean cloud and Sun glint mask image naming conventions are as above but with "cmask" and "smask" in place of "params", respectively.

Read Routines

The routines provided for reading the netCDF files were written in IDL (Interactive Data Language from Research Systems, Inc., Boulder, Colorado). Of course, any computer programming language that has access to netCDF libraries can be used. The IDL procedures are provided in the read directory.

Name Purpose
read_netcdf_latlon Read latitude and longitude netCDF files.
read_netcdf_cloudmask Read cloud mask netCDF file.
read_netcdf_params Read parameter image netCDF file.
read_netcdf_means Read parameter statistical mean and standard deviation file.

Additional documentation is provided in the individual procedures.

NOTE: The APP-x netCDF files are compressed, and must be uncompressed before reading.

References

Cracknell, A.P. 1997.The Advanced Very High Resolution Radiometer. London: Taylor & Francis.

Key, J., 2001, The Cloud and Surface Parameter Retrieval (CASPR) System for Polar AVHRR Data User's Guide. Space Science and Engineering Center, University of Wisconsin, Madison, WI, 62 pp.

Key, J. and A.J. Schweiger, 1998, Tools for atmospheric radiative transfer: Streamer and FluxNet, Computers and Geosciences, 24(5), 443-451.

Maslanik, J.A., J. Key, C. Fowler, T. Nguyen, 2000. AVHRR-derived regional cloud and surface conditions during SHEBA and FIRE-ACE. J. Geophys.l Res. , in press.

Maslanik, J., C. Fowler, J. Key, T. Scambos, T. Hutchinson, and W. Emery, 1998. AVHRR-based Polar Pathfinder products for modeling applications. Annals of Glaciology 25:388-392.

Meier, W.N., J.A. Maslanik, J.R. Key, and C.W. W. Fowler. 1997. Multiparameter AVHRR-derived products for Arctic climate studies, Earth Interactions , Vol. 1.