Clementine Image Processing with ISIS

PROCESSING CLEMENTINE DATA WITH ISIS

1. Acquiring Clementine data:

Clementine data have been archived by NASA/JPL on 88 CD-ROM volumes. These CD-ROMs can be ordered, or portions of the data may be viewed and/or ordered directly in a variety of formats from the Planetary Data System (PDS) Clementine Navigator Web page (at http://wundow.wustl.edu/clem/frontpage.html). Using the Navigator, the user can identify images based on their location, the camera and filter used, and a variety of other search parameters. A listing of desired images (with essential identifying information such as center latitude and longitude, filter, and volume id) can be prepared with the Navigator and used to find specific Clementine images. The user can place an order for images to be delivered via ftp or on CD-ROMS. Additional documentation about the Clementine mission and all acquired datasets are available at or through the same Web site.

In conducting a search for Clementine data, it is useful to know that the Clementine file names yield useful information. With a listing of desired images from the Navigator, one can identify the CD-ROM volume (e.g., VOL_ID=CL_0067) and the orbit number (e.g., REVolution=287) of the data from the filenames. For example, the image name LUA2111P.023 indicates that the target was the Moon (L), the filter was the UVVIS A (UA) filter, the image number was 2111, the data are stored on the luxxxxxp CD-ROM directory (this also indicates a latitude range) (P), and the orbit number was 23 (.023). With these parameters in hand, one can corroborate the listing produced by the Navigator.

2. Level 0 Processing: Importing Data

ISIS programs to run:

                clem2isis
                naiflab
                or 
                clem0

Initial processing of Clementine data ("Level 0" processing) involves converting the raw data to ISIS format (using the ISIS program "clem2isis" to create cube files) and updating the cube labels with navigation/engineering data (i.e., spacecraft positional data, or "SPICE" data from JPL's Navigation and Ancillary Information Facility or "NAIF") for subsequent radiometric and geometric processing (using "naiflab"). These steps have been combined in the ISIS procedure "clem0.pdf". The user can run these procedures individually for each raw Clementine data file, or he/she can simply generate a list of files , start tae, and run the list of images. For example:

unix>ls lua* > raw_lua.lis      (to create list)
unix>tae                        (to start tae) 
tae>clem0.pdf fromlist=raw_lua.lis (to run the ISIS procedure
                                "clem0.pdf" with the image list) 

3. Level 1 Processing: Radiometric Calibration and Geometric Control

Radiometric calibration is the primary goal of the next step in processing the Clementine data ("Level 1" processing). To date, the primary processing efforts have been focused on the Clementine UVVIS data, so this outline will concentrate on processing those data. The 750-nm data, occurring approximately in the middle of the wavelength range of UVVIS filters acquired by Clementine, were chosen as the control base for the first single-band global mosaic. Processing and production of geometrically controlled, full-color, UVVIS image global mosaics is currently in progress. Processing information for the Clementine NIR data is still under development.

UVVIS data processing: Merging different exposures

ISIS programs to run:

        calmrg (for lub data)
        or
        expmrg (for lua, luc, lud, lue, and luf data)

An important aspect of processing UVVIS Clementine data focuses on merging image pairs that were acquired at different exposures (i.e, at different gain/exposure/offset states). Because of the importance of this process for obtaining the best possible UVVIS Clementine image mosaic, merging of image pairs is performed within Level 1 processing.

Each UVVIS Clementine scene was taken at two different exposures, so two images exist for each target area. Typically, the short-exposure image is unsaturated for even the brightest lunar features, but has a poor signal-to-noise ratio (SNR) over dark regions, whereas the long-exposure image has a much better SNR over dark regions but can be saturated over bright craters or sunlit slopes. For any given target, the most desirable image to work with is the one with the highest dynamic range of data. To determine which scene has the most information, the exposure time, gain, and offset values must be considered. Generally the image with a long-exposure time is preferred, but occasionally that image has suffered loss of data due to saturation. The areas of saturated data can be replaced with the short- exposure image of the pair, and this involves merging the exposure pairs.

To simplify this process for the user, two ISIS procedures ("calmrg" and "expmrg") automatically identify the image with the highest dynamic range in a pair of Clementine images. Calmrg is used for the 750-nm filter images (lub) and expmrg is used for all other filters (lua, luc, lud, lue). Calmrg and expmrg perform very similar functions, with one essential difference as described below. The 750-nm images are a special case because they were processed first to create a global mosaic of geometrically controlled Clementine images. Thus, the most rigorously determined geometric data (including camera pointing data, etc.) exist for the 750-nm images. The processing steps outlined here are designed to take advantage of the high level of geometric precision that is available through ISIS for the 750-nm images. The user who intends to process all wavelengths of Clementine data for a given site is encouraged to begin processing with the 750-nm (lub) data.

To use calmrg or expmrg, the user must create a single-column ascii file containing the filenames of the Clementine level-0 cube files. This can be done with the UNIX single-column directory listing command, such as

unix>ls -1 *.cub > file.lis

Although the list can contain data from more than one orbit, make sure that the list has been sorted to include all matching pairs of images together. The orbit parameter is for log filename purposes only.

tae>calmrg fromlist=75cub.lis orbit=75 dellev0=y

Calmrg uses this list of cubes as input to "clempair" which divides the pairs of images into two output lists. All images are then run through "uvviscal" for radiometric calibration, which involves dark current subtraction, readout correction, and non-linearity and flat field corrections. The keyword values for the raw/original statistics are collected from the labels of each pair of images (RAW_MAXIMUM, RAW_MEAN, & RAW_STANDARD_DEVIATION). The long-exposure image is determined by (RAW_MEAN + RAW_STANDARD_DEVIATION) value in one file that is greater than the corresponding value for the pair. The merge decision tree is as follows:

1. If long-exp RAW_MAXIMUM is less than 255, use it only.
2. If long-exp (RAW_MEAN + RAW_STANDARD_DEVIATION) is greater than

   255, use short-exp only.

3. Otherwise, merge long-exp and short-exp by coregistering the two files and mosaicking the long-exp on top of the short-exp. The ISIS special pixel values HRS/HIS in the long-exp are replaced by the short-exp valid image data.

If images must be merged, several trimming operations are performed (using the "boxfilter" program, with "trim filters") on the saturated file (long-exp) to create smooth edges in the areas that will be replaced with the non-saturated (short-exp) file. The input files are also trimmed at the edges to remove invalid data.

(A) Level-1 Processing of 750-nm (lub) Data

ISIS programs to run for lub data:

        calmrg
        spicelab 

For the special case of 750-nm (lub) images, the labels of one of the two exposure files was identified in the merge processing for the geometric control base, and the geometric information (e.g., SPICE data and camera angles) was updated ONLY for that file. The list of product ID's that have updated camera angles for the base are stored in an ascii Master File (/clemcontrolid.lis); calmrg runs the program "clemprod" to check each input file against the master file of PRODUCT_ID values. If a calmrg input file has a PRODUCT_ID that matches one in the MASTERFL, it is held as control (or "node") during subsequent processing and coregistration of the pair and its labels will be unaltered during the merging process. Calmrg fails if neither of the long/short exposure files is found in the Master File. Calmrg produces ".lis" and ".prt" files to track its progress through the processing steps for data from each orbit. The user can examine: "calmrg_fit_#orb.lis" to examine the statistics used in the image merging process (i.e., the correlation coefficients for the coregistration process); "long_exp_#orb.lis" and "short_ exp_#orb.lis" for a listing of the long- and short-exposure images identified for each orbits; "calmrg#orb.prt" for a log of processing steps performed (as in the standard ISIS "print.prt" file); and "ercm#orb.prt" for information on any warnings and/or errors encountered during calmrg processing. Images that have been processed using calmrg have attached filename extensions of ".cub.lev1".

To complete the Level-1 creation of a controlled Clementine image mosaic for 750-nm (lub) images, the user must update the labels with the geometric information (i.e., SPICE data and camera angles) for each file. Currently these geometric data are available for orbits 32-300. "Absolute truth" for geometric control is defined as the positional data for the Apollo landing sites. Note that the previous search of the master list has ensured that the Level-1 files produced by calmrg match the PRODUCT_ID's contained in the ISIS Clementine (750/lub) SPICE file for updated geometric control. After calmrg, the user should run "spicelab" on each Level-1 file to update camera angles for each file in the mosaic. *** Watch out for spicelab errors for "no record found" on your screen and in the print.prt log file---this indicates that the information was not found in the Master File, and the labels were not updated. The user should check to see if the other member of the image pair is located in the MasterFile. This can be done with the UNIX/SHELL command grep. Example: grep lub5987r.284 /clemcontrolid.lis Any lub image pair that is not in the MASTERFL needs to be processed thru expmrg as with the other filters. The camera angles will need to be updated through the process of selecting matchpoints and applying jigsaw with neighboring images where calmrg was successful. These neighboring images would be held as the geometric "truth".

tae>spicelab to=filename.cub

(B) Level-1 Processing of Color (lua, luc, lud, lue, luf) Data

ISIS programs to run: (Please read the help for each individual program for specific details of each)

        expmrg
        shiftcube
        matchpt 
        jigsaw

For Level-1 processing of data from other Clementine filters (lua, luc, lud, lue, luf), expmrg performs a very similar function to calmrg in identifying the image with the highest dynamic range. The primary difference between calmrg and expmrg is that the latter program does not refer to a master file; it identifies the long-exposure file, carries it through processing, and merges it with the paired image if necessary. Note that expmrg must be used for any UVVIS data from orbits earlier than #32 or later than #300 because they were not included in the production of the global 750-nm mosaic. A few 300 orbits were processed to fill gaps. These can be found in the MASTERFL mentioned above.

Establishing geometric control for other color data requires identification of match points between the corresponding images to the lub filter images with the updated camera angles. First, the user may need to examine the merged, Level-1 images to determine the latitude range, longitude range, and center longitude of the selected images. One useful ISIS program is "clemgroup"; given a single-column ascii list of one filter, the output of clemgroup is a five column list that contains all corresponding filters in each row.

After merging exposure pairs for all filters, running spicelab on the selected lub images for geometric control, and identifying the correlation between each individual frame and its matching filter images, the user can update the camera angles for the other filters. The ISIS programs used to accomplish this are "shiftcube" (first order camera angle adjustment of the filters to lub data), "matchpt" (automatically select match points between the pairs of filters to the lub data), and "jigsaw" (final adjustment of the camera angles of the filter images relative to the lub data that should be 'held' as "truth"-refer to hold parameter in jigsaw). This is also the process to be applied for the lub images that are not included in the MASTERFL.

In addition to updating the camera angles of the remaining filters, there is the option to take the merged lua, luc, lud, and lue images and further subpixel register the image data to the lub image. This requires running the program called 'coreg'. The first input file is the image that will be adjusted, the second file is the "truth-lub" image. A recommended goodfit value is 0.8. The image data is adjusted by a line/sample translation relative to the lub image at a subpixel accuracy. The output lua, luc, lud, and lue images must then be stacked with the lub image in a single cube, retaining the lub image labels. The lub image labels that contain the appropriate geometry information is then used to project the entire cube to a map projection for mosaicking. The stacking is performed by the application 'cubeit'. Example: cubeit from=(lua_coreg,lub,luc_coreg,lud_coreg,lue_coreg) labfile=2 The parameter labfile=2 specifies the propagation of the lub image labels to the output cube. *NOTE*-If this option is chosen, the photometric correction should be applied to the all of the individual images before coreg and cubeit. See below for this step.

References: 1) Eliason, E.M., McEwen, A.S, Robinson, M.S., Lee, E.M., Becker, T.L., Gaddis, L., Weller, L.A., Isbell, C.E., Shinaman, J.R., Duxbury, T., Malaret, E.; Clementine: A Global Multi-Spectral Map of the Moon from the Clementine UVVIS Imaging Instrument: Proceedings of the 30th Lunar and Planetary Conference, in press, 1999.

2) Isbell, C.E., Elliason, E.M., Adams, K.C., Becker, T.L., Bennett, A.L., Lee, E.M., McEwen, A.S., Robinson, M.S., Shinaman, J.R., Weller, L.A.; Clementine: A Multi-spectral Digitial Image Model Archive of the Moon: Proceedings of the 30th Lunar and Planetary Conference, in press, 1999.

(C) NIR data processing

Procedures for processing and calibrating Clementine NIR data are currently being developed and tested. This document will be updated when the procedures are finalized.

4. Level 2 (Image Map Projection) and 3 Processing (Photometry):

4.1 Photometric correction

ISIS procedure to run:

        clemphot
        

This step may be run before or after the image map projection step. It is recommended to run the photometric application to the individual images before stacking into a single output cube, if this option was chosen. The photometric correction of the Clementine data is accomplished via the clemphot procedure which applies the programs photompr and photom. Photompr is used first to create a solar correction data file (TFILE) and this is passed to photom to perform the correction. The TFILE (sun.dat) consists of multiplicative and additive values that will be applied to the input cube in PHOTOM. For processing of the UVVIS Clementine data, the "moonpr" photometric function has been empirically determined to provide the best photometric correction (McEwen, 1996). Correction factors are built into the clemphot procedure for each filter for the 'moonpr' correction function. The result of this process is photometrically corrected images with reflectance values at a phase angle of 30 degrees. The photometric function used here is very sensitive to phase angles below 15-20 degrees. Refer to the help in clemphot for further details and references for the photometric function.

4.2 Map Projection

ISIS programs to run:

        plansinu
        geom

To reproject the Clementine data from the original point-perspective view, the ISIS programs "plansinu" and "geom" are used. Plansinu creates the geometric transformation data file (TFILE) for geom to reproject the Clementine Level-1 images to a Sinusoidal Equal-Area projection and correct camera distortions. Geom then performs the actual transformation of the cube. The user must specify the same map resolution (km or deg parameter in plansinu) for all images that are to be included in the output mosaic. The same center longitude (clon) of the mosaic coverage must be entered in plansinu for each image. This value can be determined by the longitude range of the entire image set included in the mosaic. The default bilinear interpolation option in geom is recommended. The input files for plansinu and geom are either the individual lua->lue images if the camera angles were updated to the lub, or the stacked cube if the coreg option was taken.

5. Level 4 Processing: Creating a Mosaic

ISIS program to run:

mosaic

The ISIS program "mosaic" is used to construct the combined images for a given area for each filter. To begin creating a mosaic, the "init" parameter is set to "yes" to initialize the output mosaic file, and the user must define latitude and longitude boundaries. After the larger mosaic file is created, additional files may be added, and the parameter "top" is set to "yes" to place the images on top in the mosaicking sequence. Note that month1 images (orbit 32-164) were acquired at a higher resolution at the south pole and month2 images are of higher resolution at the north pole. This means that when compiling a mosaic of data from northern latitudes (defined as 0 degrees or higher), the user should select data from orbits higher than #164 to be placed on top (and vice versa, for southern latitudes of 0 degrees or lower). Images with the highest resolution should always be placed on top. For a northern mosaic, if processing proceeds from lower to higher orbit numbers, then the higher resolution images will be placed on top as desired.

6. Brightness Matching: Cosmetic Operations

ISIS programs to run:

        blend (for small numbers of images)
        or
        b4equal and equalizer
        then
        mosaic

The mosaic produced using these methods will be processed at a basic or minimal level, and will thus be the most scientifically accurate. For cosmetic purposes only, the user may wish to match overall brightness between images (especially for coverages acquired in different months) and then reproduce the mosaic. To perform frame brightness matching, a linear normalization application is recommended. The ISIS procedures "blend" (for pairs of images at a time) or "b4equal" and "equalizer" (in combination, for larger numbers of images) will calculate multiplicative and additive values to apply to images. For a large number of images, the user should first run b4equal which calculates statistics of the brightness values for each image. Then run equalizer to determine the multiplicative and additive values; Before running the pdf that equalizer generates, look at the contents to see if mult values are less than ~0.75, if so, do not use them. To possibly improve the mult values, select a high-contrast image to hold as control when running equalizer. Valid values of mult should be between 0.75 and 1.0. To run the pdf, simply type the pdf filename at the tae prompt. The cosmetic operations should be run on single-band cubes, one filter at a time. The stacked cubes can be separated out using the sfrom parameter in most of ISIS applications (dsk2dsk, for example). Refer to more detailed help on sfrom by typing 'help sfrom' at the tae prompt.