Project

General

Profile

Feature #2358

ISIS capability to work with Kaguya MI L3C5 data

Added by William Farrand over 2 years ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Applications
Target version:
Impact:

Both kaguyami2isis and mimap2isis will have new error messages. This addition may break scripts for processing images with either program.

Software Version:
Test Reviewer:

Description

I would like to be able to use ISIS to work with the Kaguya Multiband Imager (MI) L3C5 data that is just now becoming available from JAXA. It looks like there have been changes to the labels associated with this data so I don't think the "kaguyami2isis" program will work with it. Will there be an update to this program or a new program that will allow this data to be read into ISIS?

mimap2isis_L3C5_mosaic.jpg View (548 KB) Trent Hare, 2016-10-14 05:49 PM


Related issues

Related to ISIS - Bug #4505: mimap2isis - Import PDS formatted Kaguya MI MAP file to ISIS3 cube format Closed
Related to ISIS - Bug #2244: Pixel (Line/Sample) Projection Offset Issue Closed

History

#1 Updated by Tammy Becker over 2 years ago

  • Category set to Applications
  • Status changed from New to Acknowledged

Lisa Gaddis supplied background information and can assist in providing examples of the new files. She will also help contact Mikiko Ohtake (MI PI) for more information as needed, especially documentation on how these data were revised beyond the L2B2 version.

Lisa's information (Oct 27):
"Modify the kaguyami2isis program (or create a new program) to ingest the new Kaguya Multiband Imager data, level 3C5. These data were added to the Kaguya archive in June 2015. The current kaguyami2isis program is designed to work with the older, level 2B2 version of the MI frames (NOT the MI MAP, or map-projected, frames). The kaguyami2isis program is a camera model for the MI frames, and currently it searches for keywords that are not present in the newer products. The translation table for the MI data produces an error regarding these missing keywords when used...

There is no documentation of how these data were revised beyond the L2B2 version, but I believe that both the radiometric and photometric corrections have been improved..."

Triaged at a 'higher' level due to no current 'work around'.

#3 Updated by Stuart Sides almost 2 years ago

  • Target version set to 3.4.13 (FY16 R3 2016-08-31 Aug)

#5 Updated by Stuart Sides almost 2 years ago

  • Assignee set to Kelvin Rodriguez

#6 Updated by Stuart Sides almost 2 years ago

  • Assignee changed from Kelvin Rodriguez to Adam Paquette

#8 Updated by Adam Paquette almost 2 years ago

  • Status changed from Acknowledged to In Progress

#9 Updated by Adam Paquette almost 2 years ago

  • Status changed from In Progress to Assigned

#10 Updated by Adam Paquette almost 2 years ago

  • Status changed from Assigned to In Progress

#11 Updated by Stuart Sides over 1 year ago

Hi William,

The L3C5 data can be imported into ISIS using the "pds2isis" application. Be aware: There were some problems with the older L2 map data where the Sample_Projection_Offset keyword was negated. The mimap2isis application fixed this. At this time we are not aware of any problems using pds2isis on the L3 data.

Updated: With Lisa's testing, pds2isis does not produce a mapping group in the ISIS label that is correct. It will take additional testing to find what is wrong and fix it.

#12 Updated by Adam Paquette over 1 year ago

  • Status changed from In Progress to Resolved
  • Impact updated (diff)

#13 Updated by Lisa Gaddis over 1 year ago

I tested the pds2isis application on MI L3C5 data, and while it does create an ISIS cube file, the result has incorrect longitude. A mosaic created using several cubes from pds2isis shows that western-most cubes are placed on the eastern-most side of the mosaic. (From what I can tell, it does not appear that the cubes are incorrectly flipped or mirrored.) It appears that the SAMPLE_PROJECTION_OFFSET is improperly negated in the L3C5 .img file. I have only tested a single site (Alphonsus crater) that is just west of the prime meridian (~352 to 358 degrees), so I don't know if this simple fix applies to all of the L3C5 data.

I also note that the pds2isis program does not propagate the BandBin (filter name such as MV1 to MV5, MN1 to MN4, center wavelength and width of band in nm, BaseBand name, usually something like MV5 or MV2) information that is necessary for working with multispectral data in ISIS.

Ideally, the kaguyami2isis program should be modified to work on both the Multiband Imager L2B2 and L3C5 data. I believe (but have not checked it lately) that the kaguyami2isis program was written to ingest the L2B2 data without a proper ISIS3 camera model, so it does a lot of manipulation to emulate one. That work should not be necessary for the L3C5 data. To distinguish, kaguyami2isis would need to check the DataSetId (Group=Archive) or equivalent in the .img file, then possibly just remove the negative sign in front of the SAMPLE_PROJECTION_OFFSET keyword, and propagate as many keywords as possible to the output cube (especially the BandBin information).

Finally, the kaguyami2isis documentation needs clarification on which MI data it works on. Hopefully it can be modified to include the L3C5 data, but if not, the documentation should clearly state that it works on L2B2 data.

#14 Updated by Tammy Becker over 1 year ago

  • Status changed from Resolved to Feedback

#15 Updated by Stuart Sides over 1 year ago

  • Target version changed from 3.4.13 (FY16 R3 2016-08-31 Aug) to 214

#16 Updated by Trent Hare over 1 year ago

There are several confusing topics here that need to be addressed. I will try to list how I understand the programs which exist and recommendations.

1.) The documentation needs to be clarified for what Kaguya archive the program was tested against. For example, there exist 2 ISIS Kaguya MI programs: https://isis.astrogeology.usgs.gov/Application/index.html#Kaguya

  • kaguyami2isis - this program was apparently written for Kaguya MI L2B2 archive (example filename: MVA_2B2_01_02024S140E3586.img). There is no map projected section in the PDS3 label but the images are apparently map projected. Locally there is a helper script to apply map projections labels to these files called kaguyamiproc.pl in /usgs/cdev/contrib/bin/ (and now made available for outside users here: https://github.com/USGS-Astrogeology/ISIS3_scripts/tree/master/kaguyamiproc )

  • mimap2isis - this program was apparently written for Kaguya "MI MAP 02" ?? archives (example filename: MI_MAP_02_N65E328N64E329SC.img). It appears to be just a specialized version of pds2isis which has particular translation table for this archive and fixes a problem where the SAMPLE_PROJECTION_OFFSET is incorrectly negated: https://isis.astrogeology.usgs.gov/fixit/issues/1919

Anyway, the mimap2isis application is the most appropriate routine to be modified for the newer MI L3C5 archive (example filename: MIA_3C5_03_02024S140E3586SC.img). Currently, mimap2isis crashes from a label value that does not exist in the L3C5 images which use to appear in the "MAP" archive. example run:

mimap2isis from=MIA_3C5_03_02024S140E3586SC.img to=MIA_3C5_03_02024S140E3586SC.cub
PROGRAMMER ERROR No value or default value to translate for translation group [StretchedFlag] in file [/usgs/cpkgs/isis3/data/kaguya/translations/mimapArchive.trn].


if pds2isis were to be updated to better support MI L3C5 images, two things would need to added (as Lisa mentioned).
1.) ability to propagate BandBin section from PDS3 (which is recommended for any PDS ingested image which has this section)
2.) And to correct the wrongly negated SAMPLE_PROJECTION_OFFSET, a update to the file $ISIS3DATA/base/translations/pdsProjectionLineSampToXY.def
Group = Selection
Keyword = "DATA_SET_ID"
Pattern = "MI_L3C"
xMult = 1.0
yMult = 1.0
xOff = 0.5
yOff = 0.5
End_Group

Note I imagine mimap2isis (like pds2isis) will also have the dreaded one-pixel PDS ingest issue. The mapping above for xOff and yOff is the correct setting for most PDS3 products (and hopefully Kaguya). if implemented in pds2isis, this should fix the current one-pixel offset in pds2isis for MI L3C5 images: https://isis.astrogeology.usgs.gov/fixit/issues/2244

#17 Updated by Lynn Weller over 1 year ago

Not trying to muddle things up here...

  • kaguyami2isis - this program was apparently written for Kaguya MI L2B2 archive (example filename: MVA_2B2_01_02024S140E3586.img). There is no map projected section in the PDS3 label but the images are apparently map projected. To help with this, there is a non-public helper script to apply map projections labels to these files called kaguyamiproc.pl available locally in /usgs/cdev/contrib/bin/

This was initially the case, but ultimately Janet was able to get kernels for the MI data and the kaguyami2isis program was updated/modified to ingest MI VIS and NIR PDS EDR's and have a camera model applied. I worked with this data and from what I recall, there never was mapping information on the IMG labels - that was added via the kaguyamiproc.pl script after the fact and when kernels and a camera model were unavailable.

At any rate, I think Trent is correct in suggesting the appropriate application to use for ingesting the LC35 data is mimap2isis since that data are map projected.

#18 Updated by Trent Hare over 1 year ago

mimap2isis was checked against L3C5 images for ingest parameter values and negated offsets and appears to ready for prime-time. See attached image for 27 image mosaic overlain on LOLA/TC hillshade.

Note the the non-standard PDS NULL data keyword in the L3C5 images (called OUT_OF_IMAGE_BOUNDS_VALUE) must manually added to the run. e.g.
setup corrected version:
isis /work/projects/isis/latest/m02358/isis

mimap2isis from=MIA_3C5_03_02024S140E3586SC.img to=MIA_3C5_03_02024S140E3586SC.cub setnullrange=yes nullmin=-30000 nullmax=-30000

tickets to be added as a result of this.
(1) clean up documentation for mimap2isis and kaguyami2isis (which archive (and version), how to set NULLs, and why mimap2isis has FROM, IMAGE, and TO -- IMAGE appears to do nothing).
(2) add note to pds2isis ticket (https://isis.astrogeology.usgs.gov/fixit/issues/2244 ) to also check this program once it is updated

#19 Updated by Tammy Becker over 1 year ago

  • Related to Bug #4505: mimap2isis - Import PDS formatted Kaguya MI MAP file to ISIS3 cube format added

#20 Updated by Tammy Becker over 1 year ago

  • Related to Bug #2244: Pixel (Line/Sample) Projection Offset Issue added

#21 Updated by Stuart Sides about 1 year ago

  • Target version changed from 214 to 3.5.1 (Sprint 1)

#23 Updated by Stuart Sides 8 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.2 (2017-01-31 Jan)

#24 Updated by Makayla Shepherd 8 months ago

  • Status changed from Feedback to In Progress

#25 Updated by Adam Paquette 7 months ago

  • Status changed from In Progress to Resolved

#26 Updated by Makayla Shepherd 7 months ago

  • Status changed from Resolved to Closed

Closed per Trent's successful test and Stuart's code review.

Also available in: Atom PDF