Project

General

Profile

Bug #4507

mrf2isis - Import PDS formatted MiniRF level1 or level2 image cube into Isis format

Added by Tammy Becker about 1 year ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
John Bonn
Category:
Applications
Target version:
Impact:
Software Version:
Test Reviewer:
Story points:
1

Description

Mentor: Trent Hare

The user test needs to specifically address importing the level2 (map) into ISIS

Note:

  • One person should have this figured out and done before other devs pick up the other tickets related to pixel offset.

DoD:

  • mrf2isis produces valid pixel offset.
  • Update the configuration file (ask Stuart or Trent).
  • Document a short description of the process for completing this ticket so other devs can see how to approach the other offset tickets.

Related issues

Related to ISIS - Bug #2244: Pixel (Line/Sample) Projection Offset Issue Closed

History

#1 Updated by Tammy Becker about 1 year ago

  • Related to Bug #2244: Pixel (Line/Sample) Projection Offset Issue added

#2 Updated by Tammy Becker about 1 year ago

  • Tracker changed from Bug to UserTestPlan

#3 Updated by Jason Laura about 1 year ago

  • Status changed from Acknowledged to New

#4 Updated by Stuart Sides 10 months ago

  • Target version deleted (225)

#5 Updated by Tammy Becker 8 months ago

  • Status changed from New to Acknowledged
  • Target version set to 3.5.1 (Sprint 1)

#6 Updated by John Shinaman 8 months ago

  • Test Reviewer set to John Shinaman

#7 Updated by John Shinaman 8 months ago

App tests for level2 data DO NOT check the labels appropriately to check pixel offset.

#8 Updated by Stuart Sides 8 months ago

  • Tracker changed from UserTestPlan to Bug

#10 Updated by Stuart Sides 8 months ago

User test plans are done. These have been converted to bugs, so they will be resolved.

#11 Updated by Ian Humphrey 7 months ago

  • Story points set to 1
  • Description updated (diff)

#12 Updated by John Bonn 7 months ago

  • Assignee set to John Bonn

#13 Updated by John Bonn 7 months ago

  • Status changed from Acknowledged to In Progress

#14 Updated by John Bonn 7 months ago

PDS to ISIS mapping is defined in:
/usgs/cpkgs/isis3/data/base/translations/pdsProjectionLineSampToXY.def
We are looking to confirm the xOff and yOff values within this file.

From the file:

#  This file is used by the ProcessImportPds class to define the offset and 
#  multiplication factors to convert between  sample/line and x/y.
#  
#  The Keyword defines which keyword is searched for in the pds file.  The
#  Pattern is then used to match the value in the defined keyword.  If there
#  is a match,  the given values are used.
#  The defaults used in ProcessImportPds are:
#      xMult  =  -1.0
#      yMult  =   1.0
#      xOff   =  -0.5
#      yOff   =  -0.5
#
# Equations used are
#
#     upperLeftX = xmult * (sampleOffset + xOff) * PixelResolution
#     upperLeftY = ymult * (lineOffset + yOff) * PixelResolution
#
#  Anytime a new group is added to this file, all 6 keywords need to be
#  defined (Keyword,Pattern,xMult,yMult,xOff,yOff).
#

To find conversion for the data file you are looking at there will be an entry in the pdsProjectionLineSampToXY.def file similar to:

# CHANDRAYAAN MINIRF MONOSTATIC RADAR
Group = Selection
Keyword = "DATA_SET_ID"
Pattern = "CHAN1-L-MRFFR-5-CDR-MAP-V1.0"

The "Keyword" should match an entry in the source data file and the "Pattern" is an associated regex-style expression to match on the Keyword's value.

In this example the source data file (FSL_CDR_LV2_00200_1Q.LBL) looked like this:

DATA_SET_ID = "CHAN1-L-MRFFR-5-CDR-MAP-V1.0"

Within the associated Group in the pdsProjectionLineSampToXY.def file there are

xOff    = -0.5
yOff    = -0.5

entries that may need to be changed.

In talking to Trent we need to do following steps:

1) assume overrides in pdsProjectionLineSampToXY.def are correct.
2) remove "STANDARD PDS" entry from pdsProjectionLineSampToXY.def - we assume default entry is not used. Document default values in pdsProjectionLineSampToXY.def
3) correct the default values for xOff and yOff in void ProcessImportPds::GetProjectionOffsetMults()
4) document pattern matching in pdsProjectionLineSampToXY.def (substring or exact string)
5) make test case for each PDS ingestion program
6) notify user if overrides are used. We need to specify how to do notification (entry in print file, new label, force user to use command line option to enable override)

Approach for fixing for each program (This is to ensure each program is using the new PDS defaults. The concern is some programs may have the defaults hard-coded.) :
1) Input programs should already have tests that use catlab to verify input. Verify a catlab test exists and check that it does not exclude the UpperLeftX or UpperLeftY in the input/*.DIFF files for the test. Mrf2isis has an example test for programs that do not use catlab.
2) change to new PDS defaults
3) confirm test cases fail with new PDS defaults (data set with overridden PDS values should still pass)
4) check failing test cases UpperLeftCornerX and UpperLeftCornerY are each offset by PixelResolution
5) update test case to pass.

mrf2isis is reading ProcessImportPds::GetProjectionOffsetMults. Note that there's an entry in the pdsProjectionLineSampToXY.def file to override defaults so mrf2isis is currently using correct values and test for the correct values. This means the mrf2isis will not need any change.

Note: Currently mrf2isis level2 tests do not compare the output cube labels to the truth cube labels. In reviewing the UpperLeft[X,Y] values in the truth file I noticed they did not match the output values. We know the output is generated using the correct entries in the pdsProjectionLineSampToXY.def file so I'm assuming the output values are correct and will update the truth values to match. A test will also be added to compare the output and truth labels.

# CHANDRAYAAN MINIRF MONOSTATIC RADAR
Group = Selection
Keyword = "DATA_SET_ID"
Pattern = "CHAN1-L-MRFFR-5-CDR-MAP-V1.0"
xMult = -1.0
yMult = 1.0
xOff = 0.5
yOff = 0.5
End_Group
# LRO MINIRF MONOSTATIC RADAR
Group = Selection
Keyword = "DATA_SET_ID"
Pattern = "LRO-L-MRFLRO-5-CDR-MAP-V1.0"
xMult = -1.0
yMult = 1.0
xOff = 0.5
yOff = 0.5
End_Group

Note that the other mission specific PDS import programs were tested as part of this ticket and have been updated, specifically tickets #4513, #4516, #4517, #4518. Once a deployment plan has been made the truth files for the hirdr2isis #4513, and crism2isis #4518 can be regenerated (after the code update ad .def update) and checked in - they have been fully tested for this change and the test programs that will test the change (specifically that check UpperLeft[X,Y]) are already present and tested. The ProcessImportPds::GetProjectionOffsetMults() and .def file changes will need to be checked in/updated at the same time.

During testing the mical program also came up as a candidate for PDS conversion (there was a Line and Sample change) but it was determined to not be releated to the PDS data but was instead a change in how tile sizes were calculated. The actual test ignored changes to these values (via the .DIFF file).

Deployment

The pdsProjectionLineSampToXY.def is in the data directory and in use by processors so any changes to it need to be coordinated with the processors. Also the other programs in #2244 will probably need to be deployed at the same time. The plan for deployment is as follows:

1) Deploy a new copy of pdsProjectionLineSampToXY.def as pdsProjectionLineSampToXY_V2.def file with the default PDS3 entry removed. The updated programs will reference the new pdsProjectionLineSampToXY_V2.def file and the ProcessImportPds::GetProjectionOffsetMults method will be updated with the correct defaults. The currently deployed programs will continue using the existing values in pdsProjectionLineSampToXY.def so they will not be impacted.

2) When the new code is deployed it will reference the updated pdsProjectionLineSampToXY_V2.def and have the correct updated hardcoded default values. The old pdsProjectionLineSampToXY.def will be deprecated but will be left in place fo the old versions of the programs. Note that test data will be updated when the new code is deployed. If old versions of the programs are tested against the new test data they may fail the tests.

#15 Updated by John Bonn 7 months ago

  • Status changed from In Progress to Blocked

#16 Updated by Trent Hare 7 months ago

I agree with the roll-out as John has described. But given that pdsProjectionLineSampToXY.def is used from the ISIS3DATA area, once this is changed, might it impact users who update (rsync) "DATA" but not ISIS? Should this file be versioned? e.g.
pdsProjectionLineSampToXY_v2.def or time tagged?

#17 Updated by Stuart Sides 7 months ago

  • Status changed from Blocked to In Progress

#18 Updated by John Bonn 7 months ago

  • Status changed from In Progress to Resolved

#19 Updated by Stuart Sides 5 months ago

  • Status changed from Resolved to Closed

Closed after testing by Trent

#20 Updated by Stuart Sides 4 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.1 (2017-08-08 Aug)

Also available in: Atom PDF