Project

General

Profile

Feature #4345

Handle Voyager IMG/LBL files in voy2isis

Added by Brian Burns about 1 year ago. Updated about 1 month ago.

Status:
Feedback
Priority:
Normal
Category:
Applications
Target version:
Impact:

Pvl translations and translation table now account for label input in voy2isis

Test Reviewer:

Description

Would it be possible for voy2isis to handle IMG/LBL files from the PDS?
http://pds-rings.seti.org/voyager/iss/calib_images.html

$ voy2isis from=C1625003_CALIB.LBL to=foo.cub
USER ERROR Input file [C1625003_CALIB.LBL] does not appear to be a Voyager EDR.

$ voy2isis from=C1625003_CALIB.IMG to=foo.cub
USER ERROR Unable to set PDS file. Decompressed input file [C1625003_CALIB.IMG] does not appear to be a PDS product.

$ voy2isis from=C1625003.IMQ to=foo.cub
(works)

It looks like the PDS split out the label information from the IMQ files to LBL files, and uncompressed the images to IMG files.

I've been using the IMG/LBL files in a project and just started using ISIS - if it's an involved fix then I'll see if I can switch to using the IMQ files, though the archives are organized differently.

Thank you


Related issues

Duplicates ISIS - Question #4421: Voyager - voy2isis throws **USER ERROR** Unable to set PDS file. Decompressed input file does not appear to be a PDS product. Feedback

History

#1 Updated by Tammy Becker about 1 year ago

  • Status changed from New to Acknowledged
  • Assignee set to Tammy Becker

Hi Brian,

I am not familiar with the "PDS rings node" version of Voyager "ISS", and, I suspect that ISIS3 voy2isis isn't either: A _CALIB.LBL and .IMG filename sounds suspiciously like processed images (modified keyword labels) that ISIS3 voy2isis currently does not expect at ingestion. I will have to ask our internal PDS node staff here in Astro what this version is all about.

As you have already pointed out, the "PDS Imaging node" version
(on volume vg_0017/jupiter/c1625003.imq) with voy2isis works fine; in fact the suite of applications including radiometric calibration seems to work ok:

voy2isis from=c1625003.imq to=c1625003.cub
spiceinit from=c1625003.cub
findrx from=c1625003.cub
remrx from=c1625003.cub to=c1625003_remrx.cub (optional step to remove the reseaus)
voycal from=c1625003_remrx.cub to=c1625003_cal.cub

I will network with our internal PDS staff as well as developer staff regarding voy2isis. This most likely will require a modification to the translation table that voy2isis refers to. This modification would have to be done very carefully in order to avoid causing a problem with the imaging node version (.imq).

When this gets addressed will depend on our developer resources. We will keep you posted through this ticket. In the meantime, I hope processing the pds imaging node version will be sufficient??!

#2 Updated by Brian Burns about 1 year ago

Thanks Tammy,

Yes, I'll see if I can switch to using the IMQ files, unless it happened to be an easy fix - I guess no one has been using ISIS with the reorganized/processed Voyager datasets, as it looks like they were created in 2012. They include four versions of each image - RAW, CLEANED (reseau marks removed), CALIB (flatfield correction), and GEOMED (geometric distortion removed).

I would need to redownload the IMG/LBL archives anyway, as I hadn't kept the LBL files, so it wouldn't be too much trouble to get the IMQ archives - the main issue is that the organization is different, but I'll see if I can work around that. If I'd known about ISIS I would have started with the IMQ archives though!

For completeness I'll include the labels from an IMQ file and a corresponding LBL file - they do seem to have changed a bit...

Q1625003.IMQ:

CSD3ZF0000100000001NJPL3IF0PDS200000001 = SFDU_LABEL

/*          FILE FORMAT AND LENGTH */
RECORD_TYPE                      = VARIABLE_LENGTH
RECORD_BYTES                     = 836
FILE_RECORDS                     = 861
LABEL_RECORDS                    = 55

/*          POINTERS TO STARTING RECORDS OF MAJOR OBJECTS IN FILE */
^IMAGE_HISTOGRAM                 = 56
^ENCODING_HISTOGRAM              = 58
^ENGINEERING_TABLE               = 61
^IMAGE                           = 62

/*          IMAGE DESCRIPTION  */
SPACECRAFT_NAME                  = VOYAGER_1
MISSION_PHASE_NAME               = JUPITER_ENCOUNTER
TARGET_NAME                      = JUPITER
IMAGE_ID                         = '0638J1-005'
IMAGE_NUMBER                     = 16250.03 /*FLIGHT DATA SUBSYSTEM (FDS)*/
IMAGE_TIME                       = 1979-02-28T20:35:47Z
EARTH_RECEIVED_TIME              = 1979-02-28T21:12:48Z
INSTRUMENT_NAME                  = NARROW_ANGLE_CAMERA
SCAN_MODE_ID                     = '1:1'
SHUTTER_MODE_ID                  = BOTALT
GAIN_MODE_ID                     = LOW
EDIT_MODE_ID                     = '1:1' /*FULL RESOLUTION*/
FILTER_NAME                      = VIOLET
FILTER_NUMBER                    = 1
EXPOSURE_DURATION                = 0.4800 <SECONDS>
NOTE                             = "NA EAST-WEST GLOBAL THERMAL MAP IN 2 COLORS
                                   (V,O) + WA"

/*          DESCRIPTION OF THE OBJECTS CONTAINED IN FILE */
OBJECT                           = IMAGE_HISTOGRAM
 ITEMS                           = 256
 ITEM_TYPE                       = VAX_INTEGER
 ITEM_BITS                       = 32
END_OBJECT

OBJECT                           = ENCODING_HISTOGRAM
 ITEMS                           = 511
 ITEM_TYPE                       = VAX_INTEGER
 ITEM_BITS                       = 32
END_OBJECT

OBJECT                           = ENGINEERING_TABLE
 BYTES                           = 242
 ^STRUCTURE                      = 'ENGTAB.LBL'
END_OBJECT

OBJECT                           = IMAGE
 ENCODING_TYPE                   = HUFFMAN_FIRST_DIFFERENCE
 LINES                           = 800
 LINE_SAMPLES                    = 800
 LINE_SUFFIX_BYTES               = 36
 SAMPLE_TYPE                     = UNSIGNED_INTEGER
 SAMPLE_BITS                     = 8
 SAMPLE_BIT_MASK                 = 2#11111111#
 ^LINE_SUFFIX_STRUCTURE          = 'LINESUFX.LBL'
END_OBJECT
END


----------------------------------------------------------------

C1625003_CALIB.LBL:

PDS_VERSION_ID                  = PDS3
RECORD_TYPE                     = FIXED_LENGTH
RECORD_BYTES                    = 1600
FILE_RECORDS                    = 802
^VICAR_HEADER                   = ("C1625003_CALIB.IMG", 1)
^IMAGE                          = ("C1625003_CALIB.IMG", 2)
^VICAR_EXTENSION_HEADER         = ("C1625003_CALIB.IMG", 802)

DATA_SET_ID                     = "VG1/VG2-J-ISS-2/3/4/6-PROCESSED-V1.0"
PRODUCT_ID                      = "C1625003_CALIB.IMG"
PRODUCT_CREATION_TIME           = 2012-06-06T16:00:00
SOURCE_PRODUCT_ID               = ("C1625003_CLEANED.IMG",
                                   "DC1_NA_11_X100.IMG",
                                   "FICOR77_VG1_NA_VIOLET.DAT",
                                   "VGRSCF.DAT")
PRODUCT_TYPE                    = CALIBRATED_IMAGE

/* Image Description  */

INSTRUMENT_HOST_NAME            = "VOYAGER 1"
INSTRUMENT_HOST_ID              = VG1
INSTRUMENT_NAME                 = "IMAGING SCIENCE SUBSYSTEM - NARROW ANGLE"
INSTRUMENT_ID                   = "ISSN"
MISSION_PHASE_NAME              = "JUPITER ENCOUNTER"
TARGET_NAME                     = "JUPITER"
IMAGE_ID                        = "0638J1-005"
IMAGE_NUMBER                    = "16250.03"
IMAGE_TIME                      = 1979-02-28T20:35:47.00
EARTH_RECEIVED_TIME             = 1979-02-28T21:12:48
SCAN_MODE_ID                    = "1:1"
SHUTTER_MODE_ID                 = "BOTALT"
GAIN_MODE_ID                    = "LOW"
EDIT_MODE_ID                    = "1:1"
FILTER_NAME                     = "VIOLET"
FILTER_NUMBER                   = "1"
EXPOSURE_DURATION               = 0.4800 <SECOND>
START_TIME                      = 1979-02-28T20:35:46.52
STOP_TIME                       = 1979-02-28T20:35:47.00
SPACECRAFT_CLOCK_START_COUNT    = "16250:02:792"
SPACECRAFT_CLOCK_STOP_COUNT     = "16250:03:001"

NOTE                            =
"NA EAST-WEST GLOBAL THERMAL MAP IN 2 COLORS (V,O) + WA."

DESCRIPTION                     = "This image is the result of calibrating the
corresponding CLEANED image (C1625003_CLEANED.IMG). It was created using the
VICAR software package on a Compaq Alpha running OpenVMS. Routine FICOR77
subtracted a background 'dark current' image (DC1_NA_11_X100.IMG) and
converted from dimensionless raw numbers to two-byte integers that are
proportional to I/F. The REFLECTANCE_SCALING_FACTOR value listed below defines
the conversion factor.

Note that this image contains geometric distortion, so the values for the
horizontal and vertical pixel fields of view listed below are only
approximate. See the corresponding GEOMED file (C1625003_GEOMED.IMG) for a
geometrically corrected version. See file DOCUMENT/PROCESSING.TXT for more
information about the image processing."

OBJECT                          = VICAR_HEADER
  HEADER_TYPE                   = VICAR
  BYTES                         = 1600
  RECORDS                       = 1
  INTERCHANGE_FORMAT            = ASCII
  DESCRIPTION                   = "VICAR format label for the image."
END_OBJECT                      = VICAR_HEADER

OBJECT                          = IMAGE
  LINES                         = 800
  LINE_SAMPLES                  = 800
  SAMPLE_TYPE                   = LSB_INTEGER
  SAMPLE_BITS                   = 16
  SAMPLE_DISPLAY_DIRECTION      = RIGHT
  LINE_DISPLAY_DIRECTION        = DOWN
  HORIZONTAL_PIXEL_FOV          = 5.2250E-04 <DEGREE> /* approximate */
  VERTICAL_PIXEL_FOV            = 5.2250E-04 <DEGREE> /* approximate */
  HORIZONTAL_FOV                = 0.4180     <DEGREE> /* approximate */
  VERTICAL_FOV                  = 0.4180     <DEGREE> /* approximate */
  REFLECTANCE_SCALING_FACTOR    = 1.0000E-04
END_OBJECT                      = IMAGE

OBJECT                          = VICAR_EXTENSION_HEADER
  HEADER_TYPE                   = VICAR
  BYTES                         = 1600
  RECORDS                       = 1
  INTERCHANGE_FORMAT            = ASCII
  DESCRIPTION                   = "Continuation of the VICAR header."
END_OBJECT                      = VICAR_EXTENSION_HEADER

END

#3 Updated by Stuart Sides 9 months ago

  • Target version set to 3.5.1 (Sprint 1)

#4 Updated by Tammy Becker 9 months ago

  • Assignee deleted (Tammy Becker)

Unassigned Tammy to allow for developer assignment.

#7 Updated by Stuart Sides 7 months ago

  • Test Reviewer set to Robert Sucharski

#8 Updated by Stuart Sides 7 months ago

  • Duplicates Question #4421: Voyager - voy2isis throws **USER ERROR** Unable to set PDS file. Decompressed input file does not appear to be a PDS product. added

#9 Updated by Adam Paquette 5 months ago

  • Assignee set to Adam Paquette

#10 Updated by Adam Paquette 5 months ago

  • Status changed from Acknowledged to In Progress

#11 Updated by Stuart Sides 5 months ago

  • Status changed from In Progress to Blocked

Blocked until Adam returns

#12 Updated by Stuart Sides 5 months ago

  • Status changed from Blocked to Acknowledged
  • Assignee deleted (Adam Paquette)

Adam has most of this work complete, please see him before starting work on this issue

#13 Updated by Christopher Combs 5 months ago

  • Assignee set to Christopher Combs

#14 Updated by Christopher Combs 5 months ago

  • Status changed from Acknowledged to In Progress

#15 Updated by Christopher Combs 5 months ago

  • Status changed from In Progress to Resolved
  • Impact updated (diff)

#17 Updated by Tammy Becker 5 months ago

/usgs/shareall/tbecker/VoyagerRingsNode/ = A collection of test data from the PDS rings node archive. Some files in this directory represent a number of problems we have encountered from the original PDS archive that are being addressed by a separate voy2isis ticket (#4421). We are interested in seeing if this processed archive has dealt with the corrupted or unavailable keywords we have encountered with the original set.

Bad Image Id: /pds_san/PDS_Archive/Voyager/vg_0036/saturn/c4362xxx/c4362638.imq
Out-of-bounds Err: /pds_san/PDS_Archive/Voyager/vg_0037/s_rings/c4391xxx/c4391013.imq
Decompression Err: /pds_san/PDS_Archive/Voyager/vg_0002/uranus/c2670xxx/c2670415.imq
Cannot Read Data: /pds_san/PDS_Archive/Voyager/vg_0034/saturn/c4281xxx/c4281101.imq
NAIF Err: /pds_san/PDS_Archive/Voyager/vg_0009/neptune/c0986xxx/c0986348.imq (not necessarily related, still a good test under this ticket).

Also, it will be extremely important to note the stage of processing these ringsnode-archive images have been through in order for ISIS to recognize..as an example, if a user has imported a C0261155_CALIB.IMG, this image version has been calibrated, so please don't apply the ISIS voycal, or the C0261155_GEOMED.IMG, geometrically corrected (distortion correction), don't apply the ISIS noproj application and treat this image as an "ideal camera". voycal and noproj might need to recognize a keyword label?

An example document archived with this ringsnode data describing the versions of images:

http://pds-rings.seti.org/volumes/VGISS_5xxx/VGISS_5213/DOCUMENT/PROCESSING.TXT

#18 Updated by Stuart Sides 3 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.2 (2017-01-31 Jan)

#19 Updated by Makayla Shepherd 3 months ago

  • Status changed from Resolved to In Progress

#20 Updated by Tammy Becker about 1 month ago

  • Status changed from In Progress to Feedback

I have tested voy2isis on the Raw version of the 'rings node version' and it creates an output cub, but there is no output instrument group on the labels. It seems there is no translation at all. With the Raw version a full ingestion should be possible to proceed with spiceinit.

I have also tested the other versions (CALIB, GEOMED, etc) and the voy2isis failure with clear message that these versions are currently not supported is sufficient for now.

work area: /work/users/tbecker/IsisTesting/M04421_voy2isis/vg_0024/europa/C2062524_RAW.cub

The input is locate: /usgs/shareall/tbecker/VoyagerRingsNode/C2062524_RAW.LBL, IMG

Also available in: Atom PDF