Project

General

Profile

Question #4421

Voyager - voy2isis throws **USER ERROR** Unable to set PDS file. Decompressed input file does not appear to be a PDS product.

Added by Brian Burns about 1 year ago. Updated about 1 month ago.

Status:
Feedback
Priority:
Normal
Category:
Applications
Target version:
Test Reviewer:
Story points:
2

Description

I realize this is probably a problem with the data instead of ISIS, but wanted to post it here -

$ wget http://pds-rings.seti.org/volumes/VG_0xxx/VG_0014/JUPITER/C1549XXX/C1549450.IMQ
Length: 212274 (207K)

$ voy2isis from=c1549450.imq to=c1549450.cub
**USER ERROR** Unable to set PDS file.  Decompressed input file [c1549450.img] does not appear to be a PDS product.

NASAView (Windows) can open the IMG file it extracted and it looks fine. And the labels parse okay in the IMQ file, though the IMAGE_ID looks broken -

IMAGE_ID = '?? R KL? '

I tried changing that to '0123456789', then voy2isis worked,

but then spiceinit gave an error -

$ spiceinit from=C1549450.cub TSPK=kernels/spk/jup100.bsp SPK=kernels/spk/Voyager_1.a54206u_V0.2_merged.bsp
**ERROR** No Camera Kernels found for the image [C1549450.cub].

The image number and time look okay -

IMAGE_NUMBER = 15494.50 /FLIGHT DATA SUBSYSTEM (FDS)/
IMAGE_TIME = 1979-02-03T16:25:23Z

and subsequent IMQ files in the sequence work fine with voy2isis and spiceinit, until about 10 images later and get the same error with voy2isis -

$ voy2isis from=C1549510.IMQ to=C1549510.cub
**USER ERROR** Unable to set PDS file.  Decompressed input file [C1549510.img] does not appear to be a PDS product.

Has anyone come across this before, or know what the problem might be?

Thanks


Related issues

Related to ISIS - Bug #4651: voy2isis fails on saturn images from the original PDS archive Closed
Related to ISIS - Bug #4414: voy2isis error SPICE(INVALIDTIMESTRING) The input string contains an unrecognizable substring Acknowledged
Duplicated by ISIS - Feature #4345: Handle Voyager IMG/LBL files in voy2isis Feedback

History

#1 Updated by Tammy Becker about 1 year ago

  • Category set to Applications
  • Status changed from New to Acknowledged

#3 Updated by Tammy Becker 9 months ago

  • Related to Bug #4651: voy2isis fails on saturn images from the original PDS archive added

#4 Updated by Stuart Sides 9 months ago

  • Target version set to 3.5.1 (Sprint 1)

#6 Updated by Stuart Sides 7 months ago

  • Status changed from Acknowledged to Assigned
  • Assignee set to Marjorie Hahn

#7 Updated by Marjorie Hahn 7 months ago

  • Status changed from Assigned to In Progress

#8 Updated by Stuart Sides 6 months ago

  • Duplicated by Feature #4345: Handle Voyager IMG/LBL files in voy2isis added

#9 Updated by Stuart Sides 6 months ago

  • Related to Bug #4414: voy2isis error SPICE(INVALIDTIMESTRING) The input string contains an unrecognizable substring added

#10 Updated by John Bonn 6 months ago

I just made a fix that has a very remote change of fixing this. You might test again with revision 7648 or later.

#11 Updated by Ian Humphrey 6 months ago

  • Story points set to 2

#12 Updated by Tammy Becker 6 months ago

I tried the recent fix to voy2isis using /work/projects/isis/latest/m04651/isis/bin/voy2isis (15May2017).

I attempted voy2isis on the two images reported. Both are from the original PDS archive (not the PDS-rings-SETI archive noted by the original reporter):
/pds_san/PDS_Archive/Voyager/vg_0014/jupiter/c1549xxx/c1549450.imq
/pds_san/PDS_Archive/Voyager/vg_0014/jupiter/c1549xxx/c1549510.imq

voy2isis from=c1549450.imq to=c1549450.cub
USER ERROR Unable to set PDS file. Decompressed input file [c1549450.img] does not appear to be a PDS product.

and the 2nd image (also from the original PDS archive)
voy2isis from=c1549510.imq to=c1549510.cub
USER ERROR Unable to set PDS file. Decompressed input file [c1549510.img] does not appear to be a PDS product.

Internal Test area: /work/projects/usertests/UserTestPlans/Apps/voy2isis/m04421

#13 Updated by Marjorie Hahn 5 months ago

The fix for this bug may involve isolating keywords that absolutely necessary for ingesting the data. This way, voy2isis can ingest the PDS data with a message that certain unnecessary keywords were missing or corrupt, without failing entirely. See below for a list of keywords that are necessary for ingestion:

Group = Instrument
  SpacecraftClockCount
  SpacecraftName
  TargetName
  StartTime
  InstrumentId
  ScanModeId
  ShutterModeId
  GainModeId
  EditModeId
  ExposureDuration
  CameraState1
  CameraState2
End_Group

Group = Archive
  MissionPhaseName
  ProductId
End_Group

Group = BandBin
  FilterName
  FilterNumber
  Center
  Width
End_Group

#14 Updated by Marjorie Hahn 5 months ago

Currently in the ISIS 3 beta, voy2isis fails to ingest 1804 Voyager images in the PDS archive. The following is a list of the different errors that occur on those images, sorted from least frequent to most frequent:

  • 4 of the files could not be decompressed.
  • 9 of those errors are due to a corrupt IMAGE_ID label.
  • 13 produced I/O Errors where ProcessImport could not read the data.
  • 160 are from array out-of-bounds errors.
  • 1618 are due to NAIF errors.

#15 Updated by Marjorie Hahn 5 months ago

Changes are available for testing at: /work/projects/isis/latest/m04421/

I have not resolved the ticket yet since there is still documentation that I need to complete.

#17 Updated by Tyler Wilson 5 months ago

Code Review

  1. Add a history entry for yourself in the voy2isis.xml file.
  2. In the fixLabels function, I would refactor this piece of code:
 while (c != '\n') {
    c = labels[i];

    // If character is out of range, image id is corrupt and must be replaced
    if (c < 0 || c > 127) {
      replaceImageId = true;
    }
    i++;
  }

Using the QChar class in Qt, you might try something like this:

 QChar c(labels[imageIndex]);
 int i = imageIndex++;

 while (c != '\n') {
    QChar c(labels[i]);

    // If character is out of range, image id is corrupt and must be replaced
    if (c.isNonCharacter() ) {
      replaceImageId = true;
      //Once this condition is triggered, there is no reason to stay in the loop until the end.
      break;
    }
    i++;
  }

There are non-symbol characters within the tradition ASCII numerical range of 0-127 which might show up in
a thrashed label as well, but the QChar class has a variety of boolean functions for determining these.

I would also add another way of escaping from that while-loop. It's almost always true that you can expect a '\n' to show up at some point, the threat of a potential infinite loop is living a little on the wild side.

#18 Updated by Marjorie Hahn 3 months ago

This ticket has been merged with #4345 and can be found in: /work/projects/isis/latest/m04345_m04421

#19 Updated by Marjorie Hahn 3 months ago

  • Assignee changed from Marjorie Hahn to Christopher Combs

#20 Updated by Stuart Sides 3 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.2 (2017-01-31 Jan)

#21 Updated by Christopher Combs about 2 months ago

Currently in the ISIS 3 beta, voy2isis fails to ingest 1804 Voyager images in the PDS archive. The following is a list of the different errors that occur on those images, sorted from least frequent to most frequent:

  • 4 of the files could not be decompressed.
  • 9 of those errors are due to a corrupt IMAGE_ID label.
  • 13 produced I/O Errors where ProcessImport could not read the data.
  • 160 are from array out-of-bounds errors.
  • 1618 are due to NAIF errors.

The following files can not be decompressed:

  • /pds_san/PDS_Archive/Voyager/vg_0002/uranus/c2670xxx/c2670415.imq
  • /pds_san/PDS_Archive/Voyager/vg_0009/neptune/c1064xxx/c1064447.imq
  • /pds_san/PDS_Archive/Voyager/vg_0034/saturn/c4302xxx/c4302805.imq
  • /pds_san/PDS_Archive/Voyager/vg_0035/saturn/c4335xxx/c4335720.imq

The images with corrupt labels are now handled by changing corrupted keywords to "Unknown"

The images with array out-of-bounds errors now create their cubes, but reach an error in TranslateVoyagerLabels():

USER ERROR Instrument ID [Unknown] does not match Narrow or Wide angle camera.

#22 Updated by Christopher Combs about 2 months ago

  • Status changed from In Progress to Resolved

#23 Updated by Christopher Combs about 2 months ago

Available for testing at /work/projects/isis/latest/m04345_m04421

#24 Updated by Tammy Becker about 2 months ago

Unable to test voy2isis from astrovm4 under

/work/projects/isis/latest/m04345_m04421/isis/

ast{58}> voy2isis -h
voy2isis: error while loading shared libraries: libicui18n.so.52: cannot open shared object file: No such file or directory

#25 Updated by Makayla Shepherd about 1 month ago

In the interest of time I've built the code for astrovm4

To test:
setisis /work/projects/isis/latest/m04345_m04421_prog24/isis

#26 Updated by Tammy Becker about 1 month ago

  • Status changed from Resolved to Feedback

I have tested voy2isis with the original compressed versions of Europa data. When the input has a corrupted input keyword, voy2isis does a good job informing the user of the corruption, including which keyword. It proceeds to create an output cube and translate what instrument labels it can.

The one issue is that this very usable message to the user is not captured in the print.prt file or in a Results group the voy2isis history. It seems to only go to the screen. The message needs to be captured in a standard ISIS method for future reference.

working area: /work/users/tbecker/IsisTesting/M04421_voy2isis/vg_0024/europa

Thank you.

Also available in: Atom PDF