Handle Voyager IMG/LBL files in voy2isis
Would it be possible for voy2isis to handle IMG/LBL files from the PDS?
$ voy2isis from=C1625003_CALIB.LBL to=foo.cub
USER ERROR Input file [C1625003_CALIB.LBL] does not appear to be a Voyager EDR.
$ voy2isis from=C1625003_CALIB.IMG to=foo.cub
USER ERROR Unable to set PDS file. Decompressed input file [C1625003_CALIB.IMG] does not appear to be a PDS product.
$ voy2isis from=C1625003.IMQ to=foo.cub
It looks like the PDS split out the label information from the IMQ files to LBL files, and uncompressed the images to IMG files.
I've been using the IMG/LBL files in a project and just started using ISIS - if it's an involved fix then I'll see if I can switch to using the IMQ files, though the archives are organized differently.
#1 Updated by Tammy Becker over 1 year ago
- Status changed from New to Acknowledged
- Assignee set to Tammy Becker
I am not familiar with the "PDS rings node" version of Voyager "ISS", and, I suspect that ISIS3 voy2isis isn't either: A _CALIB.LBL and .IMG filename sounds suspiciously like processed images (modified keyword labels) that ISIS3 voy2isis currently does not expect at ingestion. I will have to ask our internal PDS node staff here in Astro what this version is all about.
As you have already pointed out, the "PDS Imaging node" version
(on volume vg_0017/jupiter/c1625003.imq) with voy2isis works fine; in fact the suite of applications including radiometric calibration seems to work ok:
voy2isis from=c1625003.imq to=c1625003.cub
remrx from=c1625003.cub to=c1625003_remrx.cub (optional step to remove the reseaus)
voycal from=c1625003_remrx.cub to=c1625003_cal.cub
I will network with our internal PDS staff as well as developer staff regarding voy2isis. This most likely will require a modification to the translation table that voy2isis refers to. This modification would have to be done very carefully in order to avoid causing a problem with the imaging node version (.imq).
When this gets addressed will depend on our developer resources. We will keep you posted through this ticket. In the meantime, I hope processing the pds imaging node version will be sufficient??!
#2 Updated by Brian Burns over 1 year ago
Yes, I'll see if I can switch to using the IMQ files, unless it happened to be an easy fix - I guess no one has been using ISIS with the reorganized/processed Voyager datasets, as it looks like they were created in 2012. They include four versions of each image - RAW, CLEANED (reseau marks removed), CALIB (flatfield correction), and GEOMED (geometric distortion removed).
I would need to redownload the IMG/LBL archives anyway, as I hadn't kept the LBL files, so it wouldn't be too much trouble to get the IMQ archives - the main issue is that the organization is different, but I'll see if I can work around that. If I'd known about ISIS I would have started with the IMQ archives though!
For completeness I'll include the labels from an IMQ file and a corresponding LBL file - they do seem to have changed a bit...
Q1625003.IMQ: CSD3ZF0000100000001NJPL3IF0PDS200000001 = SFDU_LABEL /* FILE FORMAT AND LENGTH */ RECORD_TYPE = VARIABLE_LENGTH RECORD_BYTES = 836 FILE_RECORDS = 861 LABEL_RECORDS = 55 /* POINTERS TO STARTING RECORDS OF MAJOR OBJECTS IN FILE */ ^IMAGE_HISTOGRAM = 56 ^ENCODING_HISTOGRAM = 58 ^ENGINEERING_TABLE = 61 ^IMAGE = 62 /* IMAGE DESCRIPTION */ SPACECRAFT_NAME = VOYAGER_1 MISSION_PHASE_NAME = JUPITER_ENCOUNTER TARGET_NAME = JUPITER IMAGE_ID = '0638J1-005' IMAGE_NUMBER = 16250.03 /*FLIGHT DATA SUBSYSTEM (FDS)*/ IMAGE_TIME = 1979-02-28T20:35:47Z EARTH_RECEIVED_TIME = 1979-02-28T21:12:48Z INSTRUMENT_NAME = NARROW_ANGLE_CAMERA SCAN_MODE_ID = '1:1' SHUTTER_MODE_ID = BOTALT GAIN_MODE_ID = LOW EDIT_MODE_ID = '1:1' /*FULL RESOLUTION*/ FILTER_NAME = VIOLET FILTER_NUMBER = 1 EXPOSURE_DURATION = 0.4800 <SECONDS> NOTE = "NA EAST-WEST GLOBAL THERMAL MAP IN 2 COLORS (V,O) + WA" /* DESCRIPTION OF THE OBJECTS CONTAINED IN FILE */ OBJECT = IMAGE_HISTOGRAM ITEMS = 256 ITEM_TYPE = VAX_INTEGER ITEM_BITS = 32 END_OBJECT OBJECT = ENCODING_HISTOGRAM ITEMS = 511 ITEM_TYPE = VAX_INTEGER ITEM_BITS = 32 END_OBJECT OBJECT = ENGINEERING_TABLE BYTES = 242 ^STRUCTURE = 'ENGTAB.LBL' END_OBJECT OBJECT = IMAGE ENCODING_TYPE = HUFFMAN_FIRST_DIFFERENCE LINES = 800 LINE_SAMPLES = 800 LINE_SUFFIX_BYTES = 36 SAMPLE_TYPE = UNSIGNED_INTEGER SAMPLE_BITS = 8 SAMPLE_BIT_MASK = 2#11111111# ^LINE_SUFFIX_STRUCTURE = 'LINESUFX.LBL' END_OBJECT END ---------------------------------------------------------------- C1625003_CALIB.LBL: PDS_VERSION_ID = PDS3 RECORD_TYPE = FIXED_LENGTH RECORD_BYTES = 1600 FILE_RECORDS = 802 ^VICAR_HEADER = ("C1625003_CALIB.IMG", 1) ^IMAGE = ("C1625003_CALIB.IMG", 2) ^VICAR_EXTENSION_HEADER = ("C1625003_CALIB.IMG", 802) DATA_SET_ID = "VG1/VG2-J-ISS-2/3/4/6-PROCESSED-V1.0" PRODUCT_ID = "C1625003_CALIB.IMG" PRODUCT_CREATION_TIME = 2012-06-06T16:00:00 SOURCE_PRODUCT_ID = ("C1625003_CLEANED.IMG", "DC1_NA_11_X100.IMG", "FICOR77_VG1_NA_VIOLET.DAT", "VGRSCF.DAT") PRODUCT_TYPE = CALIBRATED_IMAGE /* Image Description */ INSTRUMENT_HOST_NAME = "VOYAGER 1" INSTRUMENT_HOST_ID = VG1 INSTRUMENT_NAME = "IMAGING SCIENCE SUBSYSTEM - NARROW ANGLE" INSTRUMENT_ID = "ISSN" MISSION_PHASE_NAME = "JUPITER ENCOUNTER" TARGET_NAME = "JUPITER" IMAGE_ID = "0638J1-005" IMAGE_NUMBER = "16250.03" IMAGE_TIME = 1979-02-28T20:35:47.00 EARTH_RECEIVED_TIME = 1979-02-28T21:12:48 SCAN_MODE_ID = "1:1" SHUTTER_MODE_ID = "BOTALT" GAIN_MODE_ID = "LOW" EDIT_MODE_ID = "1:1" FILTER_NAME = "VIOLET" FILTER_NUMBER = "1" EXPOSURE_DURATION = 0.4800 <SECOND> START_TIME = 1979-02-28T20:35:46.52 STOP_TIME = 1979-02-28T20:35:47.00 SPACECRAFT_CLOCK_START_COUNT = "16250:02:792" SPACECRAFT_CLOCK_STOP_COUNT = "16250:03:001" NOTE = "NA EAST-WEST GLOBAL THERMAL MAP IN 2 COLORS (V,O) + WA." DESCRIPTION = "This image is the result of calibrating the corresponding CLEANED image (C1625003_CLEANED.IMG). It was created using the VICAR software package on a Compaq Alpha running OpenVMS. Routine FICOR77 subtracted a background 'dark current' image (DC1_NA_11_X100.IMG) and converted from dimensionless raw numbers to two-byte integers that are proportional to I/F. The REFLECTANCE_SCALING_FACTOR value listed below defines the conversion factor. Note that this image contains geometric distortion, so the values for the horizontal and vertical pixel fields of view listed below are only approximate. See the corresponding GEOMED file (C1625003_GEOMED.IMG) for a geometrically corrected version. See file DOCUMENT/PROCESSING.TXT for more information about the image processing." OBJECT = VICAR_HEADER HEADER_TYPE = VICAR BYTES = 1600 RECORDS = 1 INTERCHANGE_FORMAT = ASCII DESCRIPTION = "VICAR format label for the image." END_OBJECT = VICAR_HEADER OBJECT = IMAGE LINES = 800 LINE_SAMPLES = 800 SAMPLE_TYPE = LSB_INTEGER SAMPLE_BITS = 16 SAMPLE_DISPLAY_DIRECTION = RIGHT LINE_DISPLAY_DIRECTION = DOWN HORIZONTAL_PIXEL_FOV = 5.2250E-04 <DEGREE> /* approximate */ VERTICAL_PIXEL_FOV = 5.2250E-04 <DEGREE> /* approximate */ HORIZONTAL_FOV = 0.4180 <DEGREE> /* approximate */ VERTICAL_FOV = 0.4180 <DEGREE> /* approximate */ REFLECTANCE_SCALING_FACTOR = 1.0000E-04 END_OBJECT = IMAGE OBJECT = VICAR_EXTENSION_HEADER HEADER_TYPE = VICAR BYTES = 1600 RECORDS = 1 INTERCHANGE_FORMAT = ASCII DESCRIPTION = "Continuation of the VICAR header." END_OBJECT = VICAR_EXTENSION_HEADER END
#17 Updated by Tammy Becker 8 months ago
/usgs/shareall/tbecker/VoyagerRingsNode/ = A collection of test data from the PDS rings node archive. Some files in this directory represent a number of problems we have encountered from the original PDS archive that are being addressed by a separate voy2isis ticket (#4421). We are interested in seeing if this processed archive has dealt with the corrupted or unavailable keywords we have encountered with the original set.
Bad Image Id: /pds_san/PDS_Archive/Voyager/vg_0036/saturn/c4362xxx/c4362638.imq
Out-of-bounds Err: /pds_san/PDS_Archive/Voyager/vg_0037/s_rings/c4391xxx/c4391013.imq
Decompression Err: /pds_san/PDS_Archive/Voyager/vg_0002/uranus/c2670xxx/c2670415.imq
Cannot Read Data: /pds_san/PDS_Archive/Voyager/vg_0034/saturn/c4281xxx/c4281101.imq
NAIF Err: /pds_san/PDS_Archive/Voyager/vg_0009/neptune/c0986xxx/c0986348.imq (not necessarily related, still a good test under this ticket).
Also, it will be extremely important to note the stage of processing these ringsnode-archive images have been through in order for ISIS to recognize..as an example, if a user has imported a C0261155_CALIB.IMG, this image version has been calibrated, so please don't apply the ISIS voycal, or the C0261155_GEOMED.IMG, geometrically corrected (distortion correction), don't apply the ISIS noproj application and treat this image as an "ideal camera". voycal and noproj might need to recognize a keyword label?
An example document archived with this ringsnode data describing the versions of images:
#20 Updated by Tammy Becker 4 months ago
- Status changed from In Progress to Feedback
I have tested voy2isis on the Raw version of the 'rings node version' and it creates an output cub, but there is no output instrument group on the labels. It seems there is no translation at all. With the Raw version a full ingestion should be possible to proceed with spiceinit.
I have also tested the other versions (CALIB, GEOMED, etc) and the voy2isis failure with clear message that these versions are currently not supported is sufficient for now.
work area: /work/users/tbecker/IsisTesting/M04421_voy2isis/vg_0024/europa/C2062524_RAW.cub
The input is locate: /usgs/shareall/tbecker/VoyagerRingsNode/C2062524_RAW.LBL, IMG
#21 Updated by Christopher Combs 2 months ago
I've updated the voyager.trn file to include the .LBL specific keys and values.
At the moment, this is using a local version of the translation file, but I will update the official ISIS3DATA version after testing.
Built on prog24 and ready for testing at /work/projects/isis/latest/m04345_m04421.
#23 Updated by Tammy Becker about 2 months ago
- Status changed from Resolved to Closed
I have tested the most recent voy2isis and it now functions as planned for now. Currently, ISIS3 will only ingest the _RAW version of the RingsNodeArchive. Upon the attempt to ingest the other processed versions from this archive, the user will be notified with a message explaining that ISIS3 does not currently support it.
The _CALIB, _CLEANED, _GEOMA, _GEOMB and _RESLOC will not be supported until resources are available to handle these processed versions appropriately within the ISIS3 environment.
This will be considered a new development and design that will need to be requested in a separate request/ticket with appropriate funding.