Project

General

Profile

Bug #5223

M3 default kernels need to be removed from the data area

Added by Lynn Weller 10 months ago. Updated 7 months ago.

Status:
Feedback
Priority:
Block
Assignee:
Category:
Applications
Target version:
Impact:

Allow Chandrayaan 1 M3 images to be spiceinited with the default SPICE data setup.

Software Version:
Test Reviewer:

Description

Spiceinit fails on ingested Chandrayaan1 M3 L1B data products when run in default mode. The available default kernels are outdated, incorrect and incompatible with the current M3 camera model in the public version of isis.

The following file should be removed from the system as soon as possible:
$chandrayaan1/kernels/ck/Chan1M3_ck_loc_solution_2015-03-25.bc
$chandrayaan1/kernels/spk/Chan1M3_spk_loc_solution_2015-03-25.bsp

These kernels were built upon a camera model that used the PDS label start and stop times, not Joe Boardman's improved start/stop times supplied in the TIM.TAB files which are written to the L1B data labels when ingested by chan1m32isis. Additionally, the light time correction had been set to ON for the input into these kernels among other things that have since been corrected. Under most circumstances spiceinit will fail for M3 images passed to it due to the differences in time on the label versus what is stored in the kernel, however there will be a number of images (perhaps 1/3 of the data) that will fit an available time span and run through successfully applying somewhat incorrect kernels.

At the very least the incorrect kernels should be removed, but that will leave users with no way to access the M3 camera model running spiceinit in default mode. User may apply a camera model by explicitly asking for cknadir pointing and pointing to a very specific JPL spacecraft kernel ($ISIS3DATA/chandrayaan1/kernels/spk/CH-1-JPL-MERGED-23-MARCH-2010-1220.BSP), but they would have to know they need to do this and the resulting geometry would render the data essentially useless.

The method by which the system kernels were generated has been reproduced using the proper start/stop times, light time correction settings, etc. and could be supplied as the default kernels. They are not smithed kernels (which are soon to come), but are based upon Joe Boardman's unreleased improved location (LOC) files that tapped into a better version of the LOLA data than what was available at the time for PDS released LOC files. M3 related work in house resected the M3 L1B archived PDS images using Boardman's improved LOC files resulting in a camera and spacecraft kernel for each image. Those kernels could be made available to users instead of the currently available incorrect kernels.

1- Please remove the noted incorrect kernels
2 - Please build a new kernels database for M3 using the image kernels available under /archive/projects/m3/JBLOC_Products/kernels/
There are 883 camera kernels having the naming format *_V03_L1B_nadir-jig_2016-04-29.bc and 883 spacecraft kernels with the format *_V03_L1B_nadir-jig_2016-04-29.bsp
Four of the L1B PDS archived images did not make it through the resection process due to a lack of data and insufficient information for the bundle adjustment to work with.

The current default kernels are being referred to as Reconstructed which can be kept for their replacement (unless there is a more appropriate term).

Test data demonstrating the problem are available under /work/users/lweller/Isis3Tests/Spiceinit/M3/. Here are the processing steps to ingest and spice (along w/ the spiceinit error):

chan1m32isis from=/pds_san/PDS_Archive/Chandrayaan_1/M3/CH1M3_0003/DATA/20090415_20090816/200906/L1B/M3G20090602T163012_V03_L1B.LBL to=M3G20090602T163012_V03_L1B.cub

spiceinit from=M3G20090602T163012_V03_L1B.cub

Error = "No pointing available at requested time
[297233397.69998] for frame code [-86520]"
End_Group

Group = Error
Program = spiceinit
Code = 1
Message = "Unable to initialize camera model"
File = spiceinit.cpp
Line = 240
End_Group


Related issues

Blocked by ISIS - Feature #5272: Modify kerneldbgen for M3 Closed

History

#1 Updated by Lynn Weller 10 months ago

  • Description updated (diff)

#2 Updated by Tammy Becker 10 months ago

  • Status changed from New to Acknowledged

#3 Updated by Lynn Weller 9 months ago

  • Priority changed from High to Block

#4 Updated by Stuart Sides 9 months ago

  • Status changed from Acknowledged to Resolved
  • Assignee set to Stuart Sides
  • Target version set to N/A

PLEASE DO NOT CLOSE this issue until the kernels have been pushed to the external download areas.
If testing shows no problems set this issue to FEEDBACK with a note.

The individual 883 CK and 883 SPK kernels have been placed into the Chandrayaan data area as "reconstructed" and the kernel databases updated. The old "solution" kernels were moved to a "SAVE" directory under the CK and SPK areas. The JPL kernels remain in their directories, but are no longer in the database. The changes can be pushed to the external site once testing has been completed.

#5 Updated by Lynn Weller 9 months ago

I tested the changes on the image in my testing area and spiceinit successfully ran. I will test all 883 files available via a cluster run to be sure they are all successful and report back when that is complete.

#6 Updated by Lynn Weller 9 months ago

  • Status changed from Resolved to Feedback

I ran all 883 M3 usable L1B files through spiceinit letting the program default and there were a number of failures (86) where "No Camera Kernels found for the image". I took one of the images and ran spiceinit pointing to the original kernels location and the kernels loaded without problem.

Attached is the list of images that failed spiceinit using default kernels (Can't attach - says extension .lis is not allowed). I have also copied the ingested images into the following directory for ease of use:
/scratch/lweller/M3/JBLOC_Products/ProcLev1/DefaultSpice/Redo/Lev1/
I can put these somewhere else if getting to /shareall is a problem.
Since I can't attach files see failures_root.lis in the same directory as the level 1 images.

Here is how I am running spiceinit:

This does not work
spiceinit from=M3G20090611T090220_V03_L1B_full.cub attach=true shape=user model=/archive/projects/m3/TOPO/LRO_LOLA_Global_LDEM_118m_mar2014_radius.cub

This works
spiceinit from=M3G20090611T090220_V03_L1B_full.cub attach=true shape=user model=/archive/projects/m3/TOPO/LRO_LOLA_Global_LDEM_118m_mar2014_radius.cub ck=/archive/projects/m3/JBLOC_Products/kernels/M3G20090611T090220_V03_L1B_nadir-jig_2016-04-29.bc spk=/archive/projects/m3/JBLOC_Products/kernels/M3G20090611T090220_V03_L1B_nadir-jig_2016-04-29.bsp

NOTE: 6 of the images in the list (M3G20090602T124100, M3G20090621T192751, M3G20090714T035832, M3G20090717T174402, M3G20090717T211502, M3G20090814T061121) were a problem for one of our collaborators - he said the times on the images did not match the time window of the kernels we gave him (these would have been based on jigsaw updated images, not these reconstructed kernels, but apparently the problem was present from the start), even though we made the kernels based on whatever time was on those images. I think the problem was with the stop times. These images had time gaps in the TIM.TAB files and had NULL lines added. Of the 883 images in the set, 212 had some number of NULL lines added and did not have problems like these 6, so I'm not sure what makes these images special.

#7 Updated by Lynn Weller 9 months ago

Note: These are not the same images Joe had issues with years ago - similar number of files, but only 4 in his list show up in this list of problem files.

#8 Updated by Stuart Sides 9 months ago

I'm looking into why these are failing.

#9 Updated by Stuart Sides 9 months ago

  • Status changed from Feedback to In Progress
  • Impact updated (diff)

#10 Updated by Lynn Weller 9 months ago

Reminder for myself: These kernels are based on bundling to Joe's LOC file. The input image to that process were spiced with ck=nadir and spk=jpl spacecraft kernels. The updated images based on that process are in the same directory as the kernels.

ckwriter and spkwriter were run in this manner to generate the kernels under /archive/projects/m3/JBLOC_Products/kernels/:

ckwriter from=M3T20090701T140357_V03_L1B.cub to=M3T20090701T140357_V03_L1B_basematchgrnd_2016_12_29.bc summary=Kernels/M3T20090701T140357_V03_L1B_basematchgrnd_2016_12_29_summary_ck.doc

spkwriter from=M3T20090701T140357_V03_L1B.cub to to=M3T20090701T140357_V03_L1B_basematchgrnd_2016_12_29.bsp summary=Kernels/M3T20090701T140357_V03_L1B_basematchgrnd_2016_12_29_summary_spk.doc type=9

#11 Updated by Lynn Weller 9 months ago

Could post #2246 have anything to do with the problem?
Just wondering about that JPL spacecraft kernel and having to explicitly use it when I want nadir pointing.

It may have nothing to do with this at all, but it occurred to me that it might when we tried to run spiceinit and just pointed to the ck kernel and let the spk kernel default and got an error.

#12 Updated by Stuart Sides 8 months ago

  • Target version changed from N/A to 3.5.2 (2017-01-31 Jan)

The newest kernels from spkwriter and ckwriter do not cover the bottom of the last line for 86 M3 images. This is probably due to the way the CK and SPK caches are created when spiceinit is run and the tables are attached to the cube. This equates to about 1/3 of an exposure duration as seen on a few of the 86 images. For the time being we need to ignore that small amount of data and get updated kernels available to all.
1) adjust the start and stop time in kerneldbgen
2) adjust the padding in ckwriter and spkwriter

In the future we need to make sure the top of line 1 and the bottom of line NL are covered by the caches.

#13 Updated by Stuart Sides 8 months ago

#14 Updated by Stuart Sides 7 months ago

  • Status changed from In Progress to Resolved

I've updated the database files for the M3 CK and SPK data areas. I expanded the time range by 0.2sec (i.e., 0.1sec before start and 0.1sec after end). This adds just under one line worth of time to the top and bottom. This should allow spiceinit to find the correct CK and SPK files. If it does not, I will need to look into the ones that fail. I'm not comfortable with expanding more than this without understanding it first.

#15 Updated by Lynn Weller 7 months ago

Working on getting all my data through spiceinit via the cluster, but there are some issues there. Hopefully within the next we'll have an all successful answer.

#16 Updated by Lynn Weller 7 months ago

  • Status changed from Resolved to Feedback

The cluster problems have been addressed and I was able to rerun my job attempting to use the new default kernels db file.

Three images failed with "No Camera Kernels found for the image":

M3G20090602T124100
M3G20090621T192751
M3G20090714T035832

They are available under /scratch/lweller/M3/JBLOC_Products/ProcLev1/DefaultSpice/Failures/. See spice.scr for how the images were run through spiceinit.

FYI (or red herring!): All of these images had NULL lines added to them upon ingestion and were ultimately an issue for Joe when he received updated kernels from us because there was a large discrepancy between the times in the kernels and those in the TIM.TAB (and therefore on the labels) file. There were three other images that were a problem for him as well, but they seemed to get through spiceinit ok. There are other images that had more lines added than these (except for M3G20090714T035832, which had the greatest # added), and they posed no problems, but I thought I'd mention Joe's problem in case it provided some insight.

#17 Updated by Stuart Sides 7 months ago

Those three are more than a minute off. That is too far out to call it a rounding problem. I'll track it down.

Also available in: Atom PDF