Project

General

Profile

Question #4907

hideal2pds fails on images with AlphaCube group in label

Added by Audrie Fennema about 1 year ago. Updated 11 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Applications
Target version:
Test Reviewer:

Description

Background: The process we use to clean noise from our cubes has many steps but includes using the ISIS fft app. This app adds the AlphaCube group to the label of the magnitude cube. After running hinoproj (or hijitter) on the cleaned cubes, we attempt to run hideal2pds on the cubes to create a PDS product and get the following error:

hideal2pds to=/tmp/ESP_025683_1900_pnode03:1276/ESP_025683_1900_RED.NOPROJ.img min=0.112181199346750 version=1.0 from=/tmp/ESP_025683_1900_pnode03:1276/ESP_025683_1900_RED.NOPROJ.cub max=0.207512324669180 type=MANUAL
Object = hideal2pds
IsisVersion = "3.4.13.7091 stable | 2016-03-15"
ProgramVersion = 2015-10-06
ProgramPath = /opt/usgs/isis/bin
ExecutionDateTime = 2017-05-24T21:53:11
HostName = pnode03
UserName = audrie
Description = "Export an Isis3 mosaicked ideal cube from a HiRISE
image to a PDS product containing related spice
data."

Group = UserParameters
FROM = /tmp/ESP_025683_1900_pnode03:1276/ESP_025683_1900_RED.NOPROJ.cub
TO = /tmp/ESP_025683_1900_pnode03:1276/ESP_025683_1900_RED.NOPROJ.img
VERSION = 1.0
TYPE = MANUAL
MIN = 0.112181199346750
MAX = 0.207512324669180
End_Group

Group = Error
Program = hideal2pds
Code = 1
Message = "The AlphaCube group values of the input Isis cube indicate that
this cube has been scaled. Unable to export scaled cubes to PDS
using hideal2pds"
File = hideal2pds.cpp
Line = 368
End_Group
End_Object
End

Here is the AlphaCube group from the label:

Group = AlphaCube
AlphaSamples = 1024
AlphaLines = 80000
AlphaStartingSample = 0.5
AlphaStartingLine = 0.5
AlphaEndingSample = 1024.5
AlphaEndingLine = 80000.5
BetaSamples = 1024
BetaLines = 80000
End_Group

Our noise cleaning process does not include scaling of the image. I'm not clear on exactly what fft does with the magnitude cube or if it involves any scaling. I can get hideal2pds to run successfully on the NOPROJ.cub by removing the AlphaCube group.

My question is, are there unintended implications to removing the AlphaCube group? We don't want to create problems with our products by doing this. Or is there a better way to get around this problem?

Thanks,
Audrie

History

#1 Updated by Tammy Becker about 1 year ago

  • Status changed from New to Acknowledged

#2 Updated by Stuart Sides about 1 year ago

  • Target version set to 3.5.1 (Sprint 1)

#3 Updated by Moses Milazzo about 1 year ago

Hi Audrie,

The ISIS3 (and most) FFTs require an image with power of 2 dimensions. I wonder if you can test this on an image that you've cropped to be, say 1024 samples X 65,536 lines instead of 80,000. It may be the case that for some reason the conversion to Phase and Magnitude domain (power of 2 padding) and back from the Phase and Magnitude domain (should undo the power of 2 padding) is creating an inconsistency between the AlphaCube and the true dimensions.

--Moses

#4 Updated by Kristin Berry about 1 year ago

  • Assignee set to Ian Humphrey

#5 Updated by Ian Humphrey about 1 year ago

Hi Audrie,

Would it be possible for you to provide the exact commands you've run when cleaning noise?

Thanks,
Ian

#6 Updated by Ian Humphrey about 1 year ago

  • Status changed from Acknowledged to In Progress

#7 Updated by Audrie Fennema about 1 year ago

Ian here are the commands we use to create the clean.cub:

fft magnitude=ESP_025683_1900_RED5_0.mag.cub phase=ESP_025683_1900_RED5_0.phase.cub from=ESP_025683_1900_RED5_0.cubenorm.cub

fx samples=101 to=ESP_025683_1900_RED5_0.circ.cub mode=outputonly lines=101 equation="1 - (1/{--pi*[1+((line - 51)/25)2]} + 1/{--pi*[1 + ((sample - 51)/25)2]})"

stretch to=ESP_025683_1900_RED5_0.circ.stretch.cub usepercentages=true from=ESP_025683_1900_RED5_0.circ.cub pairs="0.0:1 0.01:1 75:.99 80:0.9 100:0"

enlarge to=ESP_025683_1900_RED5_0.circ.enlarge.cub ons=1024 mode=total interp=cubic from=ESP_025683_1900_RED5_0.circ.stretch.cub onl=131072

fx samples=1024 to=ESP_025683_1900_RED5_0.cross.cub mode=outputonly lines=131072 equation="[1 - e--pi*{(line - (( 131072 / 2 +1))6})] * [1 - e--pi*{(sample - (( 1024 / 2 +1))6})]"

fx to=ESP_025683_1900_RED5_0.mask.cub f1=ESP_025683_1900_RED5_0.cross.cub f2=ESP_025683_1900_RED5_0.circ.enlarge.cub equation="f1 * f2 "

fx to=ESP_025683_1900_RED5_0.magfilter.cub f1=ESP_025683_1900_RED5_0.mag.cub f2=ESP_025683_1900_RED5_0.mask.cub equation="f2 * abs(f1)"

fx to=ESP_025683_1900_RED5_0.magfilter.cubenorm.cub f1=ESP_025683_1900_RED5_0.magfilter.cub equation=f1/8.5302289837547

gradient to=ESP_025683_1900_RED5_0.magfilter.sobel.cub from=ESP_025683_1900_RED5_0.magfilter.cubenorm.cub gradtype=sobel

fx to=ESP_025683_1900_RED5_0.threshold.cub f1=ESP_025683_1900_RED5_0.magfilter.sobel.cub equation="f1 > { 5 * cubeavg(f1) }"

gauss stddev=6.0 to=ESP_025683_1900_RED5_0.gauss.cub from=ESP_025683_1900_RED5_0.threshold.cub size=5

fx to=ESP_025683_1900_RED5_0.threshold.gauss.cub f1=ESP_025683_1900_RED5_0.gauss.cub equation="f1 > 0"

fx to=ESP_025683_1900_RED5_0.threshold.cross.cub f1=ESP_025683_1900_RED5_0.threshold.gauss.cub f2=ESP_025683_1900_RED5_0.cross.cub equation="f1 * f2"

fx to=ESP_025683_1900_RED5_0.magfilt.divideby.cub f1=ESP_025683_1900_RED5_0.threshold.cross.cub equation="(f1 * 14413214.0)+ 1"

fx to=ESP_025683_1900_RED5_0.magclean.cub f1=ESP_025683_1900_RED5_0.mag.cub f2=ESP_025683_1900_RED5_0.magfilt.divideby.cub equation="f1/f2"

ifft magnitude=ESP_025683_1900_RED5_0.magclean.cub to=ESP_025683_1900_RED5_0.clean.cub phase=ESP_025683_1900_RED5_0.phase.cub

Audrie

#8 Updated by Audrie Fennema about 1 year ago

Moses,

I'll try running the test that you suggested.

Thanks,
Audrie

#9 Updated by Ian Humphrey about 1 year ago

Hi Audrie,

As a temporary workaround it is okay to remove the AlphaCube group before running hideal2pds, it will not affect the integrity of your products.

You can remove the group with editlab, editlab from=yourImage options=delg grpname=AlphaCube

We are working on a solution so that you will not have to remove the AlphaCube group before running hideal2pds.

-Ian

#10 Updated by Audrie Fennema about 1 year ago

Hi Ian,

That's exactly what I've been doing. :)

Thanks for the help with this.

Audrie

#12 Updated by Ian Humphrey about 1 year ago

  • Status changed from In Progress to Resolved

hideal2pds was failing because of ifft and noproj in relation to an AlphaCube group.

When fft is run, it propagates the input cube's AlphaCube group (if it has one, e.g. it has been cropped), or it creates an AlphaCube group to allow ifft to restore the cube to the correct size. Note that fft creates output cubes where the samples and lines have to be powers of 2.

Now, when noproj is run, it propagates the labels from the match cube (if no match is provided, the input cube is the match cube).

hideal2pds was failing because it checks the input cube for an AlphaCube group and sees if it has been scaled.

If you run a cube through fft, ifft, noproj, and hideal2pds, hideal2pds fails with the error message described in the ticket. This occurs because ifft doesn't not remove the AlphaCube the fft creates. Then, when noproj is run, the sample dimensions can be larger for the output image (since the image is resampled to the ideal ccd size). Now, there is a mismatch between noproj's output and the AlphaCube it contains, so hideal2pds throws an error.

Now, ifft removes an AlphaCube if it was not originally present before running fft. noproj also removes the AlphaCube group from the output, since noproj is resampling the image it doesn't make sense to keep in an AlphaCube group (cannot un-noproj an image).

#13 Updated by John Shinaman about 1 year ago

  • Status changed from Resolved to Testing

#14 Updated by John Shinaman about 1 year ago

Tested and operating as advertised.

#15 Updated by John Shinaman about 1 year ago

  • Test Reviewer set to John Shinaman

#16 Updated by Stuart Sides 11 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.2 (2017-01-31 Jan)

#17 Updated by Makayla Shepherd 11 months ago

  • Status changed from Testing to Closed

Also available in: Atom PDF