ISIS Application Documentation
isisminer | Printer Friendly View | TOC | Home |
Run a series of algorithms (Strategies) that perform various operations on input sources (Resources)
| Overview | Parameters | Example 1 |
DescriptionThe isisminer application assists in the identification, manipulation, and output of data from a variety of data sources. One main source of input is caminfo. Processing in the isisminer program is controlled by the contents of a PVL-based, object-structured, configuration file provided in the CONFIG parameter. There are two main PVL objects used in the configuration of an isisminer program run: the IsisMiner object and the Strategy object. Strategy AlgorithmsThe IsisMiner object contains a series of Strategy objects. Strategy objects are associated with a defined algorithm that performs some operation on data called a Resource. Strategy objects contain parameters that are needed by the algorithm to process Resource data. The order of Strategy objects is important as the algorithms are executed in the order they occur in the IsisMiner object. Each Strategy object must be associated with a known Strategy Type. Other objects may exist anywhere in the CONFIG file but are ignored when parsing Strategy objects unless they are part of the algorithm. One common use for other objects in the CONFIG file is to include database profiles that are used to provide access information for the DatabaseReader strategy (see below). Data ResourcesStrategy algorithms operate on a Resource. A Resource is a data container that stores a set of PVL keyword/values that can have data storage extensions (called Assets and GisGeometries). These Assets can contain anything created by Strategy objects for further analysis or processing and could be unique to a set of Strategies. Resources have three major components: keyword/values, GIS geometries and Assets. Most of the data manipulations occur on keyword/value elements. A Resource may contain a GIS geometry that is used in algorithms that work on geometric regions using GIS operators (GIS Intersect, GIS Overlap, GIS Union, StereoPair ). Currently, GIS geometries must be in well-known-text (WKT) or well-known-binary (WKB) as conversions are restricted to text representations of geometries (format requirements are the same as functions like ST_GeomFromWKT and ST_GeomFromWKB, respectively). Resources may also contain Assets. Assets are typically another list of Resources created from Strategies that are attached to a (parent) Resource. If Assets are represented as Resource lists, they can be processed by almost all Strategy algorithms using the AssetSidebar strategy (see below). Resources can come from many sources. One of the most common input sources are databases. SQL queries can be specified directly in the DatabaseReader strategy. Each row returned when the query is executed is converted into a Resource. Other data sources are comma separated values (CSV) files (using the CsvReader strategy) where each row is a Resource and each column is a keyword/value. The keyword name in the case of a CSV file is the name of the column as specified in the header. Users can also read in ISIS control networks and process each Control Measure as a Resource. And, of course, ISIS cube files and other PVL file formats are also input sources (see PvlReader strategy). Strategies are designed to operate on a single Resource. This allows for a very narrow scope of design considerations to make coding Strategies quite easy. Control Options using Keyword Argument SubstitutionSome very clever and sophisticated Strategy configurations can be created but to make them generally useful, the user may need to change some parameters within Strategies, such as output directories or file names that can be modified at runtime without having to edit the CONFIG files directly. To allow some control options, there are two optional application parameters provided: GLOBALS and PARAMETERS. The GLOBALS parameter can specify the name of a file that contains PVL keywords and values. The complete contents of all the keywords found in the file, regardless of Object/Group structure are loaded immediately when isisminer first runs. The keywords contained in this file can be used as substitutions in all or part of many keyword value specifications used within most Strategies (described below). The PARAMETERS input parameter is provided so that users can create simple keyword/value pairs on the command line that can be used in substitutions of part of all keyword values used in Strategies in the same manner GLOBALS keywords are substituted. For any keyword that is contained in both the GLOBALS and PARAMETERS keyword set, keywords in PARAMETERS always take precedence. This allows users to override keywords in GLOBALS as needed during keyword lookup and replacement operations to fully determine values of keywords in real time when Strategies are executed. Since equal signs are not allowed in ISIS application parameters, keyword names and values are separated by a colon. Multiple keywords can be specified and are separated by the @ symbol. For example, PARAMETERS="inputdir:$PWD@outputdir:$PWD/Results" will create two keywords, inputdir and outputdir, with inputdir set to the full path of the current directory running the job and outputdir set to a subdirectory in the current directory called Results. Essentially, the GLOBALS and PARAMETERS elements creates a hierarchical lookup for keyword substitutions where PARAMETERS are searched first, then GLOBALS, for argument substitution of elements within a keyword value. At times, when individual Resources drive output operations (e.g., AssetSidebars), the individual Resource keywords become available for substitution as well and are the first set of keywords searched before PARAMETERS and GLOBALS. Argument substitutions are specified using the %n symbol in keyword values. Here, the n is a number from 1 to the number of values corresponding to an associated keyword with the same name but with Args appended to it. The nth value of this keyword is itself a keyword where its value is substituted in the corresponding %n occurrence in the original keyword. For example, in the CSVReader strategy, the name of a file to read the contents from is specified in the CsvFile keyword. To allow arbitrary specification of the input file without editing the CONFIG file to change it for each use of the CONFIG file, the PARAMETERS option can be used to specify the path to a named file or the entire file name by creating a keyword called CsvFileArgs. This is the general form for creating a keyword value substitution that uses two associated keywords with the first being any keyword and the second keyword having Args appended to the first keyword which contains other keywords that will be used to substitute arguments with. See the complete stereo example provided for isisminer that demonstrates many of the concepts discussed here and in the Strategy descriptions below. It will also provide output and describe the results of a MESSENGER MDIS stereo processing run. Descriptions of StrategiesThe following section describes all the Strategy algorithms currently available in IsisMiner. Strategies iterate through all active Resources. One of the features in isisminer that provides a lot of flexibility in the system is that Resources are never deleted unless users provide explicit instructions to do so (see ResourceManager). Each Resource has an inherent state that indicates if it is active, meaning it is available for processing, or inactive. Strategies like the Limit or Filter will evaluate values of keywords for boolean equations or strings, respectively, where true conditions keep Resource states as active, and false conditions set their states to inactive. Inactive Resources are not processed in subsequent Strategies unless explicitly reactivated (again using the ResourceManager). Strategies typically only use active Resources when normally processed. Exceptional is when a Strategy provides options to select an additional processing set of Resources (see GisOverlap, StereoPair and Sidebar). Those Resources accept a complete isisminer configuration where the object specification can be embedded within the Strategy object or a file name can be provided that contains a complete isisminer configuration (much like provided in CONFIG). This design allows users to progressively select Resources for processing and reactivate the entire list at any point in the execution of Strategies. The Strategies below provide the user with many options to help identify only data with specified/desired characteristics for evaluation and subsequent processing. Special Strategy KeywordsThere are some keywords available that can be used in every Strategy beyond their unique keywords described below. Many of these keywords provide additional processing control or options that produce information in the form of text to the standard output stream. The following tables provide descriptions of the keywords that can be used in all Strategies. Some will apply only under certain conditions which are described in the context of the descriptions. All of these keywords are optional except where indicated. Some miscellaneous keywords are available that provide general information and/or assist users in debugging Strategies that appear to result in unexpected behavior. Here are other keywords that users can set and are available in all Strategies.
The following table describes keywords that pertain to manipulation of GIS geometries. They are available to all Strategies that use GIS geometries but only apply where indicated.
Types of Strategies
AssetSidebar Strategy DescriptionThe AssetSidebar strategy provides the ability to process Resource Assets with most other isisminer Strategies.This Strategy can be used to add Assets to any active Resource. By providing the Operation keyword with a value of Create, the user can then process an Asset to be added to the active Resources (with the PvlReader or CsvReader strategies, for example). Note that when using create as a value for the Operation keyword, the specified Strategy in the AssetSidebar strategy will not be applied to any existing Assets contained in the active Resources. In fact, any Asset that exists in a Resource of the same name will be deleted and replaced with the results of the AssetSidebar processing. Note that some Strategies create Assets directly in the course of executing its algorithm and do not require an AssetSidebar to create them. When creating Assets, users have several options to create the Resource list for the Asset. Using the CreateSource keyword with a value of Copy, a copy of all Resources in the input list is generated prior to executing the Strategies to process the Assets. The active/deactive status of each new Resource in this list is managed independently from the list outside of the AssetSidebar processing it. However, keywords, other Assets, and the GIS Geometry remain identical to those of the parent Resource. To create a completely independent copy of the Resources, set CreateSource to Clone. This option clones all the status and data of each Resource and creates a completely new Resource. To start from scratch and create an Asset by other means, use None. Defaults to Copy the Resource list. This Strategy may also be used to apply a specified Strategy to any active Resource's Assets. For Operation set to Process, the specified Strategy in the AssetSidebar will be applied to the active Resources' Assets and replace the processed Asset in active Resources as new Assets. This mode is particularly useful for processing Assets created directly by other Strategies.
The following is an example of an AssetSidebar strategy
Object = Strategy
Name = TestCreateAsset
Type = AssetSidebar
Asset = AssetA
Operation = Create
CreateSource = None
Description = "Adds AssetA (Assets read from test.csv) to
the active Resources"
Object = IsisMiner
Object = Strategy
Name = ReadAssetTest
Type = CsvReader
CsvFile = "test.csv"
HasHeader = True
SkipLines = 0
IgnoreComments = False
Delimiter = ", "
EndObject
EndObject
EndObject
Return to List of Strategy Types
Calculator Strategy DescriptionThe Calculator provides the ability to do calculations on Resource keywords and to store the results in a keyword that is immediately added to the Resource. If any equation fails for a Resource, then that Resource is discarded and no further equations are executed for that Resource.There are two ways equations can be specified. The basic method requires two keywords to compute, execute, and store the results in each Resource. The Equation keyword stores the complete equation to execute for each Resource and uses the Result keyword value to name the keyword to store the result of the equation. An alternative form uses a PVL Group named Equations which contains a sequence of keywords whose values are equations. A keyword of the same name is created in each Resource with the value of each successfully computed equation. Values of previous equations can be used in subsequent equations. No other keyword may exist in the Equations group as every keyword is expected to be a valid equation. Currently this Strategy supports scalars in the form of integers and doubles. In addition, it supports the circle constant pi and Euler's constant e. Functions, such as sin and cos, are followed by parentheses and a list of arguments. Operators such as * and + are placed in between their arguments. If a token is not recognized as a scalar, operator, or function, then it is assumed to be a keyword, and will be replaced by the corresponding value in the Resource. All trigonometric functions expect angles in radians, not degrees so use the rads() function! The following table shows all currently supported operators and functions sorted by precedence (0 is the highest precedence). Functions have a precedence of 1 and operators have a precedence of 2 or higher, according to order of operations. All examples are valid equations using 'x' and 'y' as variables:
The following is an example of a Calculator strategy:
Object = Strategy
Name = Trigonometry
Type = Calculator
Description = "Calculate trigonometric functions of angle."
Group = Initializers
Sine = 0
Cosine = 0
Tangent = 0
EndGroup
Group = Equations
Sine = "sin(angle)"
Cosine = "cos(angle)"
Tangent = "tan(angle)"
EndGroup
EndObject
Return to List of Strategy Types
Control Network Reader Strategy DescriptionThe CnetReader strategy creates Resources from a provided control network file. Each Resource is created from a ControlMeasure group in the control network. The keywords of the ControlMeasure group become the keywords of the Resource.If the Identity keyword is not provided or its value is an empty string, each Resource is named with a unique number (starting at 0).
The following is an example of a CnetReader strategy:
Object = Strategy
Name = ControlNetworkReader
Type = CnetReader
CnetFile = "%1/Alph_VIS.net"
CnetFileArgs = "inputdir"
Identity = "%1_%2"
IdentityArgs = (PointId, SerialNumber)
Description = "Read from a control network file"
EndObject
Return to List of Strategy Types
CSV Reader Strategy DescriptionThe CsvReader strategy creates Resources from the rows of a Comma-Separated Values (CSV) file. Each row becomes a Resource with the keyword names from the CSV file header. The keyword values become the values of the corresponding columns in that row. If the CSV file does not have a header, the default keyword names are Column0, Column1, etc.The CSV file must have the following properties. The rows must be delimited by a new line. The columns must be delimited by a single character. The number of columns in each row must be the same for all rows. If IgnoreComments is set to True, then Comments must be on their own line and be preceded by a #.
The following is an example of a CsvReader strategy:
Object = Strategy
Name = ReadFilterData
Type = CsvReader
CsvFile = "%1/csvreader_data.csv"
CsvFileArgs = "inputdir"
HasHeader = False
SkipLines = 0
IgnoreComments = False
Delimiter = ","
Identity = "%1_%2"
IdentityArgs = ( SourceProductId, StereoSource )
EndObject
Return to List of Strategy Types
CSVWriter Strategy DescriptionThe CsvWriter strategy provides users with the ability to export Resources to a Comma-Separated Values (CSV) file. Each row corresponds to a Resource and each column corresponds to a keyword specified by the Keywords parameter. If the Header parameter is set to True, the keyword names are stored in the first row of the CSV file. Each Resource is written to the CSV file as a row with the corresponding keyword values written to each column. If a Resource does not have a specified keyword, the value of the DefaultValue parameter is written instead.Resource geometries can be written to the CSV file as keywords using the GisGeometryKey parameter in the format specified by the GisType parameter. The two supported formats are well-known binary (WKB) and well-known text (WKT). Note: Writing a geometry to a CSV file in WKT format while using the default comma delimiter may result in the CSV file being unreadable by CsvReader. Change the delimiter to a different character if using WKT format. Rows in the CSV file will be delimited by a new line. Columns are delimited by the character specified by the Delimiter parameter, which defaults to a comma.
The following is an example of a CsvWriter strategy:
Object = Strategy
Type = CsvWriter
Name = mdismla
CSVFile = "mdis_mla_ridelong.lis"
Mode = Create
Header = True
Keywords = (YearDoy, SourceProductId, StartTime, EtStartTime,
ExposureDuration, CenterLongitude, CenterLatitude,
PixelResolution, MeanGroundResolution,
IncidenceAngle, EmissionAngle, PhaseAngle,
SubSolarGroundAzimuth, SubSpacecraftGroundAzimuth,
ParallaxX, ParallaxY, ShadowX, ShadowY)
Delimiter = ","
DefaultValue = "NULL"
EndObject
Return to List of Strategy Types
DatabaseReader Strategy DescriptionThe DatabaseReader strategy provides direct SQL queries to databases. Data rows retrieved from a query are stored as Resources. Using the Target parameter, new Resources can either be stored on their own as Resources, or as Assets in existing Resources. Using an Asset target is really a shortcut approach to creating Assets in Resources that normally would require an AssetSidebar strategy.The access details of the database can be stored in a configuration file in a profile of the Database object. The configuration file containing the Database object is specified by the DbConfig parameter. The database profile is specified by the DbProfile parameter. A special parameter called DbFile is provided for SQLite files. This shortened specification for SQLite files eliminates the need for creating an elaborate Database configuration for file-based databases. Generally, any valid SQL query can be used that returns rows from the results of the query issued to any database. Any database that is supported by the Qt system configuration shipped with each ISIS release can accept SQL queries. This typically includes PostgreSQL, MySQL and SQLite. When specifying the query in the Query, some special quoting may be needed in order to comply with SQL syntax. Before a query is sent to the database, all occurrences of " and ' in the query string are replaced with double quotes [ " ] and single quotes [ ' ], respectively.
The following is an example of a valid configuration file named mdis.conf with a DatabaseReader strategy within an IsisMiner object and a Database object:
Object = IsisMiner
Object = Strategy
Name = MDISSelection
Type = DatabaseReader
DbConfig = mdis.conf
DbProfile = MDIS
Query = "select i.SourceProductId, i.YearDoy,
g.StartTime, i.EtStartTime, i.ExposureDuration,
g.CenterLongitude, g.CenterLatitude, i.ObservationType,
g.PixelResolution, g.MeanGroundResolution,
g.IncidenceAngle, g.EmissionAngle, g.PhaseAngle,
g.SubSolarGroundAzimuth, g.SubSpacecraftGroundAzimuth,
g.ParallaxX, g.ParallaxY, g.ShadowX, g.ShadowY,
p.GisFootprint
from Polygon p
INNER JOIN Image i
ON p.SourceProductId=i.SourceProductId
INNER JOIN Geometry g
ON p.SourceProductId=g.SourceProductId
INNER JOIN Statistics s
ON p.SourceProductId=s.SourceProductId
where (i.Center=747.7 or i.Center=748.7)
order by g.PixelResolution"
Target = Resource
Identity = "%1"
IdentityArgs = "SourceProductId"
GisGeometryRef = GisFootprint
GisType = WKB
RemoveGisKeywordAfterImport = True
EndObject
EndObject
Object = Database
DefaultProfile = "MDIS"
Group = Profile
Name = "MDIS"
Dbname = "Messenger_DOY_2011095_TO_2013011_gis.db"
Type = SQLite
Description = "MDIS Sqlite database"
EndGroup
EndObject
Return to List of Strategy Types
Filter Strategy DescriptionThe Filter strategy provides users with the ability to activate (Include) or deactivate (Exclude) Resources that have or do not have keywords with certain values.All comparisons are case-insensitive. The keyword used to filter Resources is specified in the Keyword key in the Filter Strategy object definition found in the Strategy object. Only the first value in the named Resource Keyword will be checked unless the [optional] CheckAll keyword is set to True. Values contained in the Include keyword in the Strategy object will be compared with values in the Resource key specified in the Keyword key. If a value exists in both keywords, then the Resource is retained (as active). Values contained in the Exclude keyword in the Strategy object that exist in the Resource key will deactivate a Resource. Both Include and Exclude keywords may exist in the Filter Strategy object. In this case, values are first checked in the Include list then the Exclude list. Whichever occurs first, that action is taken for the Resource. If none of the keyword values exist in either Include or Exclude keywords, then disposition of the Resource depends on the lists. If the Include list is not empty and the Exclude list is empty, the Resource is deactivated. Otherwise, the Resource is retained as active. If needed, users can provide a regular expression string that will use Qt's QRegExp class implementation to apply to keyword values. If given, this test is applied after the Include and before the Exclude test to determine if the value satisfies the regular expression. If after all the tests are applied and this string was supplied, the Resource is discarded since it did not succeed in its test. If neither Include nor Exclude keywords exist, then no action is taken for the Resources.
The following is an example of a Filter strategy:
Object = Strategy
Name = TossIgnores
Type = Filter
Description = "Deactivate all ControlMeasures that are ignored"
Keyword = Ignore
Exclude = True
EndObject
Return to List of Strategy Types
GISIntersect Strategy DescriptionThe GisIntersect strategy provides users with the ability to select Resources that intersect with a given geometry. Resource geometries must be created when Resources are created (typically from CsvReader and DatabaseReader). Users can define a geometry using the GisGeometry keyword. GisType specifies the type or format of geometry provided in this keyword. Resources which overlap the given geometry are activated and Resources which do not overlap the given geometry are deactivated. Overlaps are determined using the GIS intersect operator.
The following is an example of a GisIntersect strategy:
Object = Strategy
Name = DeactiveNonintersecting
Type = GisIntersect
Description = "Define bounding box (lower left, upper right (lon lat))
around Degas"
GisType = "WKT"
GisGeometry = "LINESTRING(225.00 30.00, 240.00 45.00)"
BoundingBox = True
ComputeRatio = True
EndObject
Return to List of Strategy Types
GisOverlap Strategy DescriptionThe GisOverlap strategy provides users with the ability to determine Resources that have overlapping GIS footprints. This Strategy algorithm is useful to identify images that have common coverage which is the first requirement for determining stereo (see also StereoPair strategy).Parameters to this algorithm can define the ratio (or percentage) of common overlap for the two GIS geometries. As implied, all Resources must have GIS geometries in order to participate in the overlap determination. Overlap candidates are determined from other Resources for every incoming active Resource and a list of Resources that are determined from another isisminer Strategy configuration. The configuration can either be specified directly in the GisOverlap Strategy object (an embedded IsisMiner object called a CandidateMiner) or come from an external file called CandidateConfigFile. Candidate overlap Resources can be either part of the incoming Resource list (active and inactive) or from a completely different source. Once overlaps are determined for a Resource, the candidate and its source are merged/combined into a single Resource and added to an Asset of the primary Resource. Keywords from both Resources are combined into a single Resource by renaming each keyword from both Resources and propagating them to the merged Asset Resource. An A is appended to the source Resource keywords and a B is appended to the candidate Resource keywords before they are merged to distinguish keywords. The suffix of keyword sources can be specified by the user using the MergeSuffixA and MergeSuffixB keywords, respectively. The identity of the merged Resource is a combination of the two Resource identities separated by an underscore character. Once the list of overlapping Resources is determined, the user can opt to immediately process the overlaps. If a second isisminer configuration is provided called OverlapMiner or OverlapConfigFile, the list of overlapping Resources that are contained combined Resources and geometries is run though the Strategy list in the OverlapMiner configuration. This option is useful to eliminate the need for intermediate storage which could cause memory issues.
The following is an example using the GisOverlap strategy. In this example we are reading image data from an SQLite database containing WAC and NAC MESSENGER data created from caminfo runs. It determines all the images overlapping a square geometry around Debussy crater.
Object = IsisMiner
Object = Strategy
Name = MDISSelection
Type = DatabaseReader
DbConfig = wac_nac_overlaps.conf
DbProfile = MDIS
Mode = Select
# Select strings that need double quotes must use " since they cannot
# be stored in keyword values. These strings will be prescanned before
# issuing to the DB and they will be properly replaced.
Query = "select i.SourceProductId, i.YearDoy, i.File,
g.StartTime, i.EtStartTime, i.ExposureDuration, i.ObservationId,
g.CenterLongitude, g.CenterLatitude, i.ObservationType,
g.PixelResolution, g.MeanGroundResolution,
g.IncidenceAngle, g.EmissionAngle, g.PhaseAngle,
g.SubSolarGroundAzimuth, g.SubSpacecraftGroundAzimuth,
ParallaxX, ParallaxY, ShadowX, ShadowY,
p.GisFootprint
from Polygon p
INNER JOIN Image i
ON p.SourceProductId=i.SourceProductId
INNER JOIN Geometry g
ON p.SourceProductId=g.SourceProductId
INNER JOIN Statistics s
ON p.SourceProductId=s.SourceProductId"
# Resource targets are for reading multiple rows from the DB. The fields
# from each row are created as Resources.
Target = Resource
Identity = "%1"
IdentityArgs = "SourceProductId"
# Can specify a GIS keyword that will be converted on input
GisGeometryRef = GisFootprint
GisType = WKB
RemoveGisKeywordAfterImport = True
EndObject
Object = Strategy
Type = GisIntersect
Name = SelectDebussy
GisType = WKT
# Define bounding box (lower left, upper right (lon lat)) around Debussy
# crater
GisGeometry = "LINESTRING(11.56 -34.91, 13.83 -33.03)"
BoundingBox = True
ComputeRatio = True
RatioRef = DebussyOverlapRatio
ShowProgress = True
EndObject
# Since there is no CandidateMiner specified so the current active list is
# processed, recursively. Meaning each Resource is checked for overlap
# of all other active Strategies
Object = Strategy
Type = GisOverlap
Name = DebussyOverlaps
Description = "Check for image overlaps near Debussy"
ShowProgress = True
OverlapMinimum = 0.5
OverlapMerge = Intersection
EndObject
Object = Strategy
Type = AssetSidebar
Name = WriteOverlapFiles
Description = "Write CSV files of overlap data"
Asset = GisOverlap
Object = IsisMiner
Object = Strategy
Type = CsvWriter
Name = WriteOverlaps
Description = "Write individual files with overlap data"
SkipEmptyLists = True
CsvFile = "overlaps/%1.csv"
CsvFileArgs = "Identity"
Keywords = ("SourceProductIdA", "SourceProductIdB",
"OverlapRatioA", "OverlapRatioB",
"YearDoyA", "YearDoyB",
"CenterLongitudeA", "CenterLatitudeA",
"CenterLongitudeB", "CenterLatitudeB",
"GisOverlapCentroidX", "GisOverlapCentroidY",
"PixelResolutionA", "PixelResolutionB",
"EmissionAngleA", "EmissionAngleB", "IncidenceAngleA",
"IncidenceAngleB", "PhaseAngleA", "PhaseAngleB")
EndObject
EndObject
EndObject
EndObject
Object = Database
DefaultProfile = "MDIS"
Group = Profile
Name = "MDIS"
Dbname = "Messenger_Monochrome_DOY_2011095_TO_2014304_gis.db"
Type = SQLite
Description = "MDIS Sqlite database"
EndGroup
EndObject
End
Return to List of Strategy Types
GISUnion Strategy DescriptionThe GIS Union strategy provides users with the ability to create unions between GIS geometries. Each geometry which the Strategy operates on is unioned with the other supplied geometries, and the output is a union which best fits all of the input geometries.If the union geometry is present and good, an entry will be added to the Resource with the parameter RatioRef as the key and the overlap ratio as the value. The double parameters OverlapMinimum and OverlapMaximum indicate the acceptable range for the overlap ratio. Geometries with overlap ratios which do not fit the constraints will be discarded.
The following is an example of a GisUnion strategy:
Object = Strategy
Name = UnionHighOverlap
Type = GisUnion
Description = "Union remaining geometries which have a high overlap ratio."
OverlapMinimum = 0.65
OverlapMaximum = 1.0
RatioRef = "HighOverlapRatio"
EndObject
Return to List of Strategy Types
IsNumeric Strategy DescriptionThe IsNumeric strategy provides users with the ability to check if Resource keyword values are numeric. Resources with a non-numeric value are deactivated.A keyword value is considered numeric if it can be converted to a floating point number. If the keyword does not exist in the Resource, then its value is not considered numeric.
The following is an example of an IsNumeric strategy:
Object = Strategy
Name = RemoveNotApplicable
Type = IsNumeric
Description = "Deactivate all responses which did not respond to rating."
Keyword = Rating
EndObject
Return to List of Strategy Types
Limit Strategy DescriptionThe Limit strategy provides users with the ability to check if Resource keyword values are within user-defined limits. Values that are outside of the limits result in Resource deactivation.If keyword values are within the limits, the Resource disposition is set to activated. If the keyword is not within the limits provided, the Resource disposition is set to deactivated. This Strategy uses the Calculator strategy. Any equation constructed that is valid for the Calculator strategy is valid for Limit. This includes the use of Equations group to use more than one equation. The difference here is the value of the last equation is used to evaluate a true (!0) or false (0) limit for the Resource. Also note, that a side effect of using the Equations group is that any new keyword created by an equation remains in the Resource.
Following are few examples of Limit strategies:
Object = Strategy
Type = Limit
Name = EmissionAngle
Equation = "((emissionangle > 30) && (emissionangle < 80))"
EndObject
This shows an example of using a series of equations to determine limits:
Object = Strategy
# This algorithm will select only images where the center latitude
# and longitudes are within the defined region provided in the GLOBALS
# parameter file.
Name = SelectRegion
Type = Limit
Description = "Find all images with center image geometry in the
defined region"
# This set of equations will result in 1 if test are good, meaning the
# clat/clon is in the defined region. If any test fails, InRegion = 0
# (false) and the Resource is disabled. Note this also defines the GIS
# bounding box.
Group = Equations
InRegion = 1
InRegion = "InRegion * (CenterLongitude > RegionLowerLeftLongitude)"
InRegion = "InRegion * (CenterLatitude > RegionLowerLeftLatitude)"
InRegion = "InRegion * (CenterLongitude <= RegionUpperRightLongitude)"
InRegion = "InRegion * (CenterLatitude <= RegionUpperRightLatitude)"
EndGroup
EndObject
Return to List of Strategy Types
NumericalSort Strategy DescriptionThe NumericalSort strategy provides users with the ability to sort Resources according to a numerically-valued keyword (SortKey). Resources may be sorted in Ascending or Descending order according to the value specified in Order.SortKey must have a numerical value for each Resource that will be sorted. See the IsNumeric strategy. Order may be specified as either Ascending or Descending. If not provided, the Resources will be sorted in Ascending order.
The following is an example of a NumericalSort strategy:
Object = Strategy
Name = RankSort
Type = NumericalSort
Description = "Sorts the Resources in ascending order by Rank"
SortKey = Rank
Order = Ascending
EndObject
Return to List of Strategy Types
PDS Table Creator Strategy DescriptionThe PdsTableCreator strategy is used to create and write a PDS table file from column Resources. When the PDS table format file is created the column Resources are converted to PdsColumns, and all active Resources are written to the file.Records are delimited by a new line. Fields are delimited as specified by the Delimiter parameter, which defaults to a comma.
The following is an example of a PdsTableCreator strategy:
Object = Strategy
Name = TestAppendTableWriter
Type = PdsTableCreator
PdsFormatFile = "%1/TestGeneralFormat.txt"
PdsFormatFileArgs = "inputdir"
PdsTableFile = "%1/TestAppend.TAB"
PdsTableFileArgs = "outputdir"
Mode = Append
Delimiter = ","
DefaultValue = "NULL"
EndObject
Return to List of Strategy Types
PDS Table Format Strategy DescriptionThe PdsTableFormat strategy is used to create and write a PDS table format descriptor file for a PDS table. When the PDS table format file is created, the columns are created according to the byte size of the Resource, and then that data is stored in the format file. The Resources in the column are delimited by the delimiter.Records are delimited by a new line. Fields are delimited as specified by the Delimiter parameter, which defaults to a comma.
The following is an example of a PdsTableFormat strategy:
Object = Strategy
Name = PdsTableFormat
Type = PdsTableFormat
PdsFormatFile = "POINTCLOUDTAB.FTM"
PdsFormatFileArgs = "outputdir"
Delimiter = ","
Column = ("POINT_ID", "STATUS")
DataType = ("CHARACTER", "CHARACTER")
Unit = ("NONE", "NONE")
DataDescription = ("Unique point identifier.", "Status of point")
POINT_ID_BYTES = 32
STATUS_BYTES= 32
EndObject
Return to List of Strategy Types
PDS Table Reader Strategy DescriptionThe PdsTableReader strategy loads a PDS formatted table using a label containing a description of the columns in the table (contained in PdsLabelFile).If the Identity keyword is not provided or its value is an empty string, each Resource created will be named the row number. Rows are read (starting at 0) from the PDS table data in PdsTableFIle.
The following is an example of an PdsTableReader strategy:
Object = Strategy
Name = PdsTableReaderTestDefault
Type = PdsTableReader
PdslabelFile = "%1/index.lbl"
PdslabelFileArgs = "inputdir"
TableName = "INDEX_TABLE"
Identity = "%1"
IdentityArgs = File_Name
UseFormattedName = False
Description = "Test the default functionality of PdsTableReader"
EndObject
Return to List of Strategy Types
PVL Reader Strategy DescriptionThe PvlReader strategy creates Resources from a list of paths to PVL files (contained in FromList). Each PVL file becomes a Resource where the keywords in the PVL file become the keywords of the Resource. Resources may be created against any provided constraints ( Includes, Excludes, and KeyListFile), which describe which keywords become part of the created Resource.If the Identity keyword is not provided or its value is an empty string, each Resource created will be named as PvlN. N is the position number (starting at 0) of the PVL file contained in FromList. The basename, Pvl, can be changed by setting PvlBaseName. For example, if FromList contains PVL Files ~/somepvl.pvl and ~/someotherpvl.pvl, their default Identity values would be Pvl0 and Pvl1, respectively. The Includes keyword operates on PVL Objects and Groups (containers). When provided by itself, the Includes keyword will exclusively include the keywords belonging to the containers specified. Since Groups must be contained in Objects, you cannot include a Group without also including its containing Object. Similarly, you must include the Root Object since all PVL files have a Root Object container. For example:
Object = Outer
Id = 001
Group = ExtraInfo
Name = outside
Time = today
EndGroup
Group = OtherInfo
Info = "No information present"
EndGroup
Group = Important
Level = High
EndGroup
Object = Inner
Data = "Some test data"
EndObject
EndObject
To include the ExtraInfo Group, the Includes keyword should be specified,
which will include the Id, Name, and Time keywords in the created Resource.
Includes = (Root, Outer, ExtraInfo) The Excludes keyword operates on PVL Objects and Groups (containers). When provided by itself, the Excludes keyword will exclude the keywords belonging to the containers specified, while all other containers and keywords will be included. When both Includes and Excludes are provided, an Object and all of its keywords and Groups will be included except for any excluded Groups (Inclusions are applied to Objects while exclusions are applied to Groups). Referring to the Includes example, to include the Outer Object except for its OtherInfo Group, you would specify the following:
Includes = (Root, Outer)
Excludes = OtherInfo
The KeyListFile keyword operates on PVL Keywords and expects a file containing PVL Keywords. If provided, only the keywords in the file will be included in the created Resource. Note that these keywords will only be included if their containers are included or not excluded.
The following is an example of an PvlReader strategy:
Object = Strategy
Name = PvlReader
Type = PvlReader
FromList = "%1/testpvl.pvl"
FromListArgs = "inputdir"
Identity = "%1"
IdentityArgs = Id
KeyListFile = "%1/keystoinclude.lis"
KeyListFileArgs = "inputdir"
Description = "A PvlReader Strategy that includes the keywords contained in keystoinclude.lis
when creating the Resource from testpvl.pvl"
EndObject
Return to List of Strategy Types
Resource Manager Strategy DescriptionThe ResourceManager strategy provides users with the ability to apply simple operations to Resources. These operations are ResetDiscard, ToggleDiscard, DeleteDiscard, DeleteAsset, and HasAsset. For more information, please see the table below. For DeleteAsset and HasAsset, the Asset is specified in the Operations Keyword as
The following is an example of a ResourceManager strategy:
Object = Strategy
Name = SwitchActivated
Type = ResourceManager
Description = "Swap activated (Included) and deactivated (Excluded) entries in a Resource."
Operations = ToggleDiscard
EndObject
A second example with multiple management operations
# Clear the entire Resource list (useful in Sidebar Strategies)
Object = Strategy
Name = ClearResources
Type = ResourceManager
Description = "Delete all Resources"
Operations = ( ResetDiscard, ToggleDiscard, DeleteDiscard )
EndObject
Return to List of Strategy Types
Run Command Strategy DescriptionThe RunCommand strategy allows shell commands to be executed for each Resource. In addition, there is frequently a need to run commands prior to and after individual Resource processing. For this purpose, groups called PreCommands and PostCommands contain commands that are executed before and after, respectively, processing each Resource.Commands can be formulated using the keyword argument substitution options available for using global parameter pools for replacing arguments in commands with Resource keyword values. One keyword, CommandArgs, is available to list all the Resource and global pool variables that are to be used in replacement. All commands can use a %1 to %n where n is the total number of keywords list in the CommandArgs keyword. This one list of keywords is used to substitute for all commands.
The following is an example of a RunCommand strategy. The " text is required and should be included in the configuration of your command. The " and ' are substituted prior to issuing the command to the shell for execution. This is required because of PVL syntax rules.
Object = Strategy
Name = RunManyCommands
Type = RunCommand
Description = "Tests running many command"
CommandArgs = (outputdir, SourceProductId, StartTime)
# Ensure the output file exists
Group = PreCommands
Command0 = "touch %1/runmanycommands.txt"
EndGroup
# Append the Resource name and start time
Group = Commands
Command1 = "echo "Resource name %2 and date %3." >> %1/runmanycommands.txt"
EndGroup
EndObject
Return to List of Strategy Types
Sidebar Strategy DescriptionThe Sidebar strategy allows complete isisminer configurations to be run within a single Strategy. The configuration can be loaded from a file or from an embedded IsisMiner object. The configuration can be run without affecting the active state of Resources with the SavePoint keyword. It can be run without deleting or creating Resources with the SaveDelete keyword.Sidebar strategies are useful for running Strategies that have the potential to throw out Resources or alter a current state that may be undesirable. For these cases, you can use a Sidebar that saves all the Resources, activated and deactivated, their current states and preserves any operations that may alter or discard Resources. It can also be used to process an entirely new set of Resources while retaining the current state of existing Resources. Sidebar strategies can be inserted anywhere in the sequence of Strategies but are best placed strategically at critical points in the processing sequence where preservation of the current active Resource state is needed.
The following is an example of a Sidebar strategy:
Object = Strategy
Name = SetUpResources
Type = SideBar
SavePoint = True
SaveDelete = False
StrategyConfigFile = "%1/setup.conf"
StrategyConfigFileArgs = "inputdir"
EndObject
Return to List of Strategy Types
StereoPair Strategy DescriptionThe StereoPair strategy provides users with the ability to determine, from Resources that have overlapping GIS footprints, stereo pairs that pass a series of constraints. This Strategy algorithm takes direct advantage of both the GisOverlap and Calculator strategies in characteristics of those Strategies are incorporated into this Strategy. The configuration for the GisOverlap and Calculator strategies (i.e., all keyword parameters) are included in the StereoPair configuration as their functionality is used in the determination of ranked (best) stereo pairs. GisOverlap performs the preliminary requirement of image overlap determination and ratio limits, disposition of overlap geometry and merging of the stereo pair Resources into one Resource. The Calculator strategy computes the stereo strength for a pair from the results of ImageStrength and StereoStrength keyword sets. Each keyword in the groups evaluates to values greater than or equal to 0 and less than or equal to 1 where 1 is the highest value for a keyword (unless a multiplier is used).Parameters for this algorithm define two levels of stereo constraints that are described in [Becker 2015]. The first level of constraints are typically for constraining photometric (incidence, emission, phase) and geometric angles. The constraints for these limits are located in the ImageStrength group. Each keyword there should exist in each Resource or have a keyword map specified in the KeywordMap group. The second level constraints contained in the StereoStrength group are stereo limits and are applied to values computed from the stereo algorithm with contributions from both pair Resources. The format for keywords in both of these groups are four element vectors but only 3 values are required. For example, to limit the emission angle between 0 and 50 degrees, the form of the keyword in the ImageStrength group is Emission = (0.0, 50, 15.0 [, 1.0]). The first value in this keyword is the minimum value, the second is the maximum value and the third value is the value for emission that provides the best stereo conditions. The last optional value is a multiplier of the computed strength, or rank, for each keyword limit that can be used to weight the computed strength of the limit beyond the 0 - 1 range. The rank of each keyword is computed using the following scheme based upon this form of limit keywords in both ImageStrength and StereoStrength: LimitKey = (t0, t1, t2, t3) where all t0 through t3 values are numerical numbers and t0 is the minimum, t1 is maximum, t2 is the best value in the range and t3 is a weighting factor. The formula to compute the rank of each keyword is as follows:
range = max(abs(t2-t0), abs(t1-t2))
rank = 1 - abs(t2-value)/range
rank *= t3
The end result of rank is then stored in a keyword comprised of the name
of the keyword with Rank appended to it. For example if the Emission
keyword above is provided as a limit, and an image has an Emission angle
value of 30 degrees, the rank computed and stored in EmissionRank
= 1 - abs(15 - 30)/max(abs(15-0),abs(50-15)) = 0.57 * 1 (t3).
All keyword ranks for a stereo Resource are summed for each group and the
values are available in ImageStrengthRank and
StereoStrengthRank. The ImageStrengthRank is computed as
the average of the two image's rank. These values are typically summed in
the Equation keyword in the Strategy definition to specify how to rank the
strength of an individual stereo pair. These values can be sorted and
reported to users (using an AssetSidebar).
The typical equation used to compute the rank of a stereo pair,
StereoPairRank in this example, is a combination of the two ranks:
# Now specify how the stereo quality is determined.
Equation = "imagestrengthrank * 1.0 + stereostrengthrank * 1.0"
Result = StereoPairRank
The StereoPairRank will contain an indication of the strength of
a stereo pair. This keyword and image keywords are combined in a Resource
made up of both Resources by merging the keywords from the Resources. The
keywords from each Resource have a unique suffix appended to the name of
the keyword before they are added. The GisOverlap keywords
MergeSuffixA and MergeSuffixB may be used to specify
keyword suffixes for each image as they are merged. MergeSuffixA
is used to identify the primary stereo image, MergeSuffixB the
secondary or matched stereo image. Defaults for merge suffix keywords are
A and B respectively.
If data is provided by caminfo, then the values required by this Strategy are provided at the center image pixel coordinate. This may not provide accurate results, particularly for the StereoStrength group of keywords, because the values are likely not of the latitude/longitude within the overlapping region of the images. However, the computation of the StereoAngle could be more accurate if you were to take advantage of GisOverlap OverlapMerge options. The centroid (longitude/latitude) values for all of these options is computed and stored in new keywords GisOverlapCentroidX (longitude) and GisOverlapCentroidY (latitude) making this coordinate the same point in both stereo image pairs. By setting a parameter in the KeywordMap group as CenterLongitude = GisOverlapCentroidX and CenterLatitude = GisOverlapCentroidY the StereoAngle is computed from a common surface point. If desired, the user can specify whether to use the stereo angle in the computation of the VerticalPrecision stereo keyword rather than the ParallaxHeightRatio. Take note the equation to compute StereoPairRank can be changed as desired or weights can be provided in the fourth limit keyword value to emphasize or de-emphasize any limit.
The following is an example using the StereoPair strategy. In this example we are reading image data from an SQLite database containing WAC and NAC MESSENGER data created from caminfo runs. It determines stereo from overlapping images in the vicinity of Degas crater on Mercury.
Object = IsisMiner
Name = DegasStereoPair
RequiredParameters = ( inputdir, outputdir )
Object = Strategy
Name = ReadDegasPerimeter
Type = CsvReader
Description = "Read Degas data"
CsvFile = "%1/degaus_stereo_perimeter.csv"
CsvFileArgs = "inputdir"
#Ignore the first several lines including the header.
HasHeader = True
Identity = "%1"
IdentityArgs = "SourceProductId"
# Can specify a GIS keyword that will be converted on input
GisGeometryKey = GisFootprint
GisType = WKB
RemoveGisKeywordAfterImport = False
EndObject
Object = Strategy
Type = StereoPair
Name = DegasStereoPairs
Description = "Find all stereo pairs in the Degas crater region"
OverlapMerge = None
ShowProgress = True
#Debug = True
PixelPrecisionMatch = 0.2
Asset = StereoPairs
`
# Now specify how the stereo quality is determined.
Equation = "imagestrengthrank * 1.0 + stereostrengthrank * 1.0"
Result = StereoPairRank
# First level contraints/strengths
Group = ImageStrength
Incidence = ( 30.0, 65.0, 50.0 )
Emission = ( 0.0, 55.0, 15.0 )
Phase = ( 15.0, 120.00, 30.0 )
Resolution = ( 25.0, 5000.0, 200.0 )
EndGroup
# Second level constraints/strengths
Group = StereoStrength
ResolutionRatio = ( 1.0, 2.5, 1.0 )
ParallaxHeightRatio = ( 0.1, 1, 0.4 )
ShadowTipDistance = ( 0.0, 2.58, 0.0 )
OverlapRatio = ( 0.1, 1.0, 0.5 )
EndGroup
# This shows the keyword mappings to values required for stereo
# matching and constraint values
Group = KeywordMap
Resolution = PixelResolution
Emission = EmissionAngle
Incidence = IncidenceAngle
Phase = PhaseAngle
EndGroup
# Select candidates from input list. Matches over itself
Object = CandidateMiner
Object = Strategy
Name = PairCandidates
Type = ResourceManager
Description = "Choose all Resources to determine stereo"
Operations = ResetDiscard
EndObject
EndObject
EndObject
Object = Strategy
Type = AssetSidebar
Name = WriteStereoFiles
Description = "Process all stereo pair Assets"
Asset = StereoPairs
Object = IsisMiner
Object = Strategy
Name = SortStereoPairs
Type = NumericalSort
Description = "Sort stereo pairs by StereoPairRank"
SortKey = StereoPairRank
Order = Descending
EndObject
Object = Strategy
Type = CsvWriter
Name = WritePairs
Description = "Write individual files with stereo pair data"
SkipEmptyLists = True
CsvFile = "%1/%2.csv"
CsvFileArgs = ( "outputdir", "Identity" )
Keywords = ("SourceProductIdA", "SourceProductIdB", "StereoPairRank",
"ImageStrengthRank", "StereoStrengthRank",
"ParallaxHeightRatio", "ShadowTipDistance", "ResolutionRatio",
"DeltaSolarAzimuth", "DeltaSpacecraftAzimuth",
"VerticalPrecision", "StereoAngle",
"ImageStrengthRankA", "ImageStrengthRankB",
"IncidenceRankA", "IncidenceRankB",
"EmissionRankA", "EmissionRankB",
"PhaseRankA", "PhaseRankB",
"OverlapRatioA", "OverlapRatioB",
"YearDoyA", "YearDoyB",
"CenterLongitudeA", "CenterLatitudeA",
"CenterLongitudeB", "CenterLatitudeB",
"PixelResolutionA", "PixelResolutionB",
"EmissionAngleA", "EmissionAngleB",
"IncidenceAngleA", "IncidenceAngleB",
"PhaseAngleA", "PhaseAngleB")
EndObject
EndObject
EndObject
EndObject
End
CategoriesHistory
|
Parameter GroupsInputs
|
Inputs: CONFIG
Description
This file contains a series of Strategy objects enclosed in an object named IsisMiner. The main documentation contains many different examples of the format of this file.
| Type | filename |
|---|---|
| File Mode | input |
| Filter | *.conf |
Inputs: GLOBALS
Description
All keywords contained in this file are loaded in the global variable pool. Unique keywords that do not exist in Resources can be used as keyword value substitutions in your Strategies.
| Type | filename |
|---|---|
| File Mode | input |
| Default | None |
Inputs: PARAMETERS
Description
This parameter allows the user to add global parameters to each Strategy so that they can be used in translation of keyword ARG values. This parameter has the form:
PARAMETERS = "key1:value1[@key2:value2...]"Defaults to None.
| Type | string |
|---|---|
| Default | None |
Example 1From a global image dataset, find all stereo pairs within a 10x10 degree map section DescriptionThis example has been used to process MESSENGER MDIS global image dataset. The purpose of this isisminer run is to
File naming conventions used for these section definition files are EquiYYYHXXXED10 where
The section in this example includes Degas crater located at 37.08 degrees latitude and 232.75 positive east longitude. This falls in the section defined in the file Equi035N235ED10.def. Here is the contents of this file:
# Define sub region in 10 degree increments below the NP
# Define lower left and upper right coordinates
Region = "Equi035N235ED10"
RegionGisGeometry = "LINESTRING(230 30, 240 40)"
RegionBoundingBox = True
RegionCenterLongitude = 235
RegionCenterLatitude = 35
RegionLowerLeftLongitude = 230
RegionLowerLeftLatitude = 30
RegionUpperLeftLongitude = 230
RegionUpperLeftLatitude = 40
RegionUpperRightLongitude = 240
RegionUpperRightLatitude = 40
RegionLowerRightLongitude = 240
RegionLowerRightLatitude = 30
End
This file is provided in the GLOBALS parameter to isisminer. This
will make all keywords in the file available for use in keyword
argument substitution, CSV output, and
Calculator and Limit
Strategies.
To process the global image dataset requires treating data at the poles slightly different when identifying stereo. The incidence angles are consistently high, up to 90 degrees. This requires a slight change to the stereo configuration so two files are required to process the data - one for equatorial data between 65N and 65S degrees and one for polar from 65N/S to 90 N/S degrees. The isisminer configuration file must consider this when processing stereo and it is driven by the map section input file as to which one is selected. The easiest way to do this is to provide the name of the stereo configuration file as a PARAMETER to isisminer. The main isisminer CONFIG file is named mdis_overlaps_and_stereo.conf and its contents with comments is shown in the mdis_overlaps_and_stereo.conf file below. This configuration file uses the GISOverlap strategy to identify all images that overlap one another and creates a CSV file of those images and a lot of relevant data for computing stereo. The stereo image pairings/sets are determined using a special processing feature of GisOverlaps. It has a provision for immediately processing the result of the overlap determination for each image. Just so happens that the output CSV file contains all the information needed to compute stereo. As each image overlap set is determined and written to the output CSV file, GisOverlap runs the stereo configuration file on that CSV file using the StereoPair strategy. Its configuration is contained in the file mdis_global_stereo_equatorial.conf and is provided in the command line to isisminer in PARAMETERS using the StereoConfig parameter. The contents of mdis_global_stereo_equatorial.conf is shown below. Finally, there is a complete configuration that has command line control depending upon which latitude region is being processed and input/output directories. To process the entire global dataset of MDIS images collected by MESSENGER, 267,769 of them, will require vast computing Resources. It is highly recommended that a processing cluster be used if available. Below is the command line used to process this map section. The results summarized here show the output of the command and a summary of the results below.
Creating Strategies...
Finished creating 24 strategies...
Running DatabaseReader::MDISSelectRegionOct2014 (TimeIn: 02:46:51.626)
Description: Strategy::MDISSelectRegionOct2014 is running a DatabaseReader algorithm.
248586 of 0 processed in DatabaseReader::MDISSelectRegionOct2014 (TimeOut: 03:18:36.966)
ElapsedTime(s): 1905
Running DatabaseReader::MDISSelectRegionMay2015 (TimeIn: 03:18:36.966)
Description: Strategy::MDISSelectRegionMay2015 is running a DatabaseReader algorithm.
19183 of 0 processed in DatabaseReader::MDISSelectRegionMay2015 (TimeOut: 03:20:59.204)
ElapsedTime(s): 142
Running IsNumeric::FindBadOnes (TimeIn: 03:20:59.204)
Description: Identify images with invalid center pixel geometry
267474 of 267769 processed in IsNumeric::FindBadOnes (TimeOut: 03:21:00.209)
ElapsedTime(s): 0
Running ResourceManager::DeleteBadOnes (TimeIn: 03:21:00.209)
Description: Delete Resources with bad values in required keywords
295 of 0 processed in ResourceManager::DeleteBadOnes (TimeOut: 03:21:00.247)
ElapsedTime(s): 0
Running Limit::SelectRegion (TimeIn: 03:21:00.247)
Description: Find all MDIS images with center image geometry in the defined region
557 of 267474 processed in Limit::SelectRegion (TimeOut: 03:21:20.489)
ElapsedTime(s): 20
Running RunCommand::MakeOutputDirectory (TimeIn: 03:21:20.489)
Description: Ensure output directories exists
Running Mkdir1 PRE command: mkdir -p /scratch/kbecker/MESSENGER/Region10D/Global/Images
Running Mkdir2 PRE command: mkdir -p /scratch/kbecker/MESSENGER/Region10D/Global/Overlaps/Equi035N235ED10
2 of 0 processed in RunCommand::MakeOutputDirectory (TimeOut: 03:21:20.976)
ElapsedTime(s): 0
Running CsvWriter::WriteRegionImages (TimeIn: 03:21:20.976)
Description: Write images that are in the defined region to a file
557 of 557 processed in CsvWriter::WriteRegionImages (TimeOut: 03:21:21.398)
ElapsedTime(s): 0
Running GisOverlap::ImageOverlap (TimeIn: 03:21:21.398)
Description: Find all overlaps images in specfied region
557 of 557 processed in GisOverlap::ImageOverlap (TimeOut: 04:42:28.181)
ElapsedTime(s): 4866
Session complete in 01:55:36.555 of elapsed time
This run completed in about 2 hours. This is typical for most regions
when run on machines with similar CPU processing power.
There were 557 images identified in this region and the results are stored in
/scratch/kbecker/MESSENGER/Region10D/Global/ImagesEqui035N235ED10_images.csv.
For each image, and average of several hundred images were
determined to overlap. There are now 557 files in the directory
/scratch/kbecker/MESSENGER/Region10D/Global/Overlaps/Equi035N235ED10
that have a base name as the image name and _overlaps (example:
EW1067331895I_overlaps.csv has 213 overlapping images) with a
total of 230,389 image overlaps found. In the directory
/scratch/kbecker/MESSENGER/Region10D/Global/Stereo/Equi035N235ED10,
there are 308 files (249 had no stereo images or did not pass
level1 image constraints for stereo) with similar file naming
conventions used in the Overlaps directory (example:
EW1067331895I_stereopairs.csv has 11 stereo images) where a total
of 7,497 stereo images were identified.
Command Line
isisminer
config=/scratch/kbecker/MESSENGER/Region10D/Global/mdis_overlaps_and_stereo.conf
parameters='DBDir:/work/projects/messenger/DB@outputdir:/scratch/kbecker/MESSENGER/Region10D/Global@stereoconfig:/scratch/kbecker/MESSENGER/Region10D/Global/mdis_global_stereo_equatorial.conf'
globals=/scratch/kbecker/MESSENGER/Stereo/Region10D/Deffiles/Equi035N235ED10.def
This command executes isisminer with the required parameters for
our example.
Data FilesLinks open in a new window.
|