Project

General

Profile

Feature #374

cnetmerge % progress a little misleading

Added by Lynn Weller over 6 years ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
John Bonn
Category:
API
Target version:
Impact:

cnetmerge will provide better progress notifications

Software Version:
Test Reviewer:
Story points:
1

Description

I'm running cnetmerge on a couple of beefy networks (one is a portion of the other) and within a few minutes or so of running it says 50% progress, but the program run for over an hour. I thought it might have got hung up, but decided not to kill it because it appeared to be doing something when I ran "top".

Steps to reproduce:

Data you can try this on (warning, big files, takes cnetmerge 1h 13m to run):
/work/users/lweller/LMMP/NPole/Jig/Rings

command line:
cnetmerge inputtype=cnets cnet=JigOut_NPole_Final_SubReg_Del_21.net cnet2=Ring_2Deg_NPole_Final_SubReg_Final.net onet=NPole_Final_SubReg_Del_23.net networkid=LROC_NAC_NPOLE_NET description="LROC NAC NORTH POLE NETWORK, LAT 85.5N-90.0N" duplicatepoints=merge overwritepoints=true overwritemeasures=true

History

#1 Updated by Steven Lambright over 6 years ago

Currently the progress is working on a % of files processed basis. The first file requires no merging, so 1/2 are done and you get 50% progress. There is no intermediate progress steps betweeen cnet files which is why it is so off for you. The progress is designed for large file lists.

Perhaps the progress should work like automos where it gives you a progress for every input file.

#2 Updated by Lynn Weller about 6 years ago

I'm not sure whats best for this program, but the current way of accounting for progress isn't making things clear for me. I guess I use the progress bar to give me some indication of how much more time the program might take to finish the process, not it's progress through a list. So when I see 50% after 3 minutes I figure it might only take another 3 minutes to complete the run. Maybe the automos suggestion is appropriate for this program. Then I might have seen it was going to take a while for the second net if it's progress was going at a slower pace.

#3 Updated by Anonymous over 4 years ago

  • Target version deleted (150)

#4 Updated by Stuart Sides 10 months ago

  • Tracker changed from Bug to Feature
  • Assignee set to John Bonn
  • Target version set to 3.5.1 (Sprint 1)

The percent processed should be based on each file. Example:
File 1 of 4
0%... 100%
File 2 of 4
0% ... 100%
File 3 of 4
0% ... 100%
file 4 of 4
0% ... 100%

#5 Updated by John Bonn 10 months ago

  • Status changed from Acknowledged to In Progress

#6 Updated by John Bonn 10 months ago

  • Impact updated (diff)

#7 Updated by John Bonn 10 months ago

  • Status changed from In Progress to Resolved

#8 Updated by John Bonn 10 months ago

Note cnetmerge has changed "cnet1" parameter to "base" and is now invoked as:

./cnetmerge inputtype=cnets base=JigOut_NPole_Final_SubReg_Del_21.net cnet=Ring_2Deg_NPole_Final_SubReg_Final.net onet=junk networkid=LROC_NAC_NPOLE_NET description="LROC NAC NORTH POLE NETWORK, LAT 85.5N-90.0N" duplicatepoints=merge overwritepoints=true overwritemeasures=true

#9 Updated by Lynn Weller 10 months ago

This needs to be built for astrovm4 before I can test it:
setisis /work/projects/isis/latest/m00374/isis/
/work/projects/isis/latest/m00374/isis/bin/isiscomplete: error while loading shared libraries: libicui18n.so.52: cannot open shared object file: No such file or directory
/work/projects/isis/latest/m00374/isis/bin/isiscomplete: error while loading shared libraries: libicui18n.so.52: cannot open shared object file: No such file or directory
/work/projects/isis/latest/m00374/isis/bin/isiscomplete: error while loading shared libraries: libicui18n.so.52: cannot open shared object file: No such file or directory
/work/projects/isis/latest/m00374/isis/bin/isiscomplete: error while loading shared libraries: libicui18n.so.52: cannot open shared object file: No such file or directory

#10 Updated by Lynn Weller 9 months ago

  • Status changed from Resolved to Feedback

#11 Updated by John Bonn 9 months ago

  • Status changed from Feedback to Resolved

I've updated the build to run on vm4

#13 Updated by Lynn Weller 8 months ago

  • Status changed from Resolved to Feedback

Sorry it has taken so long to test the changes.

I passed cnetmerge a list of 10 networks to merge and the progress on merging didn't make sense to me:
cnetmerge: Merging file 1 of 9
cnetmerge: Merging file 2 of 9

and so on. I can maybe guess why it is this way (perhaps network 1 in the list is considered the base to which the other 9 networks are merged into?) but we should consider changing this as it will most definitely be a source of confusion to the user. I immediately did a line count on my file to see if there were only 9 lines in it (nope, 10). Then I calculated the total number of points and measures in my input networks to be sure they matched that of the output (they do). It would be better not to have to go through such hurdles. Seems to me the counter should be based on the total number of networks being merged.

Yes, this should be changed. When I pass the program a list of two networks it reports
cnetmerge: Merging file 1 of 1

it just doesn't make sense.

I have a small set of data that can be used for testing as opposed to the large networks supplied in my original post. See files under /work/users/lweller/Isis3Tests/CnetMerge/m00374/. There are 10 small networks there. The file nets.lis contains all ten files and twonet.lis contains two. My last two network test had the command line:

cnetmerge inputtype=lis clist=twonets.lis onet=MergeTwo.net networkid=test description=test

#14 Updated by Lynn Weller 8 months ago

  • Test Reviewer set to Lynn Weller

#15 Updated by Ian Humphrey 7 months ago

  • Story points set to 1

#16 Updated by John Bonn 7 months ago

Update to count from 2..10 for 10 files as requested.

#17 Updated by John Bonn 7 months ago

  • Status changed from Feedback to Resolved

#18 Updated by Lynn Weller 7 months ago

  • Status changed from Resolved to Closed

Better - thank you!

#19 Updated by Stuart Sides 4 months ago

  • Target version changed from 3.5.1 (Sprint 1) to 3.5.1 (2017-08-08 Aug)

Also available in: Atom PDF