Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#2697 closed defect (fixed)

data catalog does not show all raster layers, while showing some non-existing

Reported by: pvanbosgeo Owned by: grass-dev@…
Priority: normal Milestone: 7.2.0
Component: wxGUI Version: unspecified
Keywords: datacatalog Cc:
CPU: Unspecified Platform: Linux

Description

In various of my mapsets,

  • raster layers are not showing in the Data catalog, or
  • some raster layers are showing, others are not, or
  • some rasters are not showing, some rasters that show do actually not exist (see example of this case in attached screenshot)

In all cases, vector layer are showing. In some mapsets, all raster layers are showing as well. I can't find really a pattern, nor anything out of the ordinary in the folders of the database.

Attachments (2)

example.png (129.5 KB) - added by pvanbosgeo 4 years ago.
layers shown in data catalog, compared to the layers actually present in the mapset (here the cats folder is shown as example)
example2.png (113.0 KB) - added by pvanbosgeo 4 years ago.
This example shows the layers in the data catalog and in the 'add selected layers in layer tree' menu.. clearly showing that the data catalog is not showing most layers

Download all attachments as: .zip

Change History (26)

Changed 4 years ago by pvanbosgeo

Attachment: example.png added

layers shown in data catalog, compared to the layers actually present in the mapset (here the cats folder is shown as example)

Changed 4 years ago by pvanbosgeo

Attachment: example2.png added

This example shows the layers in the data catalog and in the 'add selected layers in layer tree' menu.. clearly showing that the data catalog is not showing most layers

comment:1 Changed 4 years ago by pvanbosgeo

Currently, no data is showing at all in the data catalogue

comment:2 Changed 4 years ago by annakrat

Some time ago I rewrote loading the layers and it was working perfectly for me (Ubuntu). Any error? Does it work for others?

comment:3 in reply to:  1 ; Changed 4 years ago by martinl

Replying to pvanbosgeo:

Currently, no data is showing at all in the data catalogue

Confirmed, there is something wrong, the condition source:grass/trunk/gui/wxpython/datacatalog/tree.py#L160 seems to be never fulfilled (loc_count == len(locations)).

comment:4 in reply to:  3 ; Changed 4 years ago by martinl

Replying to martinl:

Confirmed, there is something wrong, the condition source:grass/trunk/gui/wxpython/datacatalog/tree.py#L160 seems to be never fulfilled (loc_count == len(locations)).

In my case I have 36 locations, the loop hangs when scanning 32nd location. 33rd is never scanned.

comment:5 in reply to:  4 Changed 4 years ago by annakrat

Replying to martinl:

Replying to martinl:

Confirmed, there is something wrong, the condition source:grass/trunk/gui/wxpython/datacatalog/tree.py#L160 seems to be never fulfilled (loc_count == len(locations)).

In my case I have 36 locations, the loop hangs when scanning 32nd location. 33rd is never scanned.

I can't replicate it, I tried it on different computers and operating systems. There might be something special about your locations. I am afraid you have to investigate further.

comment:6 Changed 4 years ago by pvanbosgeo

Unlike martinl I have only few locations in my GRASS databases (3 in one, 8 in another), with between 2 and 20 mapsets per location. The data catalogue shows empty in both databases.

On the other hand, when creating a new GRASS database, the data catalogue shows the locations/map sets. There are no error messages or warnings, so I am not sure how to further investigate this?

comment:7 Changed 4 years ago by veroandreo

I also don't see anything at all in Data catalog. I tried creating a new location and mapset in the GUI, but nothing appears. I have 9 locations with 3-10 mapsets each in my grassdata.

I'm using grass71 r67485 under fedora 22.

comment:8 in reply to:  7 Changed 4 years ago by pvanbosgeo

Replying to veroandreo:

I also don't see anything at all in Data catalog. I tried creating a new location and mapset in the GUI, but nothing appears. I have 9 locations with 3-10 mapsets each in my grassdata.

And if you create a whole new GRASS database?

I'm using grass71 r67485 under fedora 22.

comment:9 Changed 4 years ago by veroandreo

Yes, sure!

I created a new grassdata and pasted the North Carolina sample Location inside it. Now, Data catalog shows it perfectly. After that, I created a new Location in this new grassdata but I could only see it in the Data catalog after restarting GUI.

comment:10 in reply to:  9 ; Changed 4 years ago by annakrat

Replying to veroandreo:

grassdata but I could only see it in the Data catalog after restarting GUI.

Yes, we need a reload button

Martin, any chance you could try to debug it? I can't do anything about it right now, unless someone can send me a grass database which is not visible in the catalog.

comment:11 in reply to:  10 Changed 4 years ago by martinl

Replying to annakrat:

Martin, any chance you could try to debug it? I can't do anything about it right now, unless someone can send me a grass database which is not visible in the catalog.

hopefully tomorrow...

comment:12 Changed 4 years ago by martinl

It seems that it hangs on source:grass/trunk/gui/wxpython/datacatalog/tree.py#L169 when reading locations with more maps. Setting timeout to 0.1 (r67531) helped on my PC. Testing welcome.

comment:13 Changed 4 years ago by martinl

Does r67531 fixes originally reported issue?

comment:14 Changed 4 years ago by martinl

Component: DefaultwxGUI
Keywords: datacatalog added

comment:15 in reply to:  12 ; Changed 4 years ago by annakrat

Replying to martinl:

It seems that it hangs on source:grass/trunk/gui/wxpython/datacatalog/tree.py#L169 when reading locations with more maps. Setting timeout to 0.1 (r67531) helped on my PC. Testing welcome.

I am not sure if this solves the problem. I guess it won't show then the correct results? I am wondering why the process hangs, is it just because there is a lot of maps or because g.list or g.mapsets have some other problem? How many maps do you have in the biggest location, so that I can try to reproduce it?

comment:16 in reply to:  15 ; Changed 4 years ago by martinl

Replying to annakrat:

Replying to martinl: I am not sure if this solves the problem. I guess it won't show then the correct results? I am wondering why the process hangs, is it just because there is a lot of maps or because g.list or g.mapsets have some other problem? How many maps do you have in the biggest location, so that I can try to reproduce it?

it hanged on nc_spm_08_grass7 location with 6142 maps (modis included). The location is 19th from 34 locations.

comment:17 in reply to:  15 ; Changed 4 years ago by martinl

Replying to annakrat:

I am not sure if this solves the problem. I guess it won't show then the correct results? I am

I checked nc_spm_08_grass7 location in datacatalog and the result seems to be OK.

comment:18 in reply to:  17 ; Changed 4 years ago by veroandreo

Replying to martinl:

Replying to annakrat:

I am not sure if this solves the problem. I guess it won't show then the correct results? I am

I checked nc_spm_08_grass7 location in datacatalog and the result seems to be OK.

Works for me, too. I can see all locations, mapsets and their content. It seems slow though with the mapset that has more than 15000 raster maps, but maybe that's fine :)

comment:19 in reply to:  16 Changed 4 years ago by annakrat

Replying to martinl:

Replying to annakrat:

Replying to martinl: I am not sure if this solves the problem. I guess it won't show then the correct results? I am wondering why the process hangs, is it just because there is a lot of maps or because g.list or g.mapsets have some other problem? How many maps do you have in the biggest location, so that I can try to reproduce it?

it hanged on nc_spm_08_grass7 location with 6142 maps (modis included). The location is 19th from 34 locations.

I can now reproduce it, I will look at it later today. Thanks

comment:20 Changed 4 years ago by annakrat

It should be now fixed in r67532, there was a deadlock because queue.get must be before joining the process, there is an example in python documentation.

Regarding the original issue, the new implementation (of loading maps) which isolates the environment fixes it. So I suggest to close it after you confirm it works.

comment:21 in reply to:  18 Changed 4 years ago by annakrat

Replying to veroandreo:

Replying to martinl:

Replying to annakrat:

I am not sure if this solves the problem. I guess it won't show then the correct results? I am

I checked nc_spm_08_grass7 location in datacatalog and the result seems to be OK.

Works for me, too. I can see all locations, mapsets and their content. It seems slow though with the mapset that has more than 15000 raster maps, but maybe that's fine :)

I was testing with 12000 maps and 35 locations on a computer with 8 cores and SSD disk and it took less then 1 second to display the tree. It's parallelized on the level of locations, so if you have all maps in one location, it won't help you much. And depends on your computer specifications. What exactly is slow for you?

comment:22 in reply to:  20 Changed 4 years ago by pvanbosgeo

Replying to annakrat:

It should be now fixed in r67532, there was a deadlock because queue.get must be before joining the process, there is an example in python documentation.

Regarding the original issue, the new implementation (of loading maps) which isolates the environment fixes it. So I suggest to close it after you confirm it works.

I can confirm the issue has been fixed. Great work, thanks a lot! I am closing the ticket.

comment:23 Changed 4 years ago by pvanbosgeo

Resolution: fixed
Status: newclosed

comment:24 Changed 4 years ago by neteler

Milestone: 7.1.07.2.0

Milestone renamed

Note: See TracTickets for help on using tickets.