Opened 19 years ago
Closed 17 years ago
#841 closed defect (fixed)
transformTo() memory leak
Reported by: | hobu | Owned by: | warmerdam |
---|---|---|---|
Priority: | high | Milestone: | |
Component: | OGR_SRS | Version: | unspecified |
Severity: | normal | Keywords: | |
Cc: |
Description (last modified by )
IRC Log: hobu_work any ideas why this would just continue to consume gobs of memory? hobu_work http://rafb.net/paste/results/ILVkKn59.html hobu_work what am I forgetting to destroy? FrankW hobu_work: I don't see any obvious source of problems. Perhaps there is an issue in transformTo. hobu_work I modified it to this http://rafb.net/paste/results/o6fBo256.html and took out all of the Clone()'s and it still grows just as fast (this is a 300k oracle table I'm dumping out to WKT) FrankW Can you dump the table to shapefile, and run against that instead? That would help isolate whether it is an oracle problem or not. FrankW Another quick test would be to try removing the transformTo() call and see if that makes a difference. hobu_work removing the TransformTo does not slow the memory consumption down... hobu_work I take that back... removing the TransformTo (both of them) *does* limit the memory consumption hobu_work like so... http://rafb.net/paste/results/bPsGrA42.html FrankW Ah, I didn't notice you were calling get_zone() each time. FrankW If you enable the main transformTo is there still a leak? hobu_work yes FrankW Are these lines? Polygons? Points? hobu_work Poly's hobu_work with small 4-40 pts in them hobu_work is looking at OGRErr OGRGeometry::transformTo( OGRSpatialReference *poSR ) FrankW I looked through transform() which it calls, and it seems OK. FrankW Ahh FrankW The problem may be with cleaning up the transformer which is created and destroyed each time. hobu_work the delete poCT; statement? FrankW Yes, but it is possible that the OGRCoordianteTransformation destructor isn't as clean as it ought to be. FrankW Yes, I think I see. FrankW the new srs is assigned to the geometries, but the ~OGRGeometry() method does not delete the SRS if the reference count falls to zero, though it does dereference it. FrankW This is because some driver implementations personally destroy the OGRSpatialReference for themselves (which they assign to their geometries) assuming that no one else could be still referencing them. FrankW So I broke the normal reference counting rules for geometries to avoid causing crashes in esoteric circumstances. FrankW You are welcome to file a bug on this (include my text above), but I'm not likelyi to fix it right away. FrankW Of course, there could be more going on. I'll likely need to prepare a small c++ test program I can easily run under valgrind to be sure. Python Code: import sys #sys.path.insert(0,r'D:\cvs\gdal\gdal\pymod\ng\build\lib.win32-2.3') sys.path.insert(0,r'D:\cvs\gdal\gdal\pymod\\') import os os.environ['GDAL_DATA']=r'd:\cvs\gdal\gdal\data' os.environ['PROJ_LIB']=r'c:\proj\nad' import ogr import osr zones = {10:[-126,-120], 11:[-120,-114], 12:[-114,-108], 13:[-108,-102], 14:[-102,-96], 15:[-96,-90], 16:[-90,-84], 17:[-84,-78], 18:[-78,-72], 19:[-72,-66], 20:[-66,-60] } ref = osr.SpatialReference() ref.ImportFromEPSG(4326) def get_zone(point): #point2 = point.Clone() point.TransformTo(ref) minx = maxx = point.GetEnvelope()[0] #point.Destroy() for i in zones: min,max = __builtins__.map(float,zones[i]) if minx > min and minx < max: min_utmzone = 26900+i if maxx > min and maxx < max: max_utmzone = 26900+i if min_utmzone == max_utmzone: return min_utmzone esri_lines = ['PROJCS["USA_Contiguous_Albers_Equal_Area_Conic_USGS_version",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Albers"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-96.0],PARAMETER["Standard_Parallel_1",29.5],PARAMETER["Standard_Parallel_2",45.5],PARAMETER["Latitude_Of_Origin",23.0],UNIT["Meter",1.0]]'] in_ref = osr.SpatialReference() in_ref.ImportFromESRI(esri_lines) out_ref = osr.SpatialReference() f = open(r'd:\scott_psu.txt','wb') lyr = ogr.Open('OCI:username/pw@oranri_design.cssm.iastate.edu:SEGMENT').GetLayer(0) feat1 = lyr.GetNextFeature() while feat1: geom = feat1.GetGeometryRef() geom.AssignSpatialReference(in_ref) #geom.AssignSpatialReference(in_ref) # zone = get_zone(geom) zone = 26910 out_ref.ImportFromEPSG(zone) #geom.TransformTo(out_ref) outstring = '%s,%s,%s,%s,%s\n' outstring = outstring %(feat1.STATE, feat1.COUNTY, feat1.PSU, zone, geom.ExportToWkt()) f.write(outstring) feat1.Destroy() feat1 = lyr.GetNextFeature() f.close()
Change History (4)
comment:3 by , 19 years ago
Problem confirmed with the following program under Valgrind. The leak is one OGRSpatialReference per shape processed. The problem is that the OGRCoordinateTransformation clones the spatial reference, it is attached to the geometry, and then coordinate transformation is deleted before the geometry. But the geometry will not actually delete the spatial reference, it just dereferences it. We can't "fix" this without a careful pass through all the OGR drivers to make them honour reference counting on spatial references too. #include <stdio.h> #include "cpl_conv.h" #include "ogrsf_frmts.h" int main() { OGRRegisterAll(); OGRDataSource *poDS; poDS = OGRSFDriverRegistrar::Open( "data/esri/shape/world/world.shp", FALSE ); OGRLayer *poLayer = poDS->GetLayer(0); OGRSpatialReference *poWGS84; poWGS84 = new OGRSpatialReference(); poWGS84->SetFromUserInput( "WGS84" ); OGRFeature *poFeat; while( (poFeat = poLayer->GetNextFeature()) != NULL ) { OGRGeometry *poGeom = poFeat->GetGeometryRef(); OGRSpatialReference *poER; poER = new OGRSpatialReference(); poER->SetFromUserInput( "+proj=eqc +datum=WGS84" ); poGeom->assignSpatialReference( poWGS84 ); poGeom->transformTo( poER ); delete poFeat; delete poER; } delete poWGS84; delete poDS; delete OGRSFDriverRegistrar::GetRegistrar(); CPLFinderClean(); }
comment:5 by , 17 years ago
Description: | modified (diff) |
---|---|
Resolution: | → fixed |
Status: | assigned → closed |
I'm pretty sure Frank did a pass through the drivers about 1.5 years ago and has cleaned this issue up. Closing for now.
Note:
See TracTickets
for help on using tickets.