Opened 14 years ago

Closed 14 years ago

#3721 closed enhancement (fixed)

Rasdaman format

Reported by: cjucovschi Owned by: ilucena
Priority: normal Milestone: 1.8.0
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords:
Cc:

Description

we have written a GDAL driver to add the rasdaman (www.rasdaman.org) database as another "data format" that GDAL can read, and we would like to contribute this to the GDAL project.

please find attached the patch to the gdal source code (rasdaman.patch) as well as an attempt to create a configure.in

[1] The connect string syntax follows the WKT Raster pattern and goes like this:

rasdaman:

query='select a[$x_lo:$x_hi,$y_lo:$y_hi] from MyImages as a' tileXSize=512 tileYSize=512 [host='localhost'] [port=7001] [database='RASBASE'] [user='rasguest'] [password='rasguest']

To test it one should do the following: # install packages mentioned in http://kahlua.eecs.jacobs-university.de/trac/rasdaman/wiki/Download

# git clone git://kahlua.eecs.jacobs-university.de/rasdaman.git rasdaman_src

# cd rasdaman_src

# ./configure --prefix=SOME_DIR

# make -j 10; make install

# cd SOME_DIR/bin

# ./create_db.sh

Note that the current user should have rights for creating a postgres database.

# ./start_rasdaman.sh

# ./insert_demo.sh (run it and it will give you an example which you should just copy paste and run)

congrats you have rasdaman set up and have 3 dataset in it (rgb, mr, mr2).

a sample GDAL query would be: ./gdalinfo "rasdaman: query='select a[\$x_lo:\$x_hi,\$y_lo:\$y_hi] from rgb as a'"

Attachments (2)

configure.in (84.1 KB ) - added by cjucovschi 14 years ago.
rasdaman.patch (25.9 KB ) - added by cjucovschi 14 years ago.

Download all attachments as: .zip

Change History (17)

by cjucovschi, 14 years ago

Attachment: configure.in added

comment:1 by Even Rouault, 14 years ago

Constantin, for licencing reasons, could you update the patch (in rasdamandataset.h and rasdamandataset.cpp) with the usual X/MIT licence headers used in GDAL and appropriate author/copyright mentions, if you agree with those license terms ? It would also be good if you can send an email to the gdal-dev list stating that you have read and accepted the Legal paragraph of http://trac.osgeo.org/gdal/wiki/rfc3_commiters.

An help page (frmts/rasdaman/frmt_rasdaman.html) would also be welcome for users of the driver. It doesn't necessarily need to be big, but at minimum it should contain the syntax to open a rasdaman DB.

by cjucovschi, 14 years ago

Attachment: rasdaman.patch added

comment:2 by cjucovschi, 14 years ago

updated the patch so that the sources contain the necessary headers, also created the html page. The email will be sent soon. How can I proceed with configure.in? What version of autotools is used? The one I have gives the following error messages and makes the ./configure script unusable.

libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/ltversion.m4'
libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
libtoolize: `AC_PROG_RANLIB' is rendered obsolete by `LT_INIT'
configure.in:63: error: possibly undefined macro: AC_COMPILER_PIC
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.in:64: error: possibly undefined macro: AC_LD_SHARED
configure.in:92: error: possibly undefined macro: AC_HAVE_LONG_LONG
configure.in:93: error: possibly undefined macro: AC_UNIX_STDIO_64
configure.in:139: error: possibly undefined macro: AC_CHECK_FUNC_CUSTOM
autoreconf: /usr/bin/autoconf failed with exit status: 1

comment:3 by Even Rouault, 14 years ago

From the current configure, I can see it was generated with autoconf 2.65. I'm not sure if ./autogen.sh actually works, but autoconf did last time I tried.

comment:4 by ilucena, 14 years ago

Owner: changed from warmerdam to ilucena

comment:5 by ilucena, 14 years ago

I just committed r20351 with the patch. The configure.in needed some fixing but it is working OK. I am going to do some tests now, even writing a auto_test script.

./configure --with-rasdaman=yes
...
  OCI support:               yes
  GEORASTER support:         yes
  SDE support:               no
  Rasdaman support:          yes
  DODS support:              no
  SQLite support:            yes
  SpatiaLite support:        no
  DWGdirect support          no
  INFORMIX DataBlade support:no
  GEOS support:              yes
  VFK support:               yes

  SWIG Bindings:          no

  Statically link PROJ.4:    no
  enable OGR building:       yes
  enable pthread support:    no
  hide internal symbols:     no
gdalinfo --format rasdaman
Format Details:
  Short Name: RASDAMAN
  Long Name: RASDAMAN
  Help Topic: frmt_various.html#RASDAMAN

comment:6 by Even Rouault, 14 years ago

Ivan, I haven't checked by testing, but by review I think that the linking of Win32 builds must be broken by the (unconditionnal) addition of -DFRMT_rasdama to EXTRA_FLAGS in frmts/makefile.vc. GDALAllRegister() will try to call GDALRegister_RASDAMAN()

comment:7 by ilucena, 14 years ago

You are right. I fixed the frmts/makefile.vc on r20352. Thanks.

comment:8 by ilucena, 14 years ago

By the way, is Rasdaman and the driver ready for Windows?

comment:9 by ilucena, 14 years ago

I tried to run the scripts you proposed but I got an error on the last one. It seems like it is looking for some specific files to run the script.

% insertdemo.sh localhost 7001 ./rasdaman rasadmin rasadmin
insertdemo.sh: rasdaman demo data insert script v2.0
insertdemo.sh: using host localhost, image directory ./rasdaman, and user/passwd rasadmin/rasadmin
mr...Error: cannot find input file ./rasdaman/mr_?.pgm
Exception:  ODMG General

Are you going to provide those files or can just replace it by any any other dataset set?

I tries to make up some sample data but I've got this error here:

% gdalinfo "rasdaman: query='select a[\$x_lo:\$x_hi,\$y_lo:\$y_hi] from rgb as a'"
Execution error 355 in line 1, column 30, near token rgb: Collection name is unknown.
ERROR 4: `rasdaman: query='select a[$x_lo:$x_hi,$y_lo:$y_hi] from rgb as a'' does not exist in the file system,
and is not recognised as a supported dataset name.

gdalinfo failed - unable to open 'rasdaman: query='select a[$x_lo:$x_hi,$y_lo:$y_hi] from rgb as a''.
Open GDAL Datasets:
  1 N DriverIsNULL      -1 512x512x0

comment:10 by ilucena, 14 years ago

==[IDE]== Aug 26, 2010 8:25:20 AM Committing... commit -m "remove warnings, replace sdt::cerr by CPLError and sprintf by CPLSprintf" /home/ilucena/Dev/gdal/frmts/rasdaman/rasdamandataset.cpp Sending Dev/gdal/frmts/rasdaman/rasdamandataset.cpp Transmitting file data ... Committed revision 20455. ==[IDE]== Aug 26, 2010 8:25:28 AM Committing... finished.

comment:11 by Even Rouault, 14 years ago

Ivan,

your change from sprintf -> CPLSPrintf is incorrect and should be reverted.

They are used like this :

* sprintf(pszAlreadyAllocatedBuffer, pszFormatString, arg1, ... )
* const char* pszStaticBuffer = CPLSPrintf(pszFormatString, arg1, ... )

So they have not at all the same signatures, and with your change, the result strings are left uninitialized after CPLSPrintf() call.

By the way, the "char x_lo[10], x_hi[10], y_lo[10], y_hi[10]" would not be sufficient for huge images. "2147000000" needs 10 characters + the nul terminator, so the size should be increased to 11.

The change from std::err to CPLError() is mostly OK, except that you shouldn't use CE_Fatal that cause an abort() to be issued. You should use CE_Failure instead so that the process can cleanly recover. And CPLError(CE_xxxx, CPLE_xxxx, error.what()) is dangerous practice if error.what() contains formatting patterns like "%s". You should use CPLError(CE_xxxx, CPLE_xxxx, "%s", error.what()) instead

The "Open GDAL Datasets" message is due to a dataset leak in the error code path of the driver. In the catch (r_Error error) { }, there should be a delete rasDataset before return NULL, but it's probably more the responsibility of the driver author to correct this and ensure that the various member variables of the dataset are correctly initialized in the constructor so that the destructor can be safely called at any point.

comment:12 by ilucena, 14 years ago

OK. I am correcting it right now.

in reply to:  9 comment:13 by cjucovschi, 14 years ago

The problem with inserting the demo images is the path you specify ./rasdaman When you run the insertdemo.sh, it gives you an example like: ./insertdemo.sh localhost 7001 /home/XXX/tmp/rasdaman_src/bindist/share/rasdaman/examples/images rasadmin rasadmin

that's what you should run. The right path is basically $(SOME_DIR)/share/rasdaman/examples/images

Replying to ilucena:

I tried to run the scripts you proposed but I got an error on the last one. It seems like it is looking for some specific files to run the script.

% insertdemo.sh localhost 7001 ./rasdaman rasadmin rasadmin
insertdemo.sh: rasdaman demo data insert script v2.0
insertdemo.sh: using host localhost, image directory ./rasdaman, and user/passwd rasadmin/rasadmin
mr...Error: cannot find input file ./rasdaman/mr_?.pgm
Exception:  ODMG General

Are you going to provide those files or can just replace it by any any other dataset set?

I tries to make up some sample data but I've got this error here:

% gdalinfo "rasdaman: query='select a[\$x_lo:\$x_hi,\$y_lo:\$y_hi] from rgb as a'"
Execution error 355 in line 1, column 30, near token rgb: Collection name is unknown.
ERROR 4: `rasdaman: query='select a[$x_lo:$x_hi,$y_lo:$y_hi] from rgb as a'' does not exist in the file system,
and is not recognised as a supported dataset name.

gdalinfo failed - unable to open 'rasdaman: query='select a[$x_lo:$x_hi,$y_lo:$y_hi] from rgb as a''.
Open GDAL Datasets:
  1 N DriverIsNULL      -1 512x512x0

comment:14 by Even Rouault, 14 years ago

r20515 /trunk/gdal/ (6 files in 2 dirs): Rasdaman: fix configure; fix memory leaks (#3721)

comment:15 by ilucena, 14 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.