SE_ROW_ID in SDE not initialized for unregistered SDE tables

[mapserver@…]

I found a bug in the SDE Support of mapserver. According to my tests there is a
problem since version 4.1. 4.0 works fine. Any following versions returning the
error message below with the same configuration:

Warning: [MapServer Error]: msSDELayerWhichShapes(): SE_queryinfo_set_columns():
Invalid parameter value passed to function. (-66) in
/home/nicol/public_html/mapserver/dev/sde_datenquelle.php on line 59

I did the tests on Linux and Windows boxes.
The client version matches the server version. In both cases i user ArcSDE 8.3
The problem is reproduceable with (php) mapscript and the cgi version.
The SDE data wasn't "upgraded" from an older SDE instance.

Attached you can find some outputs from sdelayer (client) and sdetable (server):

sdelayer -o describe -i 5151 -s enzo -D sundb -u oliver -p xxxx -l
OLIVER.NLGRENZEN,SHAPE

ArcSDE   8.3      Build 284 Thu Jul 17 14:55:03 PDT 2003
Layer    Administration Utility
---------------------------------------------------------
Table Owner        : OLIVER
Table Name         : NLGRENZEN
Spatial Column     : SHAPE
Layer id           : 63
Entities           : a
Layer Type         : SDE
I/O Mode           : NORMAL
User Privileges    : SELECT, UPDATE, INSERT, DELETE
Layer Configuration: DEFAULTS

./sdetable -o describe  -i 5151 -s enzo -D sundb -u oliver -p xxx -t
OLIVER.NLGRENZEN

ArcSDE   8.3      Build 284 Thu Jul 17 14:55:03 PDT 2003
Attribute        Administration Utility
-----------------------------------------------------

Table OLIVER.NLGRENZEN:
Column name             Attribute type   Null?      Length,DPs    RowID Column?
------------------------------------------------------------------------------
NL_ID                   SE_STRING        NULL            2
NL_NAME                 SE_STRING        NULL           25
MIN_X                   SE_INTEGER       NULL            7
MIN_Y                   SE_INTEGER       NULL            7
MAX_X                   SE_INTEGER       NULL            7
MAX_Y                   SE_INTEGER       NULL            7
COLOR                   SE_SMALLINT      NULL            2
SHAPE                   SE_SHAPE         NULL            0

Hope this helps to find the problem.
Let me know if you need futher informations
Nicol

[hobu]

Nicol,

This bug exists in 4.1 as well as 4.2?  In the 4.2 release, I upgraded the code
internals to use the newer-style query methods of the C API.  SE_queryinfo is
now used instead of SE_SQL_CONSTRUCT.  Also at 4.2 was the addition of
supporting versioned queries (see the wiki
http://mapserver.gis.umn.edu/cgi-bin/wiki.pl?MapServerSDE for more details on
how to set it up in the mapfile).

At 4.1 (correct me if I'm wrong Steve), the code was altered to support shared
connections, which decreased the connection load.  There was a bug in this code
that causes segfaults when invalid connections were freed (bug 767).  This bug
has reappeared to me when I'm using the ArcSDE 9.0 client library, and I will be
reopening it, but I don't think it is related to your problem.

Could you send along the layer block of you mapfile that defines this layer? 
The COLOR field intrigues me, as it of type SE_SMALLINT, and I don't know that
the query of this field type supported and may have been dropped in the
refactoring that went on in 4.1 and 4.2.  Also, if you have a typo (like not
including the schema name of the table?) in the layer definition,
SE_queryinfo_set_columns will attempt to bind and complain with this error.

[mapserver@…]

Howard,

yes, the problem exists in 4.1, 4.2.x and the current cvs version.
here is the Map-File i use:

MAP
  NAME "sde"
  EXTENT 3280454.00000 5237810.50000 3921575.00000 6103807.00000
  SIZE 300 300
  WEB
    IMAGEURL "/~nicol/mapserver/phpmapserver/tmp/"
    IMAGEPATH /home/nicol/public_html/mapserver/phpmapserver/tmp/
  END
  LAYER
    NAME deutschlandkarte
    CONNECTIONTYPE SDE
    CONNECTION "enzo,5151,dbsun,oliver,xxx"
    DATA "OLIVER.NLGRENZEN,SHAPE"
    STATUS DEFAULT
    TYPE polygon
    CLASS
      NAME Grenze
      COLOR 192 192 192
      OUTLINECOLOR 0 0 0
    END
  END
END

[hobu]

Nicol,

Two things to try...  Try changing the connection string to
CONNECTION "enzo,5151,dbsun,oliver,xxx,SDE.DEFAULT"

It appears as though you are only drawing geometry instead of both geometry and
attributes.  I don't know for sure, but something might have changed in relation
to the TEMPLATE variable since 4.0.  Try defining a TEMPLATE variable to see if
the queryinfo complaint is being raised because it expects that we are querying
data.  

[mapserver@…]

Howard,
unfortunately both things didn't solve the problem. Neither change the
connection string nor adding the TEMPLATE variable.
May be i found an other interesting thing.

I did a:
$Layer->open();
print_r ($Layer->getItems() );
$Layer->close();

with php-mapscript. Mapserver 4.0 returns the following array:
>Array
(
    [0] => NL_ID
    [1] => NL_NAME
    [2] => MIN_X
    [3] => MIN_Y
    [4] => MAX_X
    [5] => MAX_Y
    [6] => COLOR
    [7] => SHAPE
    [8] => SE_ROW_ID
)

When i execute the same script with Mapserver 4.2 i got this array:
Array
(
    [0] => NL_ID
    [1] => NL_NAME
    [2] => MIN_X
    [3] => MIN_Y
    [4] => MAX_X
    [5] => MAX_Y
    [6] => COLOR
    [7] => SHAPE
    [8] =>
)

'SE_ROW_ID' is missing! Perhaps this is the invalid parameter
SE_queryinfo_set_columns() refers to.

Nicol

[hobu]

Is there anyway I could get access to the database to test with a debugger?  I
have some ideas where the bug might be, but I don't want to muck around with
things blindly if I can help it.  

Also, can you run the mapfile through map2img to see if the error might be
limited to PHP?  IE, are you only attempting to do things with PHP instead of
CGI with respect to SDE?  There might have been changes that have been made that
didn't ripple into the PHP support of SDE...

[mapserver@…]

Howard,

the problem also shows up with the cgi version and is not limited to php-mapscript.
I set up a account for you on our sde machine. Find more informations in a
separat mail which goes direct to you.
Nicol

[hobu]

Ok, I think I've figured out what was going on.  The table that you have wasn't
"registered" in SDE land.  This means that there is a separate set of metadata
tables that SDE uses to maintain the layer.  At 4.2, I put some logic in there
to allow you to have the SE_ROW_ID be named other things.  A test was missing to
throw back SE_ROW_ID if the table wasn't registered.  I've added this to both
the CVS HEAD and 4.2 branches.  The next release of 4.2 will have this patch. 
Please test with the version that is now in your mapserver-4.2.3 directory.  The
version I left in there yesterday had a bunch of debugging and other crap in there.

My test on your machine appears to fix this problem.  I've updated CVS to fix
it, but I'll wait for confirmation before closing the bug.  Thanks for the access

[mapserver@…]

Hi Howard,

i did some tests with your new version. Everything is working fine! I think you
could close the bug. Thanks for your time and the fix. 
Nicol