strptime() not available in VC++

[dmorissette]

A call to strptime() has been added to maplexer.l recently but this function is
not available in VC++ on Windows.

Assefa was going to look at providing a replacement for Windows.  

Note: this replacement function should be #ifdef'd the same way as was done for
strncasecmp() (i.e. #ifdef NEED_STRNCASECMP, etc.).

[dmorissette]

Note that Ned H. has replaced the strptime() call with sprintf() in CVS, so the
current CVS needs to be fixed.

[dmorissette]

Updates on this:

1- Ned has reverted the changes in CVS

2- Yewondwossen Assefa wrote:

> 
> Daniel,
> 
>  I have looked into that few days back and found several sites 
> distributing a source code for strptime. All the ones I found had some 
> sort of licence/copyright notices. I discussed this with Steve since he 
> mentionned to me that he might know some sites distributing the code. I 
> am waiting on that to see. If we don't find one without the licences, I 
> will integrate one of those I found.
> 

[dmorissette]

There seems to be a time/strptime.c in the Glibc 2.0 source covered by the LGPL
license.  Does someone know if it's safe to distribute a LGPL'd file as part of
MapServer's source? Since we distribute the source we should be safe, right?

[dmorissette]

Another possibility:

I found out that libcurl contains a function called curl_getdate() which does a
job similar to strptime. It is in curl/lib/getdate.y which is completely public
domain (no copyright).

When linking with curl we could use curl_getdate(), and when curl is not
included then we could enable the local copy.  I dunno if there could be thread
safety issue since it's a bison-generated file, but curl is quite clean AFAIK so
I would tend to trust that it's safe.

[sdlime]

Can we pull it out into it's own function and just dump that in mapstring.c? 
May be tough though because bison uses flex to generate tokens. Adding more 
configure complexity is a bad idea unless it's really necessary. 

[dmorissette]

getdate.c is about 60k (it's a bison-generated file) so we can't just copy it in
mapstring.c.

*If* we go with that file then we should probably copy getdate.y to the
mapserver source and let the makefile run bison on it. We could possibly store
the generated getdate.c to CVS as well since it is unlikely to change often.
Some work would be required to remove curl-specific stuff though but probably
not that much.

But before we go there, we should find out if the function does what you need.
I'm not sure what you were using strptime() for exactly. Was it to parse WMS
time strings?  Here is the docs for curl_get_date():
http://curl.haxx.se/libcurl/c/curl_getdate.html

[sdlime]

I looked over the lib curl function and there may be some possiblities there. I
worry about collisions with the expression parser. There must be a tokenizer
someplace, hopefully not flex based. The other issue is the format that get_date
returns a parsed date, as time_t, number of seconds since Jan 1st, 1970. I
assume with dates earlier than 1970 it will return negative values. Problem is
that there is a definite limit to how old a date you can represent, someplace
around 1910 I would guess. I'd hate to presume what sort of attribute data folks
might have that could test that limit. 

Steve

[dmorissette]

Good point about the time_t.  Perhaps if we go with that function we should
modify it to return a 'struct tm' like strptime(), or a new timeObj.

[sdlime]

Man, I waffle on struct tm or time_t. Especially as I worked through your email 
this morning. Any arithmetic (sp?) operations are cumbersome with the tm 
structure. Resolutions becomes an issue, for example, what's the difference 
between 2 dates returned as, seconds, days, years? Futhermore how do you deal 
with that in expressions. Yuck.

On the flipside working with time_t is great cause it's absolute, and just a 
single number. But there's a limit on representable dates and comparisons on 
just parts of a date become difficult.

Argh...

[dmorissette]

I suggested something in my email reply a few minutes ago that may help,
basically using a double to sotre the time in seconds instead of the 32 bits
time_t.  That would give us more precision than we need, but that doesn't solve
the issue of "the difference between 2 dates returned as, seconds, days, years".

(Arghh... the bugzilla server won't let me submit large comments... am I the
only one with that problem?)

[dmorissette]

(continued...)

Maybe to solve that case as well we could store our time as 2 values:
 struct {
    struct  tm tm_time;  /* Used for parsing/display only */
    double  tmin;   /* Seconds since the Epoch where that time unit starts*/
    double  tmax;   /* Seconds since the Epoch where that time unit ends*/
  } msTime;

For a time specified as date (e.g. TIME=2004-03-10) then tmin would be the
number of seconds for 2004-03-10T00:00 and tmax would correspond to
2004-03-10T23:59.   When time is precise to the seconds then tmin and tmax would
be equal.  

Then all comparison operations could be performed on tmin and tmax... or did I
miss something?

[sdlime]

But none of the functions that generate time in seconds will generate anything 
other than time_t will they? That is, we have no way to generate more than 32-
bit time values regardless of how they are stored.

I had also been thinking about storing reduced resolution versions of time, but 
there may be some serious performance issues with this.

[dmorissette]

Duh!  Even libcurl's getdate() uses mktime() to turn the struct tm into a
time_t... I was hoping that it did contain its own algorithms so we're out of
luck unless we reimplement mktime().

I've been doing some reading/reseearch and found refs to potential open source
replacements for the system time.h stuff... I'll let you know what I find.

[dmorissette]

It almost seems that nobody has implemented a solution for the 32 bits time_t
problem in real life systems yet. I found lots of talk about the 32 bits time_t
problem, and two interesting resources with source code, but they don't seem
very widely used or well supported:

- libtai - public domain - http://cr.yp.to/libtai.html
  The lib isn't ported to Windows and doesn't come with configure and install
  scripts. However we could likely extract the code we need and package it 
  in MapServer (e.g. maptime.c).  Porting to Windows shouldn't be a serious
  issue.

- Curtis Dyreson from U of Arizona has a paper discussing the problem and 
  sample code of a high performance algorithm that can cover a range of 
  36 billion years. The problem is that they provide code that works only 
  for years A.D. and write that "B.C. input algorithms are essentially the 
  same as the A.D. algorithms, except that different tables are used and
  the code is slightly different"... unfortunately that statement doesn't
  help much.  Anyway, the paper is at:
http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol24/issue1/spe874.pdf
... and I found a copy of the source at:
http://ftp.se.kde.org/pub/databases/misc/tsql/fastIO/fastIO.tar.z


Other than that I found a rather simple version of mktime() in Linux's
include/linux/time.h that we could very easily adapt to return doubles...
however I'm wondering what it's missing since all other versions of this
function are much more complex. It might be good enough for what we need anyway.
 Unfortunately there may be a blocker problem: this kernel code is probably
GPL'd even if that's not stated in the source file, so we cannot just copy it
into the mapserver source.  :(  If only we could find a reference to the
original Gauss algorithm that the source refers to.  I'll attach the source to
this bug.

[dmorissette]

mktime() function from Linux' include/linux/time.h

[sdlime]

Can someone verify what time functions are normally found on Windows? We should
really avoid reinventing the wheel. I wonder what MySQL uses? Their time range
is from 1000-1-1 to 9999-12-31 (or something like that).

Steve

[sdlime]

Have you guys worked or thought about this more? I was thinking that if we were 
able to use the libcurl getdate function that we'd probably have to somehow 
delimit dates differently from strings. That's would be necessary because the 
MapServer lexer wouldn't be involved in the date parsing at all. Any 
suggestions?

Steve

[sgillies@…]

What's the status of this bug?  What are y'all doing on Win32 to work around
it?

[sgillies@…]

Daniel,

I _really_ appreciate that you have been rigorous about writing messages
that get logged with your code commits.  I am slowly but surely finding
what needs to be rolled back to get MapServer to compile with MSVC.

Once I have finished, what would y'all think if I committed and tagged
the source?  Once the time stuff is done, the developers could merge their
code into the revision that I'd tagged and we'd be back in business.

Sorry to be a pest about this but I've really been counting on being able
to work on the development MapServer under Win32 at the end of this week.

[dmorissette]

I have created msStrptime() in maptime.c that calls strptime() on Unix and
produces an error on Windows, and replaced the call in maplexer.l.  You'll need
a new maplexer.c and then you should hopefully be able to build that file on
Windows (I can't test on Windows myself)

That doesn't fix the current bug: it just allows you to build on Windows until a
replacement is in place.

[dmorissette]

Time stuff pushed to the 4.4 release.

[dmorissette]

Assefa, with the time stuff done, I believe this can be marked fixed?

[assefa]

There is a local copy of strptime that is build with mapserver for windows usage.
Marking this as Fixed.