Use web-style query string syntax for parameterizing /vsicurl/ targets and open options

[sgillies]

On gdal-dev there was a proposal to add syntax for parameterizing open/connection options for remote datasets like this:

/vsicurl/option1=val1[,optionN=valN]*,url=https://example.com/foo.tif

In a following discussion (see ​https://lists.osgeo.org/pipermail/gdal-dev/2017-October/047334.html), arguments were made for a syntax based on the more common and standard query strings of the http(s) URI scheme. The example above would become:

/vsicurl?option1=val1&optionN=valN&url=https://example.com/foo.tif

The advantage of the web-style syntax is that parsers are likely exist for every language used to serve or consume web content. This is not the case for GDAL-specific syntax. GDAL users and their applications might be able to profit from standards like URI Templates ​https://tools.ietf.org/html/rfc6570 if we use web-style syntax.

There are at least two questions that haven't been asked and discussed:

1. Do the URI parsers mentioned above work only by accident? These /vsicurl/ identifiers have no URI scheme and so may be under specified with respects to ​https://tools.ietf.org/html/rfc3986.

2. Is GDAL exposed to excessive percent-encoding of URIs if we use web-style syntax. Maybe. But maybe since GDAL is becoming web middleware, it needs to handle percent encoding and more.

[Even Rouault]

In 40627:

/vsicurl/: change extended syntax to use web-style query string syntax (fixes #7095)

[sgillies]

Even, I notice that you opted for

/vsicurl/option1=val1&optionN=valN&url=https://example.com/foo.tif

instead of

/vsicurl?option1=val1&optionN=valN&url=https://example.com/foo.tif

Was the ? a problem?

For a user of Python's URL parser, this change has no effect. Here are the parsed results of a pre-40627 filename:

>>> from urllib.parse import urlparse
>>> urlparse('/vsicurl/option1=val1,optionN=valN,url=https://example.com/foo.tif')
ParseResult(scheme='', netloc='', path='/vsicurl/option1=val1,optionN=valN,url=https://example.com/foo.tif', params='', query='', fragment='')

The filename is opaque to the parser. Using & to delimit parameters but no ? to delimit the list of parameters from the rest of the filename leads to the same result.

>>> urlparse('/vsicurl/option1=val1&optionN=valN&url=https://example.com/foo.tif')
ParseResult(scheme='', netloc='', path='/vsicurl/option1=val1&optionN=valN&url=https://example.com/foo.tif', params='', query='', fragment='')

It remains opaque.

[Even Rouault]

In 40658:

/vsicurl: use proper web-style URI with '?' delimiter (fixes #7095)

[sgillies]

Thanks, Even!