This page describes techniques used for preparing OSGeo4W packages.
Table of Contents
OSGeo4W packages are each bzip2 compressed .tar files intended to be unpacked in the OSGeo4W root directory. They come with setup.hint files designed to describe dependencies and a package description.
Package Directory Layout
The installed OSGeo4W directory tree is described in DirectoryLayout.
OSGeo4W packages are generally structured like packages for Cygwin as described at http://cygwin.com/setup.html . And support two architectures x86 (32bit) and x86_64 (64bit).
Packages prepared for OSGeo4W are currently uploaded to (requires shell access to OSGeo download server):
This tree can be reviewed at: http://download.osgeo.org/osgeo4w/<arch>/release
Each package directory should have a single setup.hint file, and one or more versions of that package as a .tar.bz2 file. These should normally have names like gdal-1.5.0-7.tar.bz2 where "1.5.0" is the version of GDAL that was packaged, and "-1" indicates the version of the packaging attempt. The directory can also contain corresponding source snapshots named like gdal-1.5.0-7-src.tar.bz2. Note that the basename should exactly match the directory in which the package is found. So the "gdal" directory has gdal-<version>-<revision>.tar.bz2 package files.
Sometimes a single project is packaged as several packages. In this case component packages are normally created as subdirectories of the primary directory. So, for instance the Oracle 10g driver for GDAL is kept in the release/gdal/gdal-oracle10g directory and packaged as gdal-oracle10g-1.5.0-1.tar.bz2.
Note: Please use
bzip2 from osgeo4w's msys (%OSGEO4W_ROOT%\apps\msys\bin). Don't use 7zip as setup.exe can't understand the archives it creates, and there is mixed success with gnuwin32 bzip2 and cygwin bzip2. Linux tar/bzip2 has no reported problems. (see ticket #101)
Web Application Package Guidelines
Here are some important guidelines:
- the /osgeo4w/apps/ directory exists to hold applications. Your application must be installed in a subdirectory of the /osgeo4w/apps/ directory, preferably with a meaningful name (in lowercase).
- the Apache alias "/ms_tmp/" exists to access temporary web-accessible files, which points to the directory /osgeo4w/tmp/ms_tmp/
- the Apache alias "/cgi-bin/" points to the /osgeo4w/bin/ folder. e.g. mapserv.exe lives in the /bin folder.
- the /osgeo4w/httpd.d/ directory exists to hold Apache Alias Files. This directory is automatically scanned by Apache when it starts. The following is an alias example for an application named 'foobar':
Alias /foobar/ @osgeo4w@/apps/foobar/htdocs/ AllowOverride None Options Indexes FollowSymLinks Multiviews Order allow,deny Allow from all
The name of the file is very important. Apache will only load files that match the following pattern:
Normally, you would replace * with your application name. In the above example, the application would use httpd_foobar.conf in the /OSGEO4W/httpd.d/ directory. This would allow the user to connect to http://localhost/foobar/ to access the application.
- All web applications should include a small .pkg file in the /osgeo4w/apache/htdocs/ folder so that a link to the application is displayed on the OSGeo4W index page (localhost). This 'Index Package File' provides a small HTML snippet that is automatically included in the OSGeo4W index page. This HTML snippet should provide a brief description of your application and provide links to access the aliases you have provided via the Apache Alias File. The Index Package File follows a strict naming convention:
Only files that end in .pkg.html will be included in the index page. Here is an example of a .pkg file (named 'foobar.pkg.html):
<pre><h3>Foobar Application</h3> <blockquote><p><a href="/foobar/">Foobar demo</a></p></pre>
Each package directory has a setup.hint file which normally contains the short and long description for the package, and the list of other packages on which it depends. Note that there is normally only one setup.hint file for a package even if several versions of the package exist. A normal setup.hint file might look like::
sdesc: "The Libgeotiff library, commandline tools and supporting tables." ldesc: "The Libgeotiff library, commandline tools and supporting tables. Used for manipulating GeoTIFF files." maintainer: FrankWarmerdam category: Libs Commands requires: msvcrt libtiff
This indicates the short and long description for the package, the list of categories it will appear in for the installer user interface, and the package it requires. The maintainer field is an OSGeo4W addition and is used to add a wiki link to the package listing.
Packages may take post-install actions by including a .bat file in the /etc/postinstall directory as part of the package. The filename should be named etc/postinstall/<package>.bat. The bat file will be run with the environment variables OSGEO4W_STARTMENU, OSGEO4W_DESKTOP, OSGEO4W_DESKTOP_LINKS, OSGEO4W_MENU_LINKS, OSGEO4W_HOME, and OSGEO4W_HOME_MSYS defined, and with OSGEO4W_HOME/bin already in the path.
One very common thing to do in postinstall actions is to insert/update configuration files to include the installed path to the OSGeo4W tree. This is normally done with a postinstall file, and here are 2 examples::
- hardcoded paths exist in the /httpd.d/httpd_gmap.conf file of the gmap package:
Alias /gmap/ "C:/mypath/apps/gmap/htdocs/" <Directory "C:/mypath/apps/gmap/htdocs/"> AllowOverride None Options Indexes FollowSymLinks Multiviews Order allow,deny Allow from all </Directory>
- the first step is to replace the hardcoded path in the file with the @osgeo4w@ variable, in your favourite text editor. For example the above file would then look like:
Alias /gmap/ "@osgeo4w@/apps/gmap/htdocs/" <Directory "@osgeo4w@/apps/gmap/htdocs/"> AllowOverride None Options Indexes FollowSymLinks Multiviews Order allow,deny Allow from all </Directory>
- the second step is to create a postinstall script named gmap.bat (that lives at /etc/postinstall/ of the gmap package) to replace that variable with the actual user's install path:
textreplace -sf httpd.d/httpd_gmap.conf -df httpd.d/httpd_gmap.conf -map @osgeo4w@ %OSGEO4W_ROOT%
Note: As of April 2008 textreplace will update files in place, so the infile/outfile syntax is not needed. Also
-std is shorthand for for
-map @osgeo4w@ %OSGEO4W_ROOT% so the above becomes simply:
textreplace -std -t httpd.d/httpd_gmap.conf
textreplace and python
Add the following to the beginning of your postinstall bat so that textreplace uses double backslashes (
Another postinstall task that packages might do is to create shortcuts that will be available to launch a certain command. As its is the case for the textreplace executable, there is an utility binary xxmklink.exe that is available in the installer (part of the msvcrt package). By typing xxmklink, you can see all the options supported by this utility. Here are some examples on how it can be used in a post install file:
if not %OSGEO4W_MENU_LINKS%==0 ( mkdir "%OSGEO4W_STARTMENU%" mkdir "%OSGEO4W_STARTMENU%\Apache" xxmklink "%OSGEO4W_STARTMENU%\Apache\OSGEO4W-Apache-Install.lnk" %OSGEO4W_ROOT%\Apache\bin\httpd.exe " -k install -n \"Apache OSGEO4W Web Server\"" )
if not %OSGEO4W_MENU_LINKS%==0 xxmklink "%OSGEO4W_STARTMENU%\ttt.lnk" cmd.exe "" . "my_desc" 1 "%OSGEO4W_ROOT%\OSGeo4W.ico"
A Desktop icon:
if not %OSGEO4W_DESKTOP_LINKS%==0 xxmklink "%OSGEO4W_DESKTOP%\OpenEV.lnk" "%OSGEO4W_ROOT%\bin\openev.bat" " " \ "OSGeo4W OpenEV" 1 "%OSGEO4W_ROOT%\apps\openev\icon.ico"
The environment variables
OSGEO4W_DESKTOP_LINKS are set by the installer (since 1.0.6-5) and reflect whether or not the corresponding option was checked.
Alternativly the msvcrt package shortcuts can be created with
nircmd (see nircmd). Example from the qgis-dev postinstall:
REM get short path without blanks for %%i in ("%OSGEO4W_ROOT%") do set O4W_ROOT=%%~fsi if not %OSGEO4W_MENU_LINKS%==0 mkdir "%OSGEO4W_STARTMENU%" if not %OSGEO4W_MENU_LINKS%==0 nircmd shortcut "%O4W_ROOT%\bin\nircmd.exe" "%OSGEO4W_STARTMENU%" "QGIS Desktop 1.9.0" "exec hide %O4W_ROOT%\bin\qgis-dev.bat" "%O4W_ROOT%\apps\qgis-dev\icons\QGIS.ico" if not %OSGEO4W_MENU_LINKS%==0 nircmd shortcut "%O4W_ROOT%\bin\nircmd.exe" "%OSGEO4W_STARTMENU%" "QGIS Browser 1.9.0" "exec hide %O4W_ROOT%\bin\qgis-dev-browser.bat" "%O4W_ROOT%\apps\qgis-dev\icons\QGIS.ico" if not %OSGEO4W_DESKTOP_LINKS%==0 mkdir "%OSGEO4W_DESKTOP%" if not %OSGEO4W_DESKTOP_LINKS%==0 nircmd shortcut "%O4W_ROOT%\bin\nircmd.exe" "%OSGEO4W_DESKTOP%" "QGIS Desktop 1.9.0" "exec hide %O4W_ROOT%\bin\qgis-dev.bat" "%O4W_ROOT%\apps\qgis-dev\icons\QGIS.ico" if not %OSGEO4W_DESKTOP_LINKS%==0 nircmd shortcut "%O4W_ROOT%\bin\nircmd.exe" "%OSGEO4W_DESKTOP%" "QGIS Browser 1.9.0" "exec hide %O4W_ROOT%\bin\qgis-dev-browser.bat" "%O4W_ROOT%\apps\qgis-dev\icons\QGIS.ico"
Note: that this also uses
nircmd to start the batch job, which has the advantage that there is no command line window flashing up when a desktop application starts.
The postinstall scripts can run any DOS command or installed binary with desired arguments.
Packages may also take pre-remove actions by including a .bat file in the /etc/preremove directory as part of the package. The filename should be named etc/preremove/<package>.bat. The bat file will be run with the environment variables OSGEO4W_STARTMENU, OSGEO4W_DESKTOP, OSGEO4W_HOME and OSGEO4W_HOME_MSYS defined, and with OSGEO4W_HOME/bin already in the path.
A common application for this would be the removal of the shortcuts created in the post install actions.
Some packages need to set particular environment variables. For the OSGeo4W command shell this can be accomplished by adding an initialization file in etc/ini. A typical example looks like::
These scripts are run in random order by %OSGEO4W_ROOT%\OSGeo4W.bat when establishing the environment for the OSGeo4W shell. The OSGEO4W_ROOt environment variable will always be set on the way in.
Application batch file template
Sample of how to start an application with standard environment set. Replace
foo.exe with your application executable and save as
As of pkg-msvcrt v1.0.1-6 (2012-Feb-07) root is determined dynamically at run-time.
foobar.bz2:bin/foo.bat.tmpl. Post-Install will set the proper OSGEO4W_ROOT.
@echo off call "%~dp0\o4w_env.bat" start "Foo window title" /B "%OSGEO4W_ROOT%\bin\foo.exe" %* @echo on
@echo off call "~dp0\..\..\bin\o4w_env.bat start "Foo window title" /B "%OSGEO4W_ROOT%\apps\foo\foo.exe" %* @echo on
The setup.exe actually downloads one of the setup.ini files (32bit or 64bit) with a list of all packages each time it is run. These files should be recreated on the server by running the script invoking
osgeo4w-regen.sh on the web each time packages are uploaded. This produces a test versions of these files (http://download.osgeo.org/osgeo4w/<arch>/setup_test.ini), which can be used with the setup program (by invoking setup-test.bat in the OSGeo4W shell). Once this has been tested, using the link
osgeo4w-promote.sh to copy this file to setup.ini, making it live for regular users.
- Generate test versions of setup.ini:
- Verify package version numbers (see top of PackageListing)
- Test by invoking setup-test.bat in the OSGeo4W shell
- Promote test setup to live:
Note: the nightly build of QGIS (package
qgis-dev) is build and uploaded around 00:00 UTC / 16:00 PST in the process the
setup.ini} is updated and promoted automatically.
Please ensure that you prepare a wiki page linked off PackageListing with info on your package. It should include a pointer back to the upstream project, the name of the packager (a link to a page with contact info on the packager), and special notes on where things are placed within OSGeo4W.
Problematic Package Versions
You may find that your package may not be listed as the current version. An example is the following, where the installer believes apache-2.2.8-3 is the latest version, but apache-2.2.14 is the correct latest version:
apache-2.2.8-1.tar.bz2 apache-2.2.8-2.tar.bz2 apache-2.2.8-3.tar.bz2 apache-2.2.14-1.tar.bz2
To solve this you can specify the current and previous versions in the setup.hint file (see more info at http://cygwin.com/setup.html#setup.hint) such as:
sdesc: "Apache Web Server" ldesc: "Apache Web Server" category: Web requires: msvcrt php curr: 2.2.14-1 prev: 2.2.8-3