Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#491 closed task (fixed)

Buildbot 0.7.11p3 upgrade

Reported by: Mateusz Łoskot Owned by: Mateusz Łoskot
Priority: major Milestone:
Component: Systems Admin Keywords: buildbot
Cc:

Description

Following [mhttp://lists.osgeo.org/pipermail/sac/2009-October/002038.html announcement] on the list, I'm attempting to upgrade OSGeo Buildbot installation. This ticket is dedicated to record the upgrade process.

Next, I'm going to move slaves, particularly heavy slaves, off buildbot.osgeo.org to buildtest.osgeo.org

Change History (5)

comment:1 Changed 10 years ago by Mateusz Łoskot

Owner: changed from sac@… to Mateusz Łoskot
Status: newassigned

comment:2 Changed 10 years ago by Mateusz Łoskot

OSGeo Buildbot Upgrade Procedure

  • Purge all logs and stop buildbot instances
[buildbot@xblade14 ~]$ ./buildbot_clean_logs.sh
[buildbot@xblade14 ~]$ ./buildbot_stop.sh
  • Situation check before upgrade
[mloskot@xblade14 ~]$ buildbot --version
Buildbot version: 0.7.10p1
Twisted version: 8.1.0

[mloskot@xblade14 ~]$ more /usr/bin/buildbot
#!/usr/bin/python
# EASY-INSTALL-SCRIPT: 'buildbot==0.7.10p1','buildbot'
__requires__ = 'buildbot==0.7.10p1'
import pkg_resources
pkg_resources.run_script('buildbot==0.7.10p1', 'buildbot')
  • Upgrade Buildbot software to most recent version (here 0.7.11p3)
[mloskot@xblade14 ~]$ sudo easy_install --upgrade buildbot
Searching for buildbot
Reading http://pypi.python.org/simple/buildbot/
Reading http://buildbot.net/
Reading https://sourceforge.net/project/showfiles.php?group_id=73177&package_id=73212&release_id=665379
Reading http://buildbot.sourceforge.net/
Best match: buildbot 0.7.11p3
Downloading http://pypi.python.org/packages/source/b/buildbot/buildbot-0.7.11p3.zip#md5=dccc17e201ee1bc20e12fbaad1ffcff2
Processing buildbot-0.7.11p3.zip
Running buildbot-0.7.11p3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-p0Danp/buildbot-0.7.11p3/egg-dist-tmp-eZ8a_u
warning: no files found matching 'docs/buildbot.html'
zip_safe flag not set; analyzing archive contents...
buildbot.status.web.baseweb: module references __file__
buildbot.status.web.grid: module references __file__
buildbot.clients.debug: module references __file__
buildbot.scripts.runner: module references __file__
buildbot.test.test_maildir: module references __file__
buildbot.test.runutils: module references __file__
buildbot.test.test_shell: module references __file__
buildbot.test.test_slavecommand: module references __file__
buildbot.test.test_mailparse: module references __file__
Removing buildbot 0.7.10p1 from easy-install.pth file
Adding buildbot 0.7.11p3 to easy-install.pth file
Installing buildbot script to /usr/bin

Installed /usr/lib/python2.4/site-packages/buildbot-0.7.11p3-py2.4.egg
Processing dependencies for buildbot
Finished processing dependencies for buildbot
[mloskot@xblade14 ~]$
  • Situation check after upgrade
[mloskot@xblade14 ~]$ buildbot --version
Buildbot version: 0.7.11p3
Twisted version: 8.1.0

[mloskot@xblade14 ~]$ cat /usr/bin/buildbot
#!/usr/bin/python
# EASY-INSTALL-SCRIPT: 'buildbot==0.7.11p3','buildbot'
__requires__ = 'buildbot==0.7.11p3'
import pkg_resources
pkg_resources.run_script('buildbot==0.7.11p3', 'buildbot')
  • Prepare current console environment to run buildbot script in upgrade mode. Check if custom osgeo module is loadable.
[buildbot@xblade14 ~]$ . ./buildbot_env.sh 
OSGEOBUILDHOME=/osgeo/buildbot
PYTHONPATH=/osgeo/buildbot/usr/lib/python2.4/site-packages:/osgeo/buildbot/etc/slaves:

[buildbot@xblade14 ~]$ python
Python 2.4.3 (#1, Jun 13 2006, 16:41:18) 
[GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import osgeo
>>> 
[2]+  Stopped                 python
[buildbot@xblade14 ~]$
  • For every <PROJECT> execute buildbot in upgrade mode - NOTE: inside <PROJECT> directory!!!
[buildbot@xblade14 ~]$ cd $OSGEOBUILDHOME/<PROJECT>/master

[buildbot@xblade14 master]$ buildbot upgrade-master ./
/osgeo/buildbot/<PROJECT>/master/public_html/index.html has old/modified contents
 writing new contents to /osgeo/buildbot/<PROJECT>/master/public_html/index.html.new
/osgeo/buildbot/<PROJECT>/master/public_html/buildbot.css has old/modified contents
 writing new contents to /osgeo/buildbot/<PROJECT>/master/public_html/buildbot.css.new
upgrade complete
/usr/lib/python2.4/site-packages/buildbot-0.7.11p3-py2.4.egg/buildbot/status/web/builder.py
/usr/lib/python2.4/site-packages/buildbot-0.7.11p3-py2.4.egg/buildbot/status/web/waterfall.py
  • Make a dry-run on one of projects
[buildbot@xblade14 ~]$ ./buildbot_start.sh proj4 
OSGEOBUILDHOME=/osgeo/buildbot
PYTHONPATH=/osgeo/buildbot/usr/lib/python2.4/site-packages:/osgeo/buildbot/etc/slaves:
Starting BuildBot for proj4 in /osgeo/buildbot/proj4
   master...Following twistd.log until startup finished..
...
2009-10-10 10:39:29-0700 [-] Log opened.
2009-10-10 10:39:29-0700 [-] twistd 8.1.0 (/usr/bin/python 2.4.3) starting up
2009-10-10 10:39:29-0700 [-] reactor class: <class 'twisted.internet.selectreactor.SelectReactor'>
2009-10-10 10:39:29-0700 [-] Starting factory <buildbot.slave.bot.BotFactory instance at 0xb7af9e2c>
2009-10-10 10:39:29-0700 [Broker,client] message from master: attached
The buildslave appears to have (re)started correctly.
OK
[buildbot@xblade14 ~]
  • Start all instances
[buildbot@xblade14 ~]$ ./restartall.sh

Done.

comment:3 Changed 10 years ago by Mateusz Łoskot

Buildbot upgrade archived in SVN: r2617 r2618 r2619 r2620 r2621 r2622 r2623 r2624 r2625 and r2627 r2627

comment:4 Changed 10 years ago by Mateusz Łoskot

Resolution: fixed
Status: assignedclosed

Upgrade completed.

Heavy slaves migration put to separate task (#492)

comment:5 Changed 10 years ago by Mateusz Łoskot

Note, during the #492 migration I reorganised management scripts and configuration, so it's more flexible to maintain distributed instances. So, the instructions above are not re-applicable as presented, in case one needs to do it in future, because paths have changed.

The OSGeo Wiki will be updated.

Note: See TracTickets for help on using tickets.