XP Home

XP and Databases

Lessons Learned

 At the VCAPS project we found ourselves faced with the problem of XP and a large database. Our database was Object Oriented, but a relational database could be handled the same way. Remember that if you implement user stories customer value first your database tables and normalization will become stable faster.
 The key point is taking the advice of Kent Beck, act as if the database is easy to change. Relational databases were created to be flexible, so flex them. Kent also advises that when something is very difficult try doing it more often not less. That way you get good at doing it and it won't be hard any longer. Get into the habit of migrating your database often, you will make less mistakes not more.
 The VCAPS solution was to have a gold, a silver, and many bronze database versions. The gold is the one that resembles the production database. The silver is a migrated gold database. Each developer has a bronze database migrated from the silver.
 A bronze database becomes silver when the developer's code is released. A silver database becomes gold when the production database is migrated.
 It is important that everyone can easily get a copy of the gold or silver data base to use as a bronze quickly and that we keep track of migration paths. Setting up scripts to copy databases as files is very useful. You need one to make a copy of a bronze database promoting it to silver, and one to restore the current silver to the local host, which ever computer you are at.
 Each database has the same set of test user ids and passwords. Developers and database connections can use the same user ids on any of the databases.
 Each development pair will release newly developed code into the source code safe, promote their bronze database to silver, and add a migration script to a list. How you manage the list of migrations will depend on your data base software and how you access it. This list of scripts can then be executed in sequence to migrate the production database when the new
version is released to production. Independence from the application code is important since it can change causing the migration script fail. Use an integration station to control the sequence of changes.
 At any moment in time the new silver database and the currently released code version are exactly in sync. This is important and requires discipline to maintain.
 At any moment in time the gold database and the production release match up for production support. At any moment it time the data base migration scripts and the most current development release match up ready to be released.
 Developers can integrate often because it is easy to copy the silver database and check out the current released code at the same time. The unit tests will run at 100%. Developers then add their own changes, perform any database migration, and integrate until the unit tests are running at 100% again. This method is very fast and only takes a couple minutes.
 To support testing we had a script to create a new gold database with a predefined set of test data. Some data will be created by tests, but providing an example in the database helps developers create reliable migration scripts. We found it useful to create a new gold and migrate it to silver once a week to avoid the inevitable data corruption and to assure ourselves that our migration scripts were correct.
 We did have an occasional problem with migration. We didn't do as good a job of keeping track of changes as we should have. What we ended up doing near the end was to formulate our migrations as executable methods on a DB maintenance object. This makes the production migration more reliable and a non-event. Lessons learned

Wiki WikiWiki Wiki

ExtremeProgramming.org home | XP Lessons Learned | About the Author

Copyright 1998, 1999, 2001 Don Wells all rights reserved.