Exploring a single database migration

    Yesterday I was having some troubles with a database migration download step, and a Joshua Hesketh suggested I step through the migrations one at a time and see what they were doing to my sqlite test database. That's a great idea, but it wasn't immediately obvious to me how to do it. Now that I've figured out the steps required, I thought I'd document them here.

    First off we need a test environment. I'm hacking on nova at the moment, and tend to build throw away test environments in the cloud because its cheap and easy. So, I created a new Ubuntu 12.04 server instance in Rackspace's Sydney data center, and then configured it like this:

      $ sudo apt-get update
      $ sudo apt-get install -y git python-pip git-review libxml2-dev libxml2-utils
      libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev
      python-coverage python-netaddr python-mysqldb python-git virtualenvwrapper
      python-numpy virtualenvwrapper sqlite3
      $ source /etc/bash_completion.d/virtualenvwrapper
      $ mkvirtualenv migrate_204
      $ toggleglobalsitepackages

    Simple! I should note here that we probably don't need the virtualenv because this machine is disposable, but its still a good habit to be in. Now I need to fetch the code I am testing. In this case its from my personal fork of nova, and the git location to fetch will obviously change for other people:

      $ git clone http://github.com/mikalstill/nova

    Now I can install the code under test. This will pull in a bunch of pip dependencies as well, so it takes a little while:

      $ cd nova
      $ python setup.py develop

    Next we have to configure nova because we want to install specific database schema versions.

      $ mkdir /etc/nova
      $ sudo mkdir /etc/nova
      $ sudo vim /etc/nova/nova.conf
      $ sudo chmod -R ugo+rx /etc/nova

    The contents of my nova.conf looks like this:

      $ cat /etc/nova/nova.conf
      sql_connection = sqlite:////tmp/foo.sqlite

    Now I can step up to the version before the one I am testing:

      $ nova-manage db sync --version 203

    You do the same thing but with a different version number to step somewhere else. Its also pretty easy to get the schema for a table under sqlite. I just do this:

      $ sqlite3 /tmp/foo.sqlite
      SQLite version 3.7.9 2011-11-01 00:52:41
      Enter ".help" for instructions
      Enter SQL statements terminated with a ";"
      sqlite> .schema instances
      CREATE TABLE "instances" (
              created_at DATETIME,
              updated_at DATETIME,

    So there you go.

    Disclaimer -- I wouldn't recommend upgrading to a specific version like this for real deployments, because the models in the code base wont match the tables. If you wanted to do that you'd need to work out what git commit added the version after the one you've installed, and then checkout the commit before that commit.

    Tags for this post: openstack tips rackspace nova database migrations sqlite
    Related posts: Juno nova mid-cycle meetup summary: DB2 support; Nova database continuous integration; Faster pip installs; Upgrade problems with the new Fixed IP quota; Michael's surprisingly unreliable predictions for the Havana Nova release; Havana Nova PTL elections

posted at: 18:37 | path: /openstack/tips | permanent link to this entry