OpenShift, Virtualenvs, and Software Collections

For professional and personal reasons, I've been spending a lot of time with OpenShift lately.  Mainly OpenShift Enterprise, but I've been using OpenShift Online for some personal learning (read: idle technology noodling). One of the great things about it is the fact that with a single command I can spin up a brand new Python 2.7 environment with a MySQL backend and get to kicking the tires on Flask.

OpenShift runs the app in a virtualenv on the gear, Flask and any supporting modules are going to need to be installed there. The Python support in OpenShift is WSGI based so we'll need to be able to run the app locally in the same manner for testing.

Note: If you don't know what that change means, don't worry neither do I.  That's what community quickstarts are for. I met Ryan Jarvinen at Red Hat Summit this year who pointed me to his Flask/HBase application on Github as a starting point.  I’ve made major strides in understanding the new WSGI support in OpenShift on my part as a result.  If you are a WSGI guru, just shake your head and pity the coding sysadmin...

The immediate problem is my workstation runs RHEL 6.5 which ships with Python 2.6, and I've created a Python 2.7 app.  Not to worry, Software Collections to the rescue.  A collection is a self-contained version of development tools like Python, Ruby, and MariaDB that can be installed in parallel with existing system versions.  This is a much better way to go about supporting multiple versions of languages, especially when one is the language of choice for system utilities (e.g. Python). Software collections ship with all supported RHEL 6 subscriptions, so all I needed to do was install the Python 2.7 collection and enable it.

   [mmicene:~/Desktop] $ sudo yum install python27

That's it, now I've got the base SCL package for Python 2.7.  There's a list of additional packages you can add like Python27-MySQL-Python, Python27-tkinter, etc.  I'll skip these since we're going to install components into the local Python environment later.

Now that I have a matching interpreter installed, I need to enable it so I'm using the new interpreter in place of the original Python 2.6 interpreter.  I'm also going to run a bash shell so I have a full interactive terminal instead of running just a single app.

   [mmicene:~/Desktop] $ scl enable python27 bash

Ok, proof is in the pudding, here's a test from my new shell:

   [mmicene:~/Desktop] [python27] $ which python
   /opt/rh/python27/root/usr/bin/python
   [mmicene:~/Desktop] [python27] $ python --version
   Python 2.7.5

Note: I'm not going to go through rhc configuration and setup or how to create the application. You can follow along with Ryan's README or a few other blogs. From here on out, I'm working in a local git repo for an app I created and cloned from a Github repo.

We're going to create a Python virtualenv as our local development environment. This will make sure our app is self-contained and will run in OpenShift properly. We'll use very standard methods to make sure our dependencies are tracked.

Check out the setup.py in Ryan's repository. The install_requires line is how you tell OpenShift what additional packages to be installed in the gear virtualenv to support the application.  We're going to use that setup.py for our local development environment to keep everything in sync. If you need to add modules on the fly, you can use PIP or you can add them to setup.py and re-run the install.

Let’s set up a virtualenv called 'flask' from our newly SCL enabled shell:

   [mmicene:~/oso/srtracker] [python27] master(+4/-1)* 130 ± virtualenv flask
   New python executable in flask/bin/python
   Installing Setuptools.............................................................................................done.
   Installing Pip....................................................................................................................................done.
   [mmicene:~/oso/srtracker] [python27] master(+4/-1)* ± source flask/bin/activate
   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* ±
   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* ± python setup.py install

   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* 1 ± which python
   ~/oso/srtracker/flask/bin/python
   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* ± python --version
   Python 2.7.5

You can see in the PATH that we are using the Python interpreter in the virtualenv which, as you can see in the version, is a copy of the Python interpreter in the collection.

We're now set.  Our local Python matches OpenShift's, we have a virtualenv just like we'll need in OpenShift, and all of the requirements are installed. There are a few layers of redirection, but once configured, we can switch back and forth with ease.

To leave the virtualenv and disable the collection if we need to get back to normal:

   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* ± deactivate
   [mmicene:~/oso/srtracker] [python27] master(+4/-1)* ± exit
   exit
   [mmicene:~/oso/srtracker] master(+4/-1)*

To enable the collection and enter the virtualenv when it's time to start hacking again:

   [mmicene:~/oso/srtracker] master(+4/-1)* ± scl enable python27 bash
   [mmicene:~/oso/srtracker] [python27] master(+4/-1)* ± source flask/bin/activate
   [mmicene:~/oso/srtracker] [python27] [flask] master(+4/-1)* ±

Don’t forget to add the virtualenv directory to your .gitignore so you don't push it to your OpenShift app.

What happens if you're using Ruby 1.9.3 or PHP 5.4 in OpenShift?  No problem, just install the right SCL packages and adjust your local versioning toolkit (like RVM or rbenv).  Since OpenShift and SCL are providing the same software packages, it all just works.

Note: Curious about the prompt I'm using in the examples?  Hat tip to Langdon White, developer evangelist at Red Hat.  He pointed me to liquidprompt when I mentioned to him I was having issues keeping these sorts of nested virtualenv and OpenShift environments straight.   The current version didn't support SCL, but I hacked together an extra bit and submitted a pull request.