It took me a bit to piece together all the information, so I decided to share them here. To install an arbitrary version of Scipy in your Travis container (even Python 2.6) you need to:
Add the following packages via the apt addon:
addons:
apt:
packages:
- libopenblas-base
- libopenblas-dev
- liblapack-dev
- gfortran
Upgrade pip so it can install wheels:
before_install: - pip install --upgrade pip
Install any version of Scipy via pip:
install: - pip install scipy==0.18.1
In a Python 2.7 container this installs a binary package of Scipy. Very fast, very convenient. In a Python 2.6 container Scipy apparently has to be compiled from scratch. Less fast, but once it is set up, no less convenient.

