It took me a bit to piece together all the information, so I decided to share them here. To install an arbitrary version of Scipy in your Travis container (even Python 2.6) you need to:
Add the following packages via the apt addon:
addons: apt: packages: - libopenblas-base - libopenblas-dev - liblapack-dev - gfortran
Upgrade pip so it can install wheels:
before_install: - pip install --upgrade pip
Install any version of Scipy via pip:
install: - pip install scipy==0.18.1
In a Python 2.7 container this installs a binary package of Scipy. Very fast, very convenient. In a Python 2.6 container Scipy apparently has to be compiled from scratch. Less fast, but once it is set up, no less convenient.