Been a while since I last posted, heh. Been busy, unfortunately.
In any case, I ran into multiple problems recently. In testing my code for some Python scripts I was writing, they functioned fine in Python 3.5, but failed on a newer system running Python 3.6.3 (but worked somehow on 3.6.1). I ran into a small headache, because my 16.04 system only has Python 3.5 available on it.
So, I had four choices:
- Use multiple systems (virtual or bare metal) or containers to get newer Ubuntu versions, which could be a headache.
- Upgrade my own system to a newer version (which I don’t want to do, because I track LTSes for my regular-use laptop)
- Try and make multiple Python versions on my system in a way that doesn’t clobber the system packages, or
Now, fortunately for me, I’m a power user. I could have done most of these options.
I could have done option 1, because my computer can handle amazing things like VMs, or I could have used LXD to install a newer OS image and used that for testing. However, that requires me to duplicate whatever I’m working on and run it separately or run its tests separately, which can cause some… headaches. Mainly because I can’t use PyCharm IDE to run everything locally and test, or use its debugger for the newer version.
Option 2 is a no-go – my preference is to stick to LTSes for my system, and not use the cutting-edge because of problems that usually arise.
Option 3 is painful because I have to compile each version I need from source. Granted, I’d have to do the same for any newer unpackaged version, or install versions made by others which could be malicious.
This leaves option 4 – use
virtualenv for custom project directories and execution environments, or for PyCharm IDE, set up a
pyenv with a given Python version, and point PyCharm IDE at that. This option works because everything runs in userspace for these environments and doesn’t require me to do much messing around with my system.
Now, you’re probably wondering a couple of things: what are
pyenv, and why should I use it. I’ll address both. I’ll also address how to get all this set up.
pyenv, and what do I use them for?
This is a two part question. Put simply,
pyenv provides a method for creating Python installations and environments in a local directory. Usually, within the user’s home directory, in a
.pyenv folder, with a fairly complicated setup. You can also create environments based on standard Python versions and use those environments as-is. This allows you to create multiple Python setups for multiple Python versions. I mentioned that pyenv is mostly deprecated. It’s been replaced with, essentially, `python3
That said, there’s also a headache – if you want to install a specific program that needs to be its own new item via
pip3 it can cause some issues. Especially if you want to not have to alter your pristine environment. A prime example of this is the PostgreSQL administration tool called
pgadmin4 because it needs a newer Python version than what Ubuntu 16.04 has. For these cases (with Python 3.0 and above only), I also pair
pyenv with the standard
python3-virtualenv software. I used
virtualenv to create a pgadmin4 environment based on Python 3.6.3. This allowed me to make a pristine environment for pgadmin4 to modify. This has its own ups and downs, but sorta worked for what I need.
However, for my other programs and scripts I develop, I only need the standard Python 3.6.3 environment. To that end, I will only explain the use of
pyenv, and making it work.
Now, this is the fun part. I started with the instructions here, which is a pretty good guide for 15.10. It’s also valid here, so I’ll bring you the relevant steps.
- Install dependencies.
sudo apt-get update && sudo apt-get upgrade sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev git
- Download the
curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash
- Update your
.bashrcfile with the following:
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
- If you want to make this work instantly without restarting your terminal, run
source ~/.bashrc. Otherwise, restart your Terminal or SSH session.
That’s really all to make that work. However, there’s a lot of usage parts of relevance here.
Install a different Python version with
This is the easy part. It’ll download and compile Python versions from the source tarballs from Python upstream. It’s useful because you can get multiple Python environments.
Let’s start by installing an oddball Python version, like Python 2.5 (specifically, 2.5.6, the last version of the 2.5.x series) which is out of date. Just run
pyenv install 2.5.6, let it download, and then compile and install it.
But, how do I use it?
Well, let’s assume we’re in my computer, where the
python3 executable on the system itself is actually 3.5.2. I need 3.6.3. I ran
pyenv install 3.6.3 and it installed. Now, how do I use it?
Well, that is the usefulness of the
pyenv software. I can change the local Python environment temporarily. By running
pyenv local 3.6.3, the soystem will put me into the 3.6.3 directory. This is useful, because if I run
python3 after executing
pyenv local 3.6.3, the Python 3 version is 3.6.3 for that execution session. And if you want to switch back to the system’s 3.5.2 version, just run
pyenv local system to switch back to the standard System version of Python, which is 3.5.2 on a 16.04 system.
You mentioned PyCharm IDE… how do I use this with
Well thankfully, we don’t actually have to do any of the
pyenv software via PyCharm IDE. PyCharm is smart enough to be able to detect obvious locations such as those in the
pyenv system’s installation locations, and can offer them as usable environments. You just have to adjust the environment settings for PyCharm for the given project (or execution or runtime configuration) to use the alternate Python version as the runtime environment. This is fairly straightforward, but as I prefer to not develop in odd systems, I just have to point PyCharm to the local Python environment. Which is not as easy, because I ahve to add it into PyCharm. For any given PyCharm project, there’s a “Project Interpreter” option setting. You can choose the specific interpreter to use.
Which means you can choose the Python version to test and program against. When you go into there, there is a small gear icon next to the interpreter selection dropdown. Go into there, choose “More…”, and if you’re lucky, you’ll see the Python version in there. If not, we just have to add it. It’ll be in
/home/$USER/.pyenv/versions/VERSION/bin/pythonX.Y, where you replace
$USER with your username,
VERSION with the version number you installed, and
X.Y with the major and minor version numbers (for example,
3.6.3. Then PyCharm can work with that version.
It’ll be a bit more complex than this, of course, but this is the basics.