CS164 - Gym setup

The OpenAI Gym is a toolkit for developing and testing machine learning algorithms, where the intelligent agent learns to play games and complete tasks.

These tips and instructions are intended for Linux or Mac systems. If you have a Windows system, you should install VirtualBox and then an Ubuntu GNU/Linux guest OS. The instructions on that linked page are for an older version of VirtualBox and Ubuntu, so you should substitute the latest stable versions instead. At the time of writing, they are:

The simplest way to get started with the Gym seems to be with the GitHub version, and PIP (the python package manager). Let’s see if you have python. Open a command line, try running these programs, and stop when one works:

python3
python2
python

Make a note of which version you have – it will tell you as soon as it opens up:

Python 3.5.2 (default, Jun 25 2016, 21:38:40)
[GCC 5.4.0] on linux

At the >>> prompt, you can press control-D to exit Python.

If none of the python commands work, you should download and install python.

Next we want to make sure you have PIP, the python package manager. Try these commands until one works (the option is a capital V):

pip -V
pip3 -V
pip2 -V

It will display its version as well as the version of python it corresponds to:

pip 8.1.1 from /blahblah/python3.5/site-packages (python 3.5)

If you don’t have pip, you can install it using your system’s package manager:

On Ubuntu GNU/Linux, that means sudo apt-get install python3-pip or use python-pip if you want to stick to Python 2.
On Mac, the recommendation is to use Homebrew – it’s a package manager that gives you access to all kinds of open-source software that you’ll need. Once you have homebrew, use brew install python3.

Okay, at this point that you have a Mac or Linux system with some version of Python and corresponding version of PIP. You’ll also need git, if you haven’t set that up yet.

Change to a directory for your course stuff, and let’s grab the software for the OpenAI gym:

cd Desktop     # (for example)
git clone https://github.com/openai/gym.git
cd gym

The README file in the source distribution has some further tips for getting things set up and working. This one command (run from the gym directory) should get you pretty far:

pip install -e .

Don’t forget the lonely dot out there at the end of the line. Also, substitute pip3 or pip2 if that’s the version of PIP you have working.

If everything goes smoothly, you should be able to open your Python interpreter and enter from gym import make with no error messages.

% python2
Python 2.7.13 (default, Dec 17 2016, 20:05:07) 
[GCC 5.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gym import make
>>>

(On my system, I got it working best with python2, but I’m still working on python3.)

Now you can use any text editor (I have some recommendations on the Software setup page) to create a Python script, and save it with the .py extension. Copy/paste this code:

# cartdemo.py
import gym
import time

env = gym.make('CartPole-v0')
env.reset()
for _ in range(1000):
    env.render()
    time.sleep(0.1)
    env.step(env.action_space.sample()) # take a random action

Now, if you’re in the directory with that code saved as cartdemo.py, you can run it like this:

python cartdemo.py

and it should pop up a window with the cart-pole game. Your program will make random moves, and so won’t do a very good job keeping the pole upright.

If you run into trouble, post the details to the issues forum.

PDF