Pole balancing is an control benchmark historically used in engineering. It involves a pole affixed to a cart via a joint which allows movement along a single axis. The cart is able to move along a track of fixed length.

A trial typically begins with the pole off-center by a certain number of degrees. The goal is to keep the pole from falling over by moving the cart in either direction, without falling off either edge of the track. A more difficult extension of this problem involves two poles, both affixed at the same point on the cart.

Typically the controller, in this case a neural network, receives as input information about the system at each time step, such as the positions of the poles, their respective velocities, the position and velocity of the cart, etc. An even more difficult extension of this problem involves not providing the velocities explicitly as input into the neural network. Also, because there are no constraints in the physical simulation on things such as energy used or friction between the cart and the track, solutions may be found which simply involve rapid wiggling. In order to avoid such unrealistic solutions from emerging, one method is to penalize such oscillations in the fitness function.

These 3 variations: Double Pole Balancing, DP with no velocities, and DPNV with anti-wiggling are all implemented in ANJI, along with graphical visualization. Below are links to movies showing a solution to the most difficult problem, DPNV with anti-wiggling:

Double Pole Balancing Movie (Flash format)
Double Pole Balancing Movie (avi format)

Quick Start Instructions

Follow the steps below to perform an evolutionary run to solve Double Pole Balancing With Velocities.

Step
Procedure
1
Windows:
1) Open a command shell (from the Start menu select Run..., type "cmd", select OK).
2) Navigate to the anji_2_00/ directory.
3) Type "evolve.bat dpbalance.properties" and press Enter.

Unix:
1) Open a command shell.
2) Navigate to the anji_2_00/ directory.
3) Type "evolve.sh dpbalance.properties" and press Enter.

2

The format of the output is determined by the log4j configuration in xor.properties.

http://logging.apache.org/log4j/docs/manual.html

After ANJI displays the properties for the run, you should see several lines like the following scroll across your screen.

 INFO Run: start INFO Generation 0: start INFO connection->neuron id map size == 6 INFO neurons->connection id map size == 75 INFO species count: 7 INFO # chromosomes with max fitness: 0 INFO champ: id=43 score=0.0058 size=15 INFO Generation 0: end [10:04:09 - 10:04:11] [1265]

ANJI will print the ID mappings for topological features, the species count, the number of chromosomes that have solved the problem, and information about the fittest individual for that generation.

3

Upon solving the problem, something like the following should display for that final generation.

 INFO Generation 24: start INFO connection->neuron id map size == 86 INFO neurons->connection id map size == 767 INFO species count: 57 INFO max: id=4740 score=1.0 size=17 INFO # chromosomes with max fitness: 1 INFO champ: id=4740 score=1.0 size=17 INFO Generation 24: end [10:05:06 - 10:05:11] [5297] INFO generation of first solution == 24 INFO champ # connections == 8 INFO champ # hidden nodes == 1 INFO Run: end [10:04:09 - 10:05:12] [62250]

Chromosome 4740 has solved the double pole balancing problem. That is, the neural network controller has learned to keep both poles from falling for the 10,000 time steps of the trial.

4
To view a Java Swing display of the solution balancing the poles, enter the following:

Windows:
Type "dpbalance_evaluate.bat dpbalance.properties <champ-id>" and press Enter.

Unix:
Type "dpbalance_evaluate.sh dpbalance.properties <champ-id>" and press Enter.

The following Swing module will display:

The lower portion of the screen shows the track as a black line, the cart as a violet rectangle, and the poles as dark blue and light blue lines. The shorter pole is somewhat difficult to see, as they are affixed at the same point. The thick gray bar in the middle of the screen is a progress bar, showing time elasped in the trial. And the top bar controls the speed at which the movie plays. You can click and drag the clasp to the right or left to either speed up or slow down the speed of the movie.

5
This example demonstrated a run in the simpler domain of double pole balancing with velocities input into the network. For the more difficult variations, larger population sizes (e.g. 1000 or more) and slightly higher mutation rates should be used, and trials should take much longer to carry out.