Pole balancing is an control benchmark historically used in engineering. It involves a pole affixed to a cart via a joint which allows movement along a single axis. The cart is able to move along a track of fixed length.
A trial typically begins
with the pole off-center by a certain number of degrees. The goal is to keep
the pole from falling over by moving the cart in either direction, without falling
off either edge of the track. A more difficult extension of this problem involves
two poles, both affixed at the same point on the cart.
Typically the controller, in this case a neural network, receives as input information about the system at each time step, such as the positions of the poles, their respective velocities, the position and velocity of the cart, etc. An even more difficult extension of this problem involves not providing the velocities explicitly as input into the neural network. Also, because there are no constraints in the physical simulation on things such as energy used or friction between the cart and the track, solutions may be found which simply involve rapid wiggling. In order to avoid such unrealistic solutions from emerging, one method is to penalize such oscillations in the fitness function.
These 3 variations: Double Pole Balancing, DP with no velocities, and DPNV with anti-wiggling are all implemented in ANJI, along with graphical visualization. Below are links to movies showing a solution to the most difficult problem, DPNV with anti-wiggling:
Pole Balancing Movie (Flash format)
Double Pole Balancing Movie (avi format)
Quick Start Instructions
Follow the steps below to perform an evolutionary run to solve Double Pole Balancing With Velocities.
1) Open a command shell (from the Start menu select Run..., type "cmd", select OK).
2) Navigate to the anji_2_00/ directory.
3) Type "evolve.bat dpbalance.properties" and press Enter.
The format of the
output is determined by the log4j configuration in xor.properties.
ANJI will print the ID mappings for topological features, the species count, the number of chromosomes that have solved the problem, and information about the fittest individual for that generation.
Upon solving the
problem, something like the following should display for that final generation.
Chromosome 4740 has solved the double pole balancing problem. That is, the neural network controller has learned to keep both poles from falling for the 10,000 time steps of the trial.
|To view a Java Swing
display of the solution balancing the poles, enter the following:
Type "dpbalance_evaluate.bat dpbalance.properties <champ-id>" and press Enter.
The following Swing module will display:
The lower portion
of the screen shows the track as a black line, the cart as a violet rectangle,
and the poles as dark blue and light blue lines. The shorter pole is somewhat
difficult to see, as they are affixed at the same point. The thick gray
bar in the middle of the screen is a progress bar, showing time elasped
in the trial. And the top bar controls the speed at which the movie plays.
You can click and drag the clasp to the right or left to either speed
up or slow down the speed of the movie.
|This example demonstrated a run in the simpler domain of double pole balancing with velocities input into the network. For the more difficult variations, larger population sizes (e.g. 1000 or more) and slightly higher mutation rates should be used, and trials should take much longer to carry out.|