|
|
| Click on the applet and use the arrow keys to nudge the cart. If you uncheck "use network" then you can try to control the cart by yourself. Press Evolve to perform several generations of evolution. |
Overview
This is a classic control problem. You have a "pole" that is attached to a "cart" by a hinged joint. The cart can be pushed around, but must stay within a certain area. The pole must not fall down. This is a linear control problem, it's very much a case of: if the pole is falling over push the cart the other way.
In the applet, the cart is controlled by application of fixed strength forces.
Controller
A simple neural network, consisting of 4 inputs
(cart position, cart velocity, pole angle and pole angular velocity) and one output
neuron is used to control the system.
The four weights of the network are evolved using a genetic algorithm. The GA tests
each network in its population to see how well it controls the cart and pole when there
are forces randomly applied to the cart.
Using a full blown GA is probably overkill. A friend of mine found that succesfull solutions could be obtained merely be trying random weights, and that within the first 10 or 20 tries he would usually find a decent network. This is born out by the fact that the initial network can usually control the system, as it is the best out of the initial population of 20 or so individuals. However this initial network is not very good when you apply pressure to it (prod the cart using the left and right arrow keys).