Polecart

This is one of a set of benchmark problems presented in "Appendix A: Challenging Control Problems" of Neural Networks for Control, Miller, Sutton & Werbos, Eds., MIT Press, 1990.

"The pole-balancing problem is that problem of learning to balance an upright pole, sometimes called an inverted pendulum. The bottom of the pole is attached by a pivot to a cart that travels along a track as shown in Fig.2. Movement of both cart and pole is constrained to the vertical plane. The state of this system is given by the pole's angle and angular velocity and the cart's horizontal position and velocity. The only available control actions are to exert forces of fixed magnitude on the cart that push it to the left or right."

Fig. 2: The Polecart system (from Neural Networks for Control, Miller, Sutton & Werbos, Eds., MIT Press, 1990, page 486)

At NWCIL, we used the DHP Adaptive Critic method to design a controller for this (and other) of the benchmark problems.

• 'Degree' represents pole's degree angle from a vertical line
• 'X' represents cart's distance from the track center

The Polecart animation was developed to assist the researcher to visualize the pole angle and the cart position that result from application of the current controller being designed.

The following animation demonstrates the action of one of the final controller designs during the training process:

Play the Polecart Animation (flash) by Sam Siciliano