Ball-on-Plate Control

Academic implementation project comparing model-based and model-free control on a physical Ball-on-Plate system.

I implemented two controllers for moving a ball between targets and through obstacle layouts: a model-based Dynamic Programming planner and a model-free Soft Actor-Critic reinforcement-learning policy. Both controllers include live visualizations so their decisions can be inspected while the hardware is running.

Dynamic Programming

The Dynamic Programming controller uses a discrete MDP as a waypoint planner above a PID control layer.

• The plate is represented as a $30 \times 30$ grid.
• Actions select relative waypoint shifts.
• Candidate actions are evaluated with the nonlinear ball dynamics and closed-loop PID simulation.
• Transition rollouts are precomputed and cached, while value iteration runs in a background thread.

The projected heatmap shows the value function and selected waypoints while the ball navigates the obstacle layout.

Reinforcement Learning

The Reinforcement Learning controller uses Soft Actor-Critic and directly outputs plate-angle commands after training in simulation.

• The observation contains ball position, velocity, target position, and plate angles.
• The two actions are scaled to physical tilt setpoints between $-10^\circ$ and $10^\circ$.
• The reward combines target collection, distance shaping, step costs, and stuck penalties.
• Training uses eight parallel environments with periodic evaluation and checkpointing.

The live overlay visualizes neural-network activity together with the current ball and target positions.

Result

Both approaches were deployed on the same physical setup. The project produced a direct comparison between model-based planning and learned control, together with reusable tools for simulation, deployment, reward tuning, and real-time diagnostics.

Hardware photo from the Ball-on-Plate project.