The vision was a hastily assembled and implemented visual control system running on a Raspberry Pi 3 as the core part of my class' senior design project.

 

Highlights

Requirements

This project needed to be:

1. Quickly implemented through rapid prototyping and interation
2. Reasonable confidence in choosing targets
3. Low cost because I was paying for parts out of pocket

Implementation

Logic

The first step to solving a control systems problem is to iterate through a flowchart until one comes up with the simplest design which would satisfy the necessary requirements. The accompanying flowchart for the system is based on a few essential assumptions, namely that the tennis balls it was chasing were:

• Similarly sized
• Some variation of green
• The robot and the ball are on the same plane

Vision

The vision system is powered by the well-respected open-source computer vision library, OpenCV version 3. I compiled OpenCV using Adrian Rosebrock's tutorial on setting up OpenCV 3.2 on a Raspberry Pi 3.

After testing out the build using a basic tutorial, I pored over the developers guide on the OpenCV site to draw necessary target boxes for the python code to attempt to get its chosen target into. As you can see from the image, as the ball (lime as I had no tennis balls at home) is to the left or right of the center, the vision system feeds a correction value back into the control system, creating a closed loop. There is a deadzone in the center to help minimize control system oscillation.

Control

The control system is built on a multi-threaded python 3 script to help use all of the Pi3's cores. As one thread determines where the ball of interest is in the camera's frame of view & shares the resulting correction value, the other thread is using the correction to guide the solving of a linear matrix which dictated orientation for the robot's mechanum wheels. The wheels were controlled by motors connected to the 5x H-bridge motor controllers via the Pi's GPIO pins.

Testing

In testing the robot, I didn't account for object avoidance or any other essentials for a real-world system as the groups had run short of time despite my best efforts. Nonetheless, the robot was able to pick out the closest ball, pick it up and shoot it via a kinetic launcher. While I could have used more help in implementing the entire control system by myself, I still consider the entire project a success in terms of the essentials for a complex control system.