For my Unified Robotics III: Manipulation (RBE 3001) final project, my team and I integrated closed-loop inverse kinematics, trajectory generation, and computer vision to build a fully automated sorting system using the 4-DOF OpenManipulator-X. The objective was to accurately locate, pick up, and sort scattered plastic balls based on their color.

The foundation of the project relied heavily on motion planning and kinematics. We derived the Forward Kinematics using Denavit-Hartenberg (DH) parameters and computed the Jacobian matrix to map joint velocities to task-space velocities. To translate camera coordinates into joint-space commands, we implemented a numerical Inverse Kinematics solver using the Newton-Raphson method, explicitly monitoring the determinant of the Jacobian to avoid singular configurations. To ensure the physical arm moved smoothly between pick and place locations without jerking the servos, we utilized cubic and quintic polynomial trajectory generation to calculate precise, continuous waypoints.

This kinematic framework was guided by a custom MATLAB computer vision pipeline. We performed color segmentation in the HSV color space to identify specific targets, applying masking and median filters to remove visual noise before extracting the centroid data.

Because the camera perception and robot workspace were not perfectly aligned, we calculated the task-space distortion and applied a linear regression correction to reduce targeting error. By coupling this dynamic vision system with our trajectory generator, we engineered a state machine that could react to its environment. The robot continuously scanned the workspace, meaning if a ball moved, the system automatically recalculated the inverse kinematics and generated a new path to complete the sorting task.