Browsing by Author "Porter, Wesley"

Now showing 1 - 2 of 2

Deep learning based Real-time GPU-accelerated tracking and counting of cotton bolls under field conditions using a moving camera
(2018 ASABE Annual International Meeting, 2018-08) Fue, Kadeghe G.; Porter, Wesley; Rains, Glen
Robotic harvesting involves navigation and environmental perception as first operations before harvesting of the bolls can commence. Navigation is the distance required for a harvester’s arm to reach the cotton boll while perception is the position of the boll relative to surrounding environment. These two operations give a 3D position of the cotton boll for picking and can only be achieved by detection and tracking of the cotton bolls in real-time. It means detection, tracking and counting of cotton bolls using a moving camera allows the robotic machine to harvest easily. GPU-accelerated deep neural networks were used to train the convolution networks for detection of cotton bolls. It was achieved by using pretrained tiny yolo weights and DarkFlow, a framework which translates YOLOv2 darknet neural networks to TensorFlow. A method to connect tracklets using vectors that are predicted using Lucas-Kanade algorithm and optimized using robust L-estimators and homography transformation is proposed. The system was tested in defoliated cotton plants during the spring of 2018. Using three video treatments, the counting performance accuracy was around 93% with standard deviation 6%. The system average processing speed was 21 fps in desktop computer and 3.9 fps in embedded system. Detection of the system achieved an accuracy and sensitivity of 93% while precision was 99.9% and F1 score was 1. The Tukey’s test showed that the system accuracy and sensitivity was the same when the plants were rearranged. This performance is crucial for real-time robot decisions that also measure yield while harvesting.
Visual control of cotton-picking Rover and manipulator using a ROS-independent finite state machine
(2019 ASABE Annual International Meeting, 2019-07) Fue, Kadeghe; Barnes, Edward; Porter, Wesley; Rains, Glen
Small rovers are being developed to pick cotton as bolls open. The concept is to have several of these rovers move between rows of cotton, and when bolls are detected, use a manipulator to pick the bolls. To accomplish this goal, each cotton-picking robot needs to accomplish three movements; rover must move forward/backward, left/right and the manipulator must be able to move to harvest the detected cotton bolls. Control of these actions can have several states and transitions. Transitions from one state to another can be complex but using ROS-independent finite state machine (SMACH), adaptive and optimal control can be achieved. SMACH provides task level capability to deploy multiple tasks to the rover and manipulator. In this research, a cotton-picking robot using a stereo camera to locate end-effector and cotton bolls is developed. The robot harvests the bolls using a 2D manipulator that moves linearly horizontally and vertically. The boll 3-D position is determined by calculating stereo camera parameters, and the decision of the finite state machine guides the manipulator and the rover to the destination. PID control is deployed to control rover movement to the boll. We demonstrate preliminary results in a direct-sun simulated environment. The system achieved a picking performance of 17.3 seconds per boll. Also, it covered the task by navigating at a speed of 0.87 cm per second collecting 0.06 bolls per second. In each mission, the system was able to detect all the bolls but one.