Application of novel machine learning techniques and high speed 3D vision algorithms for real time detection of fruit

University of Lincoln: T. Duckett (lead), G. Cielniak, P. From


Novel digital technologies including vision systems, robotics and autonomous systems are seen as potential game changers for the horticulture sector. Visions systems can be used to assess and sense the crop to enable better decision support; robotics and autonomous systems offer new means to drive productivity. These issues apply to all soft and top fruits, but also more widely across the whole fresh produce sector. However, all picking and vision systems are dependent on the development of complex algorithms developed to identify, measure and locate fruit in real time. The development of these systems is not trivial, especially in outdoor environments where the back ground light level and quality can change within an instant.


The main objective of this project is to deploy novel machine learning technologies to detect, locate and measure (size and colour) fruit in real time. This work fundamentally underpins the development of all crop-picking robots.

The student will use and develop advanced machine learning algorithms to measure, identify and detect fruit in real time and in 3D. The algorithms will be trainable (so that a range of fruit types can be identified) and provide a world x,y,z co-ordinate of the fruit. Researchers from the Lincoln University (Kusmann, Duckett, Cielniak and Pearson, 2017, J. Field Robotics in press) have developed similar systems for broccoli. This earlier work showed that 3D cameras could be deployed in field environments however the algorithms were highly complex with relatively slow processing speed. The new challenge for this PhD project will be to minimise processing requirements to identify fruit whilst maximising processing speed and recognition fidelity. This project will initially focus on strawberry and be anticipated to include apple.