Mobile Software Tool for Counting Small Hardware Parts Using Computer Vision and Machine Learning

Small hardware parts (e.g. screws, bolts, nails) have different shapes, sizes, and materials. Sometimes, the individual packages (that the parts come in with) are different as well. These present challenges for humans to count large quantities of small hardware parts manually. If the parts are laid out on a uniform non-reflective surface, they can be counted using computer vision algorithms. These algorithms can distinguish, identify and count the objects (laying on a surface at random orientations) based on existing machine learning training datasets and real-time images collected from the mobile device’s camera. In this STTR Phase I proposal, we propose to develop a mobile software application that uses the machine learning model and real-time image analysis for object recognition and identification. This application can take pictures of the various laid-out hardware parts and count them at high speed and accuracy. The accuracy of the software critically depends on the quality and quantity of the initial repository of training images which allows us to analyze and experiment with multiple machine learning and computer vision algorithms for optimum results. Once the counting mechanism is deployed, the software can also be swiftly adapted to count new types of similar hardware parts. Lightweight Machine Learning models can be trained to count those hardware objects which are rejected by computer vision algorithms.