Today I am not writing about software testing, but about a programming project. I was interested in how machine learning works and was looking for an application example. I decided on a sorting machine with which blocks such as Lego can be sorted, called the Brick Sorter.
The project was very interesting and varied. Not only did I learn about machine learning, but I also had to design something and program it in Python.
You can watch a video of the Brick Sorter.
Hardware
- Raspberry PI 4 with camera
- 2 DC motor, 2 stepper motor, 1 servo motor
- Fischer Technik, wood and an aluminium bar for construction
Software
- Python with Flask, Keras
- You can find the project here: Link to Github
Solutions
Put the bricks in a sequence
The first problem I had to solve was to get the bricks in a sequence. This is important so that each brick can be photographed individually. I solved this via a conveyor belt and a vibrating plate. At the beginning of the conveyor belt is a small funnel in which the bricks can be laid. Only one sled is mounted on the conveyor belt, so that the bricks are not transported too quickly to the vibrating plate. I built the conveyor belt out of Fischertechnik parts.
From the conveyor belt, the bricks fall onto a vibrating plate (aluminium), which has a slight slope. To create the vibration, I attached an unbalance to a DC motor and this DC motor is attached to the plate. The bricks must also pass through a kind of curtain on the vibrating plate in order to separate the bricks even further. But most of it is achieved by the vibration.
At the end of the plate there is a gate installed, which opens for a short moment and then closes again. Then the bricks fall on a plate where the photos are taken. The gate is operated by a servo motor and the control of the individual motors is taken over by a Raspberry Pi.
Create photos in sufficient quality
In my view, the most difficult task in the construction project was to take photos of the bricks in good quality. This is very important for the good detection of the brick type. I had tried various things, but often had the problem that some things were in the background, which then interfered with the detection. In the current solution, I stretched a grey sheet of paper over the plate and adjusted the slope so that the bricks slip into the center so that they can be photographed well. It was also important that the plate is always well and evenly illuminated. The photos are taken with a Raspberry PI camera.
Determine brick type
When a brick arrives on the plate, the conveyor belt and the vibrating plate are stopped so that no more bricks fall on the photo plate. The photo is sent to a server which then uses machine learning to calculate the probability of which type of brick it is. The result is then sent back to the Raspberry Pi.
Sort brick into the right compartment
When the Raspberry Pi has received the result, which type of brick it is, the hub is aligned accordingly. Afterwards, the photo plate is slightly tilted so that the brick can slide from the photo plate over the hub into the compartment. The photo plate is then driven back to its original position. The hub and the photo plate are driven by a stepper motor.
Learning a model
I also used the construction to take photos of the different types of bricks. It was important to have enough photos of each type of brick. To achieve this, I also rotated each photo by 90, 180, 270 degrees. As a model, I used a ready-made Convolutional Neural Network (CNN), the VGG16. The top-layer was replaced by an own layer. The learned model was saved and then used for detection.
Next steps
Currently I have trained the model on seven different brick types, from 1×1 to 2×4. To further optimize sorting, more images of other brick types would have to be created, and then the model would have to be retrained.
Another point would be to further improve the detection of the bricks, so that the error rate is even lower. This would also require the model to be recreated.
My Brick Sorter summary
This project was a lot of fun for me. In addition to getting to know machine learning, I also had to deal more with the programming language Python. Another nice aspect was that I could construct physical things, such as conveyor belts or other things that were controlled with the Raspberry PI.