https://www.mdu.se/

 Modern robotics operate under unstable dynamic conditions, making them rely on continuous real- time state data for stabilizing, with the data traditionally coming from encoders. With the advances in computer vision, and Convolutional Neural Network (CNN) models showing their strong ability to extract feature information from images, a use case was set up involving a Furuta pendulum. The goal of the use case was to investigate if angle predictions from a CNN model can either act as a complement to encoder data, or completely replace the need for encoders in a real-world setting. This would be done by predicting the motor arm and the pendulum angles in real time by passing image pairs through a CNN model. An experiment rig using a dual camera setup was built, and a dataset was gathered. Early fusion was used to merge the features from both images and the traditional classification head of the CNN models was replaced with a custom regression head. Three different CNN families were trained and evaluated: ResNet, EfficientNet and MobileNet. All models used ImageNet pre-trained weights as a starting point, and then fine-tuned on the collected dataset. All models underwent quantization as a way to boost their inference speed at the cost of accuracy. Inference was done on a Raspberry Pi 5 with a neural network booster board. The ResNet-10 model showed the strongest performance overall, compared to the other CNN models. The model predicted a mean absolute error in degrees of 1.39◦ for the motor arm and 0.95◦ for the pendulum, while maintaining an inference time of 1.69 ms. These results show that CNN models can predict angles with good accuracy, and that the inference time when predicting these angles will not be a bottleneck in a real-world setting.