Open Access

Development of a Vision-Based Heuristic Control Framework for Multi-Axis Articulated Robot

1 COEP Technological University (COEP Tech), School of Engineering and Technology, Department of Manufacturing Engineering and Industrial Management, Pune, India
2 Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Robotics, Abu Dhabi, United Arab Emirates
3 Dr. Vishwanath Karad MIT World Peace University (MIT-WPU), Department of Computer Engineering and Technology, Computer Science and Engineering, Pune, India
4 SRH Berlin University of Applied Sciences, Department of Engineering and Sustainable Technology Management, Engineering and Sustainable Technology Management in Industry 4.0: Automation, Robotics & 3D Manufacturing, Berlin, Germany

Abstract

Traditional robotic teleoperation in industrial and hazardous environments often relies on physical joysticks, wearable sensor gloves, or computationally intensive depth-sensing cameras. These conventional methods inherently introduce significant deployment costs, mechanical constraints, and high computational overhead. To address these limitations, this paper presents the development and evaluation of a lightweight, low-cost monocular vision-based teleoperation framework designed specifically for a 4-degree-of-freedom (4-DOF) 3D-printed articulated robotic arm. Utilizing a standard consumer-grade RGB webcam, the proposed system extracts continuous hand skeletal keypoints in real time via the MediaPipe framework. To deliberately minimize processing overhead and eliminate the need for specialized GPU hardware, the algorithm isolates only the thumb tip (Landmark 4) and index fingertip (Landmark 8). A training-free heuristic threshold engine translates the calculated Euclidean distance between these two spatial coordinates into discrete servo commands, including Grip, Toggle, and Idle. Crucially, a software-level dynamic deadzone filter is integrated to explicitly reject sub-pixel landmark noise, successfully suppressing 85% of unwanted mechanical micro-motions during stationary poses. Hardware actuation is seamlessly executed via asynchronous PyFirmata serial communication to an Arduino Uno microcontroller. Rigorous experimental evaluation across 100,000 continuous inference cycles demonstrated a highly responsive mean end-to-end processing latency of 44.2 ms, comfortably satisfying the 100 ms upper bound for seamless Human-Robot Interaction (HRI). Furthermore, the system achieved a robust overall gesture classification F1-score of 95.2% across 5,472 controlled test cycles. Ultimately, this study validates that an optimized deterministic heuristic approach provides a highly efficient, cost-effective alternative to deep learning classifiers for real-time robotic teleoperation.

Keywords

How to Cite

PAWAR , A. S., PATIL , S. M., SHAHANE , G. J., NIBE , N. V., PATIL , S. S., & MAHAJAN , A. M. (2026). Development of a Vision-Based Heuristic Control Framework for Multi-Axis Articulated Robot. MAS Journal of Applied Sciences, 11(2), 252–267. https://doi.org/10.5281/zenodo.20343082

References

📄 Alamin, M.O., Khadir, E.M., Babiker, S.F., 2016. A vision-based teleoperation method for a robotic arm with 4 degrees of freedom. Conference of Basic Sciences and Engineering Studies (SGCAC), Conference Proceedings Book, 20-23 Februaty, Khartoum, Sudan, pp. 19-23.
📄 Amprimo, G., Masi, G., Pettiti, G., Olmo, G., Priano, L., Ferraris, C., 2024. Hand tracking for clinical applications: Validation of the Google MediaPipe Hand (GMH) and the depth-enhanced GMH-D frameworks. Biomedical Signal Processing and Control, 96(A): 106508.
📄 Chenchireddy, K., Dora, R., Mulla, G.B., Jegathesan, V., Sydu, S.A., 2024. Development of robotic arm control using arduino controller. IAES International Journal of Robotics and Automation, 13(3): 264–271.
📄 Cui, C., Sunar, M.S., Su, G.E. 2025. Deep vision-based real-time hand gesture recognition: A review. PeerJ Computer Science, 11: p.e2921.
📄 da Silva, M.V., Carvalho, M.H., Negri, J., Segreto Silva, T.H., Lahr, G.J.G., Godoy, R.V., Beckeret, M., 2025. A vision-based shared-control teleoperation scheme for controlling the robotic arm of a four-legged robot. Latin American Robotics Symposium (LARS), Symposium Proceedings Book, October 29-November 1 Monterrey, Nuevo León, Mexico: IEEE, pp. 1-6.