DEVELOPMENT OF A SOFTWARE PLATFORM FOR CONTROLLING A WIFI-ENABLED MANIPULATOR USING A CAMERA

Abstract

The purpose of this work is to create a convenient and effective program interface for controlling the manipulator, establishing communication with the camera to provide visual feedback, as well as developing a modular system that can accommodate various plugins. Methodology. The article discusses the key features of the software platform and its architecture, focusing on the interaction between the manipulator, camera, and various plugins. The manipulator, in its main function as an actuator, is equipped with clamps for work and a chamber above them (a kinematic scheme is used, a problem of reverse kinematics). The smartphone acts as a server (connected via Wi-Fi), sends commands to the manipulator. The program on the server has a modular structure: at any given time it uses only one plugin, which receives data from the manipulator through the interface, analyzes it and, depending on the functionality of the plugin, provides the appropriate commands to the manipulator. The program is developed in the Kotlin programming language using the Jetpack Compose graphics framework. It allows you to import the program to various platforms such as Linux, Mac OS, and Windows. Currently, the app only works on the Android platform. Scientific novelty. The importance of using input from a video camera in robot control, allowing tasks such as object recognition, tracking and improving decision-making, technical aspects of the application, including data exchange protocols for the manipulator and camera, plugin architecture, and user interface. Features of the modular software platform for controlling a manipulator with a video stream. Modularity, flexibility and real-time video input contribute to its practicality and relevance in the field of robotics. The practical significance of the Modular Manipulator Control Software Platform with Video Stream Implementation on Android is designed to simplify the control of robotic manipulators through a single software interface and real-time video integration to improve control and decision-making This work lies in its potential application in various industries such as manufacturing, healthcare, and automation. The modular nature of the platform simplifies integration of specific functions, making it suitable for a wide range of tasks in the field of robotics. The relevance of the platform lies in the growing demand for flexible robotic systems that can be adapted to different applications. By providing modularity and extensibility, the platform allows developers to create and integrate specific plugins for different tasks without having to reprogram the entire control system. Conclusions. The main results of the research indicate that the modular software platform successfully provides a single interface for manipulator control and real-time video stream integration. The platform’s flexibility allows users to adapt the system to different tasks, making it a valuable tool for researchers and developers in the field of robotics.