Adquisición y procesamiento de señales electromiográficas para el control de un vehículo virtual en tiempo real
Acquisition and processing of electromyographic signals for the control of a virtual vehicle in real time
José Luis Carrero-Carrero1 , Luis Enrique Mendoza2 , Zulmary Carolina Nieto-Sánchez3*
1Magíster en Controles Industriales, ingjoseluiscc@gmail.com, Orcid: 0000-0001-9546-9378, Universidad de Pamplona, Pamplona, Colombia.
2Magíster en Ingeniería Biomédica, luis.mendoza@unipamplona.edu.co, Orcid: 0000-0002-2012-9448, Universidad de Pamplona, Pamplona, Colombia.
3Doctor en Educación, zul.nieto@mail.udes.edu.co, Orcid: 0000-0001-6725-4601, Universidad de Santander, Cúcuta, Colombia.
Cómo citar:
J.L. Carrero-Carrero. L.E. Mendoza, Z.C. Nieto-Sánchez “Acquisition and processing of electromyographic signals for the control of a virtual vehicle in real time.”. Respuestas, vol. 25, no. 1, pp. 25-32, 2020.
Received on June 17, 2019; Approved on November 18, 2019
ABSTRACT
This work presents the recording and classification of the electromyographic (EMG) signals of the lower extremities, specifically of the coarse muscle, in order to control a virtual vehicle designed in blender. The system has 4 channels, with a graphic interface, which allows the control of a virtual vehicle. For the processing of the signals, different mathematical tools were used such as: Fourier analysis and wavelet analysis. These techniques were used in order to compress data, obtain characteristic patterns in each set of signals and perform digital filtering. The control of the car consists of 4 commands such as: accelerate, stop, right turn and left turn, which are the basic instructions for the real operation of a car. The results showed that it is possible to use biological signals to perform virtual controls (video game). Likewise, it was verified that the parameterization found for each group of EMG signals was satisfactory, since the percentage of errors of the 4 variables studied was 0.04% for a total of 400 executions. This error percentage corroborates that the system has great potential for possible future applications.
Keywords: Blender, Game Engine, Electromyography, Fast Fourier Transform, Wavelet Transform.
RESUMEN
Este trabajo, se presenta el registro y clasificación de las señales electromiografícas (EMG) de las extremidades inferiores, específicamente del musculo basto, con el fin de controlar un vehículo virtual diseñado en blender. El sistema tiene de 4 canales, con una interfaz gráfica, que permite el control de un vehiculo virtual. Para el procesamiento de las señales, se utilizaron diferentes herramientas matemáticas tales como: análisis de Fourier y análisis wavelet. Estas técnicas se usaron con el objetivo de comprimir datos, obtener patrones característicos en cada conjunto de señales y realizar un filtrado digital. El control del automóvil consta de 4 comandos como: acelerar, detenerse, giro derecha y giro izquierda, las cuales son las instrucciones básicas para el manejo real de un automóvil. Los resultados mostraron que es posible usar señales biológicas para realizar controles virtuales (video juego). Así mismo, se verificó que la parametrizar encontrada de cada grupo de señales EMG, fue satisfactoria, ya que el porcentaje de errores de las 4 variables estudiadas fue del 0.04% para un total de 400 ejecuciones. Este porcentaje de error corrobora que el sistema tiene gran potencialidad para posibles aplicaciones futuras.
Keywords: Blender, Motor de juegos, Electromiografía, Transformada rápida de Fourier, Transformada Wavelet.
Introduction
Electromyography (EMG) or myogram is a medical diagnostic technique that consists of a neurophysiological study of muscle bioelectrical activity [1] - [ 3]. Classically, the same term EMG also encompasses electroneurography (the study of the nerves that transmit the motor order to the muscular system). Although nowadays the word electromyography (EMG) is used more and more in this sense. The technique consists of applying small surface electrodes in the form of suckers to the muscle area to be studied [3] - [9]. These signals are measured using the difference in the behavior of the electrodes [10] - [15].
Electromyographic signals and their applications in the new fields of technology and medicine have been making great advances nowadays, as well as classification techniques. Using these two tools in this work we intend to identify the signals coming from the nervous system and bring them to an application where we can control a device or a simulation. In the techniques analyzed, we worked with the wavelet transform which consists of separating the signal into different frequency components, and then studying each component with a resolution equal to its original scale [16] - [20]. Another technique used to extract classification patterns was the Fourier transform where the behavior of the signal is simply identified based on frequency changes. Once the electromyographic signal was acquired and classified, the application with which we will make decisions based on the type of signal identified was chosen. To do this, a simulation of driving a particular car was made, using the electromyographic signal for the acceleration, braking and direction of the virtual vehicle (right and left). This simulation was implemented in the animation, molding and simulation software in 3D Blender, which allows a wide range of options in the handling of simulations and animations of this type [ref blender]. One of the great contributions of this work is the union of biological data with simulations, at the moment there are very few applications of this type. Therefore, the main objective of this article is to show a novel methodology used to record these signals, their classification and the advances of simulations obtained so far.
EMG signal processing
Acquisition
Electromyography signals were acquired from the lower extremities of the body, 4 different muscles were analyzed assigning each one to an information channel. In the acquisition of the signals, disposable surface electrodes were used, connected to a circuit in a differential configuration, amplifying the small potentials generated by the physiological activity. Our frequencies to be analyzed are in a range of 30Hz to 450Hz, which correspond to the EMG signal. Therefore, in order to analyze only this frequency interval, a band-pass filter was applied, composed of two sixth-order Butterworth analog filters, one 30Hz high-pass and one 450Hz low-pass. The result of the recording, amplification and filtering card developed in the GIBUP group is shown in figure 1. This acquisition and conditioning device is composed of 4 cards assigned to each muscle independently to be analyzed.
Once the signal is recorded, filtered and amplified it is sent to the computer using the NI 6009 data acquisition card from National Instrument for further analysis and classification.
Figure 1. Signal Acquisition and Conditioning Device
Treatment and Classification
The signal was acquired at intervals of 250ms stored in vectors of 2500 positions. Once the signal was recorded, we applied Wavelet Transformation with 4 levels of decomposition as shown in figure 2, in order to compress the signal and discard irrelevant information.
Figure 2. Decomposition levels of the Wavelet Transform.
The Discrete Wavelet Transform (DWT) repeatedly breaks down the S0(n) input signal into two lower resolution sub-signals considered as approximation (A) and detail (D). The Si(n) and Wi(n) signals are the approximation and detail respectively of a signal on level i. The approximation of the signal on level i + 1 can be calculated as:
And the signal detail at the i+1 level can be calculated as:
Eq. 2 and Eq. 3 describe the process of calculating the DWT [2]. We used 4 levels of decomposition and the approximation coefficient at this level was chosen to perform the classification processes. This level was chosen by successive tests with different types of levels. After the signal treatment process, the data classification process was performed. For the classification, a domain was used in the Discrete Fourier Transform (DFT) by dividing the N point signal into two N/2 sequences, corresponding to the odd and even S0(n) samples, respectively:
Equation (3) shows the single decimation of the S0(n) sequence. The N-point DFT can now be expressed in terms of the DFTs of the decimated sequences as follows:
But,
But,
Where F1(k) and F2(k) are the N/2 point DFTs of the sequences of Equation (1) [3]. Finally, taking the range of data from the Discrete Fourier transform where the highest concentrations of energy exist and establishing an adaptive threshold we determine the existence or not of muscle activity.
Application (virtual car control)
The application and emphasis in this work is to control a device or system without involving human movement (control using isometric contraction). In this case, a vehicle was controlled for people who, due to physical disabilities, are unable to do so.
The control of a car is a rather attractive application, but it was decided to opt for a simulator as a first stage and to analyse the results obtained. Figure 3 shows the simulator created. In this case the car is shown in a place where the application is to learn to drive so a place is shown are obstacles.
Figure 3. vehicles in the simulator
It was decided to implement the application in a 3D animation, modeling and simulation software called Blender, version 2.49. This version allows to quickly determine the stability of the control signal.
Results and analysis
The most relevant results of this work are shown below. Initially the physiological signal was acquired and successfully recorded. Figure 4 shows a recorded electromyographic signal, equivalent to 10 seconds of information at a sampling frequency of 40kbps.
Figure 4. Registered electromyographic signal
The wavelet transform is applied to the signal at decay level 4, in addition the second order symlet mother wavelet was used to filter and decrease resolution. After the digital filtering process using wavelet, we applied the Fourier transform to observe its respective frequency spectrum and detect relevant changes between the 4 groups to be classified (acceleration, braking, right and left).
Figure 5 shows the variation between a frequency spectrum with no muscle activity (Figure 5a) and a frequency spectrum with mild activity (Figure 5b). As can be seen, the spectral difference of muscle activity and non-muscle activity is clear. That is, the magnitudes of the frequency spectrum are increased when there is minimal muscular activity in isometric contraction. This means that it is possible to detect movement patterns using DFT.
Figure 5. Muscle EMG signal frequency spectrum (a) No muscular activity (b) Minimum applied stress
Finally, using the frequency spectra, a threshold is established to determine whether or not there is activity. Taking into account the classification made we implement the control commands. Below is an example of the designed graphical interface. Figure 6, shows the interface and thus observe the operation of the system. This interface allows us to easily visualize the existence or not of activity before and after performing the muscle contraction for the 4 channels.
Figure 6. Designed graphic interface.
Then, the designed interface, allows to visualize the acquired signal, as well as its respective spectrum in frequency and the location of the electrodes in the body (yellow and green dot figure 6) located in the lower left part of the interface. In order to evaluate the system's performance and the user's progress, different levels of difficulty were implemented in the handling of vehicles. Figure 7 shows the selection menu that allows to enter them.
Figure 7. Difficulty levels implemented
The simulation includes parking tests, obstacle avoidance and driving control on uncovered roads. These activities will allow to determine the efficiency of the system in normal situations when driving a vehicle.
Figure 8. Initial "Learning to Drive" test
Figure 8 shows the first scenario "learning to drive", in which the vehicle is placed in an open space in order to get to know the system and to control it.
The second scenario consists of an urban street, with its buildings and platforms around, the test consists of avoiding the obstacles and reaching the other end of the street without colliding (figure 9).
Figure 9. Second "Avoiding Obstacles" test
The third scenario consists of an urban street like the previous one, but in this case the test consists of parking the vehicle between two previously established cars, the objective is to park it between the two cars without a collision, since the test will restart any type of contact (figure 8).
Figure 10. Third "Parking" test
The fourth scenario is somewhat more extensive. Four city streets and a surrounding green area have been designed to simulate two different types of environments (Figure 11). The test does not have any particular objective, you just have to drive the vehicle and try to control it in the best way.
Figure 11. Final Test
It is important to mention that the error rate after training for the person who will use the application is 4.13%. This percentage of error depends on the training done to the person.
Conclusions
Computing tools are now being used in a wide range of applications. In this work, a connection between computer systems and physiological systems was made, and it was possible to successfully communicate and link these two excellent areas. The tools that were used the DFT and the wavelet transforms were fundamental for the correct classification of the system. In addition, as shown in the results section the difference between the frequency spectra between muscle activity and muscle inactivity was found.
Blender is a very good 3D simulation tool, although there is little documentation between Blender communication and third party software, in this work it was possible to control Blender with biological data. Finally, we left open the applications for the total implementation and thus to create a tool that allows to solve the problem of driving of vehicles to disabled people.
One of the main disadvantages was to find a way to communicate the processing software with the application software, because there are no direct communication protocols and very little information regarding this issue. This problem was solved by researching and understanding the independent functioning of the data handling of these two programs, looking for a common point for the exchange of information.
Learning to control the system is something that requires time and synchronization, because although executing the commands can be easy, the synchronization needed to control the vehicle varies depending on the tests and the user's concentration, since the system works on the basis of energy it is very easy to carry out tests on different people since it does not require an individual characteristic pattern in its classification system.
Referencias
[1] M. Speigel. “Teoría y Problemas de Análisis de Fourier”. McGraw-Hill serie de compendios Schaum, 1981.
[2] D. Zhang, “Wavelet transform. In Fundamentals of Image Data Mining”, Springer, Cham, e-book, pp. 35-44, 2019.
[3] A. Bhattacharyya, M. Sharma, R. Pachori, P. Sircar and U. Acharya, “A novel approach for automated detection of focal EEG signals using empirical wavelet transform”, Neural Computing and Applications, vol. 29, no. 8, pp. 47-57, 2018.
[4] W. Deng, S. Zhang, H. Zhao and X. Yang, “A novel fault diagnosis method based on integrating empirical wavelet transform and fuzzy entropy for motor bearing”, IEEE Access, vol. 6, pp. 35042-35056, 2018.
[5] A. Bhattacharyya, L. Singh and R. Pachori, “Fourier–Bessel series expansion based empirical wavelet transform for analysis of non-stationary signals”, Digital Signal Processing, vol. 78, pp. 185-196, 2018.
[6] A. Popov, E. Olesh, S. Yakovenko and V. Gritsenko, “A novel method of identifying motor primitives using wavelet decomposition”, 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 122-125, 2018.
[7] J. Jang, “Wavelet-based EMG Sensing Interface for Pattern Recognition”, 2019.
[8] Y. Sawires, E. Huang, A. Gomes, K. Fernandes and D. Wang, “Development of Concussion Evaluation Tools Using Life-Like Virtual Reality Environments”, In International Conference on Human-Computer Interaction, pp. 326-333, 2018.
[9] G. Bernabé, R. Hernández and M. Acacio, “Parallel implementations of the 3D fast wavelet transform on a Raspberry Pi 2 cluster”, The Journal of Supercomputing, vol. 74, no. 4, pp. 1765-1778, 2018.
[10] D. López, H. Correa, M. López and J. Sánchez, “Expert committee classifier for hand motions recognition from EMG signals”, Ingeniare: Revista Chilena de Ingenieria, vol. 26, no. 1, pp. 62-71, 2018.
[11] A. Subasi, E. Yaman, Y. Somaily, H. Alynabawi, F. Alobaidi and S. Altheibani, “Automated EMG Signal Classification for Diagnosis of Neuromuscular Disorders Using DWT and Bagging”, Procedia Computer Science, vol. 140, pp. 230-237, 2018.
[12] J. Ghofrani, et al., “Cross Comparison of Motor Unit Potential Features Used in EMG Signal Decomposition”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no 5, pp. 1017-1025, 2018.
[13] D. López, H. Correa, M. López and J. Sánchez, “Expert committee classifier for hand motions recognition from EMG signals”, Ingeniare: Revista Chilena de Ingenieria, vol. 26, no. 1, pp. 62-71, 2018.
[14] N. Wang, J. Wan and G. Strumolo, “U.S. Patent Application No. 16/087,121”, 2019.
[15] T. Binion, J. Harr, B. Fields, S. Cielocha and S. Balbach, “U.S. Patent Application No. 10/140,417”, 2018.
[16] S. Bullinger, C. Bodensteiner, M. Arens and R. Stiefelhagen, “3d vehicle trajectory reconstruction in monocular video data using environment structure constraints”, In Proceedings of the European Conference on Computer Vision (ECCV), pp. 35-50, 2018.
[17] M. García, J. Vargas and L. Isaza, “Virtual Object of Learning for Driving Through Virtual Reality with Development of Peripherals and Glasses for Virtual Reality”, International Journal of Applied Engineering Research, vol.13, no. 11, pp. 9382-9386, 2018.
[18] F. Stefan, et al., “Method for modeling a motor vehicle sensor in a virtual test environment”, U.S. Patent Application No 16/050,567, 2019.
[19] S. Mallat, "Multifrequency Channal Decompositions of Images and Wavelet Models", IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, no. 12, pp. 2091- 2110, 1989.
[20] J.G. Chacón-Rangel, A.S. Flórez-Fuentes y J.E. Rodríguez-Fernández, “La inteligencia artificial y sus contribuciones a la física médica y la bioingeniería”, Mundo Fesc, vol. 5, no. 9, pp. 60-63, 2015.
licencia de Creative Commons Reconocimiento-NoComercial 4.0 Internacional