Application Developed - eHealth

The first performance tests showed that python-based architectures and PyQtGraph were the most efficient to process the data sent through the serial port by the device. Therefore was developed an application based in RTGraph to process the captured data and plot them in real-time.

The developed application reads the data sent by the device in CVS format and process it. The data format includes the timestamp and two values for the signals. Currently the application can capture the ECG and GSR signals, and process them using the fast Fourier transform and a Low-Pass filter, provided by the library SciPy to remove the noise present. At the same time, the application show a real-time graph with both signals and allows to store the data processed in a CSV file in order to use them for a later analysis.

The application  allows input data in order to tag the datafiles to order them when the session finish. Also can be easily modified to add more signals in order to do a more complex benchmark to the e-Health platform.

This is a video that shows how the ECG signal is corrupted when the subject moves.

The eHealth application's source code  is available for download in the Bitbucket repository.

Data Capture and Real-Time Graphics

To capture data and show the corresponding graph in real time different software tools were used in order to test the performance and usability of each one of them. These tools will be presented in this section.

Matlab

Matlab is a complete mathematical and graphic processing software tool. Matlab have an IDE that uses his own programming language M. It is widely used in scientific and engineering environment. This tool can connect to the Arduino platform using an especially driver developed for this purpose. The main disadvantage is that it is a licensed software and at the same time implements an interface that confuses non-experts users.

Python

Python is a widely used general-purpose, high-level programming language (The RedMonk Programming Language Rankings, 2015) that supports multiple programming paradigm including object-oriented, imperative and functional programming or procedural styles. Its philosophy is maintain a readable code by using indentation and avoiding to use brackets to open and close blocks of code. The main advantages of python are that its interpreters are available for installation in many operating systems like Windows, Mac OS X and Linux; and a large set of standard open source libraries that you can add to improve the functionalities.

NumPy

NumPy is an open source Python library, basically is an extension to the Python programming language that add support for large multi-dimensional arrays and matrices, along with a library of high-level mathematical functions to operate on these arrays.

SciPy

SciPy is an open source Python library used by scientists, analysts, and engineers doing scientific computing and technical computing. SciPy contains modules for optimization, linear algebra, integration, interpolation, FFT, signal and image processing. SciPy adds Matlab-like functionalities to Python and is a fundamental library for scientific computing.

Matplotlib

Matplotlib is a Python 2D library which produce good quality graphs and figures in different formats useful for publications and presentations. Matplotlib use a Matlab-like interface and provide an object-oriented Application Programming Interface (API) for embedding plots into applications using general-purpose Graphic User Interface (GUI) toolkits like wxPython, Qt, or GTK+. Matplotlib tries to make easy things easy and hard things possible. Matplotlib is the Python library most widely used to plot, but unfortunately it does not have a good performance for real-time graphics.

PyQtGraph

It is a library that can be implemented for real-time processing. PyQtGraph creates quality graphics and it was developed for compatibility with Qt4 and, specifically with Qt Designer, simplifying the implementation of the chart. Its documentation is not complete, but now is extensively developed and there a lot of examples are available. This library can be exported to different operating systems.

Qt4, Qt Creator and Qt Designer

Qt4 or Qt version 4, is a cross-platform application framework that is widely used for developing application software. Qt can be run on various software and hardware platforms with little or no change in the underlying codebase. Qt Creator is the IDE for this framework and have an editor that supports syntax highlighting for various languages. Qt can connect with Python code through bindings.

RTGraph

Real Time Graph (RTGraph) a simple and lightweight Python application for plotting data from a serial port (Sepulveda, 2014). Its use all the Qt software mentioned above and was developed focused in real time processing and graph. The project is distributed under MIT license as a part of the Open Source Initiative.

Electrocardiogram (ECG) Sensor

The electrocardiogram sensor can detect the electrical and muscular functions of the heart. This sensor is composed by three connectors (positive, negative and neutral) that need to be attached to the body of the patient while lying on a flatbed.

This is one of the sensors that use the highest sample frequency due the level of accuracy needed to graph the heart activity waveform. At the same time this sensor present a high sensitivity to the movement of the subject, then if the patient moves while the sample is taken, then the values change drastically.

Like the GSR, the connectors of the sensor are attached directly to the skin of the patient but in this case the connector has a small piece of conductive gel that help to get a good conductivity.

Internally the sensor is connected to the analog pin 0 of the Arduino and can measure values between 0 and 1023. The software library does not have a minimal sample frequency because as mentioned before, this sensor needs a high value in order to get a good waveform.

The e-Health Sensor Platform is presented as a very good option for research and study of human emotions but early tests have shown that the platform sometimes is unstable and not very reliable.

The objective of this thesis is to test the platform, modify and develop the necessary software to make it more stable and reliable for future studies related to Affective Computing specially oriented in the field of learning but not restricted only to this area.

The work includes testing the e-Health Sensor Platform with different hardware and software architectures to define which presents the best performance and obtaining 2 or more biomedical signals simultaneously.

The first part of this work is to analyse the hardware. As an intrinsic condition of this work the platform must consider the Arduino board and e-Health Shield. The only hardware that can be changed is the electrical contact sensors and the computers responsible for capturing and processing data and graph.

The second part, is to analyse the software. In this specific point there are many options that was explored both in Arduino software and processing software.

This chapter details how the prototypes were designed and the benchmark tests used to compare them, and finally how the high performance prototype was selected and used for testing purpose and data capture. In order to obtain signals noise free useful to detect emotional changes, was necessary to test different software architectures to communicate the hardware devices, process the noise, graph and store the data.

Prototype Design

The methodology is based in the creation and design of several prototypes and architectures in order to measure and compare the performance of every one.

The first prototype was based in the board Arduino Uno R3 and the e-Health Shield as the hardware components.

The software used was the Arduino IDE version 1.0.6 and the e-Health library version 2.2 because due a compatibility problem the code for the e-Health Sensor Platform cannot be compiled with the last version of Arduino IDE 1.6.1. or if the compilation did  work, some sensors especially the pulsioximeter sensor return wrong values.

Also for the first prototype was used the library ChibiOS/RT, an embedded real time operative system  in order to run concurrently two or more process in the Arduino CPU and try to get different sample frequency for every sensor.

With the first prototype working was analysed the code of the e-Health library for each sensor.

Pulsioximeter Sensor

This pulsioximeter is based in an infra-red sensor adapted to connect to e-Health board. The sensor has a display led (light emitting diode) screen which is connected pin by pin to the Arduino using a wire to the board. The sensor can get two biometric variables: SPO2 that is the level in percentage of the blood oxygen and BPM that is the bit per minute or heart frequency.

The code related to pulsioximeter is in the file eHealth.cpp of the library lines 515-545, in this code is possible to see that the values of the sensor are obtained measuring the level of every led in the display then for the lecture is necessary to read all the leds to get two variables. Also is possible to read in the line 537 that the pulsioximeter has a minimal sample frequency of 300 microseconds because the code apply a 300 microseconds delay between the lectures of every leds. And then apply an addition formula to obtain the values.

Galvanic Skin Response (GSR) Sensor

The GSR sensor get the measuring of the electrical conductance of the skin, which varies with the moisture level. This is of interest because the sweat glands are controlled by the sympathetic nervous system, so moments of strong emotion, change the electrical resistance of the skin. Skin conductance is used as an indication of psychological or physiological arousal, The Galvanic Skin Response Sensor (GSR - Sweating) measures the electrical conductance between 2 points, and is essentially a type of ohmmeter (e-Health Platform documentation).

The conductivity of the skin is measured at fingers of the palm. When a high level of sweating take place the electrical resistance drops down.
The sensor is connected to the Analog A2 pin of Arduino. This pin is the sensor value and convert all the input to an analogue value between 0 and1023.

With this sensor is possible obtain three variables

• SkinConductanceVoltaje: Defined as (sensor value * 5.0)/1,023 with a minimal sample frequency of 2 milliseconds
• SkinConductance: Defined as 2((SkindConductanceVoltaje – 0.5) / 100,000)
• SkinResistance: Defined as 1/conductance with a minimal sample frequency of 4 milliseconds

Methodologies

Due to the nature of the problem the methodology will be based on an exploration of different hardware configurations with benchmarks for each of these configurations or prototypes using one, two or three sensors at the same time.

With each prototype will be analysed different software libraries and programming techniques in order to obtain maximum performance.

Quantitative data will be obtained to compare each prototype and the pros and cons of each will be exposed. These prototypes will be based on the Arduino platform or similar platforms such as Raspberry Pi or Intel Galileo.

The highest performance prototype will be adjust and modified in order to get more than 2 biomedical signals with suitable sampling frequency for use in testing and research. Different types of sampling techniques will be analysed to establish one that be securely, stable and reliable.

Problem Statement

The e-Health Sensor Platform is presented as a very good option for research and study of human emotions but early tests have shown that the platform sometimes is unstable and not very reliable.

The objective of this thesis is to test the platform, modify and develop the necessary software to make it more stable and reliable for future studies related to Affective Computing specially oriented in the field of learning but not restricted only to this area.

The work includes testing the e-Health Sensor Platform with different hardware architectures to define which presents the best performance and obtaining more than 2 or more biomedical signals simultaneously.

Background

The human being is emotional, all our activities are influenced by our personal emotions. When we learn something new, when we talk with our partners, when we make business or when we take decisions our emotions are influencing and probably affect the final result.

For that reason the study of emotions is interesting and his outcomes are applied to several areas like learning, marketing or health (Harley, J. M., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R. 2014).

The study of the emotions using electronic devices has increased last years, there are many papers related to this topic called Affective Computing (Calvo & D’Mello. 2010).

Affective Computing is all related to study the emotions using tools like computers or other electronic devices. One of the problems in this area is the high cost of the sensors used to collect the electrical signals in order to detect emotional changes (Harley, J. M., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R., 2014).

With the rise of open source movement in both hardware and software have been developed interesting projects that lower costs and give access to new technologies and measuring sensors at reasonable prices (Santos Ribeiro A.,Salvado D., Evans G., Soares Augusto J. and Ferreira H. 2014).

Arduino is one of these platforms, basically it is a prototyping board provided with a micro controller and developed with an open architecture. This board was designed to make it possible to add "Shields" and extend its functionality.

A shield very interesting for the study of emotions is the e-Health Sensor Platform. The e-Health was designed by the company Cooking Hacks to allow researchers, developers and artists get measurements and data for biomedical research, testing or fun. This platform allows to get biomedical variables such as temperature, electrocardiograms, galvanic skin response or blood pressure, however this is not medically certified.

The e-Health Sensor Platform

The e-Health Sensor Shield V2.0 allows Arduino and Raspberry Pi users to perform biometric and medical applications where body monitoring is needed by using 10 different sensors: pulse, oxygen in blood (SPO2), airflow (breathing), body temperature, electrocardiogram (ECG), glucometer, galvanic skin response (GSR - sweating), blood pressure (sphygmomanometer), patient position (accelerometer) and muscle/eletromyography sensor (EMG).