Non-deterministic readings from BLE

[ioachim]

My project consists of the following: an esp32c3 attached to a glove that transmits data every 200ms via BLE sensor data in the following format: {"values": [296.37, 1139.18, 917.47, 2041, 2698, 2407, 2680, 2135]}. I have attached a picture of the relevant blocks. The app then sends the data to an API in JSON format, and gets back a prediction for the sign.

The issue is that the data transmission is very non deterministic. Sometimes the glove sends out “default value” (which is how we have initalized the variable). After sending that once, it sends the correct data. Sometimes it does not send the data at all, just “default value”. Sometimes it sends the correct data from the beginning. Sometimes it sends the correct data but then after a number of queries it sends the same values. Keep in mind, nothing has been changed code wise. The app and glove were just restarted. What could it be? We have tried several things: upping the MTU to 100 on both the app and the esp32c3, using a # at the end and then sending the data only when it has that # to ensure package integrity, changing the code around. Using nrfconnect the data transmits truncated, but it does transmit. Upping the MTU sends the full data with no problems. Sending a POST request to the API from another computer with the data works.

[ABG]

You need to show the glove code too

Your ble strings received event should do more than drop the incoming data into a global variable. Have it append the incoming data to a logging label with \n line separators

Why doesn't that event also push the incoming data to the Web service?

[ioachim]

Here is the code, its quite long, sorry about that

#include <Wire.h>
#include <BLEDevice.h>
#include <BLEServer.h>
#include <BLEUtils.h>
#include <BLE2902.h>
#include <esp_adc_cal.h>

#define MXC_ADDR 0x15
#define I2C_SDA 6
#define I2C_SCL 5
#define REG_OUT_X_MSB 0x03
#define REG_OUT_X_LSB 0x04
#define REG_OUT_Y_MSB 0x05
#define REG_OUT_Y_LSB 0x06
#define REG_OUT_Z_MSB 0x07
#define REG_OUT_Z_LSB 0x08
#define REG_CTRL1 0x1B
#define SENSITIVITY 1024.0
#define N_SAMPLES 20
#define calibrationSamplingTime 3000 //milliseconds
#define timeBetweenLearningSamples 100
#define iterationsLearning 500

//BLE server name
#define bleServerName "Translating_Glove_v1"
// Default UUID for Environmental Sensing Service
// https://www.bluetooth.com/specifications/assigned-numbers/
#define SERVICE_UUID "177a85bc-c232-49bd-8aea-4a67348b6cea"

using namespace std;

// Temperature Characteristic and Descriptor (default UUID)
// Check the default UUIDs here: https://www.bluetooth.com/specifications/assigned-numbers/
BLECharacteristic dataCharacteristic("ffbbe6f9-922c-42ba-a06f-b97df12efd72", BLECharacteristic::PROPERTY_INDICATE);
BLEDescriptor dataDescriptor(BLEUUID((uint16_t)0x2902));

bool deviceConnected = false;
float calibratedStartX;
float calibratedStartY;
float calibratedStartZ;
int orientationX=1;
int orientationY=1;
int orientationZ=1;
String semn;
String answer;
float x0=0,x1=0,x2=0,x3=0,x4=0,xx=0,xy=0,xz=0;
float P0=1,P1=1,P2=1,P3=1,P4=1,Px=1,Py=1,Pz=1;
float Q0=0.1,Q1=0.1,Q2=0.1,Q3=0.1,Q4=0.1,Qx=0.1,Qy=0.1,Qz=0.1;
float R0=1,R1=1,R2=1,R3=1,R4=1,Rx=1,Ry=1,Rz=1;
float K0,K1,K2,K3,K4,Kx,Ky,Kz;

float kalman0(float measurement) {
// Prediction update
P0 = P0 + Q0;

// Measurement update
K0 = P0 / (P0 + R0);
x0 = x0 + K0 \* (measurement - x0);
P0 = (1 - K0) \* P0;

return x0;
}
float kalman1(float measurement) {
// Prediction update
P1 = P1 + Q1;

// Measurement update
K1 = P1 / (P1 + R1);
x1 = x1 + K1 \* (measurement - x1);
P1 = (1 - K1) \* P1;

return x1;
}
float kalman2(float measurement) {
// Prediction update
P2 = P2 + Q2;

// Measurement update
K2 = P2 / (P2 + R2);
x2 = x2 + K2 \* (measurement - x2);
P2 = (1 - K2) \* P2;

return x2;
}
float kalman3(float measurement) {
// Prediction update
P3 = P3 + Q3;

// Measurement update
K3 = P3 / (P3 + R3);
x3 = x3 + K3 \* (measurement - x3);
P3 = (1 - K3) \* P3;

return x3;
}
float kalman4(float measurement) {
// Prediction update
P4 = P4 + Q4;

// Measurement update
K4 = P4 / (P4 + R4);
x4 = x4 + K4 \* (measurement - x4);
P4 = (1 - K4) \* P4;

return x4;
}
float kalmanX(float measurement) {
// Prediction update
Px = Px + Qx;

// Measurement update
Kx = Px / (Px + Rx);
xx = xx + Kx \* (measurement - xx);
Px = (1 - Kx) \* Px;

return xx;
}
float kalmanY(float measurement) {
// Prediction update
Py = Py + Qy;

// Measurement update
Ky = Py / (Py + Ry);
xy = xy + Ky \* (measurement - xy);
Py = (1 - Ky) \* Py;

return xy;
}
float kalmanZ(float measurement) {
// Prediction update
Pz = Pz + Qz;

// Measurement update
Kz = Pz / (Pz + Rz);
xz = xz + Kz \* (measurement - xz);
Pz = (1 - Kz) \* Pz;

return xz;
}
//Setup callbacks onConnect and onDisconnect
class MyServerCallbacks: public BLEServerCallbacks {
void onConnect(BLEServer\* pServer) {
deviceConnected = true;
Serial.println("Device Connected");
};
void onDisconnect(BLEServer\* pServer) {
deviceConnected = false;
Serial.println("Device Disconnected");
}
};

int16_t readAxis(uint8_t regMSB, uint8_t regLSB) {
Wire.beginTransmission(MXC_ADDR);
Wire.write(regMSB);
Wire.endTransmission(false);  // restart for read

Wire.requestFrom(MXC_ADDR, 2);
if (Wire.available() < 2) return 0;

uint8_t msb = Wire.read();
uint8_t lsb = Wire.read();

int16_t raw = (msb << 8) | lsb;
raw >>= 4;               // right-align 12-bit
if (raw & 0x800) raw |= 0xF000;  // sign-extend

return raw;
}

float toResistance(int reading){
float VCC=3.3;
float R_DIV=100000;
int flexADC = reading;
float flexV = flexADC \* VCC / 4095.0;
float flexR = R_DIV \* (VCC / flexV - 1);
return flexR;
}

void setup() {

Serial.begin(921600);

BLEDevice::init(bleServerName);
BLEServer \*pServer = BLEDevice::createServer();
pServer->setCallbacks(new MyServerCallbacks());
BLEService \*gloveService = pServer->createService(SERVICE_UUID);
BLEDevice::setMTU(100);
gloveService->addCharacteristic(&dataCharacteristic);
dataCharacteristic.addDescriptor(&dataDescriptor);

gloveService->start();

pServer->getAdvertising()->start();

while(!deviceConnected){}

delay(1000);

Wire.begin(I2C_SDA, I2C_SCL);

Serial.println("Starting calibration");
Serial.println("Put you hand on the table and sit still for three seconds");
long long sumx;
long long sumy;
long long sumz;
Serial.println("Calibrating in three");
delay(1000);
Serial.println("Calibrating in two ");
delay(1000);
Serial.println("Calibrating in one");
delay(1000);
Serial.println("Calibrating...");
int secondsLeft=1;

for(int i=2;i<calibrationSamplingTime+2;i++){
calibratedStartX=(calibratedStartX\*(i-1)+readAxis(REG_OUT_X_MSB, REG_OUT_X_LSB))/i;
calibratedStartY=(calibratedStartY\*(i-1)+readAxis(REG_OUT_Y_MSB, REG_OUT_Y_LSB))/i;
calibratedStartZ=(calibratedStartZ\*(i-1)+readAxis(REG_OUT_Z_MSB, REG_OUT_Z_LSB))/i;
if(i%1000==0){
Serial.print(secondsLeft);
Serial.println(" seconds left");
secondsLeft-=1;
}
delay(1);
}
}

void loop() {
String ax = String(kalmanX(readAxis(REG_OUT_X_MSB, REG_OUT_X_LSB)-calibratedStartX));
String ay = String(kalmanY(readAxis(REG_OUT_Y_MSB, REG_OUT_Y_LSB)-calibratedStartY));
String az = String(kalmanZ(readAxis(REG_OUT_Z_MSB, REG_OUT_Z_LSB)-calibratedStartZ));
String degetAratator= String(kalman0(toResistance(analogReadMilliVolts(0))));
String degetMijlociu= String(kalman1(toResistance(analogReadMilliVolts(1))));
String degetInelar= String(kalman2(toResistance(analogReadMilliVolts(2))));
String degetMic= String(kalman3(toResistance(analogReadMilliVolts(3))));
String degetMare= String(kalman4(toResistance(analogReadMilliVolts(4))));
String data="{"values": \["+ ax+","+ay+","+az+","+degetAratator+","+degetMijlociu+","+degetInelar+","+degetMic+","+degetMare+"\]}#";
//String data="T#";
Serial.print(data);
Serial.println();


  dataCharacteristic.setValue(data);    
  dataCharacteristic.indicate(); 

  delay(timeBetweenLearningSamples); 


}

I have tried that and it didn't work, it was the same behaviour

[Anke]

First of all, permissions must be requested one after the other.

[uskiara]

Dear @ioachim in addition what @ABG and @Anke have already said, it isn’t clear to me whether it is the transmission from the ESP or the receiving in your app, which is erratic.

To (possibly) find a solution my hint is to split the issue in two. First of all, load on your phone (i.e. the Android device) the ready made SerialBluetootTerminal app (SBT), free on Playstore, connect your ESP in the BLE menu of SBT and see what happens. Once you are done with that, in other words: only after you are 100% sure that the ESP32 is sending complete, non erratic or truncated frames, you can the focus on your app. Without this step you have two variables (the app and the ESP) that can lead you crazy before finding the root cause. To add more confidence in this step, simplify the ESP code and send just a constant string (i.e. simulate a complete dataset but do not compose it in runtime). Please believe me: though it seems a long way, at the end it will be faster than searching in the darkness…

Cheers

[ioachim]

The esp32 transmits the data correctly from what I have tested. I have tried using SBT but it doesnt detect the correct uuids. However, on nrfConnect it works and shows the correct values. So I dont believe this is an esp32 issue. (I have also tried the nrfConnect on diffrent phones, same result.)

I have also made a label to check whether or not it is the web POST request at fault or not. The data was also incorrect in the label.

[davidefa]

So how do you receive the ble data in your app?

The following block receives your data, but does not guarantee to elaborate your data.

I mean in the same block you should call a “ProcessData”

[ioachim]

The data is formatted on the esp32 because it felt easier to put it in json there. Here is how it is formatted: {"values": [296.37, 1139.18, 917.47, 2041, 2698, 2407, 2680, 2135]} (the values obviously vary).

[uskiara]

@ioachim

Using nrfconnect the data transmits truncated, but it does transmit.

Upon this statement of your first post I’ve understood that also with NRFC the data was transmitted truncated (therefore a problem on ESP side). But in your last post now is: “…on nrfConnect it works and shows the correct values…” what has changed ?

Then, honestly it’s the first time that I read that SBT isn’t capable to get data from a BLE, whatever are he UUID’s (but never say never…).

Anyway, may suggestion remains the same: start with a (super) simplified code, on both ESP and app sides, with the aim at obtaining a proper and trobust BLE communication.
In other words: just a constant trasmission from ESP and display into a label the received string. Nothing else. Then modify only the ESP code to transmit the variable string, continuing to simply display into a label. At the end apply what @davidefa has said: “elaborate the received string”.

BTW, please also be aware that the delay() function in some Arduino, and probably also in some ESP, implementations, completely stops the CPU, therefore killing also any transmission. It is often better to use an own MyDelay(timeout) function based on the use of millis(). Something like:

MyDelay(unsigned long timeout)

{ unsigned long now = millis();

while ((millis() - now) < timeout); // do nothing but leaves the CPU alive

}