Difficulties parsing with "list from csv row"

[ninja2]

(apologies if this is in wrong forum section, happy to be redirected if need be, as long as I don;t have to enter it all again!)

I have an Arduino (a teensy 4.1) communicating over bluetooth classic with my AI2 app. The teensy is fitted with up to five DS18B20 thermometers and runs a sketch that sends a compound CSV text string containing the temperatures over bluetooth, every 2 seconds.

Here are the sketch code pieces for transmitting the string:

#include <Streaming.h>
#define bt Serial7                    // Bluetooth on TX7/RX7 

bt << "#" << _FLOAT(DT0_degrees,2) << ","
		  << _FLOAT(DT1_degrees,2) << ","    
		  << _FLOAT(DT2_degrees,2) << ","    
		  << _FLOAT(DT3_degrees,2) << ","    
		  << _FLOAT(DT4_degrees,2) << '\n';   

My AI2 mobile App is reading the text string but I'm having trouble parsing the string using "list from csv row".

Using the "Serial Bluetooth Terminal" on my mobile shows the CSV strings are being sent correctly

A reading of -99.00 means that thermometer is not connected yet. Each string is preceded by a hash character (#).

The longest string (no thermometers connected) would be #,-99.00,-99.00,-99.00,-99.00,-99.00
for a total length of 1 + (5 x 7) + 1 = 37 characters including the \n line feed.

The shortest string (all thermometers active) would be
#,00.00,00.00,00.00,00.00,00.00
for a length of 1 + (5 x 6) + 1 = 32 characters.

In testing I noticed that reception of the whole string by bluetoothClient is inconsistent. Sometimes it receives a few bytes, sometimes 1 byte, but most times it receives the whole string correctly. To handle this I added the extra test to redirect any dud strings into dummyString (also ensures the Serial buffer is emptied)

If BytesAvailableToReceive > 30
load into textString (& lbl_String)
else
load into dummyString

But when I run the App using AI Companion I get this runtime error:

If build and load the App onto my mobile I get this similar error:

Here are my parsing blocks:

1. How can I avoid these bugs ?

In addition my attempts to parse and display the individual temperatures on the App screen is not working as shown in this screenshot:

Note the display does show lbl_byteCount, lbl_String values (yellow arrangement) showing current textString and byte count to help in debugging.

2. How can I get the parsing to work so values appear on screen?

In case they are important, here are the blocks for whole App:

A couple of other minor Q's if I may:

3. Is it possible to strip the \n character of end of textString before displaying in lbl_String (so it doesn't wap on the App screen)?

4. why does my tick counter trigger every 10ms even though the setting in TimerInterval for Clock1 is set at 500ms?

Many TIA

[ABG]

Be sure to use println() at the end of each message to send from the sending device, to signal end of message.

Only use print() in the middle of a message.

Be sure not to println() in the middle of a message, or you will break it into two short messages and mess up the item count after you split the message in AI2.

Do not rely on timing for this, which is unreliable.

In the AI2 Designer, set the Delimiter attribute of the BlueTooth Client component to 10 to recognize the End of Line character.

Also, return data is not immediately available after sending a request,
you have to start a Clock Timer repeating and watch for its arrival in the Clock Timer event. The repeat rate of the Clock Timer should be faster than the transmission rate in the sending device, to not flood the AI2 buffers.

In your Clock Timer, you should check

  Is the BlueTooth Client still Connected?
  Is Bytes Available > 0?
     IF Bytes Available > 0 THEN
       set message var  to BT.ReceiveText(-1) 

This takes advantage of a special case in the ReceiveText block:

ReceiveText(numberOfBytes)
Receive text from the connected Bluetooth device. If numberOfBytes is less than 0, read until a delimiter byte value is received.

If you are sending multiple data values per message separated by | or comma, have your message split into a local or global variable for inspection before trying to select list items from it. Test if (length of list(split list result) >= expected list length) before doing any select list item operations, to avoid taking a long walk on a short pier. This bulletproofing is necessary in case your sending device sneaks in some commentary messages with the data values.

Some people send temperature and humidity in separate messages with distinctive prefixes like "t:" (for temperature) and "h:" (for humidity).
(That's YAML format.)

The AI2 Charts component can recognize these and graph them. See Bluetooth Client Polling Rate - #12 by ABG

To receive YAML format messages, test if the incoming message contains ':' . If true, split it at ':' into a list variable, and find the prefix in item 1 and the value in item 2.

...

[ninja2]

thanks ABG, I must remember to set that DelimiterByte to 10 !! That plus use of -1 length fixed my transmission reliability. Re your other points: my code already addressed them all in my original program.

My only remaining issue is the parsing using "list from csv row". whcih causes the runtime errors shown above, and the temperature readings never appear in the labels lbl_DT0, lbl_DT1 ... etc

The CSV string is only loaded into the global list_Celsius when the string length is > 30 and < 38. So I can't see what might be causing the runtime error "Attempt to get item number 1 from a list of length 0"?

[ABG]

It would be easier to correct your blocks if you posted an .aia export.

[ninja2]

attached as requested ...

(just noticed I left some blocks disabled, I should have enabled them before attaching.)

Pathrider2_F.aia (75.5 KB)

[ABG]

I can't get to a PC for hours so here's the short answer.

Wrap your tests on the global list with a test for length of list> 6 (7?)

You are testing an empty list.

[ABG]

Your data stream is missing a comma between the leading '#' and the first reading.

It's more expedient to just strip it on arrival and not let it mess up item 1.

[ABG]

Here's a draggable replacement Clock Timer that should work:

Given the use of line feeds (\n) as delimiters, you should be okay, unless you fear BlueTooth message length limits.

In that case, you should switch to a YAML message format with datum by datum parsing:
C0:23.23
C1:23.25
C2:23.25
C4:23.25
C5:-99.00

Pathrider2_F (1).aia (75.4 KB)

[ninja2]

good spot !! After adding list length test and the missing comma It is finally working:

Stil some work to do but this is very important progress. Many thanks for your great help (again)

Here are my blocks now. I did this before seeing your next post, but I will try your suggestions shortly.

[ninja2]

I played around with your blocks, and modified the arduino sketch so the list length is always 32 chars long, including the line feed.

For now I'm still sending the introductory hash # in the first CSV slot:
"#,00.00,11.11,22.22,33.33,44.44\n"
so I reverted to "list from csv row" and adjusted the index numbers and it's working very well and reliably.
If a thermometer is missing or not detected a negative value ( > -10 ) will be transmitted:
For example: "#,11.11,22.22,33.33,-8.00,-9.00\n"

Soon I will expand my sketch + app to send + process other data over bluetooth serial. My intention is to uses a unique intro character for each type of data, similar to what you are suggesting. I am not familair with YAML yet, but I will definitely look into it.

A few more questions if I may:

1. I seems that "list from csv row" and "split textString at comma" do the same thing. Do you think they are identical in effect?

2. With and bluetooth connected the app will reliably report the temperatures every 2 secs as each compound text string is received. And the on-screen byteCount value is a steady 32, as expected. However if I push the bluetooth toggle button in my app to disconnect, wait for say 30 seconds, and re-connect, there will be a backlog in the bluetooth serial buffer . Currently the app procesees these extra strings quite quickly and catches up. For exampe the byte counter display might show counts like:
   160 .. 128 ... 96 ... 64 ... 32
   as the buffer is cleared.
   I would like to simply discard / flush all data on the serial bluetooth buffer whenever Bluetooth is re-connected. Is this possible?

3. I you look at the blocks in screen1.initialise and btn_Bluetooth.click you sill notice the repeated blocks . To avoid the repeated code how to make this a single procedure, called by both screen1.initialize and btn_Bluetooth.click. If so, how?
   The new procedure would do this part:

	if 
		BluetoothClient1 is connected 
	then 
		set bool true
		set text "connected"
		set textcolor blue
	else 
		set bool false
		set text "not connected"
		set textcolor red

4. I created a new global variable (bytesAvailable) to store the BytesAvailableToReceive value. But it's only used within the "when-Clock1.timer-do" blocks so it doesn't need to be global. How would I do tghis as a local variable?

5. I was impressed that able to drag those "draggable" blocks directly from this forum window to the AI2 blocks in another window, very easy!
   How did you make that attachment of blocks "draggable"?

thanks again

Pathrider2_H.aia (75.6 KB)

[ABG]

@ninja2

I seems that "list from csv row" and "split textString at comma" do the same thing. Do you think they are identical in effect?

Try both against text that has blanks but no quotes to enclose the text. CSV is more fussy.

[ABG]

@ninja2

I would like to simply discard / flush all data on the serial bluetooth buffer whenever Bluetooth is re-connected. Is this possible?

Run your AI2 Clock Timer faster?

[ABG]

@ninja2

To avoid the repeated code how to make this a single procedure, called by both screen1.initialize and btn_Bluetooth.click. If so, how?

See Chapter 21 of
http://www.appinventor.org/book2

[ABG]

@ninja2

I created a new global variable (bytesAvailable) to store the BytesAvailableToReceive value. But it's only used within the "when-Clock1.timer-do" blocks so it doesn't need to be global. How would I do this as a local variable?

This value is used only once in the Timer, so no need to save it.

I use a global variable to freeze a copy for Do It access.

[ABG]

@ninja2

How did you make that attachment of blocks "draggable"?

Use the right Click option to Download png against the procedure or event block enclosing the code you want, and give it a nice name in your downloads directory for the app.
These make nice small incremental backup, reusable across projects.

[ninja2]

@ABG

I use a global variable to freeze a copy for Do It access.

Can you explain this a little more, I've not come across "Do It" ?

[TIMAI2]

@ninja2

Can you explain this a little more, I've not come across "Do It"

When testing your project with companion, you can right click on a block and select Do It, which will return the current value if it contains one.

For local variables, you would need to add, then set, a label, to the local variable to capture a value.

[ABG]

@ninja2

I've not come across "Do It" ?

FAQ Section: Debugging

Introductory Testing techniques, including Do It - MIT Sample Connect -> Companion sequence - @ABG How to Read and Understand a Runtime Error Message - @ABG Common Runtime Errors - @TimAI2 Advanced Do It' to view procedure results, event arguments and local variables How to run Logcat How to use Do It in a loop Screen1.ErrorOccured message numbers SteveJG monster list of things you should not do How to show raw responseContent from your app when debugging - TimAI2 Video of copying Do…

[ninja2]

Thanks. I must look into those debugging tools, a bit further down the track.

I must say I'm very impressed with the AI2 tool (and the community). I had looked into programming an app as a developer would (e.g. Android Studio) but that was way too much to learn. AI2 is just right !

Those "draggable blocks" have me intrigued. It seems they are only .png files but somehow when they are dragged into another AI2 instance it can recreate the underlying program.

[ninja2]

My App and sketch are working quite well now, thanks to the great help in this thread.

It's definitely not perfect but if anyone's interested I've attached the AIA + App screenshot.

I also tried to upload a ZIP of the sketch but seems that's not available? It's not a simple sketch, comprising one .ino that #includes five additional .cpp/.h pairs. So for now I can only upload the .ino

It runs on a Teensy 4.1 fiited with an HC05 BT module, five DS18B20 thermometers, DS3231 RTC, SSD1306 OLED display and a piezo buzzer.

Pathrider2_k2.aia (75.9 KB)

PATHRIDER.ino (8.0 KB)