Slowdown in Retrieving tags from firebase in Real-time

Hey everyone! I'm working on retrieving multiple tags under rapidly generating timestamp buckets from the real-time database as shown below,
image_2023-05-16_213412998
I was successful in acquiring and reproducing the data on my mit application, but the rate at which the data is changing in the application is very slow that it hardly ever changes at all.
The images below are from my block editor
image_2023-05-16_213111902
image_2023-05-16_213141363
image
I would be grateful if someone can help me out!

How many sets of readings...?

The time stamps are being issued continuously so there are a lot of datasets. I need the results in real time as each timestamp is produced. from the above block diagram I can acquire the latest timestamp, but since its being produced continuously, the process of moving on to the next data has become very slow.

If your readings list is growing by 1 per 12 milliseconds, and you retrieve the entire list each time, that would bog you down.

If all you want to display is the last reading, maybe add an extra tag off the userID LASTREADING and keep refreshing it in addition to logging the separate readings?

Just have your app request userID/LASTREADING every so often.

1 Like

Can you please send me a block sample?. So I can understand more clearly, since I'm still a beginner in transferring data from firebase.

Before doing the blocks, can you get this into your DB?
9970b390f6c464b5e3b13bfdf9299f4e2d205c6e

You can possibly do this on firebase using indexing. If you index readings and return the first (last) in list

See here

https://firebase.google.com/docs/database/security/indexing-data

https://firebase.google.com/docs/database/rest/retrieve-data#section-complex-queries

3c6d5406485ed99d8de562d7058bdd68584ab480

What arrives in the tag and value of the data changed event?

All readings, or
just the latest reading?

I can't since its a continuously changing value. And whenever I close the app and open it again the firebase starts recording the sensor data at the timestamp right at that moment in real-time

latest timestamp with the set of readings

So that's only a couple of dozen bytes, right?

Is the tag 1684067825, and the value
{"bodytemp":79,"gas":1680,...} ?

That's easy to decode with the Web JSONDecodeWithDictionaries block into a dictionary, with the parts you want.

(But I don't see "bpm" or "spo" in your data tree?)

image
I changed the variables overtime as I was testing, also a new data set is being recieved with each new timestamp

In the Data Changed Event?

By a new dataset you mean which?

  • All the data so far, or
  • the latest timestamp and its fields?

If the second, can you display the value for us in a Label or capture it in a global variable for a Do It?

the latest timestamp and its fields

A setup like this should return the last reading/timestamp/data

in your firebase rules:
image

in your blocks, use a web component to query the data:
image

1 Like

Here is how to show the newly arrived data for a single timestamp, assuming the tag is the timestamp and the value is a dictionary with your keys and values:


This dictionary ties text keys to Label components, simplifying extraction later.



firebase_data_changed_timestamped.aia (2.7 KB)
P.S. The firebase component I used was set to the default MIT server, so you would have to adapt it to your server.

@ABG

how do you know the timestamp value to call back the tag?

According to the OP, it arrives all by itself in the Data Changed event, without having to ask for it, timestamp by timestamp, in real time.

I am taking it on faith that the pump has been primed.

I believe all the data arrives from "readings", which is causing the slow down.

See my solution , getting firebase to do all the work....

I conducted a little experiment on the MIT Firebase server to see if timestamped entries could be caught one by one in a Data Changed event.

It worked for me, using BlueStacks multi-emulators ...
Animation

firebase_data_changed_timestamped.aia (4.8 KB)

Note: My List Picker Elements lag behind my Tag Lists because in the interest of speed I only refresh my List Picker Elements when I use the List Pickers, not when new data arrives.