Retrieve data from csv very slow and time consuming

Many many thanks!!
I will try to setup a local SQlite and I will came back soon to give you a feedback.
I hope to solve in this way...

Just to see if it would work, I tried a simple inline filter to get cities of a selected country:


Surprisingly, this got a stack overflow (8MB).

Even just the filter part got a stack overflow:

Apparently, there's little headroom for piping at runtime.

Maybe if I compile it?

Even compiled, I got a stack overflow.
(Note I had to use the file name helper blocks for the CSV files. The Companion was more forgiving on case mismatches between file names.)

So the elegant approach fails.
Next attempt would be a trudge through the input csv table, after conversion. to load a dictionary of country cities.

Interesting. Which of the AIAs were you using to test this? This should result in tail call recursion being applied to prevent a stack overflow, so I'd like to investigate further.

I didn't preserve the failed stack overflow version, but jumped into a dictionary based version, yet to test.

I'll whip that up for you.


TEST_csv_piped.aia (779.7 KB)

I stripped the blocks down to the bare essentials.

Load the two files into global tables, one after the other.
Load the Country name picker
After picking a country, load the city picker from the city-country table.
That's where it gets the 8mb java stack overflow error.

The dictionary based version loads and filters okay, but might be speeded up for later runs by caching to a TinyDB Namespace.

I'll try that next.
TEST_csv_dictionaries.aia (780.3 KB)

And here's the ultimate extension-free TinyDB retentive version, with loading on the first run only:
TEST_csv_TinyDB_Namespace.aia (780.6 KB)

P.S. You might want to switch to List Pickers instead of Spinners.
They have optional search boxes.

I am surprised by how many cities are in even the tiniest of countries, and at how soon I lose patience with scrolling long lists.

With SQLite, returns are immediate:

sqliteCountriesCities.aia (1.8 MB)

1 Like

I forgot to ask, will this have to also run on iOS?

Dear all, very thank you for time spended to solve my problem....I appreciate!
TIMAI2 the speed of your last .aia app is impressive! I think I will use the SQ lite as in your example.
Is that a local SQlite?

Also this will help you😊

ABG thank you for your app "ultimate extension-free TinyDB retentive version".
I have tested it and in first run the time to wait is about 90s
I will probably use the solution of TIMAI2 with SQlite.

For what concern the IOS...I know that apps that use extensions can not be exported to be used in IOS...is it true?
My app uses a lot of extensions: have you a solution?

thank you!

Dear TIMAI2, cities list for each country is usually long. For this reason I used a textbox in witch user have to enter first letter of city to search, and I introduced a "compare text" to filter (se attached screenshot in first message). Will same blocks work in your SQlite blocks sequence?

In my app I have functions that need a shared database, so I used TinyWEBDB.
Is TinyWEBDB (as TinyDB) slower than SQlite?

If you suggest to substitute TinyWEBDB with SQlite, how to create the web version of SQlite? Is it the same of local one?
And how to upload it in Google App Engine server?

No, your path is clear.

Sorry ABG, I have not well understood your reply.. sorry. Could you better explain what you mean please?

As suggested, use list pickers instead of spinners, these offer a search/filter bar, alternately you can use a text box and some blocks to filter.

Not sure why you would use a tinywebdb for anything in relation to this.

@ABG means you are ok to carry on if you are not targeting iOS.

The reason I would use tinyweb is that I have an impressive Tinyweb block in my app..
substitution with SQlite = a great work

That said, what about this?

Any solution for IOS compatibility for apps with extensions?

It is currently not possible to use extensions if targeting ios.

but probably easier to use mySQL or even Google Sheets.

Is in your opinion Google sheet faster than tinyweb db?
Using gsheet as database is a possibility...

Firebase is faster...