Retrieve data from csv very slow and time consuming

Here is a version that uses google sheets, all the hard work is done at the server end, so just the time taken to return the results.

GSheetCountriesCities.aia (27.0 KB)

You must ensure that all your sheet data is formatted to "Plain Text" for the gviz query to work.

Countries
image

Cities

TIMAI2 very thank you for time you spent for my question.
I have compared your apps (SQlitebased for local and Googlesheets for web) and probably SQlite is faster.
That said, I think that both apps are great!!!

My only question is whether it's a good idea to use Google Sheets for other web features of my app (no countries/cities) where I have sensitive user data.
I've read the limits of Google Sheets (300 read/write requests per minute, 10 million cells, 50,000 characters per cell).
If I'm within the limits and encrypt both the JSON credentials and the Spreadsheet ID, my questions are:

  • From a data loss point of view, Can I trust Google Sheets?
  • Is there a chance that malicious people could access my Google Sheets and steal sensitive data?
  • Is TinywebDB slower than Google Sheets?

That is a $64000 question. Millions of users do. Depends what you mean by data loss? (see below)

Nothing is unhackable. Data transfer between AI2 and Google Sheets will be encrypted by https.

Don't use tinywebdb for production

I have read that when many users are using Google sheets for retrieve/write data, overwriting or errors in savind data are possible. It could be?

Sorry. I mean "CUSTOM" TinyWebDB. Is your opinion the same also for "custom" TinyWebDB? What are disadvantages respect GoogleSheets?

The same could be said for just about any other online/offline storage solution.

There are industry strength key:value storage solutions, why muck about with something that isn't.

I started with TinyWEBDB because it seemed like a good solution.

  • It's integrated into AppInventor and requires no programming effort (I'm not an expert).
  • There's a lot of documentation on how to use it.
  • There's good documentation explaining how to create a custom TinyWEBDB that runs in Google App Engine.
  • It's free.

If there are industry strength key: value storage solutions that have the same features, I'd love to know what they are.
Could you help me?

Firebase

AI2 Firebase Component (experimental...)

Firebase with the Web Component

Thank you for this. I will try for web needs.

For what regards my first question about local database, I think I will use SQLite, it is very fast. I have slight modified your app (after a fast read of the commands I need) and introduced a filter textbox to filter cities with first letter (see image).
Thank you very much to all!

@TIMAI2 I was reading your page related to SimpleSQLite and I have a couple of questions.

Here you say that we need to set a Different Database as Default.
Why in your app sqliteCountriesCities you have not done it?

How this blocks work? Is it an alternative way to set a Different Database as Default?

Why in your page you use .db extension, and in sqliteCountriesCities .sqlite?

How can I create by me the file cc.sqlite you shared here?

Many tanks TIMAI2!!!

Just saying that you "can" set a different database as Default, you do not have to.

In my example I simply imported the database to the current default database.

The Screen.Initialise event blocks test if the database has been imported (returns true if successful, false if it fails), then if true opens the database for "work"

db or sqlite are two of many different filename extensions used for sqlite databases.

I used a program called DB Browser for SQLite to import your csv files as tables. I use Linux but I believe there is a Windows version as well. Or just use the one I created.

Ok. And in this case, could I have overwrite problems like you well describe in your page?

Ok, I read that it is ok for Windows.
Do you know if I can use "SQLite Studio" to create the database from csv? I read that it is possible, but I don't know if DB Browser is simpler...

You can use any tool, which is able to create a sqlite database

From my webpage App Inventor Extensions: SQlite | Pura Vida Apps

There also is a nice Windows tool called SQLite Studio to create, edit and browse SQLite databases. Thank you Frederic for this tip, thank you Pawel for providing this tool.

There are several Firefox Add-Ons available to manage a SQlite database, for example the SQLite Manager, SQLite Reader or SQLite Manager.

Taifun

Setting a database as default from a different location can help to avoid overwriting the original default database. Given this is probably the only database your are going to be using (countries/cities) I don't think you will have anything to worry about ?

Thank you @Taifun

Ok, I have now understood the reason why you have not setted default DB.
Thank you!

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.