Fast Load Big table of 20 coloumn and +10000 records

How to load an offline table that contains 20 coloumns and more than ten thousands records … saved as CSV file …

I read the file using file component
Then
For each item from csv table …

And then add items to the 20 lists

The process works great but it may cause the app to freeze on some devices …

My question is …
How to make this without lag of freeze or crash?
Need to do it simply and fast

Thanks to async Procedures app won’t freeze and crash and it may be slightly faster. But with that method you will have to rearrange some blocks to make it works. It’s probably the best way. You can also use clock component to not overload UI thread, so app will not crash as well, it will also require blocks logic change.

App Inventor based frameworks are not too fast.

Could you show me an example how to add items to list using this procedure …
That would be grateful

Just try to call your algorithm inside a procedure, and call it according to the extension guide. It may be tricky as the async procedure runs on the CPU thread different than the main thread. And the NON UI thread can’t change parameters of UI objects. Just try it and see the results. To be honest clock components is easier to implement if u encounter some async procedure errors. Just try and try, it’s the best way to learn and discover new possibilities.

If you use sqlite, you can load 50k records in seconds

1 Like

Would you help me how to load data from local.db file and add this data to lists to be used in all the app not only one screen …
I previously loaded this data from csv table and separeted the process with clock timer to protect the app from crashing actually it takes almost 2 mins to load all the data and the stored in tiny.db after that to be used furtherly …

So how to load the same data using sqlite as you mentioned that it would be much faster

unfortunately the built in sqlite component does not offer an Import method… which means, the sqlite database always starts as empty database and you can’t import a preloaded database from the assets…
https://docs.kodular.io/components/storage/sqlite/
which means, also if you use the sqlite component you have to follow the same cumbersome path to load all the data…

my sqlite extension is able to import a database… App Inventor Extensions: SQlite | Pura Vida Apps
see also Q6 there App Inventor Extensions: SQlite | Pura Vida Apps

Taifun

I will show you my blocks and if your extension would help in replacing this slow method, it would be amazing.

first I load the table using this procedure, this a process of a table out of another 8 tables

then I start to fill the lists with the data from the table every coloumn in a list

and then I store all the lists in the app Tiny.DB to be used laterly in the app in the other screens …

if your extension would help … please help me how to add the values from the table to the lists.

thanks in advance

my extension will not help you in getting the data into the database
this should be done outside of your app…
let me suggest you to use https://sqlitestudio.pl/

as mentioned in Q6 here App Inventor Extensions: SQlite | Pura Vida Apps, you first prepare the database including tables and data outside of your app and then upload the prepared database into the assets of your app
in your app, the only thing which needs to be done is to import the database from the assets

you do not have to work with lists anymore, you work with database queries…
try my example app together with the Chinook example database… Follow the opt-in URL to get access to the app and to be able to download it to your device

Taifun

1 Like

if I prepared the whole data out side the app in database as I do with csv tables and uploaded to assets or imported from storage … I still to add the data from the database to the lists to bypass rebuilding the whole app now …

would it help?

it is the same amount of time to change the logic of your app now to use database queries rather than lists… this also will help you to adjust your app in the future… less blocks and clean logic… if you go the database path then do it completely to have all the advantages…

Taifun

1 Like

Ok I will adjust it in the future but for now, all what I need is to add the values from database to lists … would you help me in that to replace the previous process with database using your extension

You should go with recycler view

it can load more than 10000 items in some milliseconds

my problem is not viewing lists in list view … but extracting the lists from a CSV table and store them in lists to be used furtherly in app screens

1 Like

you must understand, that your data already is in the database
there is no need to copy the data to lists and work from there
work directly from the database… you should do that step now or stay with your current solution…

no, because it does not make sense
I can help you in adjusting your logic to work directly from the database and without lists… however please understand, that I will not be able to do the work for free…

Taifun

There is a easy process using google sheets. Thats how i use it.

Create a insert statement with values/records in googlesheet.

Create the table and insert the records while initialising the app, it takes few secs. This is done as my core data changes every other week.
You can do this as a one time activity.

See this for reference

Hope it helps

Mohan