Part of an App i'm developing downloads a series of 65 .png files from a website. These images change on a regular basis, so the other parts of my code deal with that - but essentially the app builds a list of the current .png files and then sequentially downloads them. This is by current code:
However, I'm not sure if it's the website blocking all of the requests or the code is simply running too fast to allow the ".get" to grab each file. Not all files are downloading.
I want to try and slow the .get process down somewhat. Perhaps start at 1 ".get" per second and then speed it up from there. This may improve the outcome and take load off the server where i'm grabbing the files from - it's the right thing to do.
I can't resolve how to achieve this. I don't want to use blocking code because I want the user to be able to start looking at the images as they arrive and let the ".get" thread to keep running as they do so.
Could anyone provide some suggestions that might lead me in the right direction?
PS The .png files are in the order of 20-100kb, so not massive.
Correct, the next download will not start until the previous download finishes. That is how the web component works, asynchronously - do something (GET) and when GET completed (gotText or gotFile) then you are good to go again.
Because you cannot rely on internet speed (and there is not much you can do about that) and file sizes remaining constant, this is the best method.
Using many web components and clock timers would not be practical. You could add some code/blocks (app end or server end) to test if the file has changed at all, then to decide whether it needs to be downloaded.
You could introduce a clock timer if you wanted to slow things down, and put a gap between each download, to give the user time to view the images, and vary that time interval as you go along (speed up / slow down), but never faster than each file will take to download...
Agree. I've already implemented a strategy for checking if files have changed web side and also to ensure I don't download a file I already have app side.
It's more about setting a minimum time between "gets" when it is determined a number of files need downloading.
I feel this is required. It may be okay if I was the only person using the app. But if the app is deployed to thousands of users (wishful thinking), it may have a detrimental impact on the webside server. Not good for anyone in that case.
Yes, it probably will, how bad or noticeable that would be I am not sure.
or number of connections ? if this reaches any set thresholds (e.g. 10,20 50) then a signal is sent to set the timer accordingly. (You may need another web component for this, to capture a gotText response from the server). This could be combined with the number of files that need to be downloaded (perhaps a small spike is acceptable?)
Of course, better server with more cpu, cores and ram may also be an answer. Scale up as required when number of users increases.
I did some speed testing a while ago. With wifi, I was routinely downloading a 1.2mb file in 0.4 seconds, from google drive and my own VPS server online. Now it is not exact, averaging files of 50kb, this should only take 3-4 seconds to fetch 60 files (maybe more due to app time)
@TIMAI2, I've implemented a similar code to your original suggestion and it worked well saving the images from web to my ASD. I don't feel the load speed is excessive or too slow. I did try implementing a simple delay function using a back-to-the-future catch-up delay timer, but loading the 65 images seemed to take forever and in the end it wasn't worthwhile.
However, the next issue came with displaying and scrolling through images on screen with my simple navigation buttons. Because I allow the scrolling through images, while they are still being loaded, the app ground to a halt as i tried to naviate while it was downloading.
Probably not a problem with the downloading part, but moreso the way the set.image.picure function works in the background. Thankfully, I found Colintree's Asynchronous Image Loader - AsyncImageLoader. https://aix.colintree.cn/en/extensions/AsyncImageLoader.html That solved the problem. Now I can watch my progress bar of images downloading while I scroll through images.
Now I need to finish the app, impress the owner of the website and images and convince them to make their website more reliable in terms of making the images consistently available!
