User: anonymous   Login
Location: Unspecified
(0.0000°N, 0.0000°E)
Time: 03:25:28
(UTC+00:00)
Language:

Server overloading

Heavens-Above is being overloaded by apps for smart devices (Android and iOS) which take data from this site and reformat it for their own displays. We have tolerated this without restriction until recently, but as the numbers of apps installed grows and grows, our server is unable to cope with the load and we have already had periods when it becomes unresponsive, despite having upgraded our server hardware.

If you take a look at the daily page statistics, you will see that well over 80% of hits are for the pass listing for a single satellite and the Iridium Flares. The home page, by comparison, is going getting about 2% of the hits and that is the page I would expect most "normal" users to start with. The conclusion is that over 80% of the load on the server is caused by apps, and this is also supported by the data in the log files.

The prediction tables need a lot of processing time to produce and have to be regenerated for every user because they depend on the user's location.

What are we doing about it?

As mentioned already, we upgraded our server hardware and now have 24 CPU cores in total which all work in parallel to generate the predictions. Further hardware improvements would be expensive and probably only solve the problem for a short time.

We are appealing to app developers to reprogram their apps to make more efficient use of our server (see below), and some are already being very cooperative on this.

We have introduced a daily quota for each user on the pages which are causing the overloading. This quota is set high enough that a normal user should not be affected, but apps which abuse the server will be blocked once the quota is exceeded. Hopefully this will only be a temporary measure, but if we don't apply quotas, the site will soon become unusable for "normal" users and apps alike.

You can help by asking your app developer to do whatever they can to use the Heavens-Above site economically. Several apps seem to download far more data than they actually need to.

Message to App developers

Please make sure your apps don't abuse the Heavens-Above server. Don't download more information than the user is likely to need and try to spread the loading in time. For example, programming your app to fetch data at a predefined time is very bad, fetching data at a random time of day is much better. There is no need to refresh predictions more than once per day, and even that should be seen as an upper limit. With the exception of the ISS and other manned spacecraft, the satellite's orbits, and hence the predictions, change very infrequently. Most app users will probably not be interested in passes in the early morning before sunrise, so please make this an option that can be selected only if needed.
 

DLR