Autoupdater

[Yaguchi]

Hello everybody,

i used the Updater today and very shocked about the method which is used to download and install. A more slower method i cant image at moment. ( The Why )  

Some ideas how to do faster.

1. The updater Generate a list of needed files.
You could generate with this filelist a compressed file (like zip,rar). And download this file extract and install the files then from local Harddisk.

2. Pack updates for each version ...
for example :
psup0.3.009->0.3.011.zip

I personaly prefer the first method after generate a compressed file you can save this for some hours and if others need you dont need to regenerate . save cpu time at server. and space if longer time no update did for example autodelete for unneeded compressed files.

just idea from me.

Sincelery

Denis Theinert

[Induane]

Actually it basically works like method one, except that its smarter it: and correct me if I\'m wrong devs, :  But  many things are stored inside zip files, so the updater downloads the file needed, then puts it into the zip file where it belongs, rather than downoading a whole new zip file.  The updater is capible of updating files within zips as a result.  This is quite nice.  Also it does genereate a list of needed files and download them, but it can\'t have the server make a zip file for each person updater needs every time someone clicks update.

The updater has some problems currently .   one is speed, but this isn\'t really do to the download/install method, its due to other issues, like CS and such.  

If you want to run it at full speed, do this:

Windows: Click on the start menu and select run.  Then type command or cmd (windows 2k or newer).  Then type

cd\\
   and then
cd program files
  and then
cd planeshift crystal blue
  and then type
updater --auto

or optionally edit the icon to have the --auto option on the command.

[stfrn]

Extra compression won\'t help the server right now. Nothing we can do about it\'s networking issues but wait.

As I said in another updater post, the server doesn\'t make the zips, it holds them for people to get.

[Rage McCloud]

Yes as said it is already compressed... compressing more would do nothing... and number two is how it does it... it is a bunch or zips inside one big updater zip and it carckes the updater one open then cracks the rest open one by one... and the reason it is slow is because of lag... the server(s) has been lagging a lot recently... thats why people see monsters that are killed come back and kill them... people dissappearing... yadda yadda... it is pretty annoying but hey... what can you do cept ride it out?

/me waits for Drey to come and insult everything that He just posted about...

[DaveG]

Originally posted by Induane
If you want to run it at full speed, do this:

Windows: Click on the start menu and select run.  Then type command or cmd (windows 2k or newer).  Then type

cd\\
   and then
cd program files
  and then
cd planeshift crystal blue
  and then type
updater --auto

or optionally edit the icon to have the --auto option on the command.

OR:  Just click the \"Updater (recover mode)\" link in your start menu\'s PlaneShift folder...  

[jhessin]

Bittorrent is an excellent way to set up an updater! If you had every player use an ounce of his bandwidth you could maximize the download speed. Also when you have multiple computers on the same network with the game you can download the updates once and then share it with the rest of the computers at maximum speed. (this is the case with my network, I tried to move the files manually, but now the program won\'t run)

[DaveG]

Bittorent simply would not be feasible.  Updates are new, and would not be seeded by many, because no one would have them at first. We just need a working update server... that\'s all.  

Though, for old updates and big new updates, we could create a patch installer that could be distributed by BT like the main installer is now.

[acraig]

I recently thought about using bittorrent for the updater but came to the conclusion that it won\'t be feasable.  Bittorrent is useful on large files ~20MB.  Most of our files that we want to update are less than that.

It turns out that creating a good updater much more difficult than you would first think.  I think the current one did an outstanding job for a while and now we are thinking of ways to improve on it based on what we have learnt.

A bit of history on the updater.   We use the http system because it saved us a lot of trouble in having to write both an updater server and an updater client. By using the webserver as the updater server it allowed us to get something together quickly.  It also offered us the ability to let people have their own updater mirrors without having to run any special software.  Their webserver is all that is needed ( and the repository of files ).

However, as people mentioned we do pack a lot of stuff inside zip files.  And when it was something like light information there could be many hundereds of those files.  So the updater has to do an http request for each of those.

My work in the last week or so has been to change the updater so instead of downloading all these little files it will just download the zip of that entire directory.  So you may end up downloading a little more but it will make it much easier on the webserver to answer many many requests for small little files.  Instead it just answers one for the .zip file.   OK, this probably didn\'t make much sense but trust me on this one.

The holy grail of updating is to do some sort of binary diff so you only have to download the \'bits\' that have changed but that is beyond me at this point.  So I will try to make what we have a bit better and learn about other methods.

[Neryam]

Wait mabye I can explain about the one big file

See right now the server lags VERY bad through some netwerking problems, hardware, router and techie stuff like that. It\'s gonna be fixed.
But the point is that the server takes a LONG time to answer each request. So right now its like this:

Updater request a file
Wait.... (1 sec)
File is sent to Updater
Updater downloads the file. (2 secs)

Updater request a file
Wait.... (1 sec)
File is sent to Updater
Updater downloads the file. (2 secs)

Updater request a file
Wait.... (1 sec)
File is sent to Updater
Updater downloads the file. (2 secs)

x 200 times for a total of 600 seconds


And it would be like this instead:

Updater request a file
Wait.... (1 sec)
File is sent to Updater
Updater downloads the file. (500 secs)

Total of 501 secs.

Note this is very rough estimations of numebers.

[acraig]

Well, the new updater will work a bit like that. At least I hope it will. It will still download individual files but it will use zips more.   From what I understand you would like it to just download 1 zip file, but that cannot be done at the moment.  It would require special \'knowledge\' on the server which it doesn\'t have.

[darkw00t]

that new mirror thing Talad posted works heaps faster (took me like 15 minutes on my slower computer) so you shoudl try that

[Induane]

To avoid getting stoned, Induane uses the search function to find this post by Talad.  
..........
..........
..........
.........
Hours later, having concluded once again that the search function is useless, Induane dons a suit of stone protection, steps up, and shouts: \"WHERE IS THIS AT??\"

[Karyuu]

Sometimes, it\'s just enough to look at the stickies... :)

Posted within the Crystal Blue Problems/Bugs and Technical Help sections:
Important: READ THIS if updater is taking long time!

[Induane]

/me wonders how he survived this long without people pointing out the obvious to him. Thanks!