ok despite being a vaguely competent \"pure\" programmer, i know very little about network protocols, p2p etc. etc. so feel free to rubbish my ignorance.
my suggestion, very very briefly is letting users with fast connections run untrusted servers.
slightly less briefly, here\'s how i see it working:
the game world would be divided into many \"cells,\" say 1000 for now, but it could easily be more.
each cell would be under the ultimate control of precisely one of the trusted servers. so far so uncontroversial i hope.
however, if an untrusted source (call it X) decides they\'ve got a seriousish amount of bandwidth going spare, my hope is that this bandwidth can be used.
what would happen is that X would query some principal server P which would then give X a quick bandwidth test. if X passed the test, P would allocate X a few cells from the most overloaded of the trusted servers, according to its bandwidth. P would tell X which of the trusted servers T to send what (possibly secured) query to, and then T would respond with the current state of everything in that cell. if there were enough of the untrusted servers each cell would actually have more than one untrusted server.
how does all this speed anything up? and how does our client C interract with all this? well on logging on C queries P to get the trusted server of the cell it\'s in, T, and any untrusted servers covering that cell, say X and Y. C then does a quick ping test to see which of X, Y and T is the fastest from its particular location. (it might be necessary to weight this test in favour of untrusted servers, to avoid overloading the trusted ones)
if T\'s the fastest for it, then it sends all its data packets (movement etc.) to T and expects a response from T. of course T must also forward whatever it recieved from C to X and Y, along with any \"random\" responses it sent to C, but this forwarding will not impact on C\'s ping as T does not need to wait for a response from X and Y before responding to C.
however, if X is the fastest for it say, along with forwarding (to T and Y) whatever C sends to it and the bare minimum of what it sent to C, it must also send T everything else it sent to C, so that T can verify it is performing correctly. in addition T might occassionally ask C to forward all it\'s communications with X as an addittional security check. if T found X to be malfunctioning, it would notify P which would in turn notify all the clients in that particular cell.
although this system does not result in any substantial reduction of the bandwidth being used on the trusted servers, because a client connected to an untrusted server does not have to wait for any response from the trusted server, lag should be massively reduced.
tom