Author Topic: Suggestions for next stress test  (Read 3027 times)

Cebot

  • Hydlaa Resident
  • *
  • Posts: 161
    • View Profile
Suggestions for next stress test
« on: May 10, 2008, 04:26:10 pm »
First of all:
@Players
DO NOT CAST MAGIC!
It won't stress the server, but the clients. Some people have used 3-6 clients for the stress test and because of the magic they hardware had a very hard time
Obey the GM's/Dev's
If GM's/Dev's worldshout to stop using magic, DO it! (Same for anything else they announce)

@Devs:
Those with ssh access to the server should monitor 3 things, cpu/mem usage, network load with a network analyzer and tools like valgrind to find mem leaks
(not sure if you already did that, if so, forget about my comment :))

Maybe it might be good to switch to latest svn trunk for the stress test too, since there regularly are improvements, like khakis netcode fixes // EDIT: just saw on irc that khaki said his netcode fixes are already on laanx
(must not be a complete release, just an update to the binaries and libraries via the updater)

@Testers:
use valgrind and the like to hunt down mem leaks in the client

@GM's
Teleport players to a less laggy map please, ojaroad1 or bdroad1 for example, this might help those people who run a lot of clients, since loading plaza is a mess for many people
Maybe teleport those magic casters which won't obey your commands to a far end of the death realm where they do not bother other players with the magic :P (or an instance of DR :P)

Besides those points, I think the stress test went well and hope to see more, maybe even regular tests of this kind, since the information which can be gotten from such a test might be really helpful to you devs ;)

my 2 tria
Cebot
« Last Edit: May 10, 2008, 04:36:51 pm by Cebot »
So why do I love when I still feel pain?
When does it end, when is my work done?
Why do I fight and why do I feel that
I carry a sword, that I carry a sword through a battlefield?

VNV Nation - Joy

Prolix

  • Guest
Re: Suggestions for next stress test
« Reply #1 on: May 10, 2008, 04:36:57 pm »
I missed it again, possibly due to thinking I am gmt -5 (EST) but it could have been I did not properly account for daylight saving time which had its time change date altered in North America this year. It seems to me it went on at 4 pm when I expected it at 5. Of course I was otherwise distracted and may have missed it for that reason. Another thing that might be useful is to indicate how long it is to last, even make it last for an hour so that stragglers can show up and it is not just people on-line but people coming and going as well.

Caarrie

  • Forum Addict
  • *
  • Posts: 3369
  • We want no UNFIXED bugs!!!!!!!!!!!!!
    • View Profile
    • PlaneShift3dMods
Re: Suggestions for next stress test
« Reply #2 on: May 10, 2008, 06:15:50 pm »

@Testers:
use valgrind and the like to hunt down mem leaks in the client

The testers only do that at request of the devs. lately we have not been requested and not all have the ability to do so.

Cebot

  • Hydlaa Resident
  • *
  • Posts: 161
    • View Profile
Re: Suggestions for next stress test
« Reply #3 on: May 10, 2008, 06:34:39 pm »
a stress test could be seen as such a request, don't you think?
todays stress test was mainly for server profiling, but in future stress tests the client could be profiled at the same time too.
you are right tho, not all testers are able to use valgrind, on their website they say it's available for linux on different architectures, they don't mention windows or mac os so probably only linux users could do that, however, i know vornne and kayden have used valgrind in the past and others who use linux could use it too - i never said _all testers have to use valgrind_. Those who are able to should use the chance of a stress test to use it

And before you throw back that kayden is not a tester but a developer, i already know that :P
So why do I love when I still feel pain?
When does it end, when is my work done?
Why do I fight and why do I feel that
I carry a sword, that I carry a sword through a battlefield?

VNV Nation - Joy

Caarrie

  • Forum Addict
  • *
  • Posts: 3369
  • We want no UNFIXED bugs!!!!!!!!!!!!!
    • View Profile
    • PlaneShift3dMods
Re: Suggestions for next stress test
« Reply #4 on: May 10, 2008, 06:49:29 pm »
Those who are able to should use the chance of a stress test to use it

Testers might not find that type of stress test a good way to test their client, and as i said before we ONLY do it upon dev's asking us, the results to us dont do much it is the devs that have to deal with the results and fix them if needed. We also do the testing on the latest code in the "trunk" branch of SVN, and this client does not work with laanx most of the time so doing the test on outdated client side code might not be the best idea as there might be changes to the code that we are randomly testing.

Cebot

  • Hydlaa Resident
  • *
  • Posts: 161
    • View Profile
Re: Suggestions for next stress test
« Reply #5 on: May 10, 2008, 07:53:50 pm »
you seem to not read what i write, so please before you reply again, read again what is written, stop being against everything other people suggest and stop interpreting something into my words i have not written. thank you

Testers might not find that type of stress test a good way to test their client, and as i said before we ONLY do it upon dev's asking us
Don't refer to _all_ testers if you mean yourself only. I know vornne has done valgrind tests together with kayden
A stress test _might_ be a good chance for devs to DO REQUEST a valgrind test from the testers - and those how can and know how to do it, can do it, if you can't, don't do it, simply as that.

the results to us dont do much it is the devs that have to deal with the results and fix them if needed.
Where did I say that you testers HAVE TO analyze the output? Right, I didn't say that. Tho I can imagine that some might even be able to analyze the output and assist the devs in fixing these issues.
"IF NEEDED"? seriously, memory leaks are not needed but required to be fixed, the question is how soon they will be found and how soon someone gets to fix them. If no one reports a memory leak, then no one will fix it

We also do the testing on the latest code in the "trunk" branch of SVN, and this client does not work with laanx most of the time so doing the test on outdated client side code might not be the best idea as there might be changes to the code that we are randomly testing.
Read what I wrote:
@Devs:
Maybe it might be good to switch to latest svn trunk for the stress test too, since there regularly are improvements, like khakis netcode fixes // EDIT: just saw on irc that khaki said his netcode fixes are already on laanx
(must not be a complete release, just an update to the binaries and libraries via the updater)

Final word:
All I was doing is suggesting something that helps to improve PS.

EDIT: To not start a flameware (i have indeed better things to do than arguing with you, caarrie) I won't reply to any of your replies to this thread. I look forward to hear opinions from other people tho, no matter if player, tester, gm or dev
« Last Edit: May 10, 2008, 07:55:34 pm by Cebot »
So why do I love when I still feel pain?
When does it end, when is my work done?
Why do I fight and why do I feel that
I carry a sword, that I carry a sword through a battlefield?

VNV Nation - Joy

Caarrie

  • Forum Addict
  • *
  • Posts: 3369
  • We want no UNFIXED bugs!!!!!!!!!!!!!
    • View Profile
    • PlaneShift3dMods
Re: Suggestions for next stress test
« Reply #6 on: May 11, 2008, 05:52:56 am »
you seem to not read what i write, so please before you reply again, read again what is written, stop being against everything other people suggest and stop interpreting something into my words i have not written. thank you

I READ THE WHOLE THREAD AND WHAT YOU SAID.

Zan

  • Veteran
  • *
  • Posts: 1944
  • Just a regular guy, with an irregular soul
    • View Profile
    • Photography
Re: Suggestions for next stress test
« Reply #7 on: May 11, 2008, 07:03:10 am »
Cebot, I think that if you want to suggest changes in how the devs handle things .. that there are better mediums than a public forum available. There are also better ways of putting it that don't make you come over like "I'll tell you how you should run your own household!". Intentional or not, it is what I see when I read your post and I don't think I'm alone.

I foresee little good coming from this thread. :-\
Zan Drithor, Member of the Vaalnor Council
Tyrnal Relhorn, Captain of the Vaalguard
Thromdir Shoake, Merchant
Giorn Kleaver, Miner.

Grayne Dholm, Follower of Dakkru

peeg

  • Testers
  • Hydlaa Notable
  • *
  • Posts: 560
    • View Profile
Re: Suggestions for next stress test
« Reply #8 on: May 11, 2008, 07:05:08 am »
I agree that a stress test might be a good opportunity to do some valgrinding, but i also have to back up Caarrie: As long as Laanx doesn't run something near trunk (HEAD) it doesn't make sense. Things would be different if the next stress test would be close after a fresh release.
Switching to trunk just for a stress test seems rather impossible to me as well. Not everyone's able/willing to build a client only for a quick test.
[ PSde ] - Join the german community!
Peeg Chaoswind - Mentor in The Rangers Of Yliakum

Cebot

  • Hydlaa Resident
  • *
  • Posts: 161
    • View Profile
Re: Suggestions for next stress test
« Reply #9 on: May 11, 2008, 07:25:22 am »
@Zan, it wasn't my intention to tell the devs how to do their household, my intension was to suggest something to get as much useful information out of a stress test as possible. I apologize if it sounds like am saying "I'll tell you how you should do it".
I think the forums are a good place for it, because different people have different opinions and can add to it, also there are experienced people in the player base which have coding, profiling and debugging skills too and could assist with it (after all most devs and testers come from the player base :))

@Peeg
That is why i suggested an update, with the now working updater it won't be nearly as difficult as before. Not everyone would have to build a client but a few.
Also in my opinion a regular update of the code makes sense. Because there are constantly improvements and bugfixes which got marked as save by you testers already. A major release could then be used for major changes, ar and settings updates and the updater system could be used for bugfixes.
Otherwise I agree, not everyone is willing to build a client for a quick test (and also not everyone is able to)
So why do I love when I still feel pain?
When does it end, when is my work done?
Why do I fight and why do I feel that
I carry a sword, that I carry a sword through a battlefield?

VNV Nation - Joy

Caarrie

  • Forum Addict
  • *
  • Posts: 3369
  • We want no UNFIXED bugs!!!!!!!!!!!!!
    • View Profile
    • PlaneShift3dMods
Re: Suggestions for next stress test
« Reply #10 on: May 11, 2008, 08:10:14 am »
@Peeg
That is why i suggested an update, with the now working updater it won't be nearly as difficult as before. Not everyone would have to build a client but a few.

But it still takes a lot of time, xordan has to build the windoze and linux clients then Trymm who is in another time zone has to build the mac client, it is NOT very easy to build a client for release the art part is easy but to get it all done takes time, and testing of the official release first by the testers and devs before release. Also the testing team has regression tests to do on the client/server before we can give a go to a release.

Kemex

  • Traveller
  • *
  • Posts: 24
  • - If can be done, then i will code it -
    • View Profile
Re: Suggestions for next stress test
« Reply #11 on: May 12, 2008, 01:19:22 pm »
From mi side of view, think that at least @player suggerences is very apointed. since it will help more to those who really want to help on an Server Stress Test.
use of Vlagrind could be  helpfull for @testers who are reading this Thread.

As far as i readed, im only seeing  a few suggerences, that should be evaluated and take in consideration (if apply).


Lanarel

  • Testers
  • Hydlaa Notable
  • *
  • Posts: 782
    • View Profile
Re: Suggestions for next stress test
« Reply #12 on: May 12, 2008, 02:30:43 pm »
As I understood, the stress test was meant for the server, which was updated with most of relevant changes (improvements and test code). 'Function' of all players (and testteam members) that were asked to log in, was to get large numbers, so causes of high cpu or much network traffic, full queues, etc,  are easier to spot.
Of course anyone is free to offer advice on how to do things better, but be aware that you might be suggesting things that are already thought of, or are not relevant to the tests. On the other hand, not being told by the devs to use a tool, does not mean it might not be a good suggestion ;).

Some points that might be of relevance in this:
- 'stress'  on the client is not the same as stress on the server, so I could imagine different tests for that (which may or may not require many other players)
- distributing a trunk client as temporary update to people would probably mess up many local installs. This is not as simple as it may sound :)
- related to these: as Cebot already mentioned, Vornne did some valgrynd tests on the client. Results from those showed that (at that time, and possibly depending on HW) almost all cpu time is spent in graphics related issues in CS, such as computing collisions, determining which polygons are in view, drawing gui/labels/shadows, etc. Running valgrynd on a client while doing a stress test of the server, would not be the first step to improve on this :)

Bamko

  • Hydlaa Citizen
  • *
  • Posts: 234
    • View Profile
Re: Suggestions for next stress test
« Reply #13 on: May 12, 2008, 08:49:51 pm »
I found Cebot's original suggestion helpful, relevant and respectful.

Why are some people so sensitive that MAYBE someone has a suggestion that MIGHT be helpful to the testers (us players) and might help the Devs perform their tasks better?   

No good deed goes unpunished, eh?

Cebot, keep it up, we are cheering for you, and we know the difference between a "suggestion" and a "official notification" and at least for this non-coder (who does not even know what that V-word thing is, by the way) found it helpful and saw no reason for any other response, exccept perhaps "the above suggestion is not an official Planeshift order and infact please note: (any additional helpful or clarifying information) But keep up the good work.  The only bad suggestion is the one not said"

Like I always say.. "I would rather say it and not need to, than need to and not say it!"

Heh. I should trademark that, eh?