PlaneShift
Support => Complaint Department => Topic started by: pedenel on February 28, 2007, 09:46:19 pm
-
This is not actually a complaint but something I'd like to see improved. Would have put it in the Wishlist but that's for features in the game.
The bugtracker is incredibly hard to use and understand. I have beta tested other games which used bugtrackers but this one is incredibly tough to get to grips with. The way it is currently, it either generates large numbers of duplicate or apparently useless reports or testers shy away from using it after making one or two attempts to use it.
If resources are available, it might be worthwhile putting a little thought into improving this to make it a more productive tool.
A good example of a bugtracker was the one that Face of Mankind had during its beta testing phase. If anyone else here beta tested FoM and used that bugtracker, I'm sure they'd agree that there is no contest between that and PS's bugtracker.
If the Dev team wants to follow up on this and would like to know what aspects of the FoM bugtracker facilitated usage, I'd be glad to share what I recall.
-
What makes it hard to use? It seems fairly straight forward to me. Enter in a Summary and a description and a bug report is generated. The problem is that some of the descriptions are not sufficient to be useful. But that is an issue with the submitter and not the tool itself.
If you have specific ideas please put the forward but keep in mind that we don't have the resources to setup a full ( ie client/server based ) bug tracker and we are used to the current one now. So perhaps we can put some thought in updating the current one to make it more useful. But again, need specific things that need to be addressed.
-
It would be beneficial to have a legend key for the various drop down choices so that it was easier to choose correctly to which aspect the bug applies. I can never tell if it is client or server or whatever other category and sub-category might be available.What I am thinking of is a separate page we could have open in another window or tab with examples of what belongs where.
-
Maybe just having a really well-written and thorough guide to using the BT is all we need? With explanations of every choice and, as Bilbous suggested, examples. It would be more reading for you folks, but if you're going to be contributing with bug reports it should probably be considered mandatory anyway ;)
-
I'm thinking it needs to be at the bugtracker itself instead of on the forums because that is where it will be most useful, I suppose it doe not make too much difference but if I am in game I don't want to open the bugtracker and then sift the forums to find the right post. Sure I can book mark it but everything in its place eh?
Oh and K? No need to be formal, it is just bilbous ;) I'm too lazy to require capitalization.
-
I can't not capitalize names..! The letters, they are habit forming (http://img80.imageshack.us/img80/3034/colonuhs4.gif)
There's a "Read Documentation" link on the BT - maybe that doc page just needs to host a new guide?
-
Or maybe I should read it, not sure I ever did. I think it may be for the bugtracker software itself and not specific to PS. Just took a quick look and that is all it appears to be, a general overview that leads into the administrators guide. I stopped looking there.
-
To tell you the truth, I've used some bug-trackers as well and the PS one gives me the same user-unfriedliness feeling, although most of the options listed are pretty clear.
For example, I find it VERY hard to track down duplicates unless I give the BT an extensive read. Searching is... a pain and many times I can barely find anything useful.
Two ideas could be considered in order to improve the current system:
- 1. An extensive guideline on how to post a successful bug test - perhaps a template thingy would be in order. I recall seeing one on the forums some time ago, but still, it doesn't feel that it covers a lot of real-estate.
Once a guideline is in place (or perhaps the old one improved), an extensive example (with regard to Linux, MacOS and Windows generated reports and tools) will work wonders, basically giving people an idea of what real bug-testing is as only the devs themselves know what they are interesting in for a specific problem. Breaking it down into smaller tasks like graphical testing + example, game-mechanics testing + example and so on, might also help.
Some will follow the guide, some will not, but I believe an improvement will be seen in the long run.
- 2. Change the BT with a more user-friendly one and which allows more options. To be specific, I like the CrystalSpace bugtracker a lot. Of course... migrating all this stuff would be quite a pain, so I can see the downside.
-
Arianna is actually working on a document that we will be using for the bugtracker. It explains the full life of a bug and what status/resolution it should have at each step. I am sure we can incorporate some ideas about how to write effective and accurate bug reports. The issue with searching is also related to the quality of the existing bug reports. If they are general and not very helpful then searching will not be effective.
There is some issue between which area ( client/server ) you should pick. Sometimes even I don't know which one it should be because it might be an issue on the server but the result is a bug on the client. So for now I would not worry so much about that and let one of use direct it to the proper areas. The main thing is to be specific as you can. Give exact names on things like items or spells. For example:
"I cannot cast a spell" == bad.
"I cannot cast Life Infusion" == better
"I cast life Infusions and it says I don't have the glyphs, but I have glyphs purified and the spell is in my spell book" == even better.
Whenever the bug reports are too general it puts more onus on us to try to figure out what caused that bug. Think of it as a sniper trying to hit a target. If you tell him "Your target is over there somewhere in that group of 1000 people" he probably won't ever take a shot. If you say "Your target is the general with the pointy hat" then he has something to work for. Ok, that's somewhat of a screwed up analogy but I hope it gets the point across.
-
Thanks for the update. I've spent a bit more time looking into the bugtracker and getting more familiar with it.
Creating new reports is easy, searching to check for duplicates is the tough part as it takes some experimentation and time to learn how to search effectively. If the document Arianna could also cover how to do a proper search, I think that would help greatly. ;) Last thing I want to do is bog you guys done with duplicate reports.
Also, could it explain what the 'Status" tags mean and when they should or should not be changed/used?
I just found out that a closed bug that I added a comment to and tried to re-open was in fact fixed but the term fixed only applied if I compiled a new client from the CVS.
In other words, the bug was fixed but not in the official client. I see a lot of room for error and confusion here which would just add more work all round.
-
Thanks for the update. I've spent a bit more time looking into the bugtracker and getting more familiar with it.
Creating new reports is easy, searching to check for duplicates is the tough part as it takes some experimentation and time to learn how to search effectively. If the document Arianna could also cover how to do a proper search, I think that would help greatly. ;) Last thing I want to do is bog you guys done with duplicate reports.
Also, could it explain what the 'Status" tags mean and when they should or should not be changed/used?
I just found out that a closed bug that I added a comment to and tried to re-open was in fact fixed but the term fixed only applied if I compiled a new client from the CVS.
In other words, the bug was fixed but not in the official client. I see a lot of room for error and confusion here which would just add more work all round.
That is an issue that will be addressed. We cannot release a new client for every single bug that is fixed but we should mark them so in the bugtracker somehow to say that they will be fixed in the next release. I think that it does offer this feature ( to be closed in vesion: xxx ) but we have not made use of that yet.
-
It would be nice if people could edit summaries too.
"Mining system borked" isn't exactly normal dialog and summaries like that do nothing to help the search feature.
I could just be missing the option to edit summ. of course.
-
Ahah! Now I see a general issue. A new client cannot be released with every bug fix, this is incredibly true and has caused many games and not just this one many headaches. A solution? Dynamicly updated file systems. What do I mean? Well there's quite a few games that check the version of each file. Usualy the version of each file is collectively set within a file which is only touched during installation and during updates. This can help target specific files for updating without having to rework the entire client. This may however cost more bandwidth as each time the updater is ran it needs to check the file version, file but it shouldn't be much of a jump since it 'should' be a simple encrypted text file. :)
Then there's the problem with mirrors being up to date. This is where you need to get more dynamic by creating a 'mirror program' to update their file system as well. This can help mirrors make it a simple 'click and run' update system where they recieve what's needed from the main host and then help ease the bandwidth by offering theirs in return. Incredibly similar to a torrent but you can see the difference. :)
-
Ahah! Now I see a general issue. A new client cannot be released with every bug fix, this is incredibly true and has caused many games and not just this one many headaches. A solution? Dynamicly updated file systems. What do I mean? Well there's quite a few games that check the version of each file. Usualy the version of each file is collectively set within a file which is only touched during installation and during updates. This can help target specific files for updating without having to rework the entire client. This may however cost more bandwidth as each time the updater is ran it needs to check the file version, file but it shouldn't be much of a jump since it 'should' be a simple encrypted text file. :)
Then there's the problem with mirrors being up to date. This is where you need to get more dynamic by creating a 'mirror program' to update their file system as well. This can help mirrors make it a simple 'click and run' update system where they recieve what's needed from the main host and then help ease the bandwidth by offering theirs in return. Incredibly similar to a torrent but you can see the difference. :)
The other issue with this is that we have moved past the "released" version of the client. For example, Vengeance just did a very big inventory design change that affects both the client and the server. So it's not as simple as just rebuilding a new client. Now the proper way to do this is with branching the code tree but that has issues of it's own in making sure that bugs in one branch get fixed in the other. I think that if we can keep doing an update every 1-2 months it should not be too bad. We are also looking to totally redo how the updater works but that requires a fair bit of thought.
-
That is an issue that will be addressed. We cannot release a new client for every single bug that is fixed but we should mark them so in the bugtracker somehow to say that they will be fixed in the next release. I think that it does offer this feature ( to be closed in vesion: xxx ) but we have not made use of that yet.
Sounds good to me :D
-
Well large updates would register through the file check system and tell the program, ' Hey, there's either too many files to update or we need to update everything to be sure. ' with a simple variable. That would help full updates that are required rather then just smaller updates that would take hours to finish. x.x' *stabs most free games that don't do full releases often enough*
-
I suddenly second this complaint. Why?
For some odd reason we can't comment on bug posts anymore and we can't edit them.
"to prevent misusage".
You can now only edit and comment when you're the assigned developer or the reporter.
Other developers, GMs, bughunters and players won't be able to respond and comment on them anymore.
So yeah, this is really going to make my life as a bughunter easier.
Not.
-
That's not good ... so if I have additional information that may help shed more light on that particular bug I should start another bug report?
Thought that was what having the comments field (or something similar) in the bugtracker was meant to avoid i.e. posting another report on the same bug to provide additional information.
-
That's not good ... so if I have additional information that may held shed more light on that particular bug I should start another bug report?
Thought that was what the intention of having the comments field (or something similar) in the bugtracker was meant to avoid i.e. posting another report on the same bug to provide additional information.
This restriction is temporary until we can find a better solution for bugtracker misuse.
Greetings,
-
Apparently there was someone ravaging the entire bugtracker.
I use the bugtracker every day, a lot, and I haven't noticed that, but hey, I miss things.
So that's why I suggest this:
Use a better and professional system like Trac or BugZilla.
Simply create a new bugtracker and force everyone to make new reports there by locking the old bugtracker.
The old bugtracker will only be used for the bugs that are already there.
If you look at the issues that are 1)open 2)real 3)duplicatable 4)fixable the number of them isn't even that big.
I don't see problems with this strategy.
I see this as the only valid solution as exporting the entire bugtracker would be too much of an issue.
Changing the existing settings of the bugtracker is hard because there are so few options, it's either complete lockdown or complete anarcy, so to speak.
So, a new bugtracker.
Comments please.
-
Apparently there was someone ravaging the entire bugtracker.
I use the bugtracker every day, a lot, and I haven't noticed that, but hey, I miss things.
So that's why I suggest this:
Use a better and professional system like Trac or BugZilla.
Are those php/mysql based? Can't host scripts or executables on hydlaa.
Simply create a new bugtracker and force everyone to make new reports there by locking the old bugtracker.
The old bugtracker will only be used for the bugs that are already there.
If you look at the issues that are 1)open 2)real 3)duplicatable 4)fixable the number of them isn't even that big.
I don't see problems with this strategy.
We can't even get people to use a simple php bug tracker correctly. And there is nothing simple about creating a new bugtracker or even installing a current one. Different php versions/different mysql versions/file permissions/http access/ftp access/uploading core files/ftping over all the code to hydlaa/making sure it works/creating the different users/setting up the email system/assigning permissions/creating the different projects/creating the categories. It's not a matter of:
$./install_bugtracker.sh
So, a new bugtracker.
I'm fine if people want to set up demos of bugtrackers that they can find. But I won't install each one so people can see if they like it or not.
I changed the permissions on the current bug tracker so that only admins can change it because I was getting complaints from my team about bugs being closed/mislabeled. In that case that trumps other usage. Yes we need people to submit bugs but we are the ones that have to try to fix them.
-
Trac (http://trac.edgewall.org/report)
Requirements:
- P-Y-T-H-O-N 2.3 and higher (<--seriously, if you write that without the caps and the "-" the forum freaks out)
- ClearSilver
- SQLite or PostgreSQL, MySQL is currently in experimental phase.
BugZilla (https://bugzilla.mozilla.org/)
Requirements:
- Perl
- Mysql or PostgresSQL
Mantis (http://www.futureware.biz/mantisdemo/my_view_page.php)
Requirements:
- PHP
- MySQL
There everyone can see what a certain bug reporting tool feels like and how it works.
-
Bugzilla looks more familiar which could lessen familiarisation time and it uses MySQL which is good for this project right?
-
Again, if hosted on laanx it cannot use a lot of stuff like perl and other scripting languages. It has to be simple PHP and mysql based. I will do some tests over the weekend to see exactly what is supported on hydlaa.
Arianna is now looking over the various bug trackers to evalute them.
-
Thanks acraig, I'm sure that we'll now be able to improve on the whole bug-cycle, it will not come to a locked-down bugtracker again :)
-
Hate to throw some oil on this dying fire but,
I remember hearing something about a "testers" category. Would they have the ability to comment and edit bugs?
-
Personally, I don't see why testers should be allowed to edit a bug report created by others or comments made by other testers. Their own reports/comments should be capable of being edited though.
It has always bothered me that testers can change the status of a bug on PS's bugtracker as I've always regarded this as being something only the Dev team should be able to do.
A tester wishing to re-open a closed bug should create a new bug and link to the previous bug report by inserting the old bug number.
-
I would post a general post to all the observations that have been made in this thread.
First of all, I agree that this bugtracker is not the best tool of its kind that there is on the market. I think that its use is simple and straigthforward but the research function should be better, in order to make life easier to everybody. It would be good if it would be more customizable and also, from the admin side, to obtain different groups with different "attributes". However, this is not possible in the bugtracker today, and this is surely a limitation. Categorizing the bug should be easier and more intituive, but it is also true that not all players have the technical knowledge to know if a bug is client or server and at the moment, that hasn't been a huge problem for us, developers. This is basically what Ralleyon has said as well.
However, the main problem, which we would encounter with whatever bugtracker there is around, is how people use it.
That is why I have been putting my time in trying to create a so call "bug tracker process", as acraig stated, that should be a guide on how to use the bugtracker, for each "roles" available, and represent an unequivocable tool for following the life of a bug report.
Still, the problem is in enforcing this "life path" of the bug report. There will be always people that act indipendently and, we will try to contain that.
I would love to get this document ready as soon as possible, but I have a pending release at work, and this has caused me to have really little time for everything (I have to work overtime for the whole week, until absurd hours). However, I will try to do my best to finish it as quickly as possible. This document will try to include also a description on how to use the different fields, with the hope that everybody will notice how the bug report is changed (for example, with the "fixed in version Xxx" thingy) . It is also hard to enforce everybody to write a comment when they modify the bug report... (the bugtracker I use at work actually do that, but we don't have a really customizable tool here :-), not sure about the other proposed).
For Nurahk, I think that people *can* actually edit the summaries. So, that should be no issue, today. If you look in the bug report there is a field called "Summary". I think everybody can edit that but I am not sure, since I am admin of the bugtracker ;-)
For stevenw9, we had the updater for that purpose, but it didn't work fully perfectly so at the moment (as far as I know) we are skipping its usage. However, acraig has replied perfectly on why, in a kind of development as ours, it is hard to have a system like you describe. Vengeance's inventory is not completed (still in development) however it is committed in CVS ;-)
Now, about the *big* polemic.
First of all, if you feel that your role is the one of a bug hunter, in my opinion, you are not a "bug tracker janitor". You find bugs, you report them, you annotate which one are most important, you verify that a fix is actually a fix (and not a new bug), eventually you perform regression tests to see what else is broken and so on.
We have plenty of cases of bug reports closed too hastily, comments that are really not understandable, misplaced and people entering a "conversation" in a bug report not just to confirm the bug, bug to add new bug reports (write a new one then), and so on and so forth. Of course, closing the bugtracker was a temporary solution (even if, not everybody in the team actually was thinking of it as temporary - and it was not me >:(). I remember many many times in different "hobby oriented" projects where the people got fed up of complaints, of things not working as they should and so, and they just remove the service. Why? Because it is a privilege to have that service and not a right.
You want to cooperate? Then, observe how the *janitor* is working on the bugtracker, ask questions if you have doubts, have a dialogue and not ironic harsh comments every 5 seconds on the operate of the janitor. I have honestly (and I will alway) try to be pedagogic in each bug report that is "misplaced" or whenever a bug should have not be closed I always explain why. In the worse cases, I have tried to speak (with calm, if I could) with the interest person, pointing out *why* and *how* this should be done in a different why. However, somebody doesn't obviously like to get suggestions, since they ignore them :-)
And considering myself the "janitor", for the efforts that I put in the tool and in the process, I feel entitled to try to stear the process toward the right one.
I use the bugtracker every day too, and the latest weeks I found it really time consuming. I like to be the janitor (if you want to call it like that) and I would rather keep the task for myself, because I do have quite good experience with similar systems and I enjoy doing the task. And yes, I have been having really little time to complete my "bug process" and if instead of teaching how to use the bugtracker I would be listened a bit more, I would probably have more chances to conclude the document :thumbup:
The misuse is not just due to the limitations of the system, but rather to *how* the people use it. So we are back on the first words of this long long post.
About new tools. I have been looking at BugZilla and it looks fine. However, it will force us to run the system not where it is today, which is not the most preferred choice we have. Since we don't feel force anyway, to change bug tracker (but rather, the usage of it), maybe it is better to hunt for the "most preferred" choice (which would be compliant with the requests made by acraig).
I could write more, but I have honestly not the right amount of time to bore everybody with this monologue, so I will keep it short now.
Please, do not put me on a stake, just for the sake of doing it, because locking the bugtracker hasn't been my decision (even though I didn't disagree), and surely, it hasn't been my fault.
-
Arianna has been doing an awesome job with the BT lately, and she's more than capable of turning it around from what it once was (a confusing dump of various problems) to a very useful tool. I wouldn't doubt her for an instant (http://img151.imageshack.us/img151/7151/pleasedhc3.gif) Consider her the moderator of the BugTracker - just as you wouldn't want to misuse the forum because there are people (me and Neko) watching you, so you should be careful with what you do on the BT, because Ary knows what she's doing.
Taking care of game bugs should be as smooth of a process as possible for the developers, so if they have to spend more time than necessary cleaning up after people's bug reports, they get much less done. The documentation Arianna is writing will be a big help for everyone, luckily.
-
Any chance of a bug-wiki?
-
For what purposes? :]
-
Wikis are easy to use, and easy to search. Subjects are easily organized. Comments on subjects are easy to add. I can't say any of that about the bugtracker.
-
I personally find the BT easier to use than the majority of wiki systems, so I would disagree.
-
Wikis are good but not for bugtracking purposes, imho.
However, I am actually in line with Karyuu saying that I find the BT (even the one that we have today) easier to use than the Wikis.
I guess it is all about how much you use the tool, and still, the main problem here is not the tool :-)
-
Pardon me for bumping this thread but I have a suggestion about the bugtracker that I think would help some. Right now you can choose to place a game bug against the client or the server for bugs in the game itself. Would it be possible to add a section for the database where we could post language bugs? They do not really belong to the client where they seem to get placed because the client is not malfunctioning it is presenting exactly what it is supposed to. I am talking about spelling and grammar for the npc's. This doesn't really belong to the server either because it is sending out exactly what it is supposed to as well. The problem is the data needs to be massaged into shape.
-
When the new tracker comes into use this will not be a problem anymore.
-
I think it is inevitable that PS's bugtracker will be replaced someday. phpBugTracker (http://phpbt.sourceforge.net/), the current one, hasn't been updated for nearly 2 years and it would be wasted time for the PS team to update it themselves. Though, I do agree that the current problems with the bugtracker could be greatly reduced with a good set of guidelines, so I look forward to reading them if they are still being written.
The bugtracker that I would prefer is Trac (http://trac.edgewall.org/), it is widely used and has an entire website (http://www.trac-hacks.org/) full of plugins. There is even a phpBugTracker2TracScript (http://www.trac-hacks.org/wiki/PhpBugTracker2TracScript) which I imagine might come in handy. It also has a wiki built-in, which may sound useless for bugtracking but would enable the team to move the current PS wiki into the bugtracker to kind of focus the community on the task of bugtracking (like Edgewall's site (http://www.edgewall.org/)). PS's main website could even be moved into the bugtracker, and the forums. This is a game that is in permanent development so it makes sense to centre the community around that development.
As a sidenote it would also be a good idea to get planeshift.org/ (http://planeshift.org/), it is a non-profit game.
-
All I can say is that something new will be in place sooner than soon(tm).