Bugtracker and its inherently user-unfriendly interface

[pedenel]

This is not actually a complaint but something I'd like to see improved. Would have put it in the Wishlist but that's for features in the game.

The bugtracker is incredibly hard to use and understand. I have beta tested other games which used bugtrackers but this one is incredibly tough to get to grips with. The way it is currently, it either generates large numbers of duplicate or apparently useless reports or testers shy away from using it after making one or two attempts to use it.

If resources are available, it might be worthwhile putting a little thought into improving this to make it a more productive tool.

A good example of a bugtracker was the one that Face of Mankind had during its beta testing phase. If anyone else here beta tested FoM and used that bugtracker, I'm sure they'd agree that there is no contest between that and PS's bugtracker.

If the Dev team wants to follow up on this and would like to know what aspects of the FoM bugtracker facilitated usage, I'd be glad to share what I recall.

[acraig]

What makes it hard to use?  It seems fairly straight forward to me.  Enter in a Summary and a description and a bug report is generated.  The problem is that some of the descriptions are not sufficient to be useful.  But that is an issue with the submitter and not the tool itself. 

If you have specific ideas please put the forward but keep in mind that we don't have the resources to setup a full ( ie client/server based ) bug tracker and we are used to the current one now.  So perhaps we can put some thought in updating the current one to make it more useful.  But again, need specific things that need to be addressed.

[bilbous]

It would be beneficial to have a legend key for the various drop down choices so that it was easier to choose correctly to which aspect the bug applies. I can never tell if it is client or server or whatever other category and sub-category might be available.What I am thinking of is a separate page we could have open in another window or tab with examples of what belongs where.

[Karyuu]

Maybe just having a really well-written and thorough guide to using the BT is all we need? With explanations of every choice and, as Bilbous suggested, examples. It would be more reading for you folks, but if you're going to be contributing with bug reports it should probably be considered mandatory anyway ;)

[bilbous]

I'm thinking it needs to be at the bugtracker itself instead of on the forums because that is where it will be most useful, I suppose it doe not make too much difference but if I am in game I don't want to open the bugtracker and then sift the forums to find the right post. Sure I can book mark it but everything in its place eh?

Oh and K? No need to be formal, it is just bilbous I'm too lazy to require capitalization.

[Karyuu]

I can't not capitalize names..! The letters, they are habit forming

There's a "Read Documentation" link on the BT - maybe that doc page just needs to host a new guide?

[bilbous]

Or maybe I should read it, not sure I ever did. I think it may be for the bugtracker software itself and not specific to PS. Just took a quick look and that is all it appears to be, a general overview that leads into the administrators guide. I stopped looking there.

[Ralleyon]

To tell you the truth, I've used some bug-trackers as well and the PS one gives me the same user-unfriedliness feeling, although most of the options listed are pretty clear.

For example, I find it VERY hard to track down duplicates unless I give the BT an extensive read. Searching is... a pain and many times I can barely find anything useful.

Two ideas could be considered in order to improve the current system:

• 1. An extensive guideline on how to post a successful bug test - perhaps a template thingy would be in order. I recall seeing one on the forums some time ago, but still, it doesn't feel that it covers a lot of real-estate.
  
  Once a guideline is in place (or perhaps the old one improved), an extensive example (with regard to Linux, MacOS and Windows generated reports and tools) will work wonders, basically giving people an idea of what real bug-testing is as only the devs themselves know what they are interesting in for a specific problem. Breaking it down into smaller tasks like graphical testing + example, game-mechanics testing + example and so on, might also help.
  
  Some will follow the guide, some will not, but I believe an improvement will be seen in the long run.

• 2. Change the BT with a more user-friendly one and which allows more options. To be specific, I like the CrystalSpace bugtracker a lot. Of course... migrating all this stuff would be quite a pain, so I can see the downside.

[acraig]

Arianna is actually working on a document that we will be using for the bugtracker.   It explains the full life of a bug and what status/resolution it should have at each step.  I am sure we can incorporate some ideas about how to write effective and accurate bug reports.   The issue with searching is also related to the quality of the existing bug reports.  If they are general and not very helpful then searching will not be effective. 

There is some issue between which area ( client/server ) you should pick.  Sometimes even I don't know which one it should be because it might be an issue on the server but the result is a bug on the client.  So for now I would not worry so much about that and let one of use direct it to the proper areas.   The main thing is to be specific as you can.  Give exact names on things like items or spells.  For example:
 "I cannot cast a spell"  == bad. 
 "I cannot cast Life Infusion" == better
 "I cast life Infusions and it says I don't have the glyphs, but I have glyphs purified and the spell is in my spell book" == even better.

Whenever the bug reports are too general it puts more onus on us to try to figure out what caused that bug.  Think of it as a sniper trying to hit a target.  If you tell him "Your target is over there somewhere in that group of 1000 people" he probably won't ever take a shot.  If you say "Your target is the general with the pointy hat" then he has something to work for.    Ok, that's somewhat of a screwed up analogy but I hope it gets the point across. 

[pedenel]

Thanks for the update. I've spent a bit more time looking into the bugtracker and getting more familiar with it.

Creating new reports is easy, searching to check for duplicates is the tough part as it takes some experimentation and time to learn how to search effectively.  If the document Arianna could also cover how to do a proper search, I think that would help greatly.   Last thing I want to do is bog you guys done with duplicate reports.

Also, could it explain what the 'Status" tags mean and when they should or should not be changed/used?

I just found out that a closed bug that I added a comment to and tried to re-open was in fact fixed but the term fixed only applied if I compiled a new client from the CVS.

In other words, the bug was fixed but not in the official client. I see a lot of room for error and confusion here which would just add more work all round.

[acraig]

Thanks for the update. I've spent a bit more time looking into the bugtracker and getting more familiar with it.

Creating new reports is easy, searching to check for duplicates is the tough part as it takes some experimentation and time to learn how to search effectively.  If the document Arianna could also cover how to do a proper search, I think that would help greatly.   Last thing I want to do is bog you guys done with duplicate reports.

Also, could it explain what the 'Status" tags mean and when they should or should not be changed/used?

I just found out that a closed bug that I added a comment to and tried to re-open was in fact fixed but the term fixed only applied if I compiled a new client from the CVS.

In other words, the bug was fixed but not in the official client. I see a lot of room for error and confusion here which would just add more work all round.

That is an issue that will be addressed.  We cannot release a new client for every single bug that is fixed but we should mark them so in the bugtracker somehow to say that they will be fixed in the next release.  I think that it does offer this feature ( to be closed in vesion: xxx ) but we have not made use of that yet.

[Nurahk]

It would be nice if people could edit summaries too.

"Mining system borked" isn't exactly normal dialog and summaries like that do nothing to help the search feature.
I could just be missing the option to edit summ. of course.

[stevenw9]

Ahah! Now I see a general issue. A new client cannot be released with every bug fix, this is incredibly true and has caused many games and not just this one many headaches. A solution? Dynamicly updated file systems. What do I mean? Well there's quite a few games that check the version of each file. Usualy the version of each file is collectively set within a file which is only touched during installation and during updates. This can help target specific files for updating without having to rework the entire client. This may however cost more bandwidth as each time the updater is ran it needs to check the file version, file but it shouldn't be much of a jump since it 'should' be a simple encrypted text file.

Then there's the problem with mirrors being up to date. This is where you need to get more dynamic by creating a 'mirror program' to update their file system as well. This can help mirrors make it a simple 'click and run' update system where they recieve what's needed from the main host and then help ease the bandwidth by offering theirs in return. Incredibly similar to a torrent but you can see the difference.

[acraig]

Ahah! Now I see a general issue. A new client cannot be released with every bug fix, this is incredibly true and has caused many games and not just this one many headaches. A solution? Dynamicly updated file systems. What do I mean? Well there's quite a few games that check the version of each file. Usualy the version of each file is collectively set within a file which is only touched during installation and during updates. This can help target specific files for updating without having to rework the entire client. This may however cost more bandwidth as each time the updater is ran it needs to check the file version, file but it shouldn't be much of a jump since it 'should' be a simple encrypted text file.

Then there's the problem with mirrors being up to date. This is where you need to get more dynamic by creating a 'mirror program' to update their file system as well. This can help mirrors make it a simple 'click and run' update system where they recieve what's needed from the main host and then help ease the bandwidth by offering theirs in return. Incredibly similar to a torrent but you can see the difference.

The other issue with this is that we have moved past the "released" version of the client.  For example, Vengeance just did a very big inventory design change that affects both the client and the server.  So it's not as simple as just rebuilding a new client.  Now the proper way to do this is with branching the code tree but that has issues of it's own in making sure that bugs in one branch get fixed in the other.  I think that if we can keep doing an update every 1-2 months it should not be too bad.  We are also looking to totally redo how the updater works but that requires a fair bit of thought.

[pedenel]

That is an issue that will be addressed.  We cannot release a new client for every single bug that is fixed but we should mark them so in the bugtracker somehow to say that they will be fixed in the next release.  I think that it does offer this feature ( to be closed in vesion: xxx ) but we have not made use of that yet.

Sounds good to me