PlaneShift
Support => Technical Help: IN GAME bugs (after loading world) => Topic started by: gonger on September 17, 2019, 06:10:55 am
-
Greetings,
Since I changed the log file options (in-game: Options / Interface / Chat - Logs) only after playing for a couple of years, I have huge amounts of combat messages in my log. The file size is about 56 MB by now, and Notepad++ is having a hard time with it.
I am probably not the only one with this problem, so my question is: Does anybody have a small script that would remove combat messages from my main chat log?
Thanks in advance,
Gonger
-
I don't have a script, but what I did was search for system lines that I'd never need (combat messages, how much exp I got, etc.) and marked those lines. Then I just deleted all bookmarked lines. Worked pretty well, if not as quickly as a script.
-
Hi Shatterkiss,
Thanks for the answer. Could you add a few details? Just the outline to get me started, I will get along afterwards.
Greetings,
Gonger
-
I am guessing you are on Windows (hence Notepad++). I can give you a SED script you could use, I hear Windows nowadays can sport a BASH script so try it to see how it works. I am guessing you need to get rid of both "[Sys]" and "[BSys]" lines, the following command will do it if you have a BASH shell and also SED (or can find a way to somehow run a Linux on your machine that has SED included) and can change to the log directory containing your logfile before running it:
sed -i '/ \[B\?Sys\] >/d' *_chat.txt
The command will work with any PS character name, so be careful if you have characters whose logs you don't want to modify. Replace "*_chat.txt" with the actual filename to avoid that.
If you only want to remove "[Sys]" lines use the following command instead:
sed -i '/ \[Sys\] >/d' *_chat.txt
Replace "Sys" with "BSys" in the command above to only remove "[BSys]" lines. Good luck!
-
Thanks Migg, but this will not help me. My log lines are not tagged with "[Sys]" or "[BSys]". I just checked and for some reason, under Options / Interface / Chat-Logs, I have no value for Log Bracket.
Some examples:
(21:52:34) [NPC] Maulberlord attack Gonger Xaraha with aggressive stance
(21:52:35) [NPC] Maulbernaut thinks Gonger Xaraha is weak
(21:52:35) [NPC] Maulberlord thinks Gonger Xaraha is weak
(21:52:37) [NPC] Maulberlord casts Psychic Blow to Gonger Xaraha
(21:52:37) [NPC] Maulbernaut casts Stone Fist to Gonger Xaraha
(21:52:41) [NPC] Imago Consumer thinks Gonger Xaraha is equal
(21:52:41) [NPC] Maulbernaut casts Stone Fist to Gonger Xaraha
(21:52:42) [NPC] Maulberlord casts Psychic Blow to Gonger Xaraha
(21:54:04) [NPC] Imago Consumer emits a cloud of noxious fumes.
(15:19:58) [NPC] Maulbernaut attacks Gonger Xaraha with normal stance
(15:19:59) >You start casting the spell Missile Shower
(15:20:00) >You've gained some practice points in Heavy Armor.
(15:20:00) >Maulbernaut's smash hits Gonger Xaraha and deals 14.75 damage to you!
(15:20:00) >You've gained some practice points in Heavy Armor.
(15:20:00) >Maulbernaut's smash hits Gonger Xaraha and deals 14.88 damage to you!
(15:20:00) >You hit Maulbernaut with a shower of energy beams for 123.96 damage.
(15:20:00) >Missile Shower affected 1 target.
(15:20:00) >You've gained some practice points in Crystal Way.
(15:20:03) >You are too far away to attack!
(15:20:05) >You've gained some practice points in Sword.
But to use the Windows Shell is not a bad idea...
Gonger
-
That's far more advanced than what I did! I just opened the file in Notepad++ in the Search interface went to the Mark tab. Make sure the bookmark line box is checked and that regular expression is selected. Then I marked lines like the following:
>You gained \d\d\d experience points and
<You gained \d\d experience points and
<You start to harvest.
<You start to dig.
<You've got some
<You hit
That sort of thing. I leave some system messages (storing/dropping/etc) as they can be useful in tracking down items. If you don't mind clearing all sytem lines, you'll note that they all start with the timestamp and the a '<'. I think you can mark all of those lines with this:
\(\d{2}:\d{2}:\d{2}\) >
That will mark every line that comes into the system tab. That includes /who results, the standard emotes like greets, transactions, etc.
Once you've bookmarked everything you want to delete, just go to Search-Bookmarks-Remove Bookmarked Lines
May want to make a backup, just in case.
EDIT: Ah, you posted while I was typing. In that block above, I'd just mark lines containing the following:
\[NPC] .* casts .* to Gonger
\[NPC] .* thinks Gonger
\[NPC] .* attacks Gonger
Doing those manually is terribly slow, but shouldn't deleted quest text. You can try this, which will deleted all NPC lines that don't contain the word "says"
^\(\d{2}:\d{2}:\d{2}\) \[NPC] (?!.*says).*$
NPCs use emotes though, and those could be relevant.
Other useful lines to bookmark and delete:
<You've gained some practice points
<You are too far away
<You made
Sarras
hits you
-
Hi Shatterkiss,
Thanks again for your advice. I did not know about this Bookmark feature in Notepad++, that was most useful.
With your regular expressions and some variants, I could greatly reduce my log file's size. Plus, the latest update to Notepad++ seems to allow better handling of large files, so I should be ok for the moment.
Gonger
-
And remember to enable log brackets for the next time you need to do this ;)
-
Happy I could help! Had a few 250Mb log files myself, thanks to crafting and combat spam, until I got tired of waiting a few minutes for Notepad to open them and cleaned them up.
-
I do not know how they did it, but Kate under Linux loads the chat log file from Damola, which is about 48 MiB, in a second or so. Kate is also available for Windows. See kate-editor.org
-
I do not know how they did it, but Kate under Linux loads the chat log file from Damola, which is about 48 MiB, in a second or so.
I am using Kate 3.14.2, and it can take almost forever to open something half that size. Might be how I installed Kio libraries which causes the trouble, but Kate does give me a warning when I open >10 megabytes.
Anyways, I configured PlaneShift to split up my logs to nine or so files. Gives me some trouble reconstructing events because PlaneShift doesn't quite classify its messages consistently … you might want to split up messages to four files.
-
@Can-ned Food: The version you are using is way too old. As far as I remember I read developers at some time improved performance *a lot*. I have 19.08.1 and as I said, loading of 48 MiB of chat log file is almost instant.
-
@Can-ned Food: The version you are using is way too old. As far as I remember I read developers at some time improved performance *a lot*. I have 19.08.1 and as I said, loading of 48 MiB of chat log file is almost instant.
Hi Damola,
Following your advice I tried Kate 19.08.1 under Windows, and unfortunately I am not impressed by the performance on big files. It is not only the loading, but also the searching inside the file.
Gonger
-
Well searching indeed can take a few seconds in 48 MiB file if it searches over a long distance, which is a bit annoying as it starts searching as you type. Might be good to be able to disable "as you type".
However, I believe there is a limit as to what performance can be achieved and taking some seconds, even 10 to 20 seconds, for searching 46 MiB for a long distance search from the start of the file, is not all that surprising to me. Especially on a ThinkPad T520 with Sandybridge which is a tad bit dated meanwhile. Is Notepad++ faster with that? Searches for shorter distances are much faster.
-
I am now using Visual Studio Code (https://code.visualstudio.com/ (https://code.visualstudio.com/)). It loads fast, and searches even faster.
My only trouble is that it refuses to Word Wrap the chat log, maybe because of the control codes in /who or something similar. I will find sooner or later...
But the speed is great.
Greetings,
Gonger