Dramatically Improved Logging Performance (v6.0.3.9 beta+)

Started by Jeremy Collake, April 18, 2013, 06:29:07 AM

Previous topic - Next topic

Jeremy Collake

In builds v6.0.3.9 beta, the log is substantially faster. CPU use by the GUI in particular will be substantially decreased on systems with large or highly active logs.

Flat Log File Format

A flat file format is one that has static sized fields. This allows Process Lasso to write out, or read in, the log entries without parsing. It also allows for easy calculation of offsets to specific entries in the log.

Log File Rotation

Log file rotation means that when a log file fills, it is renamed for archiving, and a new log file is begun. This way the user can archive any number of log entries. Pruning is also substantially faster since files are just renamed.

The log files will be rotated like so:

prolasso.log  (current log)
prolasso.log.1  (last log)
prolasso.log.2  (2nd to last)
...

Export to CSV

An 'Export LOG to CSV' is being added as part of these changes.


**WARNING: Refinements continue. Bugs and incomplete features may be found in the current BETA!**
Software Engineer. Bitsum LLC.

hanemach_gt

How about compressing old log files? Unless you plan to add auto-delete feature.
<img src="[url="http://imageshack.com/a/img913/7827/On37F9.gif"]http://imageshack.com/a/img913/7827/On37F9.gif[/url]"/>

Michael Plant

I posted this (ill-advisedly?) in the beta section, but will repeat here: today's beta builds result in Process Lasso Pro throwing this error after installation: WARNING: Core util display not supported (1) on my Windows 8 64-bit machine. 6.0.3.4 downloaded, installed, and running fine.

BenYeeHua

When ProBalance is restraint the process repeat, like 1 second restraint and wait 5 second to restored, I found that the Actions log is empty for a few seconds when opening the GUI.
And also create a new entries causing the Actions log can't be scroll and showing it correctly.

Did it happen after this feature? :)
---
Some correction, the Actions log is blank for sometimes I open the GUI, and after it has been showed up, pulling the scroll bar up and down will causing the hard disk light light up for 5 secs and higher CPU usage for GUI.

Jeremy Collake

I don't intend to compress the old files, but that could be automated with an external script or utility. File system compression would also work fine on them.

Yes, they are optionally pruned.

The core utilization view error message was to help diagnose another report.

All this will be cleaned up by the time it goes final.
Software Engineer. Bitsum LLC.

BenYeeHua

Quote from: Support on April 18, 2013, 10:38:27 PM
I don't intend to compress the old files, there's not much reason to. Yes, they are optionally pruned. Everything is still under work and testing.

The core utilization view error message was to help diagnose another report.

All this will be cleaned up by the time it goes final.
And it seen like there are some issue for it, as it is beta.
High CPU usage for getting the data shown in Actions Logs.(don't has cache for it?)
I am also sawing the same entries x5 times, while the prolasso.log showing normal(only recorded 1 times).
1.Clear the log by right click on "actions log"
2.Shut the Process Lasso down
3.Open Process Lasso

Then you will see the same entries repeat showing 5 times by scrolling it at the "actions log". ;D

Jeremy Collake

Thanks guys! The situation is this:

1. The diagnostic message boxes in the prior beta are gone. So, those with that issue are ok now.
2. The new logging system is still under active development. There are LOTS of things that aren't finished, need optimized, tested, etc.. I have to go through all the code and carefully test every portion. I have no doubt some performance issues can be seen right now. As I complete the code, things will improve, and the end result will be a far superior offering. In fact, this is just the first step of larger logging changes that I'll introduce later.
Software Engineer. Bitsum LLC.

BenYeeHua

Quote from: Support on April 19, 2013, 10:52:42 AM
I apologize for previous frustration. The situation is this:

1. The diagnostic message boxes in the prior beta are gone. So, those with that issue are ok now.
2. The new logging system is still under active development. There are LOTS of things that aren't finished, need optimized, tested, etc.. I have to go through all the code and carefully test every portion. I have no doubt some performance issues can be seen right now. As I complete the code, things will improve, and the end result will be a far superior offering. In fact, this is just the first step of larger logging changes that I'll introduce later.
Ya, I just report to let you know that, and provide a step to testing it manually.
And I know you will make it nearly prefect optimized when pushing it to release version. :)

Jeremy Collake

I'm getting there, slowly. So much work to make sure things are done RIGHT :). It will probably be a while more before I go final with this new version. The product is definitely evolving to a state better than it ever has been. Soon there will be more visible changes.
Software Engineer. Bitsum LLC.

BenYeeHua

Slowly better than nothing, and I has saw so many software came out with a short time, and also bug. :)

gman68w

wasn't there an option to limit the number of repeat log entries (ex. 'X process; priority temporarily lowered' and 'priority restored' entries alternating with each other five or six times).

Jeremy Collake

Quote from: gman68w on April 24, 2013, 10:29:10 AM
wasn't there an option to limit the number of repeat log entries (ex. 'X process; priority temporarily lowered' and 'priority restored' entries alternating with each other five or six times).

No, but you can turn off the priority restored log messages (see Log Options in the General Settings of Options menu). Consolidating such series of log events is something I'm considering.
Software Engineer. Bitsum LLC.

edkiefer

Make sure "log all executed processes " is turned off . I found it on by default on last beta's and no need to have all executed processes show .

that will shorten up log actions a lot .
Bitsum QA Engineer

Jeremy Collake

Yea, it defaults to ON for now, though I believe a reset of the configuration file is occurring, else the option wouldn't change. I really have to test the upgrade and migration of the INI file.
Software Engineer. Bitsum LLC.