Bitsum Community Forum

General Category => Process Lasso => Topic started by: grahamcopley on May 30, 2011, 12:41:17 PM

Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on May 30, 2011, 12:41:17 PM
Process Lasso attempts to control a process by lowering its processor priority. But in my experience, most PC hang ups and slow downs are caused by excessive I/O and not CPU.

I've noticed that lowering the CPU priority has virtually no effect on high process I/O - but have found that I/O can be controlled by continuously suspending and resuming it (using PsSuspend). The amount of control can be achieved by varying the suspend & resume times.

Why doesn't Process Lasso use a similar technique - as it seems to work?
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: DeadHead on May 30, 2011, 02:39:32 PM
I tend to agree with what you say here, grahamcopley. The reason I found out about Process Lasso in the first place, was when I was searching the net for a way to control i/o priorities in Windows 7. To this day, that's still the no#1 feature of PL in *my* book! If anything, I would very much like to see an extension of ProBalance, to also be able to control i/o activities in a similar way it today controls processes and cpu usage.

My old thread: http://www.bitsum.net/forum/index.php?topic=715.0
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on May 30, 2011, 03:26:41 PM
It does, via the 'hard throttling'. However, this feature is currently undergoing some maintenance, so I'd advise waiting until the next version to utilize it. I have always shoved this feature on the back-burner and really not cared much about it. Further, I slapped warnings all over it. Why? Because this is a very unnatural thing to do to Windows applications. Some applications it simply will not work with. In other cases the results are unpredictable. Windows just wasn't designed to have applications acting like that. It is fine for advanced users who know what they are doing, but I've found that average users are confused by it.

This feature is hidden in the 'Less common actions' submenu of the process context menu (right-click). Like I said, I'd advise waiting until the next version to give a try. I've identified some problems that crept into the implementation and I have been meaning to address them, but keep getting side-tracked by more pressing issues. Give it a try in v5.0.0.22 v5.0.0.23 or later, I will try to have it fixed up by then.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on May 31, 2011, 09:38:02 AM
I am deferring work on the hard throttling again to v5.0.0.23, which MAY be released today. This planned minor update has some other adjustments that require testing, so it is a better time to work on this. The build v5.0.0.22 is being released now and has a bunch of small GUI fixes that require much less testing. Sorry for the delay.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: edkiefer on May 31, 2011, 01:20:28 PM
how much would I/O throttleing or any form of I/O priority would help with performance . I say this based on I/O (Hard-drives) are most likely slowest hardware in a system , wouldn't it just be better to get it over with as fast as possible ?
I guess it could be helpful in that you could do multiple app at one time have keep it down a bit so system is responsive but then it just takes longer for each app to load/finish .

I guess what I am saying is that I generally agree with grahamcopley statement above , for me I generally don't multi-task to many app at one time and with app i have run its the I/O that slows me down mostly . I just upgraded my HD and I notice mark difference in load times (my old 5yr SATA ,crapped out and replaced with WD black edition 32mb cache) .
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: DeadHead on May 31, 2011, 03:27:37 PM
Adjusting i/o priorities (read: lowering them) will afaik slow that process down, making it take longer time to finish whatever i/o activity it's currently working on, but at least for me that's ok, *if* I can continue working with a fully responsive computer while the other process is doing its thing.
The example with large archives and winrar in my other thread mentioned above is a good example of this.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on May 31, 2011, 04:12:40 PM
First, I am working on fixing up the non-recommend hard throttling. It has needed maintenance for a long time. I noticed it didn't behave like it used to during testing a while back. I'm hoping to have that done shortly. I hate that I have left it to grow weeds for so long, but very few people use it, and I'd prefer nobody use it. After all, in Vista+ lowering the priority class to Idle should suffice for most people. This should also lower the priority of the I/O, assuming the calling thread wasn't specifically raised in priority. However, you can also set the I/O priority to Very Low to be safe. That way if anything else needs to do something, it will get the resources it needs. Still, there *will* be some interruption from the lower priority threads because the OS is designed to make sure every thread gets a piece of the pie.

Whether or not I/O prioitization helps or hurts depends on what is doing the I/O, what the serving device is (e.g. hard drive), what else needs to do I/O with the device, and the system buses. In other words, there are a lot of variables. Those who expected the new I/O priotization of Windows Vista to make tremendous improvements in I/O performance and system responsiveness were generally disappointed. There is no quick and easy answer, but let me ramble off a few things that come to mind.

For whatever this is worth ;p... I suppose most of it is obvious.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on June 03, 2011, 02:03:04 PM
I've got the hard throttling mechanism rewritten so that it is operates more effectively and is likely to operate better with a larger number of processes. I'm not sure it is perfect, as I still want to continue fiddling with it, but it will definitely be usable in v5.0.0.23, which I expect to release some time in the next 12 hours.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on June 07, 2011, 06:14:31 AM
On Win XP, the processes that I'm trying to control are background backup applications, which seem to go through periods of extremely high I/O usage rate, causing severe hangs or poor responsiveness. (It does not matter if these processes take much longer to complete, but work much less noticeably.)

I've come across a CPU throttling application called BES (Battle Encoder Shirase). On these problem applications, CPU throttling of 90-99% is necessary to achieve an effective reduction in the I/O usage rates (N.B. The problem applications use very little CPU, even when unthrottled).

I've tried using v5.0.0.25, with hard throttling on the problem processes set to the HIGHEST setting, but this seems to have no effect.
- What percentage of CPU throttling does ProLasso's hard throttling achieve on its HIGHEST setting?
- Can even higher settings be made available, say 90%, 95%, 99%?
- Can you enable wildcards on the throttling list? (even though it might be considered dangerous)
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: DeadHead on June 07, 2011, 01:42:18 PM
Hm! Reading about that application gave me an idea that may or may not be suitable to implement into PL somewhere down the line... systemwide cpu throttling based on current heatlevel of the cpu, user controlled settings. :)
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on June 09, 2011, 10:37:14 PM
Yes, it takes very little CPU to issue I/O requests, so it doesn't surprise me that such high restraint is required. I will perhaps further tweak the settings, as I made them rather unaggressive in the last build. I can easily enable wildcard throttling, I will do that in v5.0.0.28 I suppose... though I may think on it, as it is awful dangerous. The highest level currently only reduces CPU utilization by 50%. I had hoped this was enough to yield control to the rest of the system. I guess not ;o.

System-wide 'hard throttling' isn't something I'd ever do, but I have long considered using additional metrics for ProBalance aggressiveness. However, it wouldn't really help in your case. The hard throttling could have a few other parameters attached to it though, only invoking under certain conditions. I'll give this some thought too.

Thanks! Sorry for the delay, it has been a hectic week.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on June 16, 2011, 05:18:32 AM
In v5.0.0.29 I increased the aggressiveness of the higher throttle levels and increased the spread between the lowest and highest. I may tune them more, but this is a first step.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: DeadHead on June 22, 2011, 03:02:05 PM
Something I've been wondering for quite some time, is *why* it is Windows (XP) many times becomes so unresponsive when there's heavy disk activity going on? What's really the cause for this?

Today at work, I was adding files to a large rar-archive (around 40 gigs of uncompressed data). The source disk as well as the destination disk are both separate physical disks (one actually consists of two disks set to raid 1), with the OS located on yet another physical disk.

I've got my pagefiles set up on all three physical disks, allocated 3 gig/disk (overkill!). My temp folder is located on a separate disk, away from the os disk.

This is a quad-core computer running XP with 3 gig ram. WinRar was handling the compression in the background, priority set to idle, and stil the computer performs rather miserably as long as WinRar is working. To me with my limited knowledge about computers, it seems I've done everything I possibly can with this setup to have this operation run smoothly; the c disk with its own pagefile and most of the software located on it really shouldn't be under such impact from the background compression imho. It's not due to heavy cpu usage; even if I set affinity to use only two out of four cores, it is the disk activity in itself that's the culprit here as far as I can tell.

Anyone care to elborate why Windows (XP, and possibly other operatingsystems as well) behaves like this? This is not an issue on my homecomputer running 64 bit Win 7, after I've set i/o priority to very low with the help of PL.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on June 23, 2011, 07:33:58 PM
It could be a bottleneck in a couple different places. My first recommendation would be to try keeping your page file on a single disk. While spread out over multiple disks is good in theory (at least in some cases), if you have a high sustained I/O load on any one of those disks, it may cause stalls with the whole system since reads and writes to the page file on that disk will be delayed.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on August 15, 2011, 08:48:40 AM
I'm using v5.0.0.38. I believe that the aggressiveness of the higher throttle levels were increased in v5.0.0.29.

However, even the highest level of throttling has no practical effect on I/O intensive tasks.

I'm not even sure that throttling is working at all. During actual throttling, shouldn't throttled processes show a status of "suspended"? - if so they don't any more.

As I indicated before, to be effective (in my case it),  probably 99% throttling will be required. Could you please raise the "Highest" setting to 99%, and publish the CPU throttling % for all the settings - Thanks.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on August 15, 2011, 11:09:33 AM
I could further increase the throttling, though of course this isn't safe. The chances of you seeing the process in suspended state are pretty random, but it won't normally show unless it happens to check while all threads are suspended. Since the suspend/resume cycles are pretty rapid, it isn't surprising it doesn't show as suspended. I have a new update I'm working on to clean up the GUI on Netbooks, where it just looks uglier than normal (to me.. though everything looks ugly on those tiny screens). I'll see what I can do about this as well.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on August 15, 2011, 11:47:25 AM
Thanks
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on August 15, 2011, 01:13:56 PM
Thinking about it as I wake up, I will do as you suggest and increase the highest level until it prevents 99% of CPU cycles in a one second interval. If that doesn't do it, then this method may not help in your case.

EDIT - I'll also see if I can provide some reliable way to double-check a process is being throttled... though you may see it as suspended most of the time after this change in aggressiveness.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on August 17, 2011, 06:19:51 PM
I massively increased the aggressiveness of all levels, except the lowest. We'll see how this does. At the highest level it is actually more than 99% restrained, with a 1 second suspend interval followed by a 10ms resume interval. That may be taking it too far, but the lower levels are more aggressive than before, so they can be used if so. Please let me know how it goes.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on September 02, 2011, 02:25:39 PM
Thanks for making the changes. On my PC the approximate timings are :-
-  Highest -   shows 77 seconds in "Suspended" status followed by 1s in "Running" status   = 99% throttling
-   High    -    shows   9 seconds in "Suspended" status followed by 1s in "Running" status   = 90% throttling

I have found that throttling does not seem to work on SugarSyncManager.exe. This is the main I/O hog that I am trying to tame, but its "Status" never changes from "Running" to "Suspended" - even on the "Highest" setting.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on September 02, 2011, 04:04:01 PM
Quote from: grahamcopley on September 02, 2011, 02:25:39 PM
I have found that throttling does not seem to work on SugarSyncManager.exe. This is the main I/O hog that I am trying to tame, but its "Status" never changes from "Running" to "Suspended" - even on the "Highest" setting.

Hmm, I will have to look into that further. Perhaps I can add some error logging, I've been needing to do so for a while. Of course, it could be that the processes manages to issue a lot of I/O in the little bit of CPU time it has, and those I/O request(s) are being fulfilled regardless of the suspension state of the process. More investigation is needed to know for sure.

Anyway, glad the levels are appropriately tuned now! Thanks for verifying that.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on October 31, 2011, 03:37:13 PM
Quote
Hmm, I will have to look into that further. Perhaps I can add some error logging, I've been needing to do so for a while. Of course, it could be that the processes manages to issue a lot of I/O in the little bit of CPU time it has, and those I/O request(s) are being fulfilled regardless of the suspension state of the process. More investigation is needed to know for sure.

Is there any progress to report in the above area?
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on October 31, 2011, 03:44:14 PM
I assume you've tried the latest version?
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on November 01, 2011, 07:06:37 AM
I'm running 5.1.0.9 beta -  but throttling still does not work on SugarSyncManager.exe (its "Status" never changes from "Running" to "Suspended").

I've tested using "PsSuspend", and this is able to Suspend/Resume the program just fine.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on November 01, 2011, 02:49:44 PM
Quote from: grahamcopley on November 01, 2011, 07:06:37 AM
I'm running 5.1.0.9 beta -  but throttling still does not work on SugarSyncManager.exe (its "Status" never changes from "Running" to "Suspended").

I've tested using "PsSuspend", and this is able to Suspend/Resume the program just fine.

Ok, thanks. This is probably a bug of some sort, probably a safety mechanism that is kicking in. Unlike PsSuspend, Process Lasso's Hard Throttling (to readers: not a recommended feature, and not the same as ProBalance) has certain safety catches. I'm sure PsSuspend 'just does it'. So, I'll investigate this particular process and/or the possible safety catches that it might be triggering.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on January 27, 2012, 09:23:54 AM
I'm using 5.1.0.44 beta. I was wondering if you have you been able to identify why throttling isn't working on certain processes?
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on March 12, 2012, 12:21:37 PM
Please will bitsum.support update progress on this topic?
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on March 12, 2012, 01:01:33 PM
I will escalate this issue. Never having been a big fan of hard throttling, I must admit it got pushed down the agenda. I also wanted to wait for the v6 beta series, which also got pushed back a little. I will look into it as soon as I can for sure.

Thanks.
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: grahamcopley on July 09, 2012, 12:09:42 PM
Please can you confirm whether v6 addresses these throttling issues, as I have not noticed any difference from v5 to v6.0.0.37? (If not, do you have any idea when they might be?).
Title: Re: Process Lasso control method is ineffective on Windows XP
Post by: Jeremy Collake on July 09, 2012, 02:57:27 PM
You know, looking at the changes though, I swear I dunno how this could happen... I can't help but WONDER if it was an awareness issue with your security software. What security software are you running? For example, if the security software considered .37 'unknown', then it might do a super deep scan on every operation, causing that slow down. Where-as by the time you woke up, x64 .38 was 'known', causing it to operate normally.

That is how these cloud security suites operate, FWIW.

I have had BIG conflicts with the guys on the malware working group... mostly over protection of innocent vendors in their battle against malware. Many of them don't give a rat's ass about collateral damage, though do try to fix it after the fact, sometimes. I am not afraid to tell them to go F themselves then they false positive on my shit 5 times in a row. I don't use that language, but I do NOT kiss ass just to get false positives that should have never happened fixed.

Now we have the cloud, were false positives come and go faster than you can track them ;o.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on July 10, 2012, 05:56:04 AM
On Win XP SP Pro SP3, I run several background processes which are I/O hungry at times (e.g. SugarSyncManager.exe). During these periods of high I/O, my PC grinds to a halt.

Using Pro-Balance and/or lowering the Process Priority has no discernible effect on the I/O. I have applied Process Lasso's CPU throttling as a means of control, but it does not seem to be able to apply itself to these processes .

When throttling is working, I expect to see the process "Status" alternate between "Running" and "Suspended" - but, with Process Lasso's CPU throttling, the process never changes to "Suspended". Using a process monitor (e.g. avir) confirms that throttling is not working. (N.B. In the past I tested throttling using "PsSuspend", which worked as expected.)

Thanks for looking into this.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on July 10, 2012, 08:53:56 PM
Quote from: grahamcopley on July 10, 2012, 05:56:04 AM
When throttling is working, I expect to see the process "Status" alternate between "Running" and "Suspended" - but, with Process Lasso's CPU throttling, the process never changes to "Suspended". Using a process monitor (e.g. avir) confirms that throttling is not working. (N.B. In the past I tested throttling using "PsSuspend", which worked as expected.)

Right, you *should* see this, though there are caveats. The algorithm I tried to design to be *safe*, but this is an unsafe operation. Thus, perhaps that is the problem, a safety catch is preventing proper or full throttling of all threads of the process. I will try to get to this in the next week.

Thanks ;)
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on October 12, 2012, 05:28:25 AM
The problem still exists at 6.0.1.63[beta]. I realise that you have been very busy of late, but would appreciate an update on any progress.

Thanks.
Title: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on October 12, 2012, 11:21:18 AM
Did we check to make sure that you are running Process Lasso with elevated rights? Perhaps the failure is a simple as these processes being elevated, and Process Lasso not? This is an option you find during install (now defaults to elevated). You can also change it in 'Options / General Settings ... / Reconfigure the way Process Lasso starts ...'. Please let me know. EDIT: Duh, forget this was XP :o

I had also ALREADY made changes that I had hoped would help your situation otherwise ... but may need to revisit them if the elevation status is OK, but malfunctions continue with the (again unrecommended) 'hard throttling'.

Readers should be aware that hard throttling has *nothing* to do with ProBalance (e.g. priority adjustments). I did explain all this I think, I hope. Anyway, it is a very different thing, and is for hard-core operations only, and is unsafe by its very nature.

Also note that there are some things that Process Lasso just can't fix.. nothing can. I/O is an entirely different problem, especially in XP. In Vista and above there are options (I/O priorities), but not so for XP. Even for Vista and above, I/O issues are very difficult to deal with.
Title: Re: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: grahamcopley on October 12, 2012, 02:03:41 PM
I'm not exactly sure which is the relevant option - but "Manage the processes of ALL users"  is being used.  If this isn't the info you wanted, please let me know and I'll provide it. Thanks again.
Title: Re: Discussion of 'hard throttling' and 'brute force' control of processes in XP
Post by: Jeremy Collake on October 12, 2012, 02:50:22 PM
Nevermind, I forgot you were in XP (duh ;p). I was busy working on code, trying to message at the same time. However, for Vista and above, it is in the Process Lasso GUI, the menu option 'Main / Run Process Lasso with elevated rights' is checked.

So, that is not relevant to you. Sorry about that :o.

I am doing some work today, preparing an important new final, so will take a look at the throttling code again if I get a chance. I just hate to think of this last resort 'advanced' option to be the solution in anything but the rarest of cases. After all, it takes very few CPU cycles to issue I/O requests, as I discovered earlier during development of hard throttling. It just isn't the 'right' way to do things I suppose I'm trying to say. I'd completely kill it off if some people didn't depend on it.