A little success story!

Started by DeadHead, March 31, 2011, 01:39:21 PM

Previous topic - Next topic

DeadHead

Hi all!

I decided to share this little story with all of you, and especially to the developer of the excellent software Process Lasso.

I work a lot with large archives, and extracting these has always been stressful to my system, not so much on the cpu, but heavy i/o loads always seem to make Windows stall to a near standstill for me. So I was searching the web for means to control i/o priority of running processes on my system, and that's when I found out about this program.

This feature alone means so much to me, and to finally being able to control it the way I want to, is reason enough to buy this program! :)

Still running lots of background extraction tasks, the difference is now I can continue to use my computer without the constant slow downs I had to tackle before.

I'm happy to be sending some well deserved money your way - keep up the good work! :)
Windows 10 Pro 64 (swedish) || Xeon 5650 @ +4 GHz || 24 gig ram || R9280 Toxic

edkiefer

Sounds good, I assume your on Vista/Win7 as I think those are only OS with support for I/O priority settings .
Do you have to manually set values or ProBalance takes care of it .

I think this would help me as I use TortoiseSVN and when I do a "update" on my project folder the HD is reading/writing and it does take a little longer for things to open when it does this . But I am on XP so that dampers that .
Bitsum QA Engineer

DeadHead

Yes, my machine runs Win7 64 bit, and the disk in question is an external WD USB harddisk.

I had to manually set the i/o priority, since it seems high i/o load won't trigger a restraint afaik? Not sure I would like it to anyway, I'm quite happy with setting this manually! :)

Perhaps there's some other way to get the same result with PL, I'm not sure.
Windows 10 Pro 64 (swedish) || Xeon 5650 @ +4 GHz || 24 gig ram || R9280 Toxic

edkiefer

Quote from: DeadHead on April 04, 2011, 12:02:03 PM
Yes, my machine runs Win7 64 bit, and the disk in question is an external WD USB harddisk.

I had to manually set the i/o priority, since it seems high i/o load won't trigger a restraint afaik? Not sure I would like it to anyway, I'm quite happy with setting this manually! :)

Perhaps there's some other way to get the same result with PL, I'm not sure.

Ahh, I see your on a external drive . Do you think you notice big improvement because your on USB transfer limit compared to if it was a SATA , internal drive ?
Anyway sounds good at least you can tweak it if you need too .
Bitsum QA Engineer

Jeremy Collake

#4
I completely missed this thread, much to my embarrasment. The last 2 weeks have been tough on me. My health has not been good at all, though I didn't want to reveal that and get anyone worried. I'm feeling better now, and am back on the ball.

Thanks for your purchase, your support is critical.

I/O priority changes in Vista+ are still a bit experimental. Honestly, I am not certain just how well they'll help things. In theory, they should, but in practice I've so far been disappointed by NT6+ I/O priorities. That said, I haven't done my performance work with them yet, so more research need be done.

It is true that ProBalance does not automatically act when I/O is high, though often high CPU utilization is often correlated with high I/O. Therefore, I have at least added that new feature in the beta to let I/O priorities at least be adjusted when ProBalance restraints occur. Of course, there are also the default I/O priorities. A bug in the current beta won't cause it to be immediately applied to existing instances (fixing that now), but new instances will adopt that I/O priority.

Some warnings: In NT6 (Vista) only the Low and Normal I/O priorities are officially supported. The extent to which they ignore the other I/O priorities in NT6.1 (Win7) is unknown at this time, which is the only reason I didn't disable it. I didn't want to later learn they expanded support for the other priorities in NT6 SP1 or NT6.1. When I determine for sure that the others are not in use, I'll disable them - as otherwise it would be quite confusing. So, remember -- ONLY use VERY LOW and NORMAL right now. That's the only two things you can count on working. The others may or may not have any effect.

Current beta memory leak: I've noticed a VERY small memory leak in the current beta. While it will take weeks or months of continued running on most systems to build to any problematic level of consumption, I still wanted to warn users of this discovery made yesterday in QA. It immediately becomes the top priority now, as I can't have memory leaks ;o. This does NOT exist in the last final/stable build.

As for the quite likely unrelated question about SATA/eSATA/USB(2?). My observations of USB 2 are that it isn't bad, though the bus CAN become overwhelmed if you have a lot of devices on it. USB3 may improve things. eSATA doesn't have this issue as it is faster and dedicated to a single device per port. If the USB bus is somehow the problem, then I/O priorities would not be of much assistance I don't think. I'd recommend moving 'real-time' input devices like keyboards and mice to a different USB controller perhaps. Their commands pre-empt those of other commands on the USB bus, to provide real-time response.

Thanks for the support and I hope to have a much improved new beta out very soon!

Software Engineer. Bitsum LLC.

DeadHead

I never thought of the fact that the usb bus could become overwhelmed because of many devices connected to it, but that may be the case for me? Besides the harddisk, I have my mouse, keyboard and two flashmemory (readyboost) sticks connected to the usb ports on the back.

The extraction software I use is WinRar, and what I did was to set i/o priority to very low for this process. Extraction most likely takes a bit longer, but to me it's worth it to have a responsive os while the extraction takes place. Besides the longer extraction times, I've seen no other "bad" issues from this change so far.

I like the idea behind this software, and my experience so far has been a very positive one, so I went ahead and purchased the Process Lasso Pro Entire Home Bundle, so I can install it on my four computers here.

Good to see you're feeling better, and looking forward to the next version! :)
Windows 10 Pro 64 (swedish) || Xeon 5650 @ +4 GHz || 24 gig ram || R9280 Toxic

Jeremy Collake

#6
Quote from: DeadHead on April 08, 2011, 06:59:21 AM
I never thought of the fact that the usb bus could become overwhelmed because of many devices connected to it, but that may be the case for me? Besides the harddisk, I have my mouse, keyboard and two flashmemory (readyboost) sticks connected to the usb ports on the back.

The extraction software I use is WinRar, and what I did was to set i/o priority to very low for this process. Extraction most likely takes a bit longer, but to me it's worth it to have a responsive os while the extraction takes place. Besides the longer extraction times, I've seen no other "bad" issues from this change so far.

I like the idea behind this software, and my experience so far has been a very positive one, so I went ahead and purchased the Process Lasso Pro Entire Home Bundle, so I can install it on my four computers here.

Good to see you're feeling better, and looking forward to the next version! :)

Thanks for your purchase and for the well-wishes. I am not feeling 'right' yet, constantly got this sick feeling that keeps me from eating. I've lost like 10 pounds in the last month, which is no good since I'm already skinny ;o. It's all good though, I'm only 32 so likely to be fine. I knew I couldn't keep going like I was, at some point a 'break' was inevitable.

I should note a couple things. For WinRAR the I/O priority definitely does help. I've seen it myself while extracting large archives. That said, the same thing can be accomplished by clicking the 'background' button during extraction. Still, better to keep it at Very Low, I agree with you on that. There should be no negative effects other than it taking a back-seat to other I/O.

USB bus: I can't say for sure at what point having so many devices really interferes with performance. However, I can say that you can notice BIG performance changes if you swap it to another USB controller or directly to the PC instead of through a hub. I wouldn't have thought so had I not repeatedly seen this myself. Transfer rate to a thumbstick, for instance, can be doubled on my PC if I move it to a secondary USB controller on the PC rather than connect it to a USB hub that has a bunch of other devices on it. The USB protocol is pretty well designed, so doesn't do THAT bad, but still... the maximum bandwidth IS *SHARED* between all devices, and those high-priority input devices absolutely can cause microsecond interferences in data transfer. USB 3 may improve things substantially. Unfortunately, there aren't that many USB 3 devices out yet. For hard drives, eSata is the way to go. Myself, I've got one of those external hard drive bays. It supports both USB and eSATA. Using eSATA I can run my primary OS off the hard drive mounted in the external 'bay'. It really runs great, and as good or close to as good as internal SATA would offer. These aren't expensive. If you don't have eSATA on your motherboard, you can buy an eSATA card or get a SATA->eSATA cable and just run that thing out a hole in the back of the PC. Firewire would also likely perform better than USB2 for hard drives, though I never did really play with it much.

Software Engineer. Bitsum LLC.

Jeremy Collake

I should have also mentioned above the PCI bus, though we are getting farther and farther out of any topic ;). This too can become overwhelmed. The PCI Latency setting may allow to you to help force devices to give up the PCI bus when they are monopolizing it. I mention because the USB devices end up on the PCI bus I believe, meaning that's the next point of possible problems. As for things like HyperTransport or PCIX, they help A LOT with I/O interconnect efficiency and bandwidth. HyperTransport supports the PCI bus protocol and is a hell of a lot faster than the old PCI busses that were at 33mhz (or was it 66mhz, I forget). That said, your USB controllers most likely get shot on the old PCI bus running at its legacy speed. I need to research this to confirm this ... So take this paragraph for what you will. Hardware isn't my native territory ;).
Software Engineer. Bitsum LLC.

edkiefer

yup, PCI was 33mhz , ISA 8mhz , AGP (graphic 66mhz ) and PCI-e multi-lane 100mhz ? , I think .
Bitsum QA Engineer

Jeremy Collake

#9
Quote from: edkiefer on April 08, 2011, 07:53:45 PM
yup, PCI was 33mhz , ISA 8mhz , AGP (graphic 66mhz ) and PCI-e multi-lane 100mhz ? , I think .

EDIT: A quick look shows that it should be put directly on HyperTransport (which like I said does PCI) for AMD chipsets, and the QUickPath I/O controller for Intel chipsets. It seems the HyperTransport I/O controller has its own USB ports, so they get directly connected to the high speed bus. Of course, this may vary between chipsets, and USB port expansion cards are put on whatever bus the slot is on (e.g. PCI or PCIX).

AMD HyperTransport info: http://www.broadcom.com/products/HT-1100
Intel QuickPath info: ... somewhere ;p
Software Engineer. Bitsum LLC.

petrossa

Quote from: jeremy.collake on April 07, 2011, 06:37:44 PM

As for the quite likely unrelated question about SATA/eSATA/USB(2?). My observations of USB 2 are that it isn't bad, though the bus CAN become overwhelmed if you have a lot of devices on it. USB3 may improve things. eSATA doesn't have this issue as it is faster and dedicated to a single device per port. If the USB bus is somehow the problem, then I/O priorities would not be of much assistance I don't think. I'd recommend moving 'real-time' input devices like keyboards and mice to a different USB controller perhaps. Their commands pre-empt those of other commands on the USB bus, to provide real-time response.

Thanks for the support and I hope to have a much improved new beta out very soon!

Sorry to hear about your health. Best wishes.

I do have about 10 usb devices and PL Management Console all beta's kept on crashing at startup. I added in TaskScheduler a delay of  a couple of minutes. No more crashes since days now.

Jeremy Collake

Quote from: petrossa on April 11, 2011, 06:22:21 AM
Sorry to hear about your health. Best wishes.

I do have about 10 usb devices and PL Management Console all beta's kept on crashing at startup. I added in TaskScheduler a delay of  a couple of minutes. No more crashes since days now.

Thank you. I think it has just been all the pressure and stress. My problems are large, but I'll leave the whining at that ;).

I have thought more about this issue and tested Process Lasso in environments of high process creation and destruction. I have determined it isn't the process creation and destruction causing the crash, but rather the high CPU contention while Process Lasso is starting. Most likely the high CPU contention causes a race condition between two threads in Process Lasso to manifest itself into a crash. I am therefore looking for this race condition. This will remain my top priority until fixed. While you can restart Process Lasso, it is still very annoying and completely unacceptable. I endeavor to do better and appreciate your patience ;)
Software Engineer. Bitsum LLC.

DeadHead

Another scenario where PL absolutely shines, is when doing heavy multitasking. Atm I'm running two browsers (ff & ie), downloading files from the net, converting videofiles to dvd (ConvertXtoDVD), a number of explorer windows open, running a virtual machine (VirtualBox running Ubuntu), streaming audio from a radiochannel on the net, working in a sign designing software and running remote sessions to my work through TeamViewer.

Not a single hiccup - the computer's running smooth all the time. Watching the graph in PL i see ProBalance kicking in on running processes when needed, assuring a nice working experience! Love this program! ;D
Windows 10 Pro 64 (swedish) || Xeon 5650 @ +4 GHz || 24 gig ram || R9280 Toxic