SmartTrim - The first smart virtual memory trimmer

[Jeremy Collake]

Bitsum's new SmartTrim RAM optimization algorithm is the first-ever halfway intelligent RAM optimization suite in the world. The entire concept of RAM optimization has been so tainted by snake-oil that Jeremy even authors an article debunking them! However, while debunking them, we realized that if we mitigated their negative effects, additional virtual memory control could indeed be useful to some users. Thus, SmartTrim was born. We're still testing it, but will have a final version out in the near future (Process Lasso v7.6). https://bitsum.com/processlasso/
(repost) #smarttrim #lasso

[edkiefer]

Bitsum unveils our first early beta with our new SmartTrim functionality. Classic RAM 'optimizers' work, or rather don't work, by forcing *everything* out of RAM. Many do this by simply allocating as much RAM as they can get, until finally they receive no more. Yes, they are that badly written.

Anyway, after they page out ALL virtual memory, the pages actively in use then get immediately paged back into RAM. You suffer a performance hit and screw up the Windows Virtual Memory Manager's efforts to fully leverage your RAM, the fastest storage you have available on your PC.

Our new technology is surgical, precise. It hits only the pigs. We've still got lots more development to finish, but an early beta always helps to put fire under our butts. Support this and other research by buying a Lasso Pro license. https://bitsum.com/processlasso/

Quick question , if you have enough ram where you never get close to say 75% but you do get page faults (Pf) shown in PL .
Would it benefit at all or even do anything ?

[Jeremy Collake]

Depends on usage.

All that 'free' RAM ends up used as a disk cache (also for new apps), that's why Windows is trying to keep some of it available. So, if you needed even *more* free RAM for a disk cache, then maybe it would help.

Make no mistake. This is not ProBalance, which applies to nearly every PC and use case.

[edkiefer]

Yes, in my case I don't notice any change , but I normally use only 30% of ram in use .
I just tried it set only 1hr just as a test .

[DeadHead]

Nice!

Now if you could work something out to force more things into Windows file cache on demand, that would be most interesting (like, forcing a game folder into the ram cache, for example).

[chris635]

I am checking this out on my lap top. How well would this work on a system with pagefile disabled. On my main system, I have 16gb of ram and turn off pagefile.

[BenYeeHua]

Nice!

Now if you could work something out to force more things into Windows file cache on demand, that would be most interesting (like, forcing a game folder into the ram cache, for example).

Except the SuperFetch has the API, I guess it is hard to force it.
(Off-topic)So far I only know Firefox has trying to speed up the boot by creating a fake prefetch file(which only speed up the boot if the file still on Hard Disk), as it will only bring negative effect for them.
https://bugzilla.mozilla.org/show_bug.cgi?id=692255

I am checking this out on my lap top. How well would this work on a system with pagefile disabled. On my main system, I have 16gb of ram and turn off pagefile.

If I am right, the "released" memory from working set will become Modified memory(still on the RAM) until you killed the processes that is released.

[Jeremy Collake]

Before I answer questions, let me say that this algorithm exists almost in framework only right now. I'm still building it out . However, the code will be so solid and safe compared to legacy brain-dead RAM 'optimizers', that I'll be almost a crime not to push out a final version quickly (disabled by default though). That means maybe the next 12 hours. I've got the translations ready, except for those last minute log entries.

Then I'll add more configuration options.

[Jeremy Collake]

Version 7.5.5 beta, building now - so uploading in about an hour - will substantially improve this new SmartTrim algorithm's implementation. It also adds aforementioned log events. I think it will turn out quite well.

Finally, a RAM optimization algorithm that doesn't pile-drive the whole virtual memory subsystem.

Please let me know of any bugs or feedback.

[Jeremy Collake]

Build out. Some tuning does remain.

My Latest Description of this New Feature (Tech Specs to Follow):

Bitsum's new SmartTrim RAM optimization algorithm is the first-ever halfway intelligent RAM optimization suite in the world. The entire concept of RAM optimization has been so tainted by snake-oil that Jeremy even authors an article debunking them! However, while debunking them, we realized that if we mitigated their negative effects, additional virtual memory control could indeed be useful to some users. Thus, SmartTrim was born. We're still testing it, but will have a final version out in the near future (Process Lasso v7.6). https://bitsum.com/processlasso/
(repost) #smarttrim #lasso

[XhenEd]

Yay! I think this will improve my Intel Atom, 2Gig netbook. (I hope so...)

[edkiefer]

Version 7.5.5 beta, building now - so uploading in about an hour - will substantially improve this new SmartTrim algorithm's implementation. It also adds aforementioned log events. I think it will turn out quite well.

Finally, a RAM optimization algorithm that doesn't pile-drive the whole virtual memory subsystem.

Please let me know of any bugs or feedback.

any trick to getting log event to show ?
I have it set to 1hr setting but not seeing log event , what log events do i need enabled ?
Maybe because there nothing to trim, i am only at 30% usage of 8gig's

[DeadHead]

any trick to getting log event to show ?
I have it set to 1hr setting but not seeing log event , what log events do i need enabled ?
Maybe because there nothing to trim, i am only at 30% usage of 8gig's

Seems to work just fine for me, but I've only tested it manually. For example, Firefox working set goes from 524 MB down to 4 MB. After a while though, FF is consuming about the same amount of memory again, so not really sure what to make of it.

This Trim-function reminds me of this other software, which I won't mention here:

"[program x] keeps memory use in check on the system without the memory being pushed to the page file. This in turn keeps the system running smoother."

[edkiefer]

ok, yes, if I trim it manually it works but the 1hr setting doesn't seem to be working .

I manually trimmed it and did lower FF like you mention and did get log events .

[chris635]

ditto for the 30 minute setting.

[chris635]

Manual in deed does work with log's.

[Jeremy Collake]

Yes, there was a malfunction in the scheduling (particularly notifying the SmartTrim thread that the config changed). I was testing this mechanism as I fell asleep, knowing I could verify/fix it when I woke.

There are also a couple unfinished parts of the algorithm, so don't judge it too quick.

I will give a full technical description that will clarify what it does and doesn't do, and for what situations it will be useful. All after I'm done with current coding.

See next beta build (hours). Thanks.

[chris635]

okay

[Jeremy Collake]

Ok .

Remember guys, you are dealing with daily betas, and I'm moving at warp speed. It's code hot off the presses and not 100% done.

With daily beta builds, you have to expect small quirks. That's why they are betas.

I did announce all this too early, but I wanted people to get excited.

The core code is quite beautiful. It's well written, and once we get it through full QA, it'll absolutely set a new standard in this area of optimization. And I don't talk BS, I will describe the algorithm in detail. I *hope* people copy me. That will result in less shitty RAM optimizers in the world.

[chris635]

Do what you need to do  . Looking forward to it, and I will help give feedback when I can.

[BenYeeHua]

Yay! I think this will improve my Intel Atom, 2Gig netbook. (I hope so...)

It depend, if you are moving the code that only run once then don't released from RAM, and you are not facing CPU performance issues, then this should help.(Which is what I want too, getting the useless data into Page File, and put more useful data into RAM)

Of cause if you are using SSD, then the effect that wrong code/memory is getting paged into the Hard Disk will be reduced, as many user can't feel the difference between reading the data from SSD and RAM, except some case.

Seems to work just fine for me, but I've only tested it manually. For example, Firefox working set goes from 524 MB down to 4 MB. After a while though, FF is consuming about the same amount of memory again, so not really sure what to make of it.

Ya, that is normal, as Firefox will get the data back from modified memory when the code is needed to run the website, then the useless or memory leak data will be put into the modified memory or Page File.

[Hotrod]

I have been waiting a very long time to see someone tackle this one in an intelligent manner. I knew it wouldn't be long before Jeremy tried it, as he was so dissatisfied with the existing attempts. He really does have a remarkable brain. Will this eventually replace PL's memory trim function, or will it run as a separate plugin?

[Jeremy Collake]

Thank you . This had been on my mind for a long time for sure.

SmartTrim will entirely replace Lasso's trim function, built into the product w/o any plug-ins.

There are other plans for it as well, but this isn't the place to describe them.

[Jeremy Collake]

Christmas disrupted me a bit, but v7.5.7 beta, building now (<1hr upload), should work well with SmartTrim scheduling and Steam based triggers. I'm still working on them, but in my book, you can't get a beta into the users' hands soon enough.

There is still lots of work to do to go final (v7.6), but we're committed to doing so this week.

[edkiefer]

SmartTrim scheduling is working now properly and logs which processes were trimmed .

[Jeremy Collake]

Great to hear, thanks for the confirmation .

[Jeremy Collake]

Version 7.6 is out. Version 8.0 will be the official SmartTrim version. But, you can use SmartTrim in v7.6.

SmartTrim framework is about 90%. There is a GUI->governor as service communication issue discovered right before release that I'll have to solve. User reported (thanks). Basically, no 'trim now' for users who installed the governor as a service. Sounds simple to fix, but there are larger implications and I need to do it right. Few users of this SmartTrim function will be running the governor as a service anyway. For those that dp: Don't, unless you have a reason. Running it as a service sounds good, but has too many disadvantages over advantages. I may only support it for the Server Edition some day (maybe, unlikely since I know people love this ability).

The algorithm itself is about 30% done, but fully functional and already 100% better than anything else on the market. First thing to do was simply be selective. Be discriminatory, don't just whack everything in memory as classic RAM 'optimizers' do! SnartTrim ignore processes that do NOT need trimming, or should not be trimmed. Why trim a process that doesn't need it, forcing hard page faults (immediately loading the memory back into RAM)? This cuts down on performance degradation from post-trim hard page faults substantially.

These simple actions already tke a RAM optimization algorithm from entrely stupid to potentially useful, under some circumstances. Remember, your 'free' RAM is used as a dsk cache.

I promised tech details, not some magic black box that works miracles. This should entice ya for now. More to come, much more.

Now, after sacrificing my mind and body for Bitsum, I believe we're in good shape, and will be in continued contact with you after I get some rest

[bertie97]

All sounds very good. 
I will see how I get on with a 30 min interval & report back.

[edkiefer]

it working, I have it set to 1hr and on FF it trimmed 2 times, then 3 said nothing to trim .
FF is only thing it trims so far , I haven't tried a lot of app yet though .

Yesterday I was playing BF4 and got a disconnect, which I first thought was maybe trim issue but it didn't trim yet and trimmed while I played .Didn't notice any issue with that .
Will keep an eye on things, so far don't notice any downside yet .

PS my FF went from 1.5gig to 475mb .yes, I got a lot of tabs open .

[BenYeeHua]

PS my FF went from 1.5gig to 475mb .yes, I got a lot of tabs open .

Same, but only when I like to open a lot of Image or News.
Too bad Firefox don't speed up the process for supporting 64-bit after they know about Chrome has released 64-bit stable, which might be useful for heavy users.

And, I guess we can start counting the TRIM count at the Insights now?

----Off-Topic
Also, when you hover the mouse over the Tray icon, why it is showing as "Lasso 7.6", but not "Process Lasso 7.6", for much better information box, or?

[edkiefer]

Same, but only when I like to open a lot of Image or News.
Too bad Firefox don't speed up the process for supporting 64-bit after they know about Chrome has released 64-bit stable, which might be useful for heavy users.

And, I guess we can start counting the TRIM count at the Insights now?

----Off-Topic
Also, when you hover the mouse over the Tray icon, why it is showing as "Lasso 7.6", but not "Process Lasso 7.6", for much better information box, or?

I actually run Palemoon, which is optimized version, it does have 64bit, but i use the 32bit version .
Not sure why it been so high, i think normally i am around 1gig, could of been I watched a lot you-tube vids or something .

[BenYeeHua]

I actually run Palemoon, which is optimized version, it does have 64bit, but i use the 32bit version .
Not sure why it been so high, i think normally i am around 1gig, could of been I watched a lot you-tube vids or something .

Check the about:memory, it should tell you more about it.
YouTube should only affect Flash Player memory usage, as they only enable HTML5 for Nightly build of Firefox only.

[edkiefer]

Yes, I didn't try about:memory , that would show detailed usage .

Ok, feedback on smarttrim . yesterday on time every hr it trimmed what it could, many times nothing to trim but I played some gaming yesterday at time to trim and noticed no log and I don't think it tried though its har to tell .It did trim 20min after, so that timing was not in sync with others .
Day before it trimmed while gaming as i was trying to see if it affects it, need more testing on this but that was only odd thing so far .

[BenYeeHua]

As the game quickly reclaim back the memory that it is using(16.67ms if the game is ticking as 60HZ), so it should not affect too much for the game, while also paged the memory leak into Page File.
(Except they are using their own cache solution and loaded into the game process, but not SuperFetch which cached as Standby and provide better memory priority.)

[edkiefer]

As the game quickly reclaim back the memory that it is using(16.67ms if the game is ticking as 60HZ), so it should not affect too much for the game, while also paged the memory leak into Page File.
(Except they are using their own cache solution and loaded into the game process, but not SuperFetch which cached as Standby and provide better memory priority.)

Yeh, I was just testing it , it did run today while gaming but didn't trim anything off game .
One time it did trim BF4 but that was only one time most its just browser if there anything to trim there .

[BenYeeHua]

Yup, I guess browser is more heavy than Game.

[Jeremy Collake]

It is designed to ignore the process currently in the foreground.

There are other exceptions and criteria as well.

Future builds (v8) will allow specific exclusions and more control. This is just the beginning.

[edkiefer]

It is designed to ignore the process currently in the foreground.

There are other exceptions and criteria as well.

Future builds (v8) will allow specific exclusions and more control. This is just the beginning.

I think it is best to ignore foreground process ,especially if its a game as maps get loaded into memory and next time it goes real fast as long as still in memory or maybe that is superfech at work .

[Jeremy Collake]

Definitely it is best to ignore the foreground process. My algorithm (SmartTrim) isn't based upon the delusion that some marketers have perpetrated: More free RAM == better  (the delusion, to be clear). Another common delusion is that RAM must be defragmented. Yet another is that processes 'leave RAM behind'.

SmartTrim is based around the idea that *sometimes* users prefer *some* memory hogging applications to go ahead and give up their working set.

I give these details out readily, knowing competitors will come and try to copy me. But the thing is -- if they couldn't come up with it themselves, what are the chances they would even duplicate it right? Further, most of these companies don't even care if their algorithms work. Seriously. They just want a button to press, a claim they can make, and a price they can set.

I hope people support Bitsum because we are a no-BS operation run by engineers.

[BenYeeHua]

Then, just provide a setting like ProBalance, "Ignore all foreground process".
Just my opinion.

[Jeremy Collake]

Yes, I'll add such a toggle. I hate to give competitors all the secret ingredients, but like I said - they couldn't do it right even if they knew exactly what to do.

[bertie97]

Smart trim seems to be behaving fine for me.  Surprising how greedy browsers can get.
I think that the system has increased smoothness, may be my imagination of course, but it does seem to feel different.

Hope the new innovation generates some more interest & sales as PL really is a life-saver when it comes to having a nice smooth working environment.  Also hope the ideas aren't ripped-off as PL is still the original & best, & deserves full recognition 

Wow, version 8 is on the horizon, how long has it been since version 1.0?

[edkiefer]

Wow, version 8 is on the horizon, how long has it been since version 1.0?

Version 1.20   released 12-20-2003

I  think I started using version 4.0

[Jeremy Collake]

Thanks for the good reports. I intend to work on the backing web pages and ancillary features ASAP.

I'm not going to set a specific date for v8.0, but it won't be long. Certainly there has been no predictable major version release history to guesstimate by.

[BenYeeHua]

And the ETA is always "when it is ready".

[bertie97]

Version 1.20   released 12-20-2003

I  think I started using version 4.0

2003 - seems almost like yesterday.  Somehow also feels like PL was around before that tho.
Not that I used PL consistently,    I lost it, forgot what it was called, failed to remember how to search for it etc etc then eventually came back to it.  Thankfully.
I don't think I could run a multi core PC without it these days.  The lumpy ride without PL just drives me crazy.
That applies on both my AMD & my Intel stuff.
Good to know of (& experience) the constant improvements after more than a decade.

[XhenEd]

I just want to refer you to the thread since this thread about PL's SmartTrim is linked: http://www.wilderssecurity.com/threads/other software-experimentation.371695/ (http://www.wilderssecurity.com/threads/other software-experimentation.371695/)

The thread is about a memory cleaning program, but it went into a broad discussion of the effectiveness of all memory cleaning programs.

[BenYeeHua]

I don't think I could run a multi core PC without it these days.  The lumpy ride without PL just drives me crazy.
That applies on both my AMD & my Intel stuff.
Good to know of (& experience) the constant improvements after more than a decade.

Who know, maybe Microsoft will become crazy one day, and providing a very good kernel for responsiveness while ignoring all the compatibility issues.

I just want to refer you to the thread since this thread about PL's SmartTrim is linked: http://www.wilderssecurity.com/threads/other software-experimentation.371695/ (http://www.wilderssecurity.com/threads/other software-experimentation.371695/)

The thread is about a memory cleaning program, but it went into a broad discussion of the effectiveness of all memory cleaning programs.

Yup, not a bad thread.

Anyways, I think the issues are
1.Clean/Trim too much processes, which causing difference processes writing/reading the Hard Disk at the same time, nothing for SSD, but worst for HDD
2.Clean/Trim the biggest process, might miss some processes that are memory leaking "slowly"(a issues for small RAM+HDD user), normal for SSD, might worst for HDD.
The effect depend on how the process read the data from virtual memory/HDD, like QQ chat software that read the chat one by one when you are scrolling, or Firefox/Chrome which read/GC the data together

[Jeremy Collake]

I just want to refer you to the thread since this thread about PL's SmartTrim is linked: http://www.wilderssecurity.com/threads/other software-experimentation.371695/ (http://www.wilderssecurity.com/threads/other software-experimentation.371695/)

The thread is about a memory cleaning program, but it went into a broad discussion of the effectiveness of all memory cleaning programs.

Thanks! I'll likely comment there myself when I get some time. First I need to finish the new SmartTrim pages I wanted to have done yesterday.

[Sparky Zelectrict]

How would ignore foreground processes would work on a system that is set to favor background services. That's a setting I always make on my own computer. (control panel->system->advanced system setting->advanced tab->performance settings->advanced tab->best performance of background services, [I think MS may have overused the word "advanced" to compensate for some other shortcoming])

My rationale is that since I use VMware Workstation constantly that it would keep my VMs running more smoothly at little expense to my actual foreground task (especially since my foreground task is often just surfing the web ).

My question is this: Can SmartTrim still tell what my foreground process is even if I've optimized the OS to favor background tasks? Will my setting have any impact on SmartTrim at all?

-----

Looking forward to $igning up for v8. Until Windows can manage processes and memory with the efficiency of *nix I'll be using Process Lasso.