Hi Sparkies,
Quotea) use PECompact and add some code to the loader which decrypts and verifies the license key stored in the registry. I am well versed in C and C++ but I'm a bit worried that making such a loader would take a lot of work, are there some examples I could start with? Is this even possible?
The PECompact Loader SDK is probably your best option, it comes with the source to the default loader.
Although you would need a good understanding of Asm, it's very well commented.
With this you can create whatever custom protection you need for your products, and you can also write your own Codec or Hook plugins to add extra layers of encryption / protection.
There is also a Hook plugin called IsDebuggerPresent available for registered users, which will detect debuggers in many different ways, simply by using one of the Anti-Debug Loaders, or can be called from your own custom Loader.
Quoteb) password protect the executable with PECompact (can I pass down the password as command line argument?) and then have another executable which handles all the registry checks etc and if that checks out ok, launching the password protected executable, passing the password in the command line.
I don't think it's a good idea to use passwords in plain-text ever. For example, if someone were to hook GetCommandLineA then they would have your password. :)
But there are many ways you could do this, such 'master and minion' protections have been around for a long time, and can be effective. ;)
BoB