RandyHale

I originally posted this topic in the WinSock area by accident. I'm reposting here, and appologizing for the duplicate post.

------

We have a command and control product that is accessed by third party software to monitor activity and/or send directives. A major component of the linkage is a DLL that exposes status values as data elements in a shared memory program section. This mechanism is very fast (millisecond response times are necessary) and is extensible, in that we can add new variables and new functions to the DLL without affecting older third party components. We have reached an impasse with Vista, which partitions session memory. Our product, which runs as a service, can no longer communicate with third party controls that run on the desktop. We can fix our controls to use a different mechanism, but we must continue to support older third party controls.

Our current solution is to cause the DLL to create a "Global\" file mapping and update the fixup vectors within the EXE so that they point at the file mapping and not the DLL shared memory. It's a tricky process, as we have to trap the WinMainCRTStartup call and then essentially perform the same functions as would be performed in a /DELAYLOAD helper function. While there isn't all that much code being executed to do this, it really is more complicated than I would like.

A much simpler mechanism would entail deleting the virtual pages encompassing the shared memory section within the DLL and mapping the "Global\" memory into that spot. Those virtual addresses are protected though, and VirtualFree doesn't have any affect on them. I have tried using EDITBIN to remove the section from the DLL, which does prevent anything from being loaded into that address range. The pages are still committed to the image though, and MapViewOfFileEx won't permit use of that address range.

Does anybody know how to remove a section from an executable EDITBIN permits you to mark it as not being used, but doesn't get rid of it. I've tried to REBASE the result, but that doesn't remove the section either. I'm thinking about creating my own bin-editor, but then I'm back to complicated solutions.

Any suggestions would be welcome.



Re: General Windows Vista Development Issues DLL Shared Memory in Vista

RandyHale

Well, it doesn't look like anybody else has had to deal with this. I¡¯m forging ahead regardless.

I've done some research and found tools called PE Packers. These are programs that compress (or encrypt) executables to reduce footprint on disk (or prevent reverse engineering). They are also used in viruses to disguise the payload from anti-virus software. This latter usage could be an issue, as some anti-virus programs block PE Packed executables as a general rule.

PE Packed modules operate by interrupting the image load process. They decrypt/unpack the module, restore the module¡¯s import, export, and relocation tables, and then let the image load complete. I'm not particularly interested in packing or encrypting our DLL, however it would seem that mapping "Global\" shared memory at load time would be an ideal solution to the DLL shared memory limitation of Vista. The export restore function of the PE un-packer could provide pointers to the shared memory and be imported into modules using the DLL.

I'll be investigating this option further. If anybody has worked with PE Packers before, I would appreciate any comments on the feasibility of this concept.