One area a lot of work has gone into is reducing heap fragmentation. [Sounds like you might be alluding to this in the paragraph before “Average Joe Six Pack”?] The basic symptom of the problem is finding that VM usage can significantly exceed the actual number of bytes that are allocated — you might not have much memory allocated, but because of the way that memory is distributed across pages it pulls in a lot more with it.

Changing the way memory is used (eg, switching to stack allocations, using arenas, avoiding small ephemeral allocations, altering allocation patterns, etc) helps avoid fragmentation, and thus overall memory consumption/paging.