Well, it turns out that people use JPS on Chromebooks
as well. Granted, a 2GB hack may be difficult to use there… But people have always surprised me: I did not expect the Chromebook use case in the first place.
In general, there are enough people who write little tools that always assume the happy path: everything fits in memory, small N so O(N³) doesn't matter, no edge case — and this is a valid space to experiment in — that there is no need for me to do the same. So yes, I first design in general my programs for scale, and then complexify with optimizations (replacing small blobs with arrays as you described is one I envision) as necessary to support use cases encountered in the wild. Knowing what scale to design for is not something easy, and there are always limitations, but I need to be aware of the next scale limitation I might encounter and have some idea ahead of time of how I could lift it, if it is possible, even if I don't plan to do so unless forced to.
As the saying goes, it is not mandatory to be crazy to work here… but it doesn't hurt, either.
Tired of IPS patching with a desktop app? Try JPS