It's not a dumb question at all.
Ultimately, it all comes down to the amount of VRAM the SNES has, and the way it is laid out. VRAM is a total of 64 kb, but remember that it has to be shared by BG1/2/3 GFX, sprite GFX, and BG1/2/3 tilemaps (which specify which block is where onscreen). (the sprite equivalent of tilemaps, OAM, is not stored in VRAM but in a separate location.)
Now, you have 4 sprite GFX slots, each of which is 4kb, which already eats 16kb.
The 4 layer 3 GFX files that are always loaded each take 2kb (since layer 3 uses half as many colors as layer 1/2), for a total of 8kb.
You also have the layer 1/2 tilemaps, 4kb each, and the layer 3 tilemap, 8kb, for a total of 16kb.
If you add this up, you have already used 40kb of the 64kb available. The remaining 24 kb is enough for exactly 6 normal GFX files. So from a purely hardware perspective, you can't load more than 6 GFX files at once, which is exactly how many LM allows.
(there are ways to reclaim some of that VRAM for FG graphics, such as shrinking the tilemaps, but AFAIK nobody has made patches to use those, since they either require significant changes to the SMW engine or have some other tricky limitations.)
Reading your question one more time, I see that you asked why the file size is 4kb. That's just an arbitrary amount that SMW devs chose since it was large enough to not get annoying to manage but small enough to allow the tileset mixing that SMW has. I'm pretty sure 2kb GFX files would work just as well. But the point is, you just can't load more than 24kb of FG/BG GFX at once.
fakeedit: damn ninjas