Just throwing my two cents in here:
I think that fundamentally there are two ways you can run these contests.
1. Set up constraints on how people make things, but allow any design goal (e.g. 24h and vanilla... at least the way they should be run)
2. Set up constraints on what types of levels and design goals are acceptable, then be as flexible as possible with how the author gets there (this contest)
I guess I just generally think that (2) is flawed. Even though people are told not to make things "for the judges", you have set up the entire premise of the contest on subjective footing. I admit that in any contest, judges are going to have their own biases, but if you group contests along (2) they are more likely to try to fit things to some model of an "ideal level" in their head.
Supposing that the purpose of these contests is to get people to make cool stuff, I would be strongly against any rule or restriction that limits what an author is allowed to aim for. E.g. for this contest, level length, time limit, puzzles, etc. The more you standardize things the more you are going to get "follow the leader", which is the exact opposite of the spirit of kaizo.
I guess that isn't helpful without providing alternative things to judge on, so here are a few:
Clear creative vision : did the level have a clear conceptual theme that was communicated to the player. Did it stick to that theme the whole time.
--good example : (you know what this looks like)
--bad example : a level which is long by virtue of changing gimmicks halfway through. a "generic" level which doesn't present any new ideas.
Coherence : did the level's design consistently match the vision communicated above.
--good example : (you know what this looks like)
--bad example : a fast paced platforming level with an out of place puzzle stuck in the middle. basically anything that feels unintentional
Depth : did the design of the level thoroughly explore the idea it is based on. did it linger on the same idea without adding new variations
--good example : a short punchy level. a long level which iterates on an idea with a lot of depth, without repeating itself
--bad example : a level which repeats itself
Pacing : does the rate at which new ideas are expressed remain relatively consistent
--good example : (you know what this looks like)
--bad example : a giant difficulty spike causing progress to grind to a halt. a level which allocates lots of time to one thought, then barrages the player with a bunch of other things near the end.
--bad example : level which goes from being extremely fast paced to slow based or vice versa, where it is clear that this wasn't done deliberately.
annoying / out of place / tedious parts : subtract x * the number of these from the final score.
I think something like this would be way better suited for basically every contest. But thats just like my opinion man.
