Banner
C3 Voting has ended! Stay tuned for the results (and Mr. Switchy's fate...)
Views: 958,597,426
Time:
12 users online:  1UPdudes, Ahrion,  Alex_X8, bwal,  Ice Man, MrDeePay, Paithus, Para_0, RanAS,  Segment1Zone2, simon.caio, Souldbminer - Guests: 36 - Bots: 74 Users: 53,079 (2,231 active)
Latest: komugi
Not logged in.
Posts by Doctor No
Doctor No's Profile - Posts by Doctor No
Pages: « 1 2 3 4 5 6 7 »
Now that the contest is officially over, I want to bump this thread for more feedback from everyone.

Based on everything I've seen/heard so far, here's what I have:

- The Olympic Scoring thing, unless I'm just not seeing or hearing people, wasn't really that much of an issue for anyone. Which is good! I liked having 5 judges more than anything, and will try to get that many judges going forward.

- There has been a lot of existential talk around what kaizo is, or what kaizo should be, following the results of the contest. Outside of the general guidelines for how I understood each category, I didn't give any of the judges a charge or guidance; their scores reflect their own, unspun opinions and evaluations. I'm all for meandering philosophical discussions about whether kaizo should be fun/not fun/punishing/technical/short, but I want the community to be able to take the results from KLDC as not just an evaluation of the levels entered, but a statement on what the judges value overall in kaizo levels and hacks.

That means getting the best possible judges and being as hands-off as possible in letting them do their thing. I firmly believe all five of our judges brought that, and they all would be happily considered for judging future kaizo contests. If you feel that other criteria should be considered when getting judges, please let me know here.

- Regarding vanilla vs. chocolate: Be on the lookout for some vanilla challenges from us in between now and 2021. #smrpg{:D}

- LEVEL LENGTH. This was the hottest topic of all for many. For some, levels were way too long. For others, shorter levels were seen to be given preference over longer levels because of the ease of beating them. I hope the results bear out that level length was not a universal boon or burden for anyone, but the call to restrict level length in the future has been sounded.

I'm honestly torn on this. On one hand, we already had each judge spending an average of 52 hours of time streaming their evaluation, which doesn't count the time all of them spent off stream writing their comments and reviewing their scores. If we hope next year's contest is bigger (which I do), that burden on judges just gets bigger, which will both keep people from wanting to judge as well as drag out the judging and results process ever longer.

On the other hand, I hear people who say that this shouldn't be a "Romhack Race Level Design Contest," and while that stings a bit given as I'm staff there as well, I understand the sentiment. People who want to make large, complex levels, and who are prepared to receive the appropriate criticism for their creations, should be allowed to do so.

Before this forum goes into the sunset, does anyone else have any final thoughts or feedback? If you'd rather share them with me directly, please hit me up on Discord or Twitter.

--------------------
I'm not a doctor.

Appreciate the feedback, Naro! A couple responses:

Originally posted by Narogugul
from what ive noticed, the 60 points for 'level design' is pretty much just an excuse to turn the judging into a more subjective judging since theres no guidelines or 'real' standards for this


- I don't think there's a way to judge any creative competition objectively. The ratios I used were directly pulled from previous competitions, kaizo and otherwise, and I'm more than happy to look at them freshly to see if they did the job. That said, the reason we need respected judges is because it ultimately will be up to their subjective opinions as to the final scoring and order. At the end of the day, KLDC is an art show, and it's impossible to truly judge art objectively at high levels.

Originally posted by Narogugul
Hopefully in the future there are at least a couple of -experienced- non streamers as judges too.


- Absolutely, unequivicaly, 100% agree with this point. One of the reasons I wanted Katerpie and Strizer as judges was to bring different perspectives to the judging pool - Katerpie filling the role of experienced non-streamer, and Strizer filling the role as experienced player who was not familiar with or to the members of the community. In hindsight, it would have been wise to have multiple backup judges so that we did not end of with four out of five judges being noted kaizo streamers.

Again, I stand by all of our judges as experts - 3 of the 5 judges are or have previously been moderators for kaizo hacks here (4 if you include Katerpie), GlitchCat plays and clears a sizable majority of the kaizo hacks submitted to this site, and Strizer is a professional QA tester who has an extensive skillset and knowledge of difficult platformers across the board. The point stands, however, that having the voice of someone who is not a streamer is crucial and will definitely be taken into strong focus when next year's judges are selected.

Originally posted by Narogugul
Point is the streaming community has a very specific view on how hacks should be played: Blind, no savestates, no spoilers, etc. This unfortunately seems to be the recipe for uncreativity.


- What I said earlier about the "philosophical" debates about kaizo are right here. What is the goal of playing a kaizo level? What is the goal of making a kaizo level?

Is it my job to settle that debate? I hope not, but it is my job to set the rules for the contest and select the judges, so my opinion on this matters a great deal. Because of that, I'm trying my very best to keep an open mind and ensure that I do not allow the contest to become about what I think kaizo is or isn't.

I would say this, however: please avoid painting any side of our community with sweeping statements when engaging this debate. I can assure you that many people who are part of the Twitch community of kaizo players and streamers are having this exact debate within their own spaces, and some of the most punishing hacks on this site are championed by some of the biggest Twitch streamers in our community. Similarly, I know there are people on this community who wish kaizo hacks were more accessible to new players.

Both sides are guilty of ivory-towering the concept of Kaizo. An explicit goal I set when asked to host this contest was, if not to tear those towers down, at least build bridges of commonality and celebration between them.

--------------------
I'm not a doctor.

Originally posted by idol
hire naro to make videos next time. pay him the big bucks. he was a great help.


Expect my budget request on your virtual desk sometime in the whenever from now.

But seriously, thank you Naro for posting your clear vids. I know ChosenTwo made some as well.

--------------------
I'm not a doctor.

Originally posted by NaroGugul
But its 'confusing' (in absence of a better word) to shape any kind of deeper thinking if the views on what the contest is about can vary so much.


Originally posted by GbreezeSunset
These are all valid ways to make Kaizo, but the Twitch community needs to understand that the long levels, puzzly and cryptic levels, and otherwise unpopular-on-twitch design tropes are an equally valid aspect of Kaizo design. My goal is not to change the perspective of others players or attempt to convince someone that their way of designing or their interpretation of Kaizo is somehow invalid or lesser than another. My goal is to have a contest and group of judges that will allow for all aspects of the Kaizo community to be appreciated.



These two things are what my biggest takeaway from this year's contest will be, apart from the structural things that I know can be improved on (judge diversity and consistency in communicating results/criteria, me rambling into a microphone more than is necessary, etc.).

I'm not sure any other contest here fundamentally questions its existence like KLDC has led us to now, but I also think that our contest is uniquely open to such observation. My job isn't to sit on a mountaintop and dictate the Ten Commandments of Kaizo to the community; I study kaizo but I don't play it and barely create it.

At the end of the day, it's up to the whole community - players, casual creators, kaizo auteurs, and everyone in between - to determine what this contest should recognize and celebrate.

Previous KLDCs have had multiple categories, so maybe that's something to explore - some sort of delineation that allows for people to enter whatever kind of level they wish and have it compared to its peers. I don't know, which is why I'm asking. I'm not good enough at anything kaizo related to dictate the terms of what makes a good level or not beyond my own personal taste. If the community can come to a consensus on what we want KLDC to celebrate, I'm happy to take it and put in the effort to throw the biggest party I can.

--------------------
I'm not a doctor.

Ok, time out.

Please stop with the individual sniping - I asked for overall feedback, not comments on any specific judge, and this is veering away from "debate over the point of the contest" to individual accusation and justification over scores, which this is not meant to be.

I think it's fair to say that I've heard the statements on judge diversity and a need for more formalized judging . No need to beat that into the ground, it has been expressed well and in ways that make it actionable. If you want to use that topic to attack or accuse judges of malpractice, do so privately and respectfully, but not here.

If anyone has other points of feedback for me, please keep them coming here.

--------------------
I'm not a doctor.

Originally posted by GbreezeSunset
Second, the second elephant: long levels. I think a good compromise would be this: make sure at least 1 judge next time doesn't mind long levels at all (even better: actively likes them), but institute a rule where levels can't be over X minutes for a full perfect run to avoid super long levels (this X number can be discussed of course). I know that pyro's rather long level scored really high, so it's obviously not impossible, but the length issue is worth mentioning regardless.


The X-minute clear limit was something on my mind. From talking to judges and reviewing comments, one of the issues with "level length" is really "forgiveness of checkpoints." You can have a 10-minute long clear level that is a pleasure to play through because it's very forgiving with checkpoints, and you can have a 2-minute clear level that is toothpicks under your fingernails because it's just 15 yumps in a row before your first checkpoint.

My interpretation, and this is reinforced after watching some blind playthroughs of past KLDC winners and standouts, is the levels that scored well because they balanced their length with checkpoints. Not saying that you need a checkpoint after every trick, because that's just essentially save-stating, but giving the player a chance to breathe after doing something impressive and moving on to a new section with new techniques.

(Also, for what it's worth after watching the results VOD, I really want to find a way to celebrate how good these levels are without people fixating on the ordinals. ALL of these levels deserve to be remembered and celebrated.)

--------------------
I'm not a doctor.

(restricted)
I see two different, but related, categories here to discuss.

The definition and weight of non-gameplay elements when evaluating levels: This is something that is a contest-by-contest evaluation, but in general, we should view levels holistically and not judge aspects of a level independent of each other. If a level that is visually beautiful and built from the ground up but has mediocre gameplay were paired off against a level with modest (not poor) graphics and superior gameplay, I believe the level with superior gameplay should win.

That said, if two levels are equal with each other in terms of gameplay, then I would consider the edge to go to the one who used visual, audio and ASM resources to enhance the overall experience of playing the level. This doesn't mean custom - a level can utilize vanilla resources and color palettes incredibly efficiently. But aesthetics should always be secondary to the gameplay experience. Whether or not a level has a lot of work done on it or not should be noted, but it should be secondary to the affect that work has on the overall quality of the level.

As GBreeze noted, this isn't a definitive answer to any question, but rather my own opinion of the matter. That said, I've certainly heard this a lot from KLDC feedback, and I can only say it's something I plan on heavily considering when creating that competitions ruleset.

Developing a consistent judging and scoring process: Ultimately what we're talking about here is writing some form of standardized scoring system set either by the host or by the site. A set "how to judge", if you will.

This is, in my mind, a separate discussion.

--------------------
I'm not a doctor.

All - I appreciate all of the feedback, and it seems that certain themes have definitely rung true.

Please know that feedback on personal opinions on what should have won or not is fine, but isn't going to help me much beyond giving me data to alter the grading scale. I loved Chroma Castle, and was very happy to see it received the scores it did. Focusing on 1st vs 10th, when the differences are as miniscule as possible, does lead me to consider formats that reward levels in ways beyond ordinals.

Please don't blame the judges for the lack of formality in how they were instructed to judge. That is entirely on me, and is something I plan on significantly improving next go-round. Asserting the judges didn't know what they were doing or weren't making intelligent decisions isn't fair to their experience and expertise, but you can chalk that up to me being an inexperienced host and not them being unaware or ill-informed on what is good or not.

If people have further feedback for me, I'm happy to take it directly in DMs - I know I've reached out to continue these talks already, and I hope you all know I'm working in good faith.

Closing this thread in the meantime to move results-based discussion back into the other threads.

--------------------
I'm not a doctor.

SUPPORT for Claustrophobia, Invictus, and Grand Poo World 2. All three stand independently of their creators as examples of high-quality chocolate kaizo game design.

In a similar mindset, I would also nominate Quickie World. Universally one of the most recommended "first" hacks, it's a great example of creative vanilla kaizo level design while still being an engaging and rewarding experience to play through.

--------------------
I'm not a doctor.

I appreciate the sentiment, but this misses the mark for me in a couple very important ways.

First, speedrunning kaizo hacks is just one of many ways to enjoy playing them - for many people (myself included), speedrunning is a massive time commitment that isn't the most enjoyable way to play. This formula punishes players who don't want to commit to the grind necessary to learn multiple speedruns. By ranking players by their participation in SRC and speedrunning as a whole, and by stating your wish is to prominently display this leaderboard, you are saying that speedrunning is the way to quantify a player's skill. It's certainly one way, but there are so many other ways to enjoy and play kaizo that putting so much emphasis on speedrunning greatly diminishes that.

Second, and I do not say this with the thought that you were moving this way, this seems like a vehicle to assign a metric to compare and contrast players by equating skill with prevalence on SRC. Going down your list, I see some of those I would consider the best players in the world up and down every position, and many of them either don't care about speedrunning hacks or don't care about submitting times. I could see this system motivating some of them to submit their times, but not for healthy reasons.

One of the great things about the kaizo community is that, while game does recognize game, the amount of "rivalries" or unhealthy competition is pleasantly small. Our rivals are the games themselves, not each other. Having some sort of ELO or RPI for kaizo players is a really neat idea, and I like that concept in principle. I also really appreciate the amount of work you clearly put in to make this. Unfortunately, "ranking" players in the way you did can breed some massive toxicity in ways that aren't healthy to the community. I'd love if more people speedran kaizo hacks, but I don't want to grow the community at the expense of those already in it.

--------------------
I'm not a doctor.

(restricted)
(restricted)
(restricted)
(restricted)
(restricted)
(restricted)
(restricted)
(restricted)
(restricted)
Pages: « 1 2 3 4 5 6 7 »
Doctor No's Profile - Posts by Doctor No

The purpose of this site is not to distribute copyrighted material, but to honor one of our favourite games.

Copyright © 2005 - 2022 - SMW Central
Legal Information - Privacy Policy - Link To Us


Menu

Follow Us On

  • YouTube
  • Twitch
  • Twitter

Affiliates

  • Super Mario Bros. X Community
  • ROMhacking.net
  • Mario Fan Games Galaxy
  • sm64romhacks