Dustin mentioned Auntie Pixelante's discussion of her frustration with being an IGF judge in comments, and I thought I'd pull it out here, since it's definitely worth a read.
I've been an IGF judge before (not this year though), and she has some cogent points. My take is: Sure, the IGF is flawed, but it's still A Good Thing. And all award systems are flawed; don't get me going on the Nebulas (which perversely encourage log-rolling and politicking among SF writers) or for that matter the Origins Awards (which now have so many categories that you wonder why some games don't win an Origins Award).
Still, the process could doubtless be improved, and I agree that at present the IGF seems to have the flaws of a jury award (relative handful of people deciding on games produces idiosyncratic results) without its virtues (debate and reasoned reflection by knowledgeable people coming to consensus).




















I have no comment on the
I have no comment on the thing itself, only that I'm glad somebody is complaining. It wouldn't be indie if everyone agreed it worked and was polite to each other, right?
More judge comments about the judging process
I consider only fair to put this article as well as well to bring more perspectives about IGF judging process, both good and bad opinion.
http://www.indiegames.com/blog/2010/01/indepth_demystifying_the_igf_j.ht...
It is curious indeed that Auntie Pixelante's opinion appeared at the same day as this indiegames' article :P.
Actually,
...while it's probably too much too ask, the indies should try and compensate for their tendency toward flamewars, rather than embrace it, because there are at least two significant groups of indies who could really use some serious debate of the calm variety. There are others, too; I'm not trying to describe Indie as a whole.
One group is almost all those people who have never made anything professionally (not their fault, of course). Regardless of their fresh perspective and often undeniable talent, they just don't have the know-how (yet). Also, some of them are still kids. At some point, they get frustrated, because some player didn't "get" their idea and they don't see why (the know-how, you see; feedback processing techniques are part of it). So, more often than not, their conclusion is: the player must have been stupid, "they don't understand my art", etc. It's even worse if the player just happens to be a judge in some contest. Stupid, AND corrupted!
In reality, their lack of know-how often leads to basic design flaws, it's that simple.
The other group is people like me, who got into mainstream industry, had some soul-crushing experiences, became frustrated, and ejected themselves (or got fired) more or less for good (e.g. I'm probably going to start looking for a new job soon; but right now - I just need a break). Believe me, there are days when it's so hard not to vomit some of those experiences on Gamasutra or somesuch. The industry is eeeeevil, it hates you, the publishers are being stupid because they don't get it, and the senior staff is not letting you advance through the ranks because they're envious. That, and of course they're corrupted with their greed, how else.
The sad reality is that the mainstream industry is just like the rest of the world: 5% assholes, 5% misfits, an occasional genius, and 90% people who just want to survive (i.e. keep their job, keep their sanity, and get paid). There is no conspiracy. It's one thing to criticise "the industry", but to *blame* it for anything is on par with blaming the Solar System for sea tides.
So, long story short, a lot of indies are frustrated and/or disappointed, and quite often - understandably so. But they're overreacting. Particularly, in this very case. So when they turn up the heat, they're doing themselves a major disservice, because they're not achieving anything, but they're ridding themselves of the ability to influence the IGF constructively.
The trite truth is that, once you've hurt someone, it's hard to undo the damage. And those guys on the other side have feelings, too, you know. For instance - just look at that blog post and the comments section. Is Anna Anthropy or Simon Carless a bad person? No, of course not. And yet, she's fuming, and he has put his corporate mask on. They're not communicating. They're just playing their parts.
And the net result is, nothing will have improved next year.
Inexorable Forces of History/Economics vs. Individual Choice
JZW, I agree with a lot of what you have to say, but disagree, at least to a degree, with the idea that "this is just the way things are" and that soul-crushing experiences are a result. Part of this is no doubt my own cynical, "angry-middle-aged-man" approach to things, but part is also, I think, informed by a sense of history, and of the evolution of other media.
The game industry doesn't have to be like this.
Almost every creative medium treats talent with some degree of oppression -- because, when you come down to it, creative talent will make compromises to see something they do published, while other people in the value chain are more focussed on money. But the degree to which this happens varies widely from industry to industry, and the particular path the game industry has followed is due not to inexorable economic forces, but to specific decisions by specific people.
As far as treatment of talent is concerned, the music industry is possibly the worst, largely because new talent tends to be young and naive, and signs contracts not knowing any better. As an example, the boilerplate contract from the labels, to this day, says that they do not have to pay royalties on a percentage of units sold due to "breakage" -- an historical holdover from wax cylinders, for godsakes, when a high percentage of shipped product did get broken, and arguable perhaps when vinyl was the rule, but totally irrelevant in an era where CDs and downloads are the main source of revenue.
The book publishing industry is perhaps the most artist-friendly, since the presumption is that, editorially at least, the author's vision holds even over the protests of editors and copyeditors; but even here, boilerplate contract language is filled with terms than anyone knowledgeable immediately negotiates out (e.g., hardcover publisher sharing in paperback right resale revenues, publisher share in revenues from translated editions).
The game industry is somewhere between the music industry and film in its inequitable treatment of developers; even the music industry doesn't try to grab IP rights from creative (which the film industry does). Film requires absurd hours, at least during actual production, but at least compensates highly for them; both film and music try to charge everything in creation to creators, so that even successful music acts often receive nothing more than statutory 'publishing' royalties on successful songs, while in the film industry there famously "is no net", and people with contracts that stipulate a share never see a dime, even for movies that are international blockbusters.
Computer game publishing, at least, began with the Trip Hawkins EA that promoted a new generation of "electronic artists," but of course EA has developed into the Antichrist; while the basic business terms under which the industry operates were set by Nintendo during the NES glory days, and make it virtually impossible for independent developers to profit.
It never seems to have occured to anyone in the value chain that ultimate growth of the medium as a whole was dependent on creating a viable ecosystem for -developers-, who lay the golden eggs and should be encouraged to lay more of them; and we are today reaping the rewards of having the likes of Bing Gordon making fundamental decisions about the financial terms of the industry, instead of having our own Ian Ballantine.
But as someone who got crushed by the industry, you're not alone; in addition to the "young turks" about whom you are most concerned, there are plenty of "old farts" around who are heartily sick of the current industry and have taken other paths. Bob Bates recent emailed me about how many of his old friends are now working on independent titles. And no wonder; the future is no longer with the EAs and Activisions of the world, but more likely with -- oh, sigh, the Zyngas, whose business ethics are, alas, no better.
After seeing that some
After seeing that some people got feedback from judges that went along the lines of "i dont like this genre, this game sucks" I decided that I'll most likely apply to be a judge next year. If my plan of going to Thailand doesn't go through (the Bureau of Consular Affairs is fucking with me and not giving me my passport) and I'm not carousing around with Thai hookers I'd love to help out. I know there are people like Michael Rose out there, but judging from the feedback given there were a couple boneheads handling the games.
She makes alot of, toward a
She makes alot of, toward a certain goal, valid points about the awards structure. After all, it's a game essentially and she's evaluating the game. But perhaps it just doesn't match the goals she's talking about (though certainly I think her apparent goals are ones I share)
It just goes to show how much people are willing to pour out credibility to a set of awards without really understanding if it's worthy of the credibility they grant it.
Damn she makes some good points, though, I'll just say. Particularly about a basically incestuous feedback loop where scene nerds reward games that forfil their scene rather than, say, is fun to players as a whole.
~~~
Philosopher Gamer Blog
& a funny take on/the shocking truth about space invaders
Industry as a case of emergent gameplay
I think a slight misunderstanding has occurred, and it's my fault. I didn't mean to imply the industry needs to be the way it is. I actually believe to the countrary, and I have some dubious bragging rights to back it up. Namely, the belief has cost me four jobs, which I either lost or quit over such matters as rigged game reviews, people who thought their teams were their property, crunching, and tax evasion. For better or worse, I'm a bit of a rioter, of the kind that writes angry rants on public forums.
The flawed Solar System analogy was meant to point at the fact that the Solar System has no will. It not so much works as happens: while each celestial body exerts tremendous influence over others, their combined influence created a process beyond anyone's direct control. Similarly, the games industry is an emergent result of each individual game developer doing what they think is best for them at the moment - falling at the bottom of some gravity well, so to speak.
There is no "natural" economic law behind talent exploitation, but the actions of each person involved have caused a feedback loop to emerge. Each individual is influenced by it, but at the same time fuels it.
This includes me looking for a new job rather than starting my own project. Not that I wouldn't want to fund one; I just cannot afford it. And it's not because I was financially exploited - the one good thing I can say about my last employer is that they paid very well. Just not well enough to make a difference.
Even large entities like EA seem to be mostly riding the flow rather than setting trends, because they're made of lots of individual people, each acting autonomously, looking for their own best in a system that is much bigger than their perception range.
For instance, one of the projects I worked on in the last few years should have been cancelled at the end of preproduction, because publisher policy required that a project meet certain criteria at that stage, and that particular project didn't. But the publisher's own employees did their best to keep the project going, because their careers depended on it to some degree. So did the future of the medium, but I don't think they noticed. The rules were set in place, in order to help improve quality, and a number of people apparently gamed those rules for their local gain (you could even say they worked in dev team's best interest - after all, we kept our jobs).
Another example is the majority of my peers from various jobs. Here where I live (eastern EU), you can be an employee, or you can be a contractor. Employees are protected by the law; for instance, overtime is limited to 150 hours a year and always paid with mandatory 50+ per cent bonus. Contractors don't enjoy such protection, but they pay lower taxes. My last employer actually gave people a choice, and of course my peers hate crunching. But which option do you think almost all of them picked? Contracting - because it means a higher paycheck next month, and that's what counts most to them.
One thing that I think really is a kind of "law of nature", is that you cannot keep entities like Zynga from appearing, because it only takes one ruthless businessperson to start it, and there are six billion potential ruthless businesspeople in the world. However, in order to keep the likes of Zynga from winning, you need to do something about the millions of Facebook users who fall for Zynga's marketing tricks, for example. Educate them, I guess.
Now, none of the above means the industry cannot change. It can, and it should, and I hope it will. But it's going to take a lot of talking to a lot of people. Talking as in - sharing experience with someone who is listening to you.
The reason why I made my first comment is that I've done a lot of talking. Some of it was polite, some of it - not so much. In my experience, polite talking sometimes worked (very, very slowly). Yelling and calling people names never did, because most assholes would escalate, while most decent people, even if they were just witnesses, would withdraw.
In short, people like Auntie Pixelante should make themselves heard, but they should try and avoid making themselves appear fearsome.
An alternative viewpoint.
I posted this in Auntie's thread as well, but here's another viewpoint from an IGF judge (Seth Spivak) who seems a little happier with the process:
http://www.gamasutra.com/blogs/SethSivak/20100108/4054/Judging_the_IGF.php
It's unfortunate that Auntie stored up all of her negative comments into one carefully spun (and well written) failbomb -- which I don't think reflects the views of the majority of the IGF contestants, judges, or even the public. It's her right, of course. But I don't really know what good attempts to destroy credibility do. The IGF organizers care about smaller games as much as anyone.
As Greg notes, you can pick holes in all awards processes 'til the cows come home. We responded to feedback last year that people wanted more judge comments, and made a herculean effort to manage and recruit 160 of them, and now a vocal minority of people are upset with the quality of some of the comments. We explained the process exactly and even let judges talk about what they did and now we have a raft of increasingly complex new judging methods being suggested.
I guess it just all goes to how important the IGF is to people's livelihood and careers - if we're picking 7% of the submitted games out of obscurity and giving them a major chance at success, people are going to be incredibly heated about how that's done. But it's certainly starting to make me question whether openness actually works across such a fractured community. Is more opacity _really_ the way forward? It's depressing if so.
So yeah, I guess this is counterspin. But it's despondent counterspin.
Simon@IGF.
Simon-
Those people are there for you. They wouldn't bother complaining if they didn't care. Some of them may seem like they're hating you now, but under the hood they're actually loving the idea of an independent game developer contest.
Calling them a "vocal minority" only lends them credibility, because that's how whistleblowers are always called. As of right now, you and them can spend the rest of the month playing the parts of image guardians and whistleblowers, respectively, but nothing good will come out of it.
They also gave you a ton of extremely useful feedback. And I think there's a lot to learn from reading what the judges themselves wrote, particularly what different judges wrote about the same game. It's not that the bulk of the feedback was bad, but I think it's amazing how much the judges' criteria differed from each other.
Also, some of the points being made seem valid to me. Just like it's been said, IGF is a good thing - but it can be even better!
Those ideas they're giving you are just seeds. They may or may not be good ideas, but the more you discuss them, the more refined they will become. Organising a contest like this is actually a game design challenge of sorts. So it's no surprise that it can take a few iterations to get it right, and that sometimes you need to do a little brainstorming.
Heh.
Fair comment, jzw. I would just prefer that our improvement processes be less painful for all concerned, but I guess that's life.
JZW, I think I get your
JZW,
I think I get your point, but your also being kind of pointless yourself.
If you can figure out a more effective discussion approach Auntie could do and present that to her and show how it'll get her what she wants much more effectively, then leave it to her, that'd probably meet what your aiming for with your posts.
But you've used the phrase 'what she should do'. What is this but repeating the mistake you ascribe to her, of telling people what to do? In the end your repeating her 'your doing it all wrong!' approach.
And possibly more effective idea is to present an approach which will get what they want, rather than tell people what they 'should' be doing. And leave them to choose whether they take it up. That's an approach I present for your consideration should you wish to consider it.
Simon,
How do you yourself test if the awards have failed in it's mission?
If you have no way of testing if somethings failed, it gives a likely false impression of rightness and success. It's very easy to feel right when you have no method of proving yourself wrong.
~~~
Philosopher Gamer Blog
& a funny take on/the shocking truth about space invaders
Don't do flamewars
I don't recall saying she shouldn't tell people what to do. I didn't use the phrase "she should". I'm also not interested in helping her "get what she wants". Clearly, she mostly wants IGF to improve, and that's a goal I support, but I'm not even sure if she has the same definition of "improve IGF" as I do.
In fact, I wasn't even referring to her specifically. the99th kindly said "It wouldn't be indie if everyone agreed it worked and was polite to each other, right?", which appears to me as a very true statement, but also a cause for worry. Hence my asessment, that "the indies should try and compensate for their tendency toward flamewars, rather than embrace it". For reasons which I tried to outline, indies are more vulnerable to flamewars than you would normally expect. I thought it was important, because a few good points were made in the recent IGF discussions, and it would be a pity to see them melt in the heat.
If you take a look at the end of my second comment, it says "people like Auntie Pixelante" rather than just "Auntie Pixelante", which is a subtle but vital distinction. I wasn't trying to dissect her post, and I don't think she would appreciate that.
Frankly, the only practical advice I offered was "don't do flamewars", but that's not exactly enlightenment. We knew flamewars were bad back in the Usenet days, and that was ten years ago (to me, at least; the Usenet as such is slightly older than me).
Somewhere in the mean time, we digressed into a brief exchange of thoughts on whether the industry is doomed to be in such a poor state as it is right now, but that was beside the main point.
I'm not asking for
I'm not asking for justification. I'm offering a vector of approach for possible consideration.
Flame wars are just a symptom. Getting rid of them simply cures the symptom. The root cause is, unless the other persons goals completely match ones own, a widespread disinterest in helping the other person meet their goals. Or that's the hypothesis I offer, anyway.
~~~
Philosopher Gamer Blog
& a funny take on/the shocking truth about space invaders