- Thu, Jul 28
- Explore Yooka-Laylee in special Toybox demo, exclusive to Kickstarter backers
- Dragon Quest Monsters: Joker 3 is Japan's best selling game so far in 2016
- Nep-Nep Connect: Chaos Chanpuru will bring Compile Heart to Vita's card table in Japan
- Dragon Ball Xenoverse 2 trailer shows off new transformations, power pole, and Flying Nimbus
- Wed, Jul 27
- Level-5 reveals Lady Layton, meet the puzzle professor's daughter
Larian Studios Interview Pt. 1: Games Journalism is Broken - PAGE 2Sean Ridgeley - Friday, March 2nd, 2012 Like (1) Share
I agree with one the comments in your blog about how you tend to follow reviewers as opposed to publications. More people should do that. I read a lot of comments whenever the topic comes up on forums or wherever, and usually there's a large amount of people that say 'All reviewers are terrible, they don't know what they're talking about, I don't read reviews, etc'.
To an extent I understand that, because there's a lot out there that gives us a bad name, but if more people put the effort into seeking out reviewers they really trust, that 90% of the time are in line with their own opinions, and can tell just by how they word things they're not being bought off or suckered into anything, and they stick with that person. Seek those people out and spread those reviews around; I don't care where they're from.
Once I started making the list of all the things that affect a review...it's an enormous list. Other than the influence of the money I guess, there's the subjectivity of the reviewer...that's where it's important you find someone you can trust, and that he speaks for you as part of a target audience and player instead of just the target audience of the magazine. That can be quite complicated to find.
What do we do when we create a box? "IGN says 8/10, fantastic." And on the back we say "IGN says this is a very good game." But who the hell is IGN? If it was a particular reviewer, we couldn't just put his name on it because chances are nobody knows who he is, but that would be much more valuable.
That's a good point. Film works that way. There are famous film critics; I guess only a handful, but still, maybe if we put more emphasis on that in games that would help.
What we would love to see is a review of the reviewers - a Metacritic for them where it shows where they are in line with games, and you can say 'Well I liked the game' and it says 'This reviewer says you wouldn't like the game'. You would have the good reviewers float to the top, and these guys could then probably run their own magazines on their own. This would be a good thing.
There's no liability for reviewers right now. We had a very big issue with a particular magazine who gave us a 5/10 for Dragon Knight Saga, which we couldn't understand at all. Development had been working 16 hour days for months and months and months, and you're waiting for your first review and that was the first that came in. It's like 'What the hell did we do wrong?' So he wrote it hasn't become a good game, but he likes the graphics. Then a couple of days later the IGN review pops up and says "The graphics are acceptable but only barely." But he gives it 7.5, so fair enough.
We called the guy with the magazine, and they say 'The guy isn't here anymore, he's a student, an intern.' 'Oh really? So you let a student review for a Metacritic review site?' This affects the initial perception people are getting from our game, because it's being listed on whatever aggregators. There should be some sense of responsibility, because this guy was just an intern who happened to play games, he's not a journalist at all, and he's going to be reviewing the thing you've been working on for years.
It depends on the intern. You can have some highly skilled ones, but some are just amateurs, yeah. And especially for an RPG - that's a big deal.
Don't get me wrong. It's not that I have something against interns, it's just...it's already a big problem with the magazine, the consistency. There's a lot of publications out there where you pick three different RPG reviews, and it's probably easy to find statements immediately conflicting with one another.
We used to listen to what reviewers were saying, especially the big ones, but we discovered there's so little consistency that we just gave up completely.
That's disappointing from my end. I understand it; it's warranted completely, but it's unfortunate for reviewers that do work hard and are consistent and do put that emphasis in.
It's another example of where reviewing reviewers...the good ones would flow to the top. And actually that should be a factor in the weighing of values on something like Metacritic, which, like it or not, has become incredibly important in our industry.
I remember [PR company] The Redner Group saying "sales teams live and die by Metacritic", and I'm inclined to believe them. You can see that emphasis from PR and publishers. If they don't get that 90, it's like 'forget it' - they don't even care.
I saw a curve once from a [big publisher] showing the correlation for an RPG between Metacritic score and sales. There's an incredibly strong relationship, which is exponential. An 80 will get you a couple of hundred thousand sales, say, where a 90 will get you a couple million if it's marketed properly.
That's a very scary thing, that one number can have such a huge impact on whether or not a studio is going to be allowed to make its next game... Studios are killed over bad Metacritic scores. That happens every day almost. If you look at what makes up the Metacritic score and how it's done...you wonder how many reviewers or sites are taking it very responsibly.
I wish the user ratings would be much more important than the critic ratings, because then if you add up all the critic ratings, what do you have, 100 guys (or girls)? Versus thousands of players. So something is twisted there.
And when you see scores being manipulated, that drives you completely bananas as an independent, small developer. Obviously if I was on the other side I would be happy if I could put shit in a box and put a nice name on it and get a 90% score.
It seems like it's more possible than ever to promote a new game, it's just a matter of how clever and creative you can be with getting people to know about what you're doing.
There should be accessibility in press in that way.
I like to think we generally do a lot to help that. If someone is working on something brand new and is extremely well done. I'm thinking of Bastion, for example. Press jumped all over that and I think for good reason.
Yeah, well that's the trend. Once people are writing about it...
It catches on, yeah. In that respect, games coverage is healthy.
I once did a press trip to the US promoting Divinity II. We had meetings with all the large [publications], and I flew over the Atlantic to meet these guys, and they said, 'Sorry, we can't see you. We're working on this preview now.' I go the next one and it's a guy who was a fan of us, he gave the original Divinity 86%, but it turned out he was working on a preview of a major game, so I sat down with the intern who was into racing games. The only reason we did the trip was because online we got no response. I can only imagine how hard it is for [a studio] with nothing behind them who have to start from scratch.
I guess it varies from publication to publication. Some of us are eager to hear about some game nobody else knows about, because then we can be the ones to start that trend. That's exciting to us. But I guess some sites are not as open and welcoming and more concerned with what they're already doing.
There's a trend now where you have sites like you guys and Rock, Paper, Shotgun and others. They're very accessible and give startups and mid-size companies like ourselves a chance to talk to the public and that's fantastic. It's exactly what I had with GameSpot or IGN back in 1999. So that's changed at the very large sites. I guess it comes with the size and growing they've been doing.
I think it's also a different model. How they make their money is not how we make our money. We pride ourselves on uniqueness; they don't really have to bother with that.
Going back to reviews: how do you manage consistency amongst them?
You mentioned how sites like IGN will hire interns and whoever. I guess they want to cover everything and get the most traffic. That's not how we operate, really. We don't spread ourselves too thin I think is the key, and because of that it's a lot easier to manage what we are doing.
And we don't take anyone on lightly; I'm not just going to hire anybody. Reviews are a sacred thing, partly because it's a trust issue. We've had experiences where we thought we trusted people, hired them on to do reviews, and then they flake. I'm probably more strict than other editors; I put more emphasis on high quality work and look for someone that gets what we're doing.
There've still been issues even with those people. One of our writers wrote this review of a flight game. I'm not sure how that got through; we're not perfect and things do slip through. But anyway, somehow he got assigned to this game and shouldn't have been. Flight games are really advanced; you need years of experience in that genre, but he's more into somewhat casual games, reviews a lot of Nintendo stuff. So I just canceled the review.
Oh, wow. Good for you.
Yeah. I'm not publishing that; that's bad for us, and the publisher is just going to be pissed. He gave it a really low score, and I had to explain to him 'just because you don't understand the game, doesn't mean it's a bad game and deserves a low score - you have to understand that'. So we had a big talk about that and he got that.
I think if everybody was doing that Metacritic would be much more useful. It's too powerful, in any case.
It's sad publishers place that much emphasis on it. But readers do it, too. Some just look at the score and that's it. It's barely about the score; the score is just there because it has to be there, I guess. We could remove that, I mean, some sites succeed and don't have scores, but I guess it helps. But in the same way the game is about the experience, the review is about all of its elements and how they come together. Our readers are pretty intelligent, but nonetheless sometimes the discussion is purely about the score - 'I would've given it this score!' 'I would've given it this score!' 'I don't agree with this score!' 'Let's just talk about the review itself...' (laughs) Like you said, it's silly to quantify a creative experience. And what I call an 8 might be a 6 to you, but if we both described it, we'd be saying the same thing, so what does it matter?
I think the lack of background in journalism is a huge part of the problem. Many editors don't know how to edit and manage and do all that and oversee and look for issues in consistency and such. I don't think writers in this field necessarily have to have gone to school to be good at what they do, but you need an editor who has gone, for the ethics, the background, the function and the form and all these things. Then they can push the other writers up to that same standard, which is what I do.
This is a crucial point. You're right. And given the scale of the industry, it's surprising how few educated journalists are doing games journalism.
I understand that it's games and it's the Internet and it's easy to get into because a lot of people are into games and they're enthusiastic about it and say, 'I know all about games; I have good taste'. Usually that doesn't work out. You have the rare writer that is really talented and doesn't need the education and has that flare, but yeah, it's rare. Usually they don't understand the journalistic side of it, the ethics, the integrity, and even down to the actual writing: the skill, the flow, various techniques.
I concur with that. We've invited journalists over here, and sometimes you see previews which have nothing to do with what you've been telling them.