TruthScape.com - RuneScape 2007 Year in Review and Report Card

Now that we’ve looked at the most important additions and changes made to RuneScape in 2007, I want to take a step back and look at the big picture. In this section I move away from grading specific changes, and instead assess various aspects of the game in more general terms. This includes some summaries of the areas covered in preceding pages, as well as general evaluations of how Jagex has done in running the game and dealing with its customers.

On the whole, I think Jagex did a very good job with content updates in 2007. We got some cool new dungeons to explore, many good new monsters (Figure 70), small expansions to Crafting, Fishing and other skills, and some fantastic items like the dragonfire shield and godsword. In particular, it is nice that Jagex followed through on its promise to create a few more updates suited towards higher-level players, even though there weren’t quite as many as some of us would have liked.

**Figure 70: Some Fine New Content in 2007**
With their combination of high difficulty and good rewards, mith dragons were a highlight of 2007.

I don’t feel the company did nearly as well in terms of updating existing content as it did in adding new content. The main problem isn’t that the updates themselves were bad—well, except for that dud of a skeletal wyvern “improvement”—but more that there were so few updates that were really noteworthy. Given just how many segments of the game are in need of attention or balancing but are left to stagnate year after year, the total number of updates was a bit low.

In general, I think this is a serious issue with Jagex’s content priority: most of its effort seems to be focused on “adding more stuff” to the game instead of improving what’s already there. The problem with this is a simple matter of manpower: it takes much more work to make a new minigame than it does to tweak and improve an existing one.

For the amount of time it took to make the Sorceress’s Garden, for example, Jagex could have probably revitalized every existing minigame in RuneScape. And some of the minigames, like chompy hunting and Shades of Mort’ton, are desperately in need of updates. It’s unconceivable that over a year after the release of a skill called “Hunter”, chompy hunting still has not been integrated into the skill, and still provides rewards that are essentially useless despite requiring hours of work. Shades of Mort’ton is hanging on by a thread, now that interest in splitbark armor has dropped down to nearly nothing; the highest-level chests still give far too many pathetic rewards like black spears and mithril plateskirts.

Now let’s consider skills—how many skills are there that max out at low skill levels and haven’t had a meaningful update in years? The highest level tree for Woodcutting is 75, but the only tree added this year is level 54 and gives the same XP as chopping oaks—why did they bother doing this? How long has it been since Mining got an update? When will we have seeds for allotment patches above level 47, or proper high-level potions?

Jagex did a pretty good job in this particular area in 2007. Many annoyances were removed that had hounded players for years, the world switcher and clan chat are fantastic features, much work went on to improve the graphical appearance of many parts of the game, and sound was improved as well. The knowledge base is working well, achievement diaries are a solid feature, and themed worlds were long overdue.

It seems that 2007 was the year that, after a slow start, Jagex finally got serious about cleaning up some of the persistent security problems in RuneScape. We were the beneficiaries of updates that eliminated many trade-related scams, Wilderness luring was (finally) made illegal and the Wilderness ditch was a welcome fix. The RWT-related changes near the end of the year also have served to make many remaining forms of scamming either impossible or just unrewarding enough that the community as a whole is a better place to be.

Jagex seems to have settled on a routine where it implements significant game updates roughly three times a month. I think this is a good balance; it gives players something to look forward to, without the updates being so frequent that it leads to confusion.

Some players seem to expect Jagex to put out new updates every single week, they don’t realize how lucky they have it that the game is updated as often as it is. If anything, I lean more towards those who suggest that less frequent, higher-quality updates would be preferable.

The name “quality assurance” implies analyzing a product to make sure that it is of high quality. While many people just think of software QA as being about bug testing, it is really far more than that. And while Jagex seems to do well with the bug testing aspect, things get dicey once you move past that part of the job.

To its credit, the Jagex QA team seems to do very well in terms of catching and eliminating game bugs prior to release. Given the number of release cycles Jagex goes through, RuneScape has very few bugs. Even some fairly complicated features have, so far, been released without any truly horrible bugs that could have led to disaster.

When dealing with a game like RuneScape, testing new content for bugs is not enough—the QA team has to also test it to ensure that the content is played as it was intended by the designers. This so-called play testing should involve testers who play the new content in the same way that real players will, looking for design flaws and exploits that need to be fixed before the update is released.

Simply put, Jagex does a bad job in this area. In a large percentage of the major updates this year, experienced players have identified major flaws and exploits within hours of features being released. I often find myself in discussions with other high-level players, all of us each pondering the same simple question: “If we found these problems in a couple of hours, how could the QA team possibly have missed them?”

A few notable examples from 2007 of issues that QA should have caught before release but didn’t:

These sorts of flaws not being caught by QA suggests either that the team is not spending enough time on play testing, or that it is out of touch with how real gamers play RuneScape. (Probably both.)

Jagex did a decent job here in some respects, and a rather poor one in others. Good examples include the godsword, which while powerful is not a weapon that obviates all other weapons, and the dragonfire shield, which is very useful but rare and expensive enough that it’s not unbalancing. On the other hand we have crossbows, which went from useless to being so good that nobody uses regular bows any more.

Also, many new monsters in the game are not properly balanced in terms of the effort needed to kill them and the drops they provide. Once again, skeletal wyverns and terror dogs top this list, which is especially bad given that they are part of Slayer.

The company has developed an absolutely horrible pattern of putting out an update, letting people scream about it for a week, applying a couple of bandaids to the problems associated with the update—and then abandoning it. This has occurred with nearly every problematic update this year, from the fishing trawler to the wyverns to dueling tournaments and the assist system.

Suppose you spent 100 hours developing a product, and when you launched it your customers identified some problems with it. Which would make more sense—spending an extra five or ten hours fixing that product so it is as good as it could be, or moving on to spend 100 hours on some other product? I wish Jagex would recognize this simple but important “bang for the buck” concept.

My understanding is that at one time, Andrew and Paul Gower and other developers frequently interacted with players on the RuneScape forums, announcing and discussing game content updates and other changes. Well, Jagex is a big company now and the Gowers are rich and important men, so I can understand that they have other things to do. But it seems that nobody else on the content team really considers interacting with players a very high priority either.

What usually happens these days is that when an update comes out there’s an announcement about it on the main website, and a thread started on the RuneScape forums, and that’s about all. If we the customers are lucky, some J mods charged with perusing the forums will throw us a bone, but interaction with actual content developers is exceedingly rare.

It’s amazing just how much confusion can be avoided and goodwill generated when those who make a product take some time to deal directly with those who pay for it. The development diaries are a nice step in the right direction, but only a small one.

Jagex’s customer service department has become a laughingstock, only what’s going on really isn’t funny. There are only two words in customer service—“customer” and “service”—which would seem to imply that the job of a company’s CS department is to, oh I dunno, service its customers. But when it comes to RuneScape, customers most often don’t get any service.

It’s always been hit or miss to try to get help from Jagex; you send a message and sometimes you get a useful reply, while other times you do not. Lately, however, CS has become so awful that it is effectively impossible to get aid on nearly any issue. It was bad enough when CS reps would send nearly useless form letters that were barely relevant to the question asked, but now they don’t even make that much effort. Most of the time you get completely useless form letters that simply say “we’re too busy to answer your question”. It’s like leaving an answering machine message and being called back by a computer.

Earlier this week I was having an exchange with a J mod, where I was trying to inform him of potential RWT abuses as a result of the Grand Exchange. I couldn’t actually tell him anything properly, because the stupid CS system only lets mere peons (er, customers) type 400 characters in a message. He couldn’t change this with the dialog we were having, so he asked me to send a new CS message that he would reply to, believing that this would hopefully let me send a more lengthy response back. I did as he asked, but he obviously never got it—and two days later I got yet another “we’re too busy to answer your question” answer from someone else (Figure 71).

**Figure 71: Customer Disservice**
Ridiculous. And by the way, unless there was some holiday I missed in early December, this nonsense is not because of the “holiday period”—it’s been going on for weeks.

Summarized simply, this situation is inexcusable. Jagex is a company that takes in tens of millions of dollars a year. It should have some way of setting up a customer service department so that paying customers can actually get service!

I’m afraid that I have to be rather blunt in this section as well, because Jagex has not been very forthright with its customers this year on a number of issues. For starters, the company lost a lot of respect from me and many others with its attempt to legalize luring and then cover up that decision. This problem was reversed, but the handling of the whole real world trading mess has been nearly as bad.

Jagex should have come out months ago and told all of its customers exactly what was going on with respect to RWT and what it was planning to do about it. Instead, it fed us piecemeal updates that left everyone confused and waiting for the other shoe to fall. While players wondered if they were going to ban trading or get rid of PKing or whatnot, Jagex kept everyone in the dark on the entire issue.

Speaking of those updates, they were sold to use with teasers and descriptions that, in retrospect, turned out to be flatly deceptive. Consider, for example, this quote from the November Behind the Scenes article:

Does anyone here really think there were any “suggestions from players, forum posts and general feedback” to limit dueling stakes to 3k every 15 minutes? I seriously doubt it.

Then there’s the Grand Exchange, which was sold to us as being a way to make players’ lives easier while allowing us to retain control of prices. In reality, it is the centralized price control mechanism for the game now, and Jagex is blatantly manipulating prices through price restrictions and even volume limits on trades.

The only reason this grade isn’t even lower is that Jagex finally shows some signs of coming clean about the RWT issue. But even so, I suspect they are holding back on us, and I really wonder how important the honesty issue is going to be going forward.