On its livestream this afternoon, Camelot Unchained boss Mark Jacobs told backers and watchers that technical issues have delayed the start of beta, which was anticipated literally any day now and has been delayed once already (as was the alpha before it). Jacobs explained that CSE discovered its ability system was unable to properly handle the complicated skill and class design the team was throwing at it, necessitating a refactoring of the underlying ability code. “We are going to be able to preserve a lot of the underlying code but have to change some aspects of it and the pipeline that an ability has to go through from button push to active in the game,” he told us.
While it’s frustrating news, Jacobs believes the retooling will ensure faster deployment of new skills in the future and will positively influence the crafting system as well. He told us that while CSE could have opened beta now and fixed the problem later, the team prefers to launch a beta with the permanent ability system in place rather than try to sneak one past the backers.
We spoke to Jacobs and Head of Code Andrew Meggs ahead of the stream to ask a few more questions; read on for their explanation.
Massively OP: I know you’ve said you don’t have all the details yet [as of the time of the interview], but can you tell non-tech readers why it took as long as it did to identify that a system of this magnitude needed a major rewrite? Wouldn’t the technical team have suspected it weeks ago with beta one bearing down on them?
CSE Head of Code Andrew Meggs: There’s a saying: “Perfect is the enemy of good.” We knew it wasn’t the best code in the game, but getting games shipped isn’t always about writing textbook-perfect code. And with a crowdfunded game, people always want to see forward progress, even if it’s on a shaky foundation. Crowds don’t get excited when your big achievement for the month is something under the hood that end-users can’t see.
So I originally made the call to grab some duct tape and keep moving forward, with the goal of getting to beta earlier and hoping to refactor later. But we hit a certain point where most things that didn’t touch abilities were moving forward in a generally good way, but the things that touched abilities were stalling and missing their dates and coming in with lots of buggy edge cases. The amount of extra time we were spending to work around brittleness in the system was exceeding what it would take to address that brittleness. And that’s when you have to make those hard calls.
CSE CEO Mark Jacobs: We actually identified the problem late last year, before we went on break. After that, among other things, we made some changes internally, as we believed that we could improve the system enough that it would be able to handle the needs of the designers and get us into Beta 1 on time. Unfortunately, while we were making progress, the deeper we went, the less likely it seemed that the system could handle what it needed to do, without us adding a lot of “throw-away” code. We could have easily made the decision to throw on lots of band-aids and called Beta 1 open. But that would have been antithetical to our promise to our Backers to do things the right way. The right way in this case is what we are doing now: not putting on band-aids, but rather healing the wound and moving forward.
We’re fortunate that when we made the decision last year, we thought that the majority of the system could be saved, and we were right. What we are doing now doesn’t involve a total rewrite of the ability system code; just some of it. Because of the importance of this system, most of the engineers have been part of the discussion regarding this, and Andrew, Marc, and Tim started doing code/design on it last week, and are continuing to work on it this week.
What’s most important about these changes is that they will allow us to have a system that can not only do what we need it to do for Beta 1, but will add flexibility (including allowing direct C# scripting to be done by Ben and myself), enabling us to add and modify abilities not only more easily but also with more power than we had before. This will also have the added benefit of lowering the amount of time programmers will be needed for ability work going forward. This has been a major problem with the current system, and that issue will be reduced.
You don’t have hard dates of course, but are we looking at weeks of delay? Months? Beta in 2017? Are we to assume crunch will go on? Are you concerned that prolonged crunch will be detrimental in the long run?
Jacobs: Certainly not 8 months. Right now, we are looking at weeks, in terms of the core ability code. The key thing to know is that the problem with the current ability system is not a technical challenge in the same way maintaining a high FPS with 1K players on your screen is a technical challenge. This fix is more of an architectural one, and this time, we have the right blueprint for it. :)
In terms of continued crunch, no, we are going to end crunch. At this point, it would be unfair to keep people working loads of extra hours while we are also reworking this system. The team gave it their best during the soft/hard crunch, and now it is time to work normal hours while the new system is being put in place.
As to a prolonged crunch being detrimental, well, the length of our crunch is nothing compared to some. However, just as making the changes to the ability system now rather than later is the right decision, so is ending the crunch now the right decision. I’m proud of the team’s effort, and I know that they would even work harder if Andrew/I asked them to, which is precisely why we will not ask them to do so any longer.
Meggs: The majority of the team’s out of crunch for now. The people working on re-abilitation have some pretty intense weeks in front of them. But I think there’s a lot of good morale internally right now. Working long hours is demoralizing when it feels like you’re paddling against the current. And working with the old code very much did feel that way. But when it feels like you’re turning the boat in the right direction, the enthusiasm to keep going comes a lot easier. It becomes about execution instead of frustration.
The delay is going to allow more to go into the first round of the beta than originally planned, correct? And make other things easier to implement (like crafting). Are any specific systems getting cut or delayed or slowed down as a result of the delay and rewrite to the ability code?
Jacobs: Depends on the length of the delay. The short answer is yes, we’ll get more things in than originally planned. The new system will benefit both abilities and crafting in both the short and long term, but those benefits will be seen over time. Right now, we are focusing on making sure that abilities are everything we need them to be, now and in the future, so the team is mostly focused on that. As a side benefit, the new ability system will help our crafting system as well, allowing me to do work on crafting that would otherwise have had to wait.
Meggs: There’s a core team of the more senior server people working all-out on this system. The other half of the team is going to be moving forward as usual with work on the client, C.U.B.E. features, crafting UI, rendering, and hopefully some terrain things. So when we go to beta, those pieces will all be farther along than if we weren’t doing this.
Obviously the time is going to come from somewhere, and in the short term that means abilities won’t be as far along as if we were just hacking things in and rolling forward despite the bugs and brittleness. But we’ve always got the end goal in sight. The reason we’re doing this is because it looked like it would take less total time to get in all the abilities we wanted. We’ll catch back up, and even get ahead.
Are you going to be issuing refunds to grumpy backers? I recall you’ve done this for past delays (and it’s been pretty rough on you guys to boot). Are you anticipating a lot of lost income? How do people go about getting a refund?
Jacobs: Our refund policy hasn’t changed since day one. All a Backer has to do is send a message to support with the proper information, and we take care of it. As to whether we will get a lot of requests? I hope not. We’ve paid out a fair amount of money so far.
In terms of it being rough, it is. Each request takes time/toll on us and frankly, hurts the game’s development in the aggregate. The worst are those people who also feel the need to vent at us at the same time and hurl all sorts of insults our way. I would understand it if people were being denied refunds and then got testy, but when their first message to us is something like, “You guys suck, I want my money back NOW!” it’s a little disheartening, though not unexpected, at times. We’ve had Backers verbally attack us in horrific ways, simply because we always ask them to provide us with their receipt for the purchase and not just take their word that they bought a pledge, even though they are sending us an email from another account. :)
On the flip side of that, what can players and backers do to help?
Jacobs: Just be patient. It would be one thing if development had stalled, but we’ve made lots of progress on the game and will continue to do so. And frankly, everything we are doing is going to be worth the wait. We’re not mindlessly adding features, selling crap, or doing other things to encourage Backers to open their wallets. We’ve been doing the exact opposite, and we hope, if nothing else, that our Backers will remember this before hitting the panic button.
I’ve got my own money riding on this game, and the amount I’ve pledged is a substantial amount of my own net worth, so between that and not getting paid for the last three years, well, suffice to say I’ve got a lot riding on this game. So, since I’m not even close to panicking yet, I hope they won’t do that either.
You’ve said you’re not adding more stretch goals — a decision that other post-Kickstarter games have eventually made as well in order to prevent feature creep (or maybe more like to stave off accusations of feature creep). That doesn’t seem to be your reasoning, though; you almost sound as if you’re withholding stretch goals as a sort of penance for CSE. Would you explain your reasoning? Why not let the backers themselves decide whether they want to throw more money at you if they think more money will help get them their game faster?
Jacobs: You’re right, it is a bit of a penance, but it is also a matter of trust. And to quote Andrew, this is a statement of focus. With this delay, albeit brief, we believe that we should stop offering additional enticements for backing this game other than what we have talked about so far. Once we get into Beta, I’ll happily put up the next Stretch Goal, but not until then. It’s one thing to ask our Backers to trust us and not to hit the panic button; it’s another thing to also say at the same time, “Look over there! Bright, shiny things!!” That’s not how we do business, and not how I would want to be treated as a Backer.
We’d like to extend our thanks to Jacobs and Meggs for their candor with us and with our readership.
