Ascent’s lead dev offers insight on the Star Citizen controversy

On the status of the Star Citizen project: A professional opinion from Ascent: The Space Game’s James Hicks

The technical concerns

In June this year, Cloud Imperium Games made an offhand announcement to the effect that it had almost finished getting its game engine to operate in 64bit 3D space. To CIG’s fanbase, this sounded like a cool new development (which it sure is!), but in Florida it set alarm bells ringing in Derek Smart’s head, here in Australia it made me very concerned, and I can guess that somewhere in England, if he heard it, David Braben responded with something like, “Come again?”

With the absence of such an announcement until that time, I had been assuming CIG either wasn’t going to need 64bit 3-D space or had done it already and wasn’t going to announce it.

If 64bit 3-D space is something you need, and you don’t have it, it’s a complete project killer. Without a solution for 32bit vs. 64bit 3-D positioning, your engine will not have enough accuracy to display spaceships in a steady position within a large environment. This is one of the core reasons games usually avoid truly huge environments, and if you are going to “go big,” you run into problems immediately. Between frames, objects will appear to shift positions. Close to the game engine’s “0,0,0,” the effect is so small you can’t see it (less than a pixel). Get a little further away and your spaceship will seem to vibrate a little. Further still and it wiggles. Further again and it’s jittering all over the place — ugly and unplayable.

On the other hand, with 64bits to work with, you can be accurate a huge distance from “0,0,0” — big enough, depending on your implementation, to work with a full-sized star system like ours, as a matter of fact.

The problem is that GPUs are crap at 64bit calculations. You either need to accept hideous performance (accept the unacceptable) or need to be painting a 32bit picture for the GPU, a snapshot based on the larger 64bit picture your game engine has… every frame. If this sounds complex, that’s because it is. Deciding to go 64bit has ramifications all down the line of your project. It complicates absolutely everything that comes afterwards.

And that’s why announcing that you’re almost ready to do it, six months after your initial projected completion date, sets off alarms in the heads of people who’ve been through this issue before.

Frankly, right after the announcement, discussion about Star Citizen went right off the rails. It’s now devolved into name calling and “lawyers at ten paces at dawn!!!!1” and I’ve got no interest in that. Chris Roberts and Derek Smart are two of my heroes. Listening to them fight, I feel like a child hiding while his parents scream at each other. It’s ugly, it’s off message for both of them, and I wish they’d cut it out.

So below, I’m going to continue the discussion from where it left the rails and talk about the technology challenges Cloud Imperium are tackling and what I can say about the project, as a somewhat informed outsider.

Who am I to judge?

I’m the developer of Ascent: The Space Game, an indie space MMO with a vast procedurally generated galaxy and a massive sandbox construction “endgame” that includes player-made cities with over 100,000 structures and player-constructed jump gates that are co-op megaprojects.

Ascent deals with many of Star Citizen’s technical challenges particularly with respect to going big, and we made design decisions that considered many of SC’s design parameters, so I’m in a position to evaluate its project status, to an extent, based on publicly available information.

The largest project I’ve managed to date was $3 million, and the most staff I’ve had is 18. So while I’m not the perfect choice to comment on a 260+ staff, $90 million project, I’ve still got some ability to make rough judgments there as well.

What’s my angle? I’ve got a few here, and I’ll be upfront with you about them. First, yes indeed I am taking time away from development, testing, bug fixes and customer support to write this dev post because it can give Ascent some publicity. No free lunch here.

Second, if Star Citizen does collapse in a heap, it will, to a large extent, take crowdfunding for computer games with it. And that’s something I need to avoid because I’ve successfully used crowdfunding before and would like to have the option again for our next game too.

Third, I want to play Star Citizen one day, and I’m not alone. Many of my own customers are also SC backers.

And if the negativity and noise kill off SC’s funding, that adds risk to the project, and makes the doomsday scenario more likely — if you will, a potentially self-fulfilling prophecy. So my interest here is in bringing the discussion back to reality and less… screaming.

So, now you know my potential biases, let’s get back to…

Going big

Once you’ve solved the 64bit issue, however you solve it (and there are a few options), you quickly run into the next problem.

GPUs, and therefore game engines, have a limited draw distance. This is called a “far clip plane.” Generally this will stretch, at most, a few kilometers out. You can push it further, but GPUs use even less accurate math to deal with depth, a pathetic 24bits of information. You can stretch the information out by changing the “near clip plane,” or how close the game engine will display things, to go out a little further.

But in Star Citizen, you need to be able to see your hands on the controls less than a meter in front of you and a gas giant that could be a million kilometers away.

I can’t tell you for sure how CIG has solved that problem, but I can tell you it looks like the studio has definitely solved it, based on its most recent gameplay videos. In Ascent, especially when you disembark your ship, you need to be able to see your hand having in front of your face as well as faraway objects like planets and suns, and we solve this issue by having two cameras (near and far cam), giving us double the accuracy. But this comes with all kinds of its own complications and issues.

Without a solution, objects that are near to each other in depth within the scene will clip into each other and flicker. We call this “Z-fighting,” and you can see it in many games actually. Anywhere you’ve seen the water at the shoreline wiggle and flicker where it gets very shallow, you’re looking right at Z-fighting.

In Ascent, we need to display oceans from hundreds of kilometers above the surface, smoothly all the way down until you splash into the water. We avoid Z-fighting with our shorelines by fading the water layer in when you get close enough to see the waves clearly, and our water shader fades out at the shoreline where it calculates that it’s too shallow to display without Z-fighting.

It’s not clear to me how Star Citizen will implement planetary landing; however, based on CIG’s “designed” rather than “procedurally generated” approach and a separate “Planetside” module, I believe that rather than smoothly fly down from space through 100km of atmosphere to the planet surface, you’ll likely get a “loading, please wait” screen, and this eliminates many of the problems, effectively making SC easier and cheaper to complete. This is a good design choice in that regard.

Next, even with a very far “far clip plane,” you still can’t draw an object 150 million kilometers away (that’s how far away the sun is from Earth, for example). Not with two cameras; not with 10. Star Citizen has also seemingly solved this issue. Once again, I don’t know how, but for Ascent we solved it by shifting the objects into view range and shrinking them proportionately for the camera. In much the same way the moon appears to be the same size as the sun, even though it’s much smaller, we can make an object appear to be further away by shrinking it. Beyond about 100 meters, human binocular vision can’t tell the difference, so this will work even with an Oculus Rift. If that’s what CI is doing, it’s got this problem sorted out too.

Lastly, rocky planets need detailed terrain for when you get in close, and land. If you’re going Earth-sized, they need a lot of terrain, and it needs to be spherical. No game engine comes with a terrain engine like that; you’re stuck with coming up with your own solution. Kerbal Space Program did it by making a six-planed cube that mapped into a sphere with stored height map data. We do it the dumb way: We wrote a spherical, Earth-sized terrain engine with procedural generation to give us a trillion or so unique Earth-sized terrains with a high level of detail. Star Citizen is likely to avoid this issue completely by having a “loading, please wait…” screen and then a designed planet surface scene for you to appear in — that is, I believe, the premise of the “Planetside” module.

The reason I think this is that all of Star Citizen’s cities and star bases are hand-built by artists and designers, and there are a limited number of these scenes. It simply isn’t viable, even with $90 million, to draw out the terrain for 100 planets by hand. Imagine every single terrain feature on earth, times one hundred. So I expect CIG will have a loading screen, and then a jaw-droppingly beautiful cityscape and terrain scene for you to land in and interact with. This is a good design choice for Star Citizen.

Ascent needed the huge planet terrain because we have one artist and one developer. We can’t make cities by hand at all; all of our cities and star bases are built by players in a sandbox construction system, so we’re stuck with the more complex solution that will never, ever look as good.

Derek Smart’s “technology panic”

In order to understand the level of technology panic Derek Smart experienced, as a backer, back in June, you need to take all of these problems into account. Why? Because he solved them in the early 1990s, before the game engines and tools we take for granted today were even thought of. To get inside his head, you need to imagine the June announcement roughly translating as, “We have now begun development of Star Citizen!” Hopefully, this brings his angry “they don’t have the technology” rants into focus.

If you were going to run into all of these problems, virtually every piece of code written prior to that announcement would need to be at the very least put under a microscope, if not refactored or even rewritten. To understand why that could’ve been the case, we need to talk about the game’s overall design.

But Smart’s vehemence is less baffling than David Braben’s apparent silence, and the gaming press either neglecting to ask him for comment or failing to elicit comment. If anyone outside CIG is in a position to judge the challenges Star Citizen faces, Braben’s name would surely head that rather short list.

My own view is this: There are many factors that can actually reduce Star Citizen’s cost, and it comes down to selecting a design that meets the mad vision but that CIG can afford to actually build. As near as I can tell, that is exactly what CIG has aimed for.

Tightly integrated vs. loosely integrated

If Star Citizen is to be a single contiguous, seamless first-person experience, with no “loading” between scenes except perhaps for jumping between star systems, I would have to join the “can’t make it for $90 million” camp. You can probably have 100 Earth-sized planets with high-resolution terrain that also contain gargantuan handmade cities. But the cities will look terrible from 200km away, or framerates will be slideshow-esque and the terrain will sure look empty — like Earth but with only one city. Or you could hand-make all of that terrain too, but I can’t even begin to estimate the cost in time or dollars or what possible gameplay value all the empty terrain could have.

Whereas, a hundred or so large hand-made planetary landing scenes is at least in theory possible on a budget of tens of millions of dollars. It will be an extraordinary challenge and an extraordinary achievement if CIG can deliver so many environments with its extremely high standard of visual fidelity. But not impossible.

Similarly, the various modules of the game don’t all need to be jammed into the same environment, or even the same engine code. Squadron 42 can meet its specification without taking place within the Persistent Universe. It could be, in theory, a completely separate game, with PU citizenship and any other rewards being assigned via Steam Achievements, or at most a web call to the PU database.

The same can be said of the FPS module, to an extent: Placing SQ42 and Star Marine into the PU means all of their code needs to solve the 64bit problem, whereas if they’re kept separate, this can be avoided.

Loosely coupled modules can share code where it benefits the gameplay of the individual modules but otherwise avoid borrowing each other’s problems. What first-person shooter can possibly need to think about distances of 1 million kilometers? Why lump the Planetside module with all of the PU’s requirements or vice versa? What benefit does the end-user experience, in exchange for what cost?

The tighter the integration, the more coordination is required between the different studios and steams in different locations and time zones. Loose integration suits such a distributed production.

Tightly integrating these modules such that they actually run within the PU, rather than switching to them from the PU where appropriate, will add complexity, cost, and risk to the project, and probably has little to no value to the user experience, unless the goal is…

“And now, we shall tick the ‘MMO’ box!”

Another potential panic point, at least for me, has been the various allusions throughout Star Citizen’s ballooning scope to things like an MMO-like persistent universe. There are many good reasons most games studios don’t attempt an MMO, and why all of the advice to indies is to avoid it like the plague. Basically? Because it’s the plague.

An MMO is the most expensive and complex design choice you can possibly make. In some ways it makes the 64bit problem look like a school project. Moving from single-player to multi-player adds complexity. Moving to large numbers of players who expect to be able to interact with each other en masse, chat, trade, work together, work against each other, connect any time of the day or night, never lose any saved data, never have their accounts hacked, never lose out to a cheater or a scammer, and never be abused in chat by a troll… well, it’s games development in “Extreme difficulty” mode. “MMO” also makes people think of complex character sheets, and massive, massive, massive replayability.

Right now, Ascent has a review up on Steam by a player who logged over 2,000 hours in the game. And he gave it a thumbs-down. When you say “MMO,” people will expect a huge amount of “content” and varied gameplay. We’ve got sandbox empire building and procedural generation to give us that content — wacky math and the players themselves build all our environments for us. We couldn’t meet content expectations making it all by hand without spending tens of millions of dollars doing it.

Ascent is a tiny indie production with one developer (soon to be two!) and one artist, and yet we’ve got two separate replicating database clusters, one for property data, one for meta and environment data. We’ve got a very complex data model that shifts between traditional tables and tables of blobs (some of which are compressed) depending on the kind of data being stored and how they might be retrieved. I spent weeks agonising over our data-locking model, only to change it – twice – since we began early access on Steam. We’re using a distributed global caching system so that when you visit someone else’s gargantuan colony, you load it not from the game servers but from the same kind of system Netflix uses to distribute movies. We’ve got a load-balanced front-end server cluster, which is what you connect to; we’re using shared memory for interprocess communication; and I had to write our own ultralight client-server encryption system (because nothing off the shelf was fast enough) and extend libraries to work with it in three different languages. We do server updates live (no downtime) to meet player expectations. We can’t do resets. Not of anything, not ever. We have three kinds of backups into three separate locations to protect players’ data and property. I routinely restore test these onto our test sever, which by the way is a complete duplicate of production because those live updates need to be backwards and forwards compatible and atomic enough that I can roll them back if something goes wrong on the live system.

Now, scale that up to Star Citizen, today running at almost one million accounts. Imagine even 30% of them want to log in at once?

Now, imagine that even I can’t imagine all of the things they’ll need or the issues they’ll face.

Now you understand why my number one concern with Star Citizen is that it’ll bring the various components of the game together, and then say, “And now, we shall tick the ‘MMO’ box!”

If you’re making an MMO, you need to start with that on your very first whiteboard session and carry it through every day of design, development, testing, and production, and then years of fixing and adding content thereafter. It blows out your technical scope and risk, and it puts huge pressures on your business model.

However, at no point (that I know of), has Cloud Imperium actually called Star Citizen an MMO. MMO-like and persistent universe imply many of the same things, but hopefully, for the sake of the game, not all of them.

What I think Star Citizen will do is not stretch itself to try to meet all of the usual MMO expectations — only those that fit the game’s core design. The amount of content will be large, but limited. To balance this, it will be of extreme quality, something no MMO universally achieves with all of its content. Think Skyrim vs. Elder Scrolls Online, but picture what ESO could’ve been if Bethsoft had stepped back from the “MMO” cliff and just said, “You can play Skyrim with your friends, trade stuff in towns with everyone else playing at the time, and even join big battles against boss dragons!” Or, you know, a better compromise design that took more than 10 seconds to come up with, something that combined the best of both worlds, and only where the advantage was to the end product.

“More than multi-player” does not have to mean “leveling and upgrade treadmill with a raiding endgame.”

The important thing to understand, is that to meets its scope as described, Star Citizen doesn’t have to cast itself bodily into the MMO abyss. Its various modules can have differing levels of multi-player elements, as appropriate to their design and where it benefits the end product. SQ42 and Star Marine could have limited, instanced co-op and/or PvP battles, with the PU acting essentially as a (mind-blowingly beautiful and immersive) lobby for them, allowing you to “zoom in” to the battles — a model that has worked beautifully in many other games. Only the PU’s property database needs to be necessarily unified and slugged with the crazy bank-like data storage and locking/transaction requirements. Planetside can be instanced and sharded to keep costs down and frame-rates up. This alone is complex enough!

Technical conclusion: Yeah, it’s possible

I’ve spoken to a lot of games industry people about Ascent and about Star Citizen. Purely from a technical standpoint, obviously both games are “possible.” And hopefully, if you’re still reading at this point, you’re at the same conclusion.

But the technical possibility, in my view and theirs, was never really in doubt. The questions usually raised are:

• Can they deliver on that scope with that budget?
• Can they deliver something that lives up to all the hype?

And that comes down to project management, and, well, Avatar, the film. We’ll get to that in a moment.

I spent years studying, getting qualified in, and then practicing project management, so I’m qualified to tell you what an unmitigated trainwreck software project management is, has always been, and hopefully will not always be. I’ll spare you the theory paper and say that the bits of a new project I look at when evaluating it that are relevant to Star Citizen are risk management and scope management.

Risk management: Star Citizen carries a lot of what we call “Technical Risk,” loosely translated as “Stuff we have no idea how to do yet.” Much of that is because nobody’s done it yet, but some of it has been done before. What I’d look for comes down to two things:

• Moving all the risky stuff to the beginning of the project, to find out how bad the damage is and get the schedule and cost settled down ASAP, and
• Researching how the things that have been done before were done before, and hopefully, if you can afford it, going and finding the people who did it and getting them to help.

CIG, from an outsider’s perspective, appears to have failed comprehensively on both fronts (64bits in June 2015 reads like a miss, and apparently Smart offered them help at the outset and was ignored — also a miss), but the company has now paid for that failure by doing everything itself. We saw a bit of chopping and changing on the engine front and a few other painful- and expensive-looking things, and going 64bit a few months ago definitely appears to have untangled the log-jam and finally allowed some more tangible progress. So while risk management to date, examined by an outsider with hindsight and a complete lack of knowledge of what’s actually gone on inside the company, looks clumsy and expensive, it doesn’t appear to have killed the project, and hopefully the worst is behind us. Unless we’re about to leap off the MMO cliff with no MMO people on staff. Don’t do that.

Scope management: The other thing that kills any project stone dead is failed scope management. This can include

• Feature creep (what everyone seems to be labeling as Scope Creep at the moment – the blatant adding of whole features as we go along), and
• Scope creep, which is more normally what we call the scope increasing in nefarious ways, such as discovering pre-requisites we didn’t think of, stumbling over technology issues, gold plating, and people sneaking scope in when you’re not looking.

Star Citizen’s official scope has stopped growing, in the sense that new “stretch goals” are no longer being added. This is good… this potentially means someone has crash-tackled Chris Roberts and convinced him that CIG should release a product before adding more features.

However, the scope is still expanding, and it will continue to do so until release day, and then it will expand some more. No plan survives first contact with the players!

Scope creep

Looking down the list of stretch goals, I think it’s clear there’s little correlation between the money raised at each point and the scope added. At $57 million we’ve got the Endeavour ship with a whole host of modules and new game features attached, and at $58 million we’re giving everybody 10k space bucks, something the database administrators can do with an hour or two’s work at most.

Some of the items on the list look as if they might cost more than one million to implement; others, less. But what is clear is that based on the short paragraph each, there’s a lot of room to interpret the scope and the potential for things to cost more than we might expect. Usually in software projects, things cost more than we expect, and that presents risk to the project.

The Endeavour’s a good example. How will we attach and remove the modules? Where will they come from? How does science work? How does medicine work? The answers to these questions can either simplify or complicate the project’s scope.

Something that will happen — often, and moreso the more complex, tightly integrated and MMO-ish the project gets — is that designers, developers, and artists will find scope hidden between the lines of the existing scope. Some possible examples:

• “Dammit, now we need to make a spherical Earth-sized terrain engine!”
• “Hey, these complex face animations cause very low framerates in the 64bit PU engine!”
• “Who knows how to network 300,000 computers together?”

And a zillion other little details that pop up with even the most mundane business software projects, let alone computer game projects, let alone the most ambitious and crazy computer game ever attempted.

Depending on how much money’s left in the bank, scope management on this project is acceptably loose (after all, players want as much scope as possible) or a ghastly failure that’s about to kill the project or something in between. If I had to guess, and I don’t, I’d guess something in between, mostly because that’s what usually happens.

And that’s the kicker. Because the other half of why Derek Smart is so terrified for the project is that he claims to have emails from people claiming to work or have worked at CIG and know the financials, and he claims they claim that there’s $8 million left in the bank.

And if that’s the case, it’s very hard to see an acceptable outcome from here.

But if — and I find this somewhat more credible — they haven’t just driven themselves sideways into such a financial brick wall, we’re still likely to see a product, and an excellent one. But can it ride its own epic hype-train all the way to the end of the line?

Avatar

And this is why I’ve brought up Avatar. Avatar is a singularly excellent film. The acting is solid. The story is hardly original but perfectly adapted, the script is tight, the sci-fi is mass-market but the very top of that class, the visual effects are utterly mind-blowing, and it’s got Sigourney Weaver. But when it came out, most people I know derided it as inferior piffle. You probably did too. Do yourself a favour and watch it again now. You might find it to be… an excellent film!

Star Citizen runs the same risk. It’s now been hyped to such an extent and had so many people’s disparate hopes pinned on it that sometimes it’s hard to see what, if anything, could live up to the hype. If SC releases, and fails to MELT YOUR EYEBALLS WITH AWESOME, it risks being savaged by press and gamers alike.

Conclusion: This looks like every other big experimental game project ever

Over the weekend, I watched a Star Citizen video that included some really neat-looking stuff (which is normal) that was beginning to look like it might be a coherent game in the not too distant future (which is not), along with a new angle on Chris Roberts’ vision of the game. The Squadron 42 cutscenes look to be played out more like the friendly NPC encounters in Skyrim — fully interactive with dialogue trees and quite natural interaction (at least at first; the 507th time my wife thanked me for bringing the claw back to where it belonged, I googled “Skyrim divorce”). Suddenly, all the big celebrity names and expensive facial mocap made sense. Squadron 42 really is shaping up to become the modern incarnation of Wing Commander that everyone dreamed about… with the kind of NPC immersion Skyrim delivered but with Mark Strong.

And the demo of the PU was not only beautiful but began to really show how the game’s big ideas can be brought together into something entirely new. It also showed me CIG has largely sorted out, worked around, or completely avoided the “Going Big” technical issues.

In conclusion, here is a short list of things that virtually every ambitious computer game ever has had to do at least one of to ship a product at all:

• Needing more time, and then yet more time
• Needing more money, and then, more money
• Cutting scope, as most deliver 50-80% of what’s initially proposed
• Downsizing the studio to complete the game on budget
• Shipping an initial “minimum viable product” and adding the rest of the cool bits with expansion packs and DLC.

The question is not will Star Citizen need to do any of these but rather how many?

And can its fans cope with that?

And can its detractors be fair about it?