Insomniac Interview: The Tech Behind Marvel’s Spider-Man

In Marvel’s Spider-Man, launching exclusively on PS4 tomorrow, fans will experience an action-packed narrative that lives up to the legacy of the character’s rich history while traversing an expansive open-world Marvel’s New York using a suite of tightly tuned gameplay mechanics. And, of course, a stunning graphical presentation crafted by the maestros at Insomniac Games.

Whether you’re patrolling the streets of Manhattan, swinging at breakneck speeds across the city, scaling the tallest building you can find, or just mingling with citizens and being a friendly neighborhood Spider-Man, there’s a lot to appreciate. Insomniac’s attention to detail brings environments to life, while every move in Spidey’s repertoire feels both fluid and powerful.

As we quickly approach September 7, we’re thrilled to have had the chance to learn more about the advanced technology behind Marvel’s Spider-Man on PlayStation 4. Here is an edited transcript of our interview with Insomniac’s Mike Fitzgerald, Director of Core, and Al Hastings, Chief Architect.

PlayStation.Blog: The open-world Marvel’s New York you’ve built here is stunning — and it’s impressive to see how fluidly it renders while Spidey swings through it at breakneck speeds. How did you accomplish this from a technical perspective?

Mike Fitzgerald: Like many open-world games, we have to carefully parcel up our world up into small sections, since the whole island of Manhattan would be impossible to fit into the PS4’s memory at one time. There are approximately 800 square tiles making up our version of Marvel’s New York, each 128 meters square, making the city several times larger than what we built for Sunset Overdrive. As you swing through it, we’re constantly unloading tiles behind you as well as loading tiles ahead of you—seamlessly transitioning so you’re always exposed to the highest detail level. And when Spider-Man is really pushing his top speed, we’re loading about one new tile every second.

PlayStation.Blog: There’s a technical term — temporal injection — that Insomniac has used to great effect in Ratchet & Clank and now in Marvel’s Spider-Man. What is temporal injection, and how have you modified its implementation between your last PS4 Pro project and this one?

Al Hastings: Temporal injection is a technique we use to create a full 4k image on the PS4 Pro without rendering every pixel on every single frame. We render slightly more than half the pixels, then “inject” them into a full-sized 4k image. By slightly jittering the locations of the rendered pixels from frame to frame, we can reconstruct a very accurate anti-aliased full 4k image over a handful of frames. There are other teams using similar techniques, but with slight variations in the implementation.

Since Ratchet & Clank, we’ve made dozens of refinements to the algorithm. These serve both to make it run faster and to minimize its shortcomings, like the tendency to make the final 4k image slightly softer than it ought to be.

PlayStation.Blog: One of the first things a Spider-Man game needs to get right is the swinging. How did Insomniac approach this development challenge? Can you describe the physics that go into the perfect Spidey swing?

Mike Fitzgerald: Part of the fun of game development is to give the player the feel of something like swinging between buildings in Manhattan without actually making them do it. This extends to the physics behind it as well – if we perfectly simulated the dynamics of a person hanging from a rope on 34th Street, the player wouldn’t feel much like a Super Hero. But with neat effects like camera movement, field of view, and careful character animation, we can give the feel of being Spider-Man in an even more convincing way.

There are some nods to realism to make it more convincing, however. For example, every web you shoot to swing from needs to be anchored to an attachment point on the edge of a building or tree. Every piece of architecture in the city is tagged with places webs can attach, and as you swing around, we find the perfect swing points to preserve your momentum and keep you flying towards your destination.

PlayStation.Blog: What resolution does Marvel’s Spider-Man render at on PS4 Pro when connected to a 4K display? How does it render when connected to a 1080p display?

Mike Fitzgerald: On a PS4 Pro, the game outputs to the TV at 4k. Internally, most of the frame is rendered to a 2816×1584 or 2560×1440 image (in expensive situations), then projected over time into the master output 4k image using the “temporal injection” technique mentioned above to accurately anti-alias the picture. If you’ve got a PS4 Pro attached to a 4k TV, this is going to be a beautiful way to show it off.

On a base PS4 or when connected to a 1080p display, the internal representation is 1920×1080 to perfectly match the TV’s expected input.

PlayStation.Blog: What details can you share about the HDR implementation for displays that support it? Any details players should watch for when playing with HDR enabled?

Al Hastings: We’ve tried to find the right balance with our HDR implementation such that the medium tones and dark tones in the image look the same as they would in non-HDR, but the brightest parts of the image really pop. Swinging around the city at night or in the rain shows off HDR nicely—the bright lights of the city really stand out against the darker background. As you progress through the game and meet up with some of the more colorful villains Spidey needs to contend with, there are some gorgeous night scenes with bright effects—but you’ll have to play the game to see what I mean.

PlayStation.Blog: Anything else players should keep their eyes peeled for while playing the game?

Mike Fitzgerald: It’s immensely fun to swing at top speed around the city, way above the ground. But every once in a while, just go for a walk at street level and take in some of the little details that Insomniac’s amazing design and environment teams have built into our Manhattan. There are lots of little shout-outs to other Marvel characters, iconic pieces of New York City architecture to climb around, and friends to take selfies with in Times Square.

When you’re done on the ground, run to the top of the tallest building you can find, perch at its highest point, listen to the murmur of the city, and take in the vista around you—especially at sunset. Give your always-busy Super Hero a quick Zen moment before jumping back into crime-fighting.

PlayStation.Blog: Do you have any advice for developers who are considering developing for PS4 and PS4 Pro?

Mike Fitzgerald: Insomniac has always had a hardware-centric engine development approach. If you know the strengths of the hardware you’re developing for, and you take advantage of those strengths as specifically as possible, you can really deliver something of stunning technical quality. For us on PS4 and PS4 Pro, this meant carefully structuring our memory layout to optimize data flow between the CPU and GPU, using unique features of the GPU hardware in our shader code, carefully scheduling different processes on different cores of the CPU, and much, much more. Getting to develop this title for only the PS4 family of consoles let us be incredibly ambitious with where we could push our technology. If you want to deliver the best product you can, dig into the documentation, use the profiling tools, think about the quirky bits of the PS4 design, and figure out how your game can take advantage of them.

PlayStation.Blog: The story in the game is fantastic, and the cutscenes are full of detail—at times it feels like watching a Marvel movie. How did you need to push your technology to get to this point?

Mike Fitzgerald: Marvel’s Spider-Man has a notably more realistic art style than our last PS4 title, Ratchet & Clank, which had a cartoon-like look and feel. Beyond changing our lighting model to more accurately reflect physical materials, we also developed specific improvements to hair, eye, and skin rendering systems.

In the animation realm, we added support for large-scale destruction, which our artists put to good use to create some epic Super Hero-scale interactions. Our facial animation system can now accurately model the response of characters’ skin as they speak and emote, and it can represent the individual movement of each of the approximately 60,000 vertices in their faces. Every character has an extreme-quality model that’s used for close-ups, cutscenes, and scripted action sequences, and we built up the motion- and facial-capture processes we use for those moments to bring out the details of our actors’ performances. The final foe you face in the game is rendered with more than one million polygons—more detail than we’ve ever had for a single character in a game before!