Development Diary: Building Communities on PS4

It took a community to build Communities for PS4.

Clan support on a console sounds easy enough to do, but consider… it’s never been done before, and maybe for good reasons. It’s big. It’s messy. It impacts everything. It’s not easy. That’s why building Communities for PS4 appealed to Khoi Do, a senior engineer on the PlayStation Network team. “I actually switched teams because of this feature,” says Do. “There were a lot of question marks. I was a little worried, but that made it fun.”

Do leads the Gambit scrum development team at Sony Network Entertainment International (SNEI), the division behind the PS4 user interface and PSN. Gambit’s team covers PS4 social features, so Communities was destined to fall squarely on them. But there was no way they could do it alone.

Communities, released in PS4 System Software 3.00, came out of a hackathon coding competition presentation by designer Dominic Graziano, together with engineers Daryl Nelson and Zack Argyle, and the idea was simple enough: make it quick and easy to connect and play with other people who share your interests. Looking for a party? Want to raid? A Community gives you a direct line to the players you need.

Right there, you’re talking about tapping into the PSN Friends list, Party, What’s New, Who’s Playing, Messages, and Notifications — components overseen by multiple teams. All that still doesn’t get you to the front door in terms of data management or feature interface.

“We worked with Tokyo for all the game-session information,” says Do. On the domestic side, Graziano started drawing up a UI while another scrum team, Cerebro, made the Community wall possible. Even with all those people already committed, it didn’t take long before Do realized delivering his Day 1 requirements depended on recruiting a third social-features team.

“I knew about it from the hackathon,” says Nick Hunter, head of the Stark scrum team. “It’s always been a feature with big aspirations.” Now it depended on his availability.

SNEI puts all its teams from around the world into one big room four times a year, and that’s where the next development cycle solidifies… after a certain amount of haggling, juggling, rearranging, and reality checks. Hunter went through all those stages, then made the call. Stark was in. “We need to get this out there,” he remembers thinking. “There’s a clear need, and there are so many benefits to having it on-platform, using real data about who’s on, who’s playing.” Stark took that on.

Then engineers settled in to do what engineers love to do: solve problems. “We are an open-source shop,” says Do. “We do a lot of valuation of whether this technology fits this problem. Then you have to get it to work.”

That meant leveraging cloud-hosted services to scale fast across availability zones, Cassandra database management to store and retrieve every post, message, invitation, and join, and Apache Tomcat open-source web server to deploy APIs. Meanwhile, the Stark team built shared components to bring all-important Now Playing and What’s New features in line with Communities’ needs. When complications came up, Do and Hunter made the 30-foot walk to each other’s desk to get it sorted out.

But they couldn’t depend on existing systems for everything. Customization, in the form of images and wallpaper, was one of the bigger asks. That tool had to be built, and the data pull had to be practical.

“This is the first time we’ve hosted user-image uploads on this scale,” says Do. They quickly latched on to edge-based content delivery networks to store all those images and uploads “closer” to the Community itself. That knocked latency issues down from an expensive fetch to the blink of an eye.

All told, “We spun up five new services for Communities,” says Do. Then everyone sat back, had a beer, and watched it go live at midnight with a code push, wondering if anyone would notice.

Since communities launched, millions of PlayStation players have joined at least 400,000 communities.

“You hope that your features get to that level of engagement,” says Hunter. “It was awesome.”

And it’s not done yet. Now that Communities is live, the Gambit team is going right back into planning for Day 2 features. The team is listening to user feedback and considering a number of features including search. “The size of my Friends list has grown,” he says. “I’m meeting more people. I’m using the platform more. It was fun to develop, and it’s fun to use.”

Now he wants to make it even better… with a little help from his friends.