Why I killed our IPv6 project…

Seven years ago we started an “IPv6 project”. The goal was to deploy IPv6 throughout our internal game studio network. After two months of analysis, I approached my boss and recommended we kill it. At least as an “IPv6 project”. It was reborn as a “clean up our network architecture, and oh by the way, add IPv6 (and a bunch of other things)”.

IPv6 for its own sake offered no value to the business at all. The problems with our network were not going to be fixed by adding IPv6. But there was an opportunity for a truly transformational project, a new global network architecture, designed for our business today, not the business we were 20 years ago. A new network, designed for collaboration,  ease of management, service agility, that happened to include IPv6 as one of its (many) new features.

Our internal network was (and still is) complicated. Change is difficult and takes too much time. It is difficult to respond quickly to new network and service needs. It is difficult for our users in various offices around the world to collaborate effectively. Our network, like so many other large networks, grew through osmosis over decades of rapid organic change.

About 20 years ago, what is now our internal network was three completely separate networks contained in just three buildings, in three countries, with about 300 users total. Throughout the regions (Americas, EU, Japan) we’ve had many major acquisitions and expansions over the years, and each acquisition brought a new pre-existing network that had to be integrated as quickly as possible into its respective regional network. In 2005, we began to inter-connect (but not integrate) the three regional networks.

Today our internal networks support 14 game development studios around the world. Almost  4000 people with at least 15000 devices use the networks every day to create award-winning video games for Playstation game consoles. (Our consumer-facing networks support all the many thousands of people who play our games each day, but that’s a story for a future post.)

The problem with our network was not that it didn’t have IPv6. The problem was that it was the result of a merger of dozens of networks over a 20 year period. When the original pieces of our networks were first conceived (in the early 1990s), the first Playstation console was still in the future, cell phones were the size of bricks, online gaming was in its infancy, and there was (practically) no world wide web.

I didn’t see this network until the early 2000s, and it was already the case that our network was the most complex network many of us had ever worked on, and our backgrounds ranged from ISPs, to government agencies, to supercomputing, to large research universities.

By around 2011 it was past time to think about re-designing the entire global network from scratch, and we began. Just getting to a new design was not an overnight process. We had to start with current and future needs, learn about IPv6, create technical guiding principles, create global governance, and even mundane things like IP address plans.

IPv6 is a key technology that will help us simplify, re-factor and re-imagine our worldwide network. Too much of the complexity of our network was due to the vision of IPv4 addresses as a precious commodity to be hoarded, and to the avoidance of network renumbering during acquisitions. We joked that from some parts of our network to another, a single packet might be NATed, re-NATed, over-NATed, under-NATed, de-NATed, and then NATed again. Not quite true, but an indication of the complexity of our inter-connected networks.

The network we are (still) building is a completely new design. We started with an empty whiteboard, network and technology architects from all our global regions, and a list of all the things that we felt were wrong with our network, and all things that were right. We designed a new network that is focused on the way our business works today, that enables easy collaboration between our development studios, and that make it easier to bring new technologies to our users.

The new network we are building (and will continue to build) is a much simpler network, with much of the legacy and IPv4 complexity left behind. In concept it is an IPv6 network, that just happens to still support IPv4 for a while. We tried to leave behind all the IPv4-thinking, such as address scarcity, the need for slicing subnets ever smaller into /26, /30, Network Address Translation, and private IP space (RFC 1918). There are no transition technology cul-de-sacs.

We still have legacy IPv4, and RFC 1918 space, and all our old networks, but our network team has been ripping out NATs, legacy firewall rules and other accumulated cruft.

Our EU network team worked with the rest of the IT organization to completely renumber thousands of devices, including desktops, Playstation devices, printers and such to the new IPv4 address plan, in support of the network simplification.

The new net (as it is being built out) will be fully dual-stacked for now, and frankly, for the foreseeable future. All IPv6 addresses are Globally Unique Addresses, and our “site” subnets are /48s so that they can be individually announced via BGP to all our network providers when or if necessary.

We’re rolling out the new network as we have time and needs. We’re building the new network a few subnets at a time, then slowly moving services, users, their devices and desktops onto the new network. All our client devices are already dual-stack capable, and we are dual-stacking servers and services when we move them, too. When the old IPv4-only subnets are empty, they are just decommissioned.

(In 2016, we merged two large companies  into a single company. In 2017 we added in another company. All the complexity I’m describing below was before that merger even started. We’re now working towards the merger of three separate company networks, I’ll get to that in a future post…)

Advertisements
  1. #1 by Infostack on February 2, 2018 - 6:09 am

    It would be interesting if a follow-up blog came out putting economic numbers and ROI behind all the technology description.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: