Archive for category best practice

IPv6 transition and the pit of doom

In case anyone hasn’t noticed, I’m not a big fan of the IPv6 transition technologies, such as Teredo, 6to4, DNS64, etc. I also don’t like “big bang”, everything today, or “flag day” cutovers either. Fortunately, for IPv6 there’s a good middle ground, dual-stack.

Consumer-facing may be different, but here’s the strategy that I like for transitioning an internal, corporate or .EDU network to IPv6. You don’t have to build anything that you won’t be using for the future, so no spending on technological cul-de-sacs or band-aids. You won’t have to install, maintain, and then eventually turn off any transition mechanisms.  Just a steady, always forward, straightforward path that will eventually lead you to 100% IPv6, and eventually allow you to turn off IPv4, if you ever so desire. You won’t have to, you just may not have many people to talk to on IPv4 in a few years.

You can begin this transition today, given time and a steady plan to refresh your technology over the course of a few years. If you have an urgent need for IPv6, you can go faster. If you don’t, or are time or money constrained, you can go slower, taking years to make the switch.

  1. boot to the head to anyone who insists on any transition technology for our internal networks, without a very convincing argument
  2. dual stack the (internal, client) networks + provide “outbound” external dual-stack paths to the public Internet
  3. dual-stack DNS and DHCP and any thing else I need to manage and operate the clients, such as Active Directory
  4. dual stack the clients – if an OS doesn’t do IPv6, chuck it or confine it to the legacy pit of doom, an IPv4-only subnet (or two)
  5. dual stack the servers – do this on your regular refresh/upgrade cycle or as-needed, or if it can’t be upgraded, it goes into the pit
  6. dual stack any remaining services – your software vendors and internal developers have had enough time to sort out dual-stack network calls, or they go into the pit
  7. At this point your network, clients, servers and services are all dual-stacked. The users can reach everything on the public Internet, old or new, as well as all your internal services on both IPv4 and IPv6. And guess what? By this time, you’ll only have two reasons to keep IPv4 around:
    • Your users might still need to reach old crufty web sites and services out there in the real world
    • You might still need to talk to some things in the pit of doom
  8. Finally, after years, you can look into turning off IPv4. Start by pouring petrol into the pit of doom and lighting it on fire.
  9. Then start turning off IPv4, first on the the services and servers, then the clients, and eventually the network. Take your time, you’re in no hurry.  You can take years, if you like.

We’re doing 1,2 and 3 this year and next. We’ve started on 4 and that will continue for the next year or so. We can start on 5 as soon as we have time. New servers will be born dual-stack starting early next year. A year or three from now we clean up 6, and we’ll be at stage 7.

Then, and only then, do we think about Step 8, turning off IPv4. If that begins by 2017, I’ll be surprised. If Step 9 doesn’t start until 2020, I don’t care.

As long as I’ve got 1 – 4 done, I have a lot less pressure and I can proceed on a rational, non-panic’ed basis to replace servers and services as part of a regular refresh cycle.  For many companies, this is 3-4 (or even 5) years.

The consumer-facing case has completely different drivers and requirements, it gets a completely different plan and schedule. But I bet it won’t have any transition technologies in it, if I can help it.

This is rather cold-blooded and (I freely admit) a bit of a pipe dream. But if I can make this work, I’ll never have to justify, pay for, create, debug, support, and then decommission any of the transition technologies. I’ll always have a fall back if there’s a problem with a particular IPv6 implementation or if we run into a show stopper on vendor support for IPv6.

I hate building things that I know I’m only building as a temporary bridge. Spending time (and money) putting in the transition strategies can be better spent just moving forward, not sideways.

IPv6 just isn’t has hard as some might lead you to believe. There are still some implementation glitches here and there, but every day there’s better vendor support and fewer bugs.

Don’t be afraid to move forward, it’s easier than you think.


Leave a comment

IPv6 transition – just get on with it!

A recent DEFCON presentation highlights the need to accelerate adoption of IPv6 on your network. You can either turn off IPv6 on all your hosts (which will break things), or get on with it and deploy IPv6 “for real”.

(TL;DR: your hosts already have dual-stack activated, not having IPv6 supported in your network opens up a  man-in-the-middle (MITM) attack. Though long known, there is now a “one click” exploit available.)

There’s been a lot of discussion on various IPv6-related mailing lists with how to drive the transition, how to transition, and which (if any) of the transition technologies should be used.

In general, NONE of the transition technologies (other than dual-stack) address this particular MITM attack. They (for the most part) leave old IPv4 nodes as-is on your network, and try to translate protocols and hide IPv6 from those old nodes (and vice versa).

Personally, I find it quite heartening that many are making good business cases for aggressive adoption of native IPv6. Some are also providing good historical evidence that we’ve made similar transitions in the past, without extensive transition technologies, with good success:

On 8/8/13 1:40 PM, Ray Hunter wrote (v6ops):
Actually I think your reasoning and reference to the IPX and Appletalk
phase out would suggest it’s easier to make a bold call: move to IPv6
ASAP for critical systems via dual stack, and for the rest you draw a
box around it and call it legacy and run it on IPv4 until it dies a natural death.

IMHO Going half way with NAT64/DNS64 just prolongs the pain and locks
you into a transition technology that is expensive and difficult to
operate for the life cycle of that box, and which has to remain in place
until the last app is migrated or switched off.

I’ve been in a fair number projects where you sometimes just have to
dare to cut the cord whilst maintaining a process to find out what has
broken. So one valid IPv6 only migration strategy might be: “If it’s
important, they’ll migrate before a flag day date. Otherwise they get
cut off.”

I cannot agree enough with the “prolongs the pain”  and “locks you into a transition technology “observations.

At work, we’re going on the assumption that we’ll be able to go dual-stack and not need any translation. So far, that looks viable for our internal networks. When we get to the consumer-facing stuff, well, we’ll see.

Leave a comment

IPv6 adoption – still doubling every year

Good News, everyone!

IPv6 adoption continues to double, year on year.  Of course, that’s only three years of baseline, but things are certainly moving in the right direction.

As this article points out, if this rate continues, more than half of Internet users could have IPv6 within 6 years. This goes along with estimates of IPv6-only customers reaching 20% by 2017.

It remains to be seen if this adoption rate can continue.  However, events such as Switzerland moving from 3% to 10% adoption in a single month are interesting. They show that a single large ISP can quickly make a huge difference in adoption rates, as they turn up large portions of IPv6 connectivity in large deployment events. I expect Comcast to quickly begin to have a similar impact on US IPv6 availability later this year.

This CircleID article calls out some winners and losers in the IPv6 adoption race. My favorite IPv6 (tunnel) provider, Hurricane Electric gets high marks.

Leave a comment

Having “the talk” with your vendors

My company buys a fair amount of IT kit each year. We visit and are visited by vendors almost weekly. Lately, “the talk” has become part of the conversation: “How’s your IPv6 support?”

We’ve had this discussion with our network vendors for quite a while, now we’re talking to the rest of the vendors: storage, cloud services, middleware platforms, monitoring, and security.

A very select few of them have answered immediately: “Of course, we’ve had it for years. What can we do to help you with your IPv6 transition?”

Others have said, “It’s on our roadmap, about X months out, would you like to be in the Beta?”

But too many have responded, “IPv6? Is that important to you? You’re the first customer who has asked about it. We’ll get back to you…”

I predict a rocky next 5 years for the vendors in the the last group.  Smaller, more agile, more forward-thinking upstarts are going to make life “interesting” for those folks.

You should have “the talk” with your vendors. If they can’t help you move forward on IPv6, you’ll need to find alternatives that can.


Leave a comment

Total Cost of Ownership of dual-stack IPv4/IPv6 and Carrier Grade NAT (CGN)

INET Denver was held last month at the same time and location as the North American IPv6 Summit.  The theme was “IPv4 Exhaustion and the Path to IPv6″.

I missed the morning talks as they conflicted with my advanced IPv6 class, but I did catch the afternoon sessions.

It seemed like there were three camps at INET Denver: people already embracing the future by deploying IPv6, people trying to avoid IPv6 as long as possible, and people who planned to make money from both of the other two camps.

Let’s talk about the “wait as long as possible” camp.

For almost two decades the argument from the “business side” of IT was there was “no compelling business reason to move to IPv6“. (That article is is from 2009 by the way, but I haven’t seen a new argument since then.) It’s true, there’s been no “killer app” that everyone demanded that was only available via IPv6. It’s also true that doing nothing was a legitimate strategy for quite a while. After all, what good is it to have a telephone (IPv6), if no one else has one? Until recently, moving to IPv6 truly didn’t have a compelling business argument. After all, doing nothing costs nothing.  Mostly.

The Internet has changed. And we’re (almost) out of IPv4 addresses, so you have to do something. Sadly, too many ISPs have tried to do what they think is the cheapest and most minimal amount of work they could get away with. That’s Carrier Grade NAT (CGN).

The economics have changed, too.  Lee Howard of Time Warner Cable had a very interesting talk where he deconstructed, and then destroyed the myth that CGN is cheaper to deploy than dual-stack.  Since he’s the Director of Network Technology for Time Warner Cable, I guess he knows more about the ISP business than most people.

Mr Howard’s talk shows that CGN will cost you in (unhappy) customers, support costs, and only delay the inevitable, when you’ll have to move to dual-stack anyway.  His talk effectively demonstrates that the infrastructure and operational costs of a CGN network are more expensive than dual-stack.

There’s your business case. Deploy IPv6 and save money. Done. Now get to work.

1 Comment

IPv6 – creating an IPv6 address plan – Global Routing Prefix, sites and subnets

This post looks at sizing the IPv6 Global Routing Prefix and creating the subnet plan for the previously defined hypothetical company.

From the prior post, remember that we have these constraints:

  • The assignment of the Global Routing Prefix is gone by an Internet Registrar according to their policies. You must justify the size of the allocation you request.
  • Subnets are “always” on a /64 boundary (host identifiers are “always” 64 bits)
  • “Sites” are groups of subnets on a /48 boundary
  • Only networks with prefixes at /48 or larger are considered “publicly routable” by most ISPs. They won’t announce routing data for anything smaller.

The first thing to look at is the needed size of the address prefix. Here’s a modified diagram from RFC 4291. This one includes the specification that the Interface ID is fixed at 64 bits.

   The general format for IPv6 Global Unicast addresses is as follows:
   |         n bits         |   m bits  |       128-n-m bits         |
   | global routing prefix  | subnet ID |       interface ID         |
   |         P bits         |   S bits  |         64 bits            |

What we need to figure out is what is the size of prefix (P bits) we need, in order to get enough subnets (S bits) to create a reasonable network architecture. There’s no real limit to the number of hosts in a subnet, but subnets are used for all kinds of things including routing and access decisions. Since this company is in North America, we’ll use policies from ARIN.

The ARIN Number Resource Policy Manual (NRPM) uses “sites” as the determining number to determine the prefix size, so let’s look at the “sites” in this company.

While there are only six office locations, there are actually more “sites”. Two locations actually have four sites each, as they each house four completely unique sub-organizations, each meeting the definition of “site” from the NRPM. Two more locations each house two sites, and the last two locations each house a single site. At least two of the locations have their own Internet connections, meaning that they must have at least /48 assignments to be able to announce their routes publicly, which is additional justification that there are multiple sites in some locations. That’s 14 sites in six locations.

In the three co-location facilities, there are independent complexes of independent consumer facing services and extensions of the office (internal) networks for DR.  At each so-lo, these consumer services are in distinct “sub-facilities”, each leased to a separate business entity and a unique site.  There are six sub-facilities spread across the three hosting locations. The sub-facilities have separate ISPs and must be able to announce their own public routes, providing additional justification that the sub-facilities must be distinct sites. Two co-lo facilities host DR sub-facilities which are also separate sites. Two co-lo’s also have internal services that are used by the office sites. This means that three co-lo facilities actually contain 10 unique sites.

That’s a total of 24 sites in all. Per NRPM Section, the allocation of a /40 prefix is justified.

We now know that P is 40 bits, the host ID is 64 bits so we have 24 bits for “subnet”. We also need to work in the /48 definition for “site”, so we end up with something that looks like this. We’re switching from /prefix notation to showing the actual IPv6 address format, which is how people will see the address plan:


In this format:

  • PPPP:PPPP:PP represents the /40 IPv6 address prefix assigned to us by ARIN.
  • CC represents an 8-bit (2 nibble) “site code” that represents a location, usage, organizational unit or other network “slice” as needed. There are 256 site codes in this plan, numbered 0x00 through 0xFF. A “site” is a /48 prefix that may be announced publicly via an ISP for Internet routing. Sites may be internal (behind a firewall, not announced) or external (publicly announced and routed) as defined by each region.
  • SSSS represents a 16-bit (4 nibble) network (subnet) number. These are the traditional “subnets” as used in IPv4, there are just more of them and they are larger. Subnets are on the /64 prefix boundary.  Subnets are unique within a single site code, but are not unique beyond site code boundaries. There are 65536 possible subnets per site code, numbered 0x0000 through 0xFFFF. Subnets are NOT publicly routable and will not be accepted by most ISPs for public routing.
  • HHHH:HHHH:HHHH:HHHH represents the 64-bit (16-nibble) host interface identifier. This is the same as the host part of an IPv4 address; it is just much larger.  Host identifiers can be assigned in many ways including SLAAC, DHCPv6 or by static assignment.

Leave a comment

IPv6 – creating an IPv6 address plan – the hypothetical company

This post begins a series that will create a sample IPv6 address plan for a medium sized company with multiple office sites and multiple hosting locations.

In prior posts, I’ve written about IPv6 address plans in general and shown an especially interesting plan from UCSD.EDU. This post begins to “do the math” for a hypothetical company to produce a concrete addressing plan. This company is similar to my employer but I’ve thrown in a few things that we don’t have that many other more typical companies will have.

For the purpose of this thought experiment, we’ll use a company that looks like this:

  1. six office locations (cities)
  2. three co-location facilities that host consumer-facing services
  3. is part of a larger multi-national, but there is no higher-level single global network

For the office locations we’ll use these criteria:

  • around 5000 employees total
  • some locations have multiple large groups
  • most users have at least four devices that need an IP address, many have six and some have 10
  • two of the locations have their own Internet connections as well as a connection on the internal WAN
  • some locations have multiple Internet connections from multiple providers

The co-location facilities look like this:

  • up to 5000 hosts (or instances) per location
  • multiple Internet connections from multiple providers
  • each location houses sub-facilities that are leased to other business units (separate private clouds and the like)
  • each sub-facility needs to be independently routable via separate ISPs (not all use the same ISPs)
  • also used for Disaster Recovery (DR) for the offices

With this information, we can begin to create the IPv6 address plan.  Creating the plan begins with determining the Assigned Global Routing Prefix and subnet sizes and concludes with subnet numbering and routing plans.

The constraints we have to work within are:

  • The assignment of the Global Routing Prefix is gone by an Internet Registrar according to their policies. You must justify the size of the allocation you request.
  • Subnets are “always” on a /64 boundary (host identifiers are “always” 64 bits)
  • “Sites” are groups of subnets on a /48 boundary
  • Only networks with prefixes at /48 or larger are considered “publicly routable” by most ISPs. They won’t announce routing data for anything smaller.

Next time, I’ll look at these constraints and how they factor into the size of a desired Global Routing Prefix and creating a site and subnet plan.

, ,

1 Comment

IPv6 – an interesting address plan (UCSD)

Last year I came across an interesting IPv6 address plan, from the University of California San Diego (UCSD.EDU).  Their networking group presented their IPv6 implementation status and address plan at the annual on-campus IT conference.  Their address plan has some interesting features that I haven’t seen elsewhere.

UCSD is a large campus and currently has an IPv4 /16 (“class B”) and multiple IPv4 /24 assignments. UCSD also has an IPv6 /32 assignment.  The campus spans about 2000 acres and serves about 30,000 students. The campus is large enough that having intra-campus geographically-based routing is useful, and there are about 30 main network nodes identified for this use.

UCSD’s address plan is:

  • 2607:f720::/32 is UCSD’s assigned IPv6 prefix
  • LR is actually vlrrrrrr in binary
    • “v” bit is 0 (zero) for this version of the address plan
    • “l” bit is “local”, meaning that any packet to or from this address is to be dropped at the campus net boundary
    • “rrrrrr” is 6 bits that indicate the campus region, or major network node
  • there are four zero bits at /40
  • UUU identifies an organizational unit (department, lab, etc)
  • SS provides 256 separate subnets per business unit
  • <host> is a 64 bit Interface ID

This plan has a few unique features I haven’t seen in any other IPv6 address plan: a versioning scheme and the “local” bit. Note that there are also 4 bits (at /40) that are defined as “0” (zero).

The “version” bit does cut the present number of addresses in half, but that still leaves an astronomical number of addresses available, with the flexibility of having completely different address plans in the future, all coexisting.

The “local” bit is a kind of RFC-1918 (or at least peudo-NAT) replacement.  Any address with the “l” bit set will be unreachable from the “outside” and unable to reach “outside” as all traffic to or from addresses with the “l” bit will be dropped at the campus network boundary.

They will also be delegating /56s or /60s to clusters, virtual machines, etc. Since UCSD (and SDSC.EDU) run a fair number of supercomputers and clusters of machines, being able to delegate large subnets is useful.

(My company is using OpenStack to build our own private cloud.  OpenStack wants to dynamically DHCP large groups of machines as needed, so I can see why UCSD is reserving these large blocks.)

Having so much address space available offers all kinds of opportunities to encode information into the addresses themselves. Only time will tell if this is a good use of the large address space or not.

1 Comment

IPv6 – address planning and the structure of an IPv6 address

Defining an IPv6 address plan is an important process.  Whatever you create will live on for years.  Some analysis and thought up front can save time and pain later.

Before we dive into addressing plans, it is useful to look at the actual structure of an IPv6 address.  While there’s lots of talk of “340,282,366,920,938,000,000,000,000,000,000,000,000 unique IP addresses”, that sort-of assumes that all addresses are usable (for any purpose).

Creating an address plan for all that would be a truly daunting task 🙂  Fortunately (for our purposes) a lot of the space is reserved and there’s some internal structure that we can take advantage of to simplify creating an address plan.

Over the years, many kinds and flavors of IPv6 addresses have been defined and some later removed (“deprecated”), such as “Site-local Unicast”. Also, restrictions or better definitions have been made for some address parts, such as the “Interface Identifier”, which will become important below.

Before we start, go read RFC4291.  Go ahead, I’ll wait.  Really, go read (or at least skim) it. You want to get a few things from this RFC…  First, the standard hex notation for IPv6 addresses (Sec 2.2).  Second, the prefix notation (Sec 2.3). And third, which will become important later, is Section 2.5.1, which specifically defines the size of the Interface ID as 64 bits.

Now let’s look at a regular old IPv6 address.  From RFC4291:

   The general format for IPv6 Global Unicast addresses is as follows:

   |         n bits         |   m bits  |       128-n-m bits         |
   | global routing prefix  | subnet ID |       interface ID         |

Where does the global routing prefix come from?  This is the address assignment for your network which comes from either your ISP for provider aggregatable address space (PA-space), or from a Regional Internet Registry (RIR)  for a provider independent address space (PI-space) assignment.

If you’re a home user, IPv6 tunnel user, or a (small) end point business, you’re likely to have your address assigned from the pool that was assigned to the upstream ISP,  provider (aggregatable) space. The drawback here is that if you change providers, your IPv6 addresses are going to change.

If you’re an ISP, not-small company or any organization that is multi-homed through multiple ISPs, you’re going to want provider independent space. Each RIR has different policies for address assignments, including the size of the assignment.

The important thing about that prefix is that you have no real control over it, either its size or its content. It is assigned to you and that’s it.

So, the Interface ID is always 64 bits, and the global routing prefix is fixed and assigned.  That means that all you really need to worry about to create an IPv6 addressing plan is the subnet ID. Everything else is of a predetermined size, and in the case of the prefix, the content is also fixed.

The IPv6 address plan is really about how big your subnet ID is, and how it is broken down by purpose, location or any other information you may want to encode. Which is we’ll look at next time.

1 Comment

IPv6 – address plans

Space is big. You just won’t believe how vastly, hugely, mind- bogglingly big it is. I mean, you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.

— The Hitchhiker’s Guide to the Galaxy, Douglas Adams

The IPv6 address space offers some challenges to the network architect. It’s vastly different in scope and scale from our address-constrained IPv4 world.  Network Address Translation (NAT), subnets-of-subnets, and other familiar workarounds just aren’t needed or helpful.

The biggest challenge in creating an IPv6 address plan may be overcoming decades of IPv4 planning experience.  So much of “best practice” in IPv4 is exactly “worst practice” in IPv6.

Over the next several posts, I’ll be looking at how to create IPv6 address plans, registrar recommendations, IETF Best Current Practices, and practical considerations. I’ve found a few interesting address plans from some research organizations, too. While most of this won’t be needed by the home user, it may help understand what you’re seeing from your home IPv6 network.

Leave a comment

%d bloggers like this: