Archive for category best practice

IPv6 – creating an IPv6 address plan – Global Routing Prefix, sites and subnets

This post looks at sizing the IPv6 Global Routing Prefix and creating the subnet plan for the previously defined hypothetical company.

From the prior post, remember that we have these constraints:

  • The assignment of the Global Routing Prefix is gone by an Internet Registrar according to their policies. You must justify the size of the allocation you request.
  • Subnets are “always” on a /64 boundary (host identifiers are “always” 64 bits)
  • “Sites” are groups of subnets on a /48 boundary
  • Only networks with prefixes at /48 or larger are considered “publicly routable” by most ISPs. They won’t announce routing data for anything smaller.

The first thing to look at is the needed size of the address prefix. Here’s a modified diagram from RFC 4291. This one includes the specification that the Interface ID is fixed at 64 bits.

   The general format for IPv6 Global Unicast addresses is as follows:
   |         n bits         |   m bits  |       128-n-m bits         |
   | global routing prefix  | subnet ID |       interface ID         |
   |         P bits         |   S bits  |         64 bits            |

What we need to figure out is what is the size of prefix (P bits) we need, in order to get enough subnets (S bits) to create a reasonable network architecture. There’s no real limit to the number of hosts in a subnet, but subnets are used for all kinds of things including routing and access decisions. Since this company is in North America, we’ll use policies from ARIN.

The ARIN Number Resource Policy Manual (NRPM) uses “sites” as the determining number to determine the prefix size, so let’s look at the “sites” in this company.

While there are only six office locations, there are actually more “sites”. Two locations actually have four sites each, as they each house four completely unique sub-organizations, each meeting the definition of “site” from the NRPM. Two more locations each house two sites, and the last two locations each house a single site. At least two of the locations have their own Internet connections, meaning that they must have at least /48 assignments to be able to announce their routes publicly, which is additional justification that there are multiple sites in some locations. That’s 14 sites in six locations.

In the three co-location facilities, there are independent complexes of independent consumer facing services and extensions of the office (internal) networks for DR.  At each so-lo, these consumer services are in distinct “sub-facilities”, each leased to a separate business entity and a unique site.  There are six sub-facilities spread across the three hosting locations. The sub-facilities have separate ISPs and must be able to announce their own public routes, providing additional justification that the sub-facilities must be distinct sites. Two co-lo facilities host DR sub-facilities which are also separate sites. Two co-lo’s also have internal services that are used by the office sites. This means that three co-lo facilities actually contain 10 unique sites.

That’s a total of 24 sites in all. Per NRPM Section, the allocation of a /40 prefix is justified.

We now know that P is 40 bits, the host ID is 64 bits so we have 24 bits for “subnet”. We also need to work in the /48 definition for “site”, so we end up with something that looks like this. We’re switching from /prefix notation to showing the actual IPv6 address format, which is how people will see the address plan:


In this format:

  • PPPP:PPPP:PP represents the /40 IPv6 address prefix assigned to us by ARIN.
  • CC represents an 8-bit (2 nibble) “site code” that represents a location, usage, organizational unit or other network “slice” as needed. There are 256 site codes in this plan, numbered 0x00 through 0xFF. A “site” is a /48 prefix that may be announced publicly via an ISP for Internet routing. Sites may be internal (behind a firewall, not announced) or external (publicly announced and routed) as defined by each region.
  • SSSS represents a 16-bit (4 nibble) network (subnet) number. These are the traditional “subnets” as used in IPv4, there are just more of them and they are larger. Subnets are on the /64 prefix boundary.  Subnets are unique within a single site code, but are not unique beyond site code boundaries. There are 65536 possible subnets per site code, numbered 0x0000 through 0xFFFF. Subnets are NOT publicly routable and will not be accepted by most ISPs for public routing.
  • HHHH:HHHH:HHHH:HHHH represents the 64-bit (16-nibble) host interface identifier. This is the same as the host part of an IPv4 address; it is just much larger.  Host identifiers can be assigned in many ways including SLAAC, DHCPv6 or by static assignment.

Leave a comment

IPv6 – creating an IPv6 address plan – the hypothetical company

This post begins a series that will create a sample IPv6 address plan for a medium sized company with multiple office sites and multiple hosting locations.

In prior posts, I’ve written about IPv6 address plans in general and shown an especially interesting plan from UCSD.EDU. This post begins to “do the math” for a hypothetical company to produce a concrete addressing plan. This company is similar to my employer but I’ve thrown in a few things that we don’t have that many other more typical companies will have.

For the purpose of this thought experiment, we’ll use a company that looks like this:

  1. six office locations (cities)
  2. three co-location facilities that host consumer-facing services
  3. is part of a larger multi-national, but there is no higher-level single global network

For the office locations we’ll use these criteria:

  • around 5000 employees total
  • some locations have multiple large groups
  • most users have at least four devices that need an IP address, many have six and some have 10
  • two of the locations have their own Internet connections as well as a connection on the internal WAN
  • some locations have multiple Internet connections from multiple providers

The co-location facilities look like this:

  • up to 5000 hosts (or instances) per location
  • multiple Internet connections from multiple providers
  • each location houses sub-facilities that are leased to other business units (separate private clouds and the like)
  • each sub-facility needs to be independently routable via separate ISPs (not all use the same ISPs)
  • also used for Disaster Recovery (DR) for the offices

With this information, we can begin to create the IPv6 address plan.  Creating the plan begins with determining the Assigned Global Routing Prefix and subnet sizes and concludes with subnet numbering and routing plans.

The constraints we have to work within are:

  • The assignment of the Global Routing Prefix is gone by an Internet Registrar according to their policies. You must justify the size of the allocation you request.
  • Subnets are “always” on a /64 boundary (host identifiers are “always” 64 bits)
  • “Sites” are groups of subnets on a /48 boundary
  • Only networks with prefixes at /48 or larger are considered “publicly routable” by most ISPs. They won’t announce routing data for anything smaller.

Next time, I’ll look at these constraints and how they factor into the size of a desired Global Routing Prefix and creating a site and subnet plan.

, ,

1 Comment

IPv6 – an interesting address plan (UCSD)

Last year I came across an interesting IPv6 address plan, from the University of California San Diego (UCSD.EDU).  Their networking group presented their IPv6 implementation status and address plan at the annual on-campus IT conference.  Their address plan has some interesting features that I haven’t seen elsewhere.

UCSD is a large campus and currently has an IPv4 /16 (“class B”) and multiple IPv4 /24 assignments. UCSD also has an IPv6 /32 assignment.  The campus spans about 2000 acres and serves about 30,000 students. The campus is large enough that having intra-campus geographically-based routing is useful, and there are about 30 main network nodes identified for this use.

UCSD’s address plan is:

  • 2607:f720::/32 is UCSD’s assigned IPv6 prefix
  • LR is actually vlrrrrrr in binary
    • “v” bit is 0 (zero) for this version of the address plan
    • “l” bit is “local”, meaning that any packet to or from this address is to be dropped at the campus net boundary
    • “rrrrrr” is 6 bits that indicate the campus region, or major network node
  • there are four zero bits at /40
  • UUU identifies an organizational unit (department, lab, etc)
  • SS provides 256 separate subnets per business unit
  • <host> is a 64 bit Interface ID

This plan has a few unique features I haven’t seen in any other IPv6 address plan: a versioning scheme and the “local” bit. Note that there are also 4 bits (at /40) that are defined as “0” (zero).

The “version” bit does cut the present number of addresses in half, but that still leaves an astronomical number of addresses available, with the flexibility of having completely different address plans in the future, all coexisting.

The “local” bit is a kind of RFC-1918 (or at least peudo-NAT) replacement.  Any address with the “l” bit set will be unreachable from the “outside” and unable to reach “outside” as all traffic to or from addresses with the “l” bit will be dropped at the campus network boundary.

They will also be delegating /56s or /60s to clusters, virtual machines, etc. Since UCSD (and SDSC.EDU) run a fair number of supercomputers and clusters of machines, being able to delegate large subnets is useful.

(My company is using OpenStack to build our own private cloud.  OpenStack wants to dynamically DHCP large groups of machines as needed, so I can see why UCSD is reserving these large blocks.)

Having so much address space available offers all kinds of opportunities to encode information into the addresses themselves. Only time will tell if this is a good use of the large address space or not.

1 Comment

IPv6 – address planning and the structure of an IPv6 address

Defining an IPv6 address plan is an important process.  Whatever you create will live on for years.  Some analysis and thought up front can save time and pain later.

Before we dive into addressing plans, it is useful to look at the actual structure of an IPv6 address.  While there’s lots of talk of “340,282,366,920,938,000,000,000,000,000,000,000,000 unique IP addresses”, that sort-of assumes that all addresses are usable (for any purpose).

Creating an address plan for all that would be a truly daunting task :-)  Fortunately (for our purposes) a lot of the space is reserved and there’s some internal structure that we can take advantage of to simplify creating an address plan.

Over the years, many kinds and flavors of IPv6 addresses have been defined and some later removed (“deprecated”), such as “Site-local Unicast”. Also, restrictions or better definitions have been made for some address parts, such as the “Interface Identifier”, which will become important below.

Before we start, go read RFC4291.  Go ahead, I’ll wait.  Really, go read (or at least skim) it. You want to get a few things from this RFC…  First, the standard hex notation for IPv6 addresses (Sec 2.2).  Second, the prefix notation (Sec 2.3). And third, which will become important later, is Section 2.5.1, which specifically defines the size of the Interface ID as 64 bits.

Now let’s look at a regular old IPv6 address.  From RFC4291:

   The general format for IPv6 Global Unicast addresses is as follows:

   |         n bits         |   m bits  |       128-n-m bits         |
   | global routing prefix  | subnet ID |       interface ID         |

Where does the global routing prefix come from?  This is the address assignment for your network which comes from either your ISP for provider aggregatable address space (PA-space), or from a Regional Internet Registry (RIR)  for a provider independent address space (PI-space) assignment.

If you’re a home user, IPv6 tunnel user, or a (small) end point business, you’re likely to have your address assigned from the pool that was assigned to the upstream ISP,  provider (aggregatable) space. The drawback here is that if you change providers, your IPv6 addresses are going to change.

If you’re an ISP, not-small company or any organization that is multi-homed through multiple ISPs, you’re going to want provider independent space. Each RIR has different policies for address assignments, including the size of the assignment.

The important thing about that prefix is that you have no real control over it, either its size or its content. It is assigned to you and that’s it.

So, the Interface ID is always 64 bits, and the global routing prefix is fixed and assigned.  That means that all you really need to worry about to create an IPv6 addressing plan is the subnet ID. Everything else is of a predetermined size, and in the case of the prefix, the content is also fixed.

The IPv6 address plan is really about how big your subnet ID is, and how it is broken down by purpose, location or any other information you may want to encode. Which is we’ll look at next time.

1 Comment

IPv6 – address plans

Space is big. You just won’t believe how vastly, hugely, mind- bogglingly big it is. I mean, you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.

— The Hitchhiker’s Guide to the Galaxy, Douglas Adams

The IPv6 address space offers some challenges to the network architect. It’s vastly different in scope and scale from our address-constrained IPv4 world.  Network Address Translation (NAT), subnets-of-subnets, and other familiar workarounds just aren’t needed or helpful.

The biggest challenge in creating an IPv6 address plan may be overcoming decades of IPv4 planning experience.  So much of “best practice” in IPv4 is exactly “worst practice” in IPv6.

Over the next several posts, I’ll be looking at how to create IPv6 address plans, registrar recommendations, IETF Best Current Practices, and practical considerations. I’ve found a few interesting address plans from some research organizations, too. While most of this won’t be needed by the home user, it may help understand what you’re seeing from your home IPv6 network.

Leave a comment

Security – why programmers should study computing history

You can now add LinkedIn, eHarmony and to the long list of major sites that have had poor password security in their user database designs.  The saddest part is that in the case of LinkedIn, at least, this was apparently completely avoidable. (I haven’t found enough details to comment on the others, yet.)

Protecting stored user passwords is not rocket science.  This problem was pretty much solved in the 80s and 90s: Use a salted one-way hash function of sufficient strength to resist a dictionary attack.

(LinkedIn’s mistake was to use hashes, but to not salt them. )

That’s it.  Really.  UNIX has been using a salted hash since about 1985, initially with a hash based on DES. Since that time, as computing speeds have increased, new (salted) hash functions based on MD5, Blowfish, and SHA-2 have all been introduced.

In other words, stored password security has been a solved problem for at least 25 years. The concept is the same, only the algorithms have needed to be updated as Moore’s Law has dictated.

This is just one reason that programmers (and sysadmins) should study history, if only the history of computer security. Oh, if you’re not a cryptologist, for security-critical functions, please use well-vetted library functions.

A few references:

, , , , , , ,

Leave a comment

Speaking as if you are being translated can help your native-language conversations

Tonight I was out after dinner with some of my colleagues from Japan. With the help of translators we were discussing both personal and work things and I noticed that the conversations were more focused than they might be when everyone speaks the same language. We have some absolutely wonderful bi-lingual folks in our offices, some of who are full-time translators, and some who often serve as translators, in addition to their regular jobs. Over the past years they’ve helped me become more adept at working with translators and be a more effective communicator.

Since as IT we are often working with our customers or users, you could almost say that we are always working with translation. The things that make working with a separate translator and a person who doesn’t speak your language will also help in your communications with others who speak your language, but not might be part of your “culture” (IT).

Having a translator in the conversation changes the way you listen, think and express your ideas. I believe that we could learn from this and improve our regular (non-translated) conversations.  When there is a translator, you especially learn to do four things: listen carefully, think about what you want to say before you say it, consider how the idea might be received by the listener and try to avoid ambiguous or unclear thoughts that might lead to is-understanding, and articulate your ideas concisely and directly.

Listening is key. You must focus not only on the words being spoken by the translator, but before that you also need to listen to the other speaker, while the translator is listening.  Watch and listen to the speaker, not the translator.  Understand the body and facial language of the speaker and get a sense from them about which ideas (in the sequence) are most important.  While they are speaking, pay most of your attention to them, not the translator.  When the translator begins speaking, pay attention to both the translator and the speaker, working to keep everyone involved in the conversation.

When it is time for you to respond, but before you speak, the most important thing is to make sure that you have a completely formed thought (or just a few) that you want to express. You need to think about the idea and how to communicate it clearly, before you open your mouth.  You shouldn’t be trying to expand on or complete your half-formed idea while you’re in the middle of a sentence. Before you speak, know what you want to say, and how you want to say it.

Now that you know what you want to say, you have to decide how to say it. Plan your sentences, plan the sequence of ideas, and consider how to avoid ambiguity or misunderstanding. This is where knowledge of the other person’s language, culture, (business) environment and your relationship with the other person is especially helpful. If I absolutely know a specific word in the other language that helps express the idea completely, I may use it to help in translation or understanding. If there is a term that I know has a special meaning or is used in the office or the company in a special way, I might want to use that word or term. If the other speaker and I have a common background, such as prior conversations or projects we’ve worked on together, I may reference those.

Finally, it is time to open your mouth. Be concise. Speak in reasonable-sized, self-contained “sound bites”. Don’t go on too long without stopping to a) give the translator time to translate and b) look for the other person to want to speak.  No long-winded sentences, no rambling thoughts. Don’t waste the translator’s efforts, don’t expect them to remember a complete five minute monologue with eight bullet points before they begin translating, and don’t make it impossible for the other speaker to interrupt if needed. While you are speaking, pay attention to the other speaker as much (or more) than the translator, looking for their reaction. This will help you understand if your ideas are being understood and how they are being accepted (or not).  All three of you are in the conversation, but it is primarily a conversation between the two speakers.

The things you need to do to effectively work with a translator can also improve your communications with other people speaking the same language: Listen well, form one or a few complete thoughts, think about how you want to say them, and express them concisely.


Leave a comment

IPv6 – source address selection

When we did our IPv6 sprint earlier this year, one of the biggest surprises (and sources of confusion) was how we needed to deal with multiple IPv6 addresses per network interface. The confusion wasn’t about having multiple addresses, it was predicting which address would be used as the source address when sending packets. Almost everyone was already familiar with “VIFs” (Virtual Interfaces) or equivalent from Solaris, Linux or other operating systems. But VIFs don’t have the problem of needing to select a source address.

The interesting issue is that the source address you must select depends on the network path between you and your destination. The same source computer shows up as different IPv6 addresses on different destination systems.

Since source addresses are the basis for many security mechanisms, such as rules on network firewalls and destination host iptables configurations, you need to know which address a source host will use in several different cases. This makes managing source-host-specific firewall and iptables rules….. complicated.

Fortunately, the need to be able to predict (or configure) the source address was recognized early on in IPv6 development and rules for selecting IPv6 source address were documented in RFC 3484 (2003). However, like many RFCs, it is a great specification, but is light on readability and explanations. The RFC also has no specification for the implementation details, such as the user interface for the “User Configuration Table” which allows the system administrator to change the default behavior.

Fortunately, at least for Linux, one of the developers for “glibc” (which implements the network stack interface), has written about these issues for Linux, and there are some good articles about the specifics of the Linux RFC 3484 implementation. That’s the good news.  The bad news is that it is still complicated.

Source address selection is controlled by the User Configuration Table, which I’ll show in a later post.  After that, I’ll cover how this adds even more weight to the argument that host-IP-based access restrictions need to be revisited (or just not used) in IPv6-capable networks.

Leave a comment

IPv6 and MacOS X Lion – “Hampered Eyeballs”

As part of the IPv6 sprint at work last month, I ended up doing a lot of IPv6 research. For my part, I spent a lot of time researching “customer issues” and MacOS issues in addition to the purely technical work.

When I started the sprint, my laptop was on MacOS X Snow Leopard, which I used for all my home IPv6 work. Halfway through the sprint, I upgraded to MacOS X Lion.

The upgrade to Lion went well, but Apple has changed the behavior of some IPv6 features, and I personally would have to consider Snow Leopard as a better IPv6 platform than Lion.

Apple didn’t “break” IPv6 in Lion, but they did introduce a new problem, which has been dubbed “hampered eyeballs”.

I’ve noticed some newly-hampered IPv6 web browsing since the upgrade.  Some sites that came back solidly on IPv6 100% of the time, now come back as IPv4 up to 20% of the time. (Thanks IPvFox!)

This has lots of implications for how consumers will see the new Internet, especially during the transition.  According to some anecdotal remarks on some IPv6 mailing lists, this is being used as an excuse by some companies to delay (even more) any IPv6 transition or even dual stacking!

This last week was Game Developer Conference in San Francisco, next week is a global IPv6 meeting in New Jersey. I should have lots more “corporate” IPv6 info on the next 10 days.


, ,


IPv6 “sprint” – background and results

The last two weeks at work have been some of the most fun in the past few years. A few months ago I moved from management back to my first love: deep technical work. In my new position I’m responsible (with a co-worker) for technical strategy, creating our Enterprise Architecture, and forward-looking technical projects. We’re also tasked with finding new ways to collaborate and take on projects as well as take a hard look to ensure that IT is supporting the rest of the business.

For some of these, we act as facilitators for IT projects, even though we aren’t in the management chain.

IPv6 has been one of my “back burner” projects for almost a year. There is a business mandate that we must have IPv6 connectivity to one of the inter-corporate networks by 1 April. A select set of our internal users need to have IPv6 connectivity to business applications that will only be available over IPv6 via this network.

To prepare for this, we had a need to ramp up IPv6 knowledge from almost nothing, to ready to plan a limited IPv6 deployment next month.

We decided to try a new project methodology (loosely) based on agile concepts: we performed IPv6 testing and deployment preparation as a “sprint”. We got 12 of our most senior system and network admins together in a large conference room with a pile of hardware, a stack of OS install disks, a new IPv6 transit connection and said, “Go!”.

No distractions, no email, no phone calls. Just 12 people off in a completely different building, in a big room with a pile of gear and the mandate to “explore IPv6” and learn enough to be comfortable planning a limited IPv6 deployment at the end.

It was great seeing people from different IT departments who usually specialize in Linux, MS Windows, VMWare, networking, security, etc. all come together to explore IPv6 on all these platforms, bring up services, test, find vendor bugs :-) and in general build a standalone IPv6 lab from scratch.

We truly did start from scratch; we started with an empty room, a bunch of tables and chairs, two pallets of PCs, assorted network kit, three boxes of ethernet cables and installation media.

Along the way, all of these people stepped out of their comfort zones, learned about each others’ specializations, and worked together for a common goal that we all created together.

At the end of the 2 weeks, we had a fully functioning dual-stack IPv4/IPv6 network:

  • Routers and switches, firewall and IPv4/6 transit from a new provider
  • Fully functioning Windows infrastructure: AD, DNS, DHCP, IIS, Exchange, etc.
  • Linux infrastructure: DNS, DHCP, syslog, apache, Splunk, Puppet (mostly)
  • Windows Server 2008 and 2008 R2, Windows 7 clients
  • Linux Centos 5 and 6 servers and desktop
  • MacOS Snow Leopard and Lion clients

All the results and everything we learned is documented in a wiki full of IPv6 configurations, hints and tips, debugging info, links to IPv6 info, lessons learned and plans for IPv6 next steps to production. I think we generated about 50-60 pages of new documentation along the way on IPv6, and about 6 pages of notes on the sprint experience itself.

The sprint wasn’t perfect, and we had a few stumbles along the way. But we learned a lot about how to run these kinds of sprints, and we’re pretty sure that we’ll have more of them in the future.

We also had two full weeks of face time with our colleagues from four sites in two states. In some cases we had never met each other in person, but had been exchanging email and tickets for years.

It was incredibly productive two weeks. We learned a lot about IPv6, each other and found new ways to work together.

, , , , ,



Get every new post delivered to your Inbox.

Join 304 other followers

%d bloggers like this: