ISP Design Guide: Separation of network functions – introduction and overview

PDF link is here


A reference guide for new & existing ISPs that need to understand network functions and separation.

“How do I add redundancy?”
“How do I scale?”
“How do I reduce downtime and operational costs?”

These are questions that I get asked practically every day as a consulting network architect that designs and builds ISPs.

In most cases the answer is the same whether the ISP uses fixed wireless broadband, copper or fiber to deliver the last mile – separation of network functions.

This illustrated guide is intended to define the topic and create visual context for each function using a network drawing. It’s the first in a new series on this subject.

A new series of content

This topic is deep and there is a lot to unpack so this will be the first segment in a series of blog posts and videos covering function separation.

Large ISPs typically already embrace the philosophy of separating network functions, so the focus of this series will be to help new or growing regional ISPs understand the design intent and the challenges/costs of running networks that don’t separate network functions.

WISP Design – OSPF “Leapfrog” path for traffic engineering

Introduction

One challenge that every WISP owner or operator has faced is how to leverage unused bandwidth on a backup path to generate more revenue.

For networks that have migrated to MPLS and BGP, this is an easier problem to solve as there are tools that can be used in those protocols like communities or MPLS TE to help manage traffic and set policy.

However, many WISPs rely solely on OSPF and cost adjustment to attempt to influence traffic. Alternatively, trying to use policy routing can lead to a design that doesn’t failover or scale well.

Creating a bottleneck on a single path

WISPs that are OSPF routed will often have a primary path back to the Internet at one or more points in the network typically from a tower that aggregates multiple backhauls.

As more towers are added that rely on this path, it can create a bottleneck while other paths are unused.

ospf-leapfrog-pic1

Creating an alternate best path by leapfrogging another router

One way to solve this problem is to use VLANs to create another subnet for OSPF to form an adjacency.

By tagging the VLAN from Tower 6 through Tower 3 and into Tower 4, a new path for OSPF is created that will cause Tower 6 and Tower 7 to take an alternate path.

This allows a WISP operator to tap into unused bandwidth while still retaining a straightforward failover mechanism.

ospf-leapfrog-pic2

Tower 6 and 7 can now use the alternative path

Now that a lower cost path exists at Layer 3 for Tower 6 and 7, the original best path become less congested and bandwidth that was previously unused can now be used to load balance traffic.

opsf-leapfrog-pic3

How to build the new path?

I often recommend “switch-centric” architecture also known as “router-on-a-stick” when building a WISP – there are a number of benefits to doing this. In this case, since the switches handle all connections, it’s fairly easy to tag a VLAN from Tower 6 to Tower 4 and create the “leapfrog” path.

opsf-leapfrog-pic4

A few closing thoughts

This works well when the towers you need to move onto an alternate path are part of a stub network – that is, they only have one way into the aggregation point before the traffic is split.

If you were trying to do this with rings that connect to other rings and had redundant paths, there could be unintended results so you need to plan the costs and paths carefully.

Labbing the topology to see what will happen is always a good idea.

Hope this is helpful for someone looking to get just a little more bandwidth out of an OSPF based WISP network.