Val:~$ whoami

I am Val Glinskiy, network engineer specializing in data center networks. TIME magazine selected me as Person of the Year in 2006.

Search This Blog

Saturday, February 24, 2018

IP fabric over unnumbered interfaces

  So, you read the industry websites, know that IP fabric is the next best thing in data center networking and decided to take a plunge and build your own. Nothing big to start with: 2 spine and 8 leaf switches. Now you realize that your IPAM system does not have API and you have to assign 16 IP addresses for transit links and 10 IPs for loopback interfaces manually. While not insurmountable task, it's tedious. Fortunately, Cisco's NXOS and Juniper's JUNOS let you configure ip unnumbered ethernet interface and now you need only 10 IPs for loopbacks.
  I created a small virtual lab of 3 NXOSv switches - 2 spines and 2 leafs - to test the concept. I could not make BGP work directly over unnumbered interfaces, so I configured OSPF to advertise loopbacks and BGP peering between loopbacks. To simplify configuration even more, I configured dynamic BGP peering on the spines.
   Why would you need BGP if you already have OSPF? You might want to run another next best thing - VXLAN with EVPN control plane.
  To run the lab you'll need Vagrant, VirtualBox, Linux machine with 32GB of memory and Vagrant package of NXOS from Cisco. You need to have CCO account and may be maintenance contract to download the image. Do not ask me to provide the image. I used NXOSv image nxosv-final.7.0.3.I7.1.boxVagrant 2.0.1, VirtualBox 5.2.6 and Ubuntu Linux 16.04LTS. Although the lab worked with earlier versions of Vagrant and VirtualBox and should run on any Linux distro.

  • Clone or download from git configuration files
  • Run "" script to build ISO files with configuration for each NXOSv switch
  • run "vagrant up". Depending on the resources, it might take up 10 minutes for all 4 switches to come up fully.
You can see boot progress by connecting to consoles: "ncat -U /tmp/", where are leaf1, leaf2, spine1 or spine2. After switches are up, you can log in by running "vagrant ssh ". You'll be presented with bash shell, to get to NXOS prompt, type "su - admin", password is admin.

Disclaimer: this is in no way shape or form production-ready configuration and was not tested for any side effects. Use it at your own risk.

Happy Labbing!