Val:~$ whoami

I am Val Glinskiy, network engineer specializing in data center networks. TIME magazine selected me as Person of the Year in 2006.

Search This Blog

Friday, October 09, 2009

BGP and BFD (Bidirectional Forwading Detection)

If you have Ethernet uplink to your ISP, the chances are high that it looks like this:



The Layer2 device could be "on the wire" provider or Ethernet over MPLS service. The problem arises when, for example, connection between Layer2 device and ISP router goes down



The BGP session with R1 on ISP router will reset immediately, unless you configured "no bgp fast-external-fallover". But R1 will rely on BGP hello messages to detect if neighbor still alive. It might take R1 up to 3 minutes detect that ISP is not available and for these 3 minutes R1 will be sending traffic to black hole instead of re-converging and sending traffic to your backup link. You have backup link, don't you?
Here are the syslog messages from R1 and ISP routers. To imitate link failure I shutdown interface on ISP

router ISP:
Oct 9 17:55:18 UTC: %LINK-5-CHANGED: Interface GigabitEthernet2/5, changed state to administratively down
Oct 9 17:55:18 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.255.1 Down Interface flap


router R1:
Oct 9 17:57:24 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.255.2 Down BGP Notification sent
Oct 9 17:57:24 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.255.2 4/0 (hold time expired) 0 bytes

Note the timestamps of first and last messages. That's not good, especially if every minute of downtime costs you a bundle. You can adjust bgp timers, but lowest you can go is 1 second and it could be hard on CPU.
BFD protocol allows you to go to microseconds level. It is very lightweight and easy to configure.
On interfaces facing Layer2 device apply command:

bfd interval 100 min_rx 100 multiplier 3
To check if BFD is configured properly:
#sho bfd neighbor

OurAddr NeighAddr LD/RD RH/RS Holddown(mult) State Int
172.31.255.2 172.31.255.1 3/7 Up 0 (3 ) Up Gi2/5

under "router bgp" configuration:

neighbor [neighbor IP] fall-over bfd

Now let's imitate link failure again. Shutdown interface on router ISP:

Oct 9 18:14:27.408 UTC: %LINK-5-CHANGED: Interface GigabitEthernet2/5, changed state to administratively down
Oct 9 18:14:27.408 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.255.1 Down Interface flap


On R1

Oct 9 18:14:27.673 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.255.2 Down BFD adjacency down


The difference now in milliseconds. The hard part is to convince your ISP to configure BFD on their side.
At this moment Cisco supports BFD on Ethernet interfaces only and only for directly connected BGP peers, i.e. no multi-hop BGP.
BFD requires UDP ports 3784 and 3785 to be open in case you have ACL applied to your uplink interface.

1 comment:

  1. Very well explain with diagram presentation is very much easy to understand and i found the article worth reading.
    MPLS training course


    ReplyDelete