Skip to content

Campus Layer 3 Leaf-Spine

Lab Topology

Preparing The Lab

  1. Log into the LabAccess jumpserver:
    1. Click on the Console Access on the lab access menu.
    2. Look for the option for the Layer 3 Leaf-Spine Lab with BGP.
    3. Type the correct option number after theWhat would you like to do?: prompt.
    4. The script will pre-configure the topology with the exception of leaf-1a and leaf-3a. leaf-3a will be configured manually, via the CLI.

Lab Tasks

  1. Configure User VLANs, SVIs, Virtual IPs, and VARP on the leaf-3a switch using the following criteria.

    Note

    Arista EOS utilizes an Industry-Standard CLI. When entering configuration commands, be sure to first type configure to enter configuration mode.

    1. Create the User Data, Voice, and Guest VLANs and define their names on the leaf-3a:

      Commands

      vlan 310
    name Access_Pod3-Data
!
vlan 320
    name Access_Pod3-Voice
!
vlan 330
    name Access_Pod3-Guest

    2. Add the User VLANs to the Member Leaf port channels on the leaf-3a:

      Commands

      interface Port-Channel51
    switchport trunk allowed vlan add 310,320,330
!
interface Port-Channel52
    switchport trunk allowed vlan add 310,320,330
!
interface Port-Channel531
    switchport trunk allowed vlan add 310,320,330

    3. Create the vARP MAC Address in Global Configuration mode on the leaf-3a:

      Command

      ip virtual-router mac-address 00:1c:73:00:00:99

    4. Create the SVIs and their Virtual Router Addresses, and add the DHCP Relay Information on the leaf-3a:

      Commands

      interface Vlan310
    description Access_Pod3-Data
    no autostate
    ip address 10.3.10.2/24
    ip virtual-router address 10.3.10.1
    ip helper-address 10.100.0.0
    ip helper-address 10.100.0.2
!
interface Vlan320
    description Access_Pod3-Voice
    no autostate
    ip address 10.3.20.2/24
    ip virtual-router address 10.3.20.1
    ip helper-address 10.100.0.0
    ip helper-address 10.100.0.2
!
interface Vlan330
    description Access_Pod3-Guest
    no autostate
    ip address 10.3.30.2/24
    ip virtual-router address 10.3.30.1
    ip helper-address 10.100.0.0
    ip helper-address 10.100.0.2

    5. Validate the configuration on leaf-3a with the following commands:

      Command

      show ip interface brief

      Expected Output

      leaf-3a#show ip interface brief
                                                                                              Address
Interface              IP Address               Status       Protocol               MTU       Owner
---------------------- ------------------------ ------------ ---------------------- --------- -------
Management0            192.168.0.17/24          up           up                     1500
Vlan10                 10.10.10.2/24            up           up                     1500
Vlan310                10.3.10.1/24             up           up                     1500
Vlan320                10.3.20.1/24             up           up                     1500
Vlan330                10.3.30.1/24             up           up                     1500
Vlan4093               10.255.2.10/31           up           up                     1500
Vlan4094               10.255.255.10/31         up           up                     1500

      Command

      show ip virtual-router

      Expected Output

      leaf-3a#show ip virtual-router
IP virtual router is configured with MAC address: 001c.7300.0099
IP virtual router address subnet routes not enabled
MAC address advertisement interval: 30 seconds

Protocol: U - Up, D - Down, T - Testing, UN - Unknown
        NP - Not Present, LLD - Lower Layer Down

Interface       Vrf           Virtual IP Address       Protocol       State
--------------- ------------- ------------------------ -------------- -------
Vl10            default       10.10.10.1               U              active
Vl310           default       10.3.10.1                U              active
Vl320           default       10.3.20.1                U              active
Vl330           default       10.3.30.1                U              active

  2. Configure BGP on the leaf-3a switch using the following criteria.

    1. Based on the diagram, configure L3 interfaces to spine-1 and spine-2. Also configure interface Loopback0 for use as the BGP Router ID:

      Commands

      interface Ethernet49
    description P2P_LINK_TO_SPINE-1_Ethernet5
    mtu 1500
    no switchport
    ip address 10.0.0.21/31
!
interface Ethernet50
    description P2P_LINK_TO_SPINE-2_Ethernet5
    mtu 1500
    no switchport
    ip address 10.0.0.23/31
!
interface  Loopback0
    description Management and Router-id
    ip address 10.255.0.8/32

      Commands

      show ip interface brief

      Commands

      leaf-3a#show ip interface brief
                                                                                   Address
Interface          IP Address             Status     Protocol             MTU      Owner
------------------ ---------------------- ---------- -------------------- -------- -------
Ethernet49         10.0.0.21/31           up           up                          1500
Ethernet50         10.0.0.23/31           up           up                          1500
Loopback0          10.255.0.8/32          up           up                         65535
Management0        192.168.0.17/24        up           up                          1500
Vlan10             10.10.10.2/24          up           up                          1500
Vlan310            10.3.10.2/24           up           up                          1500
Vlan320            10.3.20.2/24           up           up                          1500
Vlan330            10.3.30.2/24           up           up                          1500
Vlan4093           10.255.2.10/31         up           up                          1500
Vlan4094           10.255.255.10/31       up           up                          1500

    2. Based on the diagram, enable BGP and configure the neighbor relationships on leaf-3a. Configure eBGP to spine-1 and spine-2 and iBGP to leaf-3b:

      Note

      We are using a peer group to configure the neighbor attributes for the spines. This allows us to apply all BGP attributes within a group to each neighbor that is a member in a scalable method.

      Commands

      router bgp 65113
    router-id  10.255.0.8
    neighbor SPINE peer group
    neighbor SPINE remote-as 65001
    neighbor SPINE send-community
    neighbor 10.0.0.20 peer group SPINE
    neighbor 10.0.0.22 peer group SPINE
    neighbor 10.255.2.11 remote-as 65113
    neighbor 10.255.2.11 next-hop-self
      Info

      Since neighbor 10.255.2.11 remote-as 65113 specifies an iBGP peering relationship (because the ASN is the same as this switch 65113), the receiving switch may not have a route to networks more than 1 hop away, hence the switches should each advertise that they are the next hop via the neighbor 10.255.2.11 next-hop-self statement. While this scenario is only 2 iBGP peers, in a network fabric with several iBGP peers, a switch inside an AS (and not on an edge) may not have a route to a switch in any external AS.

    3. Validate the BGP configuration using the show active command while in the “Router BGP” configuration section of the CLI. Also, check to see that BGP neighbors are established

      Command

      show active

      Expected Output

      leaf-3a(config-router-bgp)#show active
router bgp 65113
    router-id 10.255.0.8
    neighbor SPINE peer group
    neighbor SPINE remote-as 65001
    neighbor SPINE send-community
    neighbor 10.0.0.20 peer group SPINE
    neighbor 10.0.0.22 peer group SPINE
    neighbor 10.255.2.11 remote-as 65113
    neighbor 10.255.2.11 next-hop-self

      Command

      show ip bgp summary

      Expected Output

      leaf-3a(config-router-bgp)#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.255.0.8, local AS number 65113
Neighbor Status Codes: m - Under maintenance
    Neighbor     V  AS       MsgRcvd   MsgSent  InQ OutQ  Up/Down  State   PfxRcd PfxAcc
    10.0.0.20    4  65001    23        28        0    0   00:11:42 Estab   12         12
    10.0.0.22    4  65001    24        31        0    0   00:11:42 Estab   16         16
    10.255.2.11  4  65113    22        26        0    0   00:12:22 Estab   21         21

  3. Configure networks on leaf-3a to advertise the user subnets to spine-1 and spine-2.

    1. Add the following networks to BGP announcements on leaf-3a:

      Commands

      router bgp 65113
    network 10.3.10.0/24
    network 10.3.20.0/24
    network 10.3.30.0/24
    network 10.255.0.8/32
    redistribute attached-host

    2. Verify that these networks are being advertised to the other Spines and Leafs:

      Command

      show ip route

      Expected Output

      leaf-3a#sh ip route

VRF: default
Source Codes:
    C - connected, S - static, K - kernel,
    O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
    E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
    N2 - OSPF NSSA external type2, B - Other BGP Routes,
    B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
    I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
    A O - OSPF Summary, NG - Nexthop Group Static Route,
    V - VXLAN Control Service, M - Martian,
    DH - DHCP client installed default route,
    DP - Dynamic Policy Route, L - VRF Leaked,
    G  - gRIBI, RC - Route Cache Route,
    CL - CBF Leaked Route

Gateway of last resort is not set

C        10.0.0.20/31
          directly connected, Ethernet49
C        10.0.0.22/31
          directly connected, Ethernet50
B E      10.1.10.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.1.20.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.1.30.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.2.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.2.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.2.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
C        10.3.10.0/24
          directly connected, Vlan310
C        10.3.20.0/24
          directly connected, Vlan320
C        10.3.30.0/24
          directly connected, Vlan330
C        10.10.10.0/24
          directly connected, Vlan10
B E      10.100.0.0/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.2/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.4/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.6/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.255.0.6/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.255.0.7/32 [200/0]
          via 10.0.0.20, Ethernet49
C        10.255.0.8/32
          directly connected, Loopback0
B I      10.255.0.9/32 [200/0]
          via 10.255.2.11, Vlan4093
C        10.255.2.10/31
          directly connected, Vlan4093
C        10.255.255.10/31
          directly connected, Vlan4094
B E      172.16.1.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.1.2/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      172.16.20.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.20.2/32 [200/0]
         via 10.0.0.20, Ethernet49
C        192.168.0.0/24
          directly connected, Management0

      Command

      show ip route bgp

      Expected Output

      leaf-3a#show ip route bgp

VRF: default
Source Codes:
    C - connected, S - static, K - kernel,
    O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
    E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
    N2 - OSPF NSSA external type2, B - Other BGP Routes,
    B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
    I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
    A O - OSPF Summary, NG - Nexthop Group Static Route,
    V - VXLAN Control Service, M - Martian,
    DH - DHCP client installed default route,
    DP - Dynamic Policy Route, L - VRF Leaked,
    G  - gRIBI, RC - Route Cache Route,
    CL - CBF Leaked Route

B E      10.1.10.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.1.20.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.1.30.0/24 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.2.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.2.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.2.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.0/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.2/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.4/31 [200/0]
          via 10.0.0.20, Ethernet49
B E      10.100.0.6/31 [200/0]
         via 10.0.0.20, Ethernet49
B E      10.255.0.6/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      10.255.0.7/32 [200/0]
          via 10.0.0.20, Ethernet49
B I      10.255.0.9/32 [200/0]
          via 10.255.2.11, Vlan4093
B E      172.16.1.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.1.2/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      172.16.20.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.20.2/32 [200/0]
          via 10.0.0.20, Ethernet49

    3. Add in multiple paths by enabling ECMP, on leaf-3a. Enter BGP configuration mode, and add:

      Commands

      router bgp 65113
    maximum-paths 4 ecmp 4

    4. Check the BGP and IP route tables on leaf-3a as well as each of the Spines and Leafs:

      Note

      ECMP is now working - notice the new status code in the show ip bgp output on leaf-3b.

      Command

      show ip route

      Expected Output

      leaf-3a#show ip route

VRF: default
Source Codes:
    C - connected, S - static, K - kernel,
    O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
    E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
    N2 - OSPF NSSA external type2, B - Other BGP Routes,
    B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
    I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
    A O - OSPF Summary, NG - Nexthop Group Static Route,
    V - VXLAN Control Service, M - Martian,
    DH - DHCP client installed default route,
    DP - Dynamic Policy Route, L - VRF Leaked,
    G  - gRIBI, RC - Route Cache Route,
    CL - CBF Leaked Route

Gateway of last resort is not set

C        10.0.0.20/31
          directly connected, Ethernet49
C        10.0.0.22/31
          directly connected, Ethernet50
B E      10.1.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.1.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.1.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
C        10.3.10.0/24
          directly connected, Vlan310
C        10.3.20.0/24
          directly connected, Vlan320
C        10.3.30.0/24
          directly connected, Vlan330
C        10.10.10.0/24
          directly connected, Vlan10
B E      10.100.0.0/31 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.100.0.2/31 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.5/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.6/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.7/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
C        10.255.0.8/32
          directly connected, Loopback0
B I      10.255.0.9/32 [200/0]
          via 10.255.2.11, Vlan4093
C        10.255.2.10/31
          directly connected, Vlan4093
C        10.255.255.10/31
          directly connected, Vlan4094
B E      172.16.1.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.1.2/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      172.16.20.1/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
C        192.168.0.0/24
          directly connected, Management0

      Command

      show ip route bgp

      Expected Output

      leaf-3a#show ip route bgp

VRF: default
Source Codes:
    C - connected, S - static, K - kernel,
    O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
    E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
    N2 - OSPF NSSA external type2, B - Other BGP Routes,
    B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
    I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
    A O - OSPF Summary, NG - Nexthop Group Static Route,
    V - VXLAN Control Service, M - Martian,
    DH - DHCP client installed default route,
    DP - Dynamic Policy Route, L - VRF Leaked,
    G  - gRIBI, RC - Route Cache Route,
    CL - CBF Leaked Route

B E      10.1.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.1.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.1.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.10.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.20.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.2.30.0/24 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.100.0.0/31 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.100.0.2/31 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.5/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.6/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B E      10.255.0.7/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50
B I      10.255.0.9/32 [200/0]
          via 10.255.2.11, Vlan4093
B E      172.16.1.1/32 [200/0]
          via 10.0.0.20, Ethernet49
B E      172.16.1.2/32 [200/0]
          via 10.0.0.22, Ethernet50
B E      172.16.20.1/32 [200/0]
          via 10.0.0.20, Ethernet49
          via 10.0.0.22, Ethernet50

Test Client Connectivity

  1. Validate connectivity from host-3 to host-4.

    1. Since the host are using DHCP on interface Ethernet1, run the show ip interface brief command on host-4 to find its IP address:

      Command

      show ip interface brief

      Expected Output

      host-4#sh ip interface brief
                                                                                 Address
Interface           IP Address            Status       Protocol       MTU        Owner
------------------- --------------------- ------------ -------------- ---------- -------
Ethernet1           10.3.10.10/24         up           up             1500
Management0         192.168.0.25/24       up           up             1500
      Note

      Since the IP address is assigned via DHCP, your device may have a different IP than what's shown above.

    2. Ping and traceroute host-4’s IP address from the CLI of host-3:

      Commands

      ping 10.3.10.10

      traceroute 10.3.10.10

    3. Verify leaf-3a’s IP address is in the traceroute path, either interface 10.0.0.21 via spine-1 or interface 10.0.0.23 via spine-2. If traffic is hashing via leaf-3b’s interface 10.0.0.25 or interface 10.0.0.27, perform the optional shutdown steps below on leaf-3b:

      Commands

      router bgp 65113
    neighbor 10.0.0.24 shutdown
    neighbor 10.0.0.26 shutdown

    4. Rerun traceroute/verification from host-3 to host-4 to verify traffic is going through leaf-3a’s, then revert the shutdown changes on leaf-3b:

      Commands

      router bgp 65113
    no neighbor 10.0.0.24 shutdown
    no neighbor 10.0.0.26 shutdown

Additional BGP Features

If you have some time, here are some additional features that can be configured in BGP and you can experiment with in the lab.

  1. Route Redistribution: Issue watch 5 diff show ip route | begin Gateway on leaf-2a and let those run while you execute the command redistribute connected below on leaf-3b. You will see new routes being injected into the route table of leaf-2a:

    Commands

    router bgp 65113
    redistribute connected

  2. Route Maps and Prefix-Lists: Below is an example of some basic Prefix-Lists and Route-Maps that can be used for BGP filtering. Note that this is just an example and will not impact route advertisement in the lab:

    Commands

    <Example>
ip prefix-list BOGON-Prefixes seq 10 permit 10.0.0.0/8
ip prefix-list BOGON-Prefixes seq 20 permit 172.16.0.0/12 ip prefix-list BOGON-Prefixes seq 30 permit 192.168.0.0/16
!
route-map BOGONS permit 10
    match ip address prefix-list BOGON-Prefixes
!
route-map BOGONS deny 20
!
route-map InboundSP1 deny 10 sub-route-map BOGONS
!
route-map InboundSP1 permit 20 set local-preference 200
!
router bgp 65113
    neighbor UpstreamSP1 route-map InboundSP1 in

  3. BFD: BFD is a low-overhead, protocol-independent mechanism which adjacent systems can use for faster detection of faults in the paths between them. BFD is a simple mechanism which detects the liveliness of a connection between adjacent systems, allowing it to quickly detect failure of any element in the connection. Note that BFD is not running on the other devices so the BFD neighbor will not come up.

    Commands

    router bgp 65113
    neighbor  <neighbor_ip>  bfd
Success

Lab Complete!

Tip

The following additional commands are useful to verify connectivity for validation and troubleshooting purposes:

  • show ip bgp summary
  • show ip bgp
  • show ip bgp neighbor
  • show run section bgp
  • show log