Skip to content

Day 2 Operations

Our multi-site L2LS network is working great. But, before too long, it will be time to change our configurations. Lucky for us, that time is today!

Cleaning Up

Before going any further, let's ensure we have a clean repo by committing the changes we've made up to this point. The CLI commands below can accomplish this, but the VS Code Source Control GUI can be used as well.

git add .
git commit -m 'Your message here'

Next, we'll want to push these changes to our forked repository on GitHub.

git push

If this is our first time pushing to our forked repository, then VS Code will provide us with the following sign-in prompt:

VS Code GitHub Sign in Prompt

Choose Allow, and another prompt will come up, showing your unique login code:

VS Code GitHub Login Code Prompt

Choose Copy & Continue to GitHub, and another prompt will come up asking if it's ok to open an external website (GitHub).

VS Code External Site Prompt

Choose Open and then an external site (GitHub) will open, asking for your login code.

GitHub Login Code Entry

Paste in your login code and choose Continue. You will then be prompted to Authorize VS Code.

GitHub Authorize VS Code

Choose Authorize Visual-Studio-Code, and you should be presented with the coveted Green Check Mark!

VS Code Git Login Complete

Whew! Alright. Now that we have that complete, let's keep moving...

Branching Out

Before jumping in and modifying our files, we'll create a branch named banner-syslog in our forked repository to work on our changes. We can create our branch in multiple ways, but we'll use the git switch command with the -c parameter to create our new branch.

git switch -c banner-syslog

After entering this command, we should see our new branch name reflected in the terminal. It will also be reflected in the status bar in the lower left-hand corner of our VS Code window (you may need to click the refresh icon before this is shown).

Now we're ready to start working on our changes 😎.

Login Banner

When we initially deployed our multi-site topology, we should have included a login banner on all our switches. Let's take a look at the AVD documentation site to see what the data model is for this configuration.

The banner on all of our switches will be the same. After reviewing the AVD documentation, we know we can accomplish this by defining the banners input variable in our global_vars/global_dc_vars.yml file.

Add the code block below to global_vars/global_dc_vars.yml.

# Login Banner
banners:
  motd: |
    You shall not pass. Unless you are authorized. Then you shall pass.
    EOF
Yes, that "EOF" is important!

Ensure the entire code snippet above is copied; including the EOF. This must be present for the configuration to be considered valid

Next, let's build out the configurations and documentation associated with this change.

make build-site-1 build-site-2

Please take a minute to review the results of our five lines of YAML. When finished reviewing the changes, let's commit them.

As usual, there are a few ways of doing this, but the CLI commands below will get the job done:

git add .
git commit -m 'add banner'

So far, so good! Before we publish our branch and create a Pull Request though, we have some more work to do...

Syslog Server

Our next Day 2 change is adding a syslog server configuration to all our switches. Once again, we'll take a look at the AVD documentation site to see the data model associated with the logging input variable.

Like our banner operation, the syslog server configuration will be consistent on all our switches. Because of this, we can also put this into our global_vars/global_dc_vars.yml file.

Add the code block below to global_vars/global_dc_vars.yml.

# Syslog
logging:
  vrfs:
    - name: default
      source_interface: Management0
      hosts:
        - name: 10.200.0.108
        - name: 10.200.1.108

Finally, let's build out our configurations.

make build-site-1 build-site-2

Take a minute, using the source control feature in VS Code, to review what has changed as a result of our work.

At this point, we have our Banner and Syslog configurations in place. The configurations look good, and we're ready to share this with our team for review. In other words, it's time to publish our branch to the remote origin (our forked repo on GitHub) and create the Pull Request (PR)!

There are a few ways to publish the banner-syslog branch to our forked repository. The commands below will accomplish this via the CLI:

git add .
git commit -m 'add syslog'
git push --set-upstream origin banner-syslog

On our forked repository, let's create the Pull Request.

When creating the PR, ensure that the base repository is the main branch of your fork. This can be selected via the dropdown as shown below:

PR Base Repository Selection

Take a minute to review the contents of the PR. Assuming all looks good, let's earn the YOLO GitHub badge by approving and merging your PR!

Tip

Remember to delete the banner-syslog branch after performing the merge - Keep that repo clean!

Once merged, let's switch back to our main branch and pull down our merged changes.

git switch main
git pull

Then, let's delete our now defunct banner-syslog branch.

git branch -D banner-syslog

Finally, let's deploy our changes.

make deploy-site-1 deploy-site-2

Once completed, we should see our banner when logging into any switch. The output of the show logging command should also have our newly defined syslog servers.

Provisioning new Switches

Our network is gaining popularity, and it's time to add a new Leaf pair into the environment! s1-leaf5 and s1-leaf6 are ready to be provisioned, so let's get to it.

Branch Time

Before jumping in, let's create a new branch for our work. We'll call this branch add-leafs.

git switch -c add-leafs

Now that we have our branch created let's get to work!

Inventory Update

First, we'll want to add our new switches, named s1-leaf5 and s1-leaf6, into our inventory file. We'll add them as members of the SITE1_LEAFS group.

Add the following two lines under s1-leaf4 in sites/site_1/inventory.yml.

s1-leaf5:
s1-leaf6:

The sites/site_1/inventory.yml file should now look like the example below:

sites/site_1/inventory.yml 
---
SITE1:
  children:
    CVP:
      hosts:
        cvp:
    SITE1_FABRIC:
      children:
        SITE1_SPINES:
          hosts:
            s1-spine1:
            s1-spine2:
        SITE1_LEAFS:
          hosts:
            s1-leaf1:
            s1-leaf2:
            s1-leaf3:
            s1-leaf4:
            s1-leaf5:
            s1-leaf6:
    SITE1_FABRIC_SERVICES:
      children:
        SITE1_SPINES:
        SITE1_LEAFS:
    SITE1_FABRIC_PORTS:
      children:
        SITE1_SPINES:
        SITE1_LEAFS:

Next, let's add our new Leaf switches into sites/site_1/group_vars/SITE1_FABRIC.yml.

These new switches will go into RACK3, leverage MLAG for multi-homing, and will have locally connected endpoints in VLANs 10 and 20.

Just like the other Leaf switches, interfaces Ethernet2 and Ethernet3 will be used to connect to the spines.

On the spines, interface Ethernet9 will be used to connect to s1-leaf5, while Ethernet10 will be used to connect to s1-leaf6.

Starting at line 64, add the following code block into sites/site_1/group_vars/SITE1_FABRIC.yml.

- group: RACK3
  nodes:
    - name: s1-leaf5
      id: 5
      mgmt_ip: 192.168.0.28/24
      uplink_switch_interfaces: [ Ethernet9, Ethernet9 ]
    - name: s1-leaf6
      id: 6
      mgmt_ip: 192.168.0.29/24
      uplink_switch_interfaces: [ Ethernet10, Ethernet10 ]
Warning

Make sure the indentation of RACK3 is the same as RACK2, which can be found on line 52

The sites/site_1/group_vars/SITE1_FABRIC.yml file should now look like the example below:

sites/site_1/group_vars/SITE1_FABRIC.yml 
---
fabric_name: SITE1_FABRIC

# Set Design Type to L2ls
design:
  type: l2ls

# Spine Switches
l3spine:
  defaults:
    platform: cEOS
    spanning_tree_mode: mstp
    spanning_tree_priority: 4096
    loopback_ipv4_pool: 10.1.252.0/24
    mlag_peer_ipv4_pool: 10.1.253.0/24
    mlag_peer_l3_ipv4_pool: 10.1.254.0/24
    virtual_router_mac_address: 00:1c:73:00:dc:01
    mlag_interfaces: [ Ethernet1, Ethernet6 ]
  node_groups:
    - group: SPINES
      nodes:
        - name: s1-spine1
          id: 1
          mgmt_ip: 192.168.0.10/24
        - name: s1-spine2
          id: 2
          mgmt_ip: 192.168.0.11/24

# Leaf Switches
leaf:
  defaults:
    platform: cEOS
    mlag_peer_ipv4_pool: 10.1.253.0/24
    spanning_tree_mode: mstp
    spanning_tree_priority: 16384
    uplink_switches: [ s1-spine1, s1-spine2 ]
    uplink_interfaces: [ Ethernet2, Ethernet3 ]
    mlag_interfaces: [ Ethernet1, Ethernet6 ]
  node_groups:
    - group: RACK1
      filter:
        tags: [ "Web" ]
      nodes:
        - name: s1-leaf1
          id: 1
          mgmt_ip: 192.168.0.12/24
          uplink_switch_interfaces: [ Ethernet2, Ethernet2 ]
        - name: s1-leaf2
          id: 2
          mgmt_ip: 192.168.0.13/24
          uplink_switch_interfaces: [ Ethernet3, Ethernet3 ]
    - group: RACK2
      filter:
        tags: [ "App" ]
      nodes:
        - name: s1-leaf3
          id: 3
          mgmt_ip: 192.168.0.14/24
          uplink_switch_interfaces: [ Ethernet4, Ethernet4 ]
        - name: s1-leaf4
          id: 4
          mgmt_ip: 192.168.0.15/24
          uplink_switch_interfaces: [ Ethernet5, Ethernet5 ]
    - group: RACK3
      nodes:
        - name: s1-leaf5
          id: 5
          mgmt_ip: 192.168.0.28/24
          uplink_switch_interfaces: [ Ethernet9, Ethernet9 ]
        - name: s1-leaf6
          id: 6
          mgmt_ip: 192.168.0.29/24
          uplink_switch_interfaces: [ Ethernet10, Ethernet10 ]


##################################################################
# Underlay Routing Protocol - ran on Spines
##################################################################

underlay_routing_protocol: OSPF

##################################################################
# WAN/Core Edge Links
##################################################################

core_interfaces:
  p2p_links:

    - ip: [ 10.0.0.29/31, 10.0.0.28/31 ]
      nodes: [ s1-spine1, WANCORE ]
      interfaces: [ Ethernet7, Ethernet2 ]
      include_in_underlay_protocol: true

    - ip: [ 10.0.0.33/31, 10.0.0.32/31 ]
      nodes: [ s1-spine1, WANCORE ]
      interfaces: [ Ethernet8, Ethernet2 ]
      include_in_underlay_protocol: true

    - ip: [ 10.0.0.31/31, 10.0.0.30/31 ]
      nodes: [ s1-spine2, WANCORE ]
      interfaces: [ Ethernet7, Ethernet2 ]
      include_in_underlay_protocol: true

    - ip: [ 10.0.0.35/31, 10.0.0.34/31 ]
      nodes: [ s1-spine2, WANCORE ]
      interfaces: [ Ethernet8, Ethernet2 ]
      include_in_underlay_protocol: true
Tip

Notice how we did not specify a filter or tags under RACK3. If the filter parameter is not defined, all VLANs/SVIs/VRFs will be provisioned on the switch. In our case, this means that VLANs 10 and 20 will both be created on our new Leaf switches. However, since they are leaf node types, no SVIs will be created.

Next - Let's build the configuration!

make build-site-1
Important

Interfaces Ethernet9 and Ethernet10 do not exist on the Spines. Because of this, we will not run a deploy command since it would fail.

Please take a moment and review the results of our changes via the source control functionality in VS Code.

Finally, we'll commit our changes and publish our branch. Again, we can use the VS Code Source Control GUI for this, or via the CLI using the commands below:

git add .
git commit -m 'add leafs'
git push --set-upstream origin add-leafs

Backing Out Changes

Ruh Roh. As it turns out, we should have added these leaf switches to an entirely new site. Oops! No worries, because we used our add-leafs branch, we can switch back to our main branch and then delete our local copy of the add-leafs branch. No harm or confusion related to this change ever hit the main branch!

git switch main
git branch -D add-leafs

Finally, we can go out to our forked copy of the repository and delete the add-leafs branch.

Great Success!

Congratulations. You have now successfully completed initial fabric builds and day 2 operational changes without interacting with any switch CLI!