{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Validating Configuration Settings with Batfish" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Network engineers routinely need to validate configuration settings of various devices in their network. In a multi-vendor network, this validation can be hard and few tools exist today to enable this basic task. However, the vendor-independent models of Batfish and its querying mechanisms make such validation almost trivial.\n", "\n", "In this notebook, we show how to validate configuration settings with Batfish. More specifically, we examine how the configuration of NTP servers can be validated. The same validation scenarios can be performed for other configuration settings of nodes (such as dns servers, tacacs servers, snmp communities, VRFs, etc.) interfaces (such as MTU, bandwidth, input and output access lists, state, etc.), VRFs, BGP and OSPF sessions, and more.\n", "\n", "Check out a video demo of this notebook [here](https://youtu.be/qOXRaVs1Uz4)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initializing our Network and Snapshot" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`SNAPSHOT_PATH` below can be updated to point to a custom snapshot directory, see the [Batfish instructions](https://github.com/batfish/batfish/wiki/Packaging-snapshots-for-analysis) for how to package data for analysis.
\n", "More example networks are available in the [networks](https://github.com/batfish/batfish/tree/master/networks) folder of the Batfish repository." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'example_snapshot'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Import packages\n", "%run startup.py\n", "bf = Session(host=\"localhost\")\n", "\n", "# Initialize a network and snapshot\n", "NETWORK_NAME = \"example_network\"\n", "SNAPSHOT_NAME = \"example_snapshot\"\n", "\n", "SNAPSHOT_PATH = \"networks/example\"\n", "\n", "bf.set_network(NETWORK_NAME)\n", "bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The network snapshot that we initialized above is illustrated below. You can download/view devices' configuration files [here](https://github.com/batfish/pybatfish/tree/master/jupyter_notebooks/networks/example). We will focus on the validation for the six **border** routers. \n", "\n", "![example-network](https://raw.githubusercontent.com/batfish/pybatfish/master/jupyter_notebooks/networks/example/example-network.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Extracting configured NTP servers\n", "This can be done using the `nodeProperties()` question." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NodeNTP_Servers
0as1border2['18.18.18.18', '23.23.23.23']
1as2border1['18.18.18.18', '23.23.23.23']
2as3border2['18.18.18.18', '23.23.23.23']
3as1border1[]
4as3border1['18.18.18.18', '23.23.23.23']
5as2border2['18.18.18.18']
\n", "
" ], "text/plain": [ " Node NTP_Servers\n", "0 as1border2 ['18.18.18.18', '23.23.23.23']\n", "1 as2border1 ['18.18.18.18', '23.23.23.23']\n", "2 as3border2 ['18.18.18.18', '23.23.23.23']\n", "3 as1border1 []\n", "4 as3border1 ['18.18.18.18', '23.23.23.23']\n", "5 as2border2 ['18.18.18.18']" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set the property that we want to extract\n", "COL_NAME = \"NTP_Servers\"\n", "\n", "# Extract NTP servers for all routers with 'border' in their name\n", "node_props = bf.q.nodeProperties(\n", " nodes=\"/border/\", \n", " properties=COL_NAME).answer().frame()\n", "node_props" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `.frame()` function call above returns a [Pandas](https://pandas.pydata.org/pandas-docs/stable/) data frame that contains the answer." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Validating NTP Servers Configuration\n", "Depending on the network's policy, there are several possible validation scenarios for NTP-servers configuration:\n", "1. Every node has at least one NTP server configured.\n", "2. Every node has at least one NTP server configured from the reference set.\n", "3. Every node has the reference set of NTP servers configured.\n", "4. Every node has NTP servers that match those in a per-node database.\n", "\n", "We demonstrate each scenario below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Validation scenario 1: Every node has at least one NTP server configured\n", "Now that we have the list of NTP servers, let's check if at least one server is configured on the border routers. We accomplish that by using ([lambda expressions](https://docs.python.org/3/reference/expressions.html#lambda)) to identify nodes where the list is empty." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NodeNTP_Servers
3as1border1[]
\n", "
" ], "text/plain": [ " Node NTP_Servers\n", "3 as1border1 []" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Find nodes that have no NTP servers configured\n", "ns_violators = node_props[node_props[COL_NAME].apply(\n", " lambda x: len(x) == 0)]\n", "ns_violators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Validation scenario 2: Every node has at least one NTP server configured from the reference set.\n", "Now if we want to validate that configured _NTP servers_ should contain at least one _NTP server_ from a reference set, we can use the command below. It identifies any node whose configured set of _NTP servers_ does not overlap with the reference set at all." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NodeNTP_Servers
3as1border1[]
5as2border2['18.18.18.18']
\n", "
" ], "text/plain": [ " Node NTP_Servers\n", "3 as1border1 []\n", "5 as2border2 ['18.18.18.18']" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Define the reference set of NTP servers\n", "ref_ntp_servers = set([\"23.23.23.23\"])\n", "\n", "# Find nodes that have no NTP server in common with the reference set\n", "ns_violators = node_props[node_props[COL_NAME].apply(\n", " lambda x: len(ref_ntp_servers.intersection(set(x))) == 0)]\n", "ns_violators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because `as1border1` has no configured NTP servers, it clearly violates our assertion, and so does `as2border2` which has a configured server but not one that is present in the reference set." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Validation scenario 3: Every node has the reference set of NTP servers configured\n", "A common use case for validating _NTP servers_ involves checking that the set of _NTP servers_ exactly matches a desired reference set. Such validation is quite straightforward as well. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NodeNTP_Servers
0as1border2['18.18.18.18', '23.23.23.23']
1as2border1['18.18.18.18', '23.23.23.23']
2as3border2['18.18.18.18', '23.23.23.23']
3as1border1[]
4as3border1['18.18.18.18', '23.23.23.23']
5as2border2['18.18.18.18']
\n", "
" ], "text/plain": [ " Node NTP_Servers\n", "0 as1border2 ['18.18.18.18', '23.23.23.23']\n", "1 as2border1 ['18.18.18.18', '23.23.23.23']\n", "2 as3border2 ['18.18.18.18', '23.23.23.23']\n", "3 as1border1 []\n", "4 as3border1 ['18.18.18.18', '23.23.23.23']\n", "5 as2border2 ['18.18.18.18']" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Find violating nodes whose configured NTP servers do not match the reference set\n", "ns_violators = node_props[node_props[COL_NAME].apply(\n", " lambda x: ref_ntp_servers != set(x))]\n", "ns_violators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As we can see, all border nodes violate this condition." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A slightly advanced version of pandas filtering can also show us which configured _NTP servers_ are missing or extra (compared to the reference set) at each node." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Nodeextra-NTP_Serversmissing-NTP_Servers
0as1border2{18.18.18.18}{}
1as2border1{18.18.18.18}{}
2as3border2{18.18.18.18}{}
3as1border1{}{23.23.23.23}
4as3border1{18.18.18.18}{}
5as2border2{18.18.18.18}{23.23.23.23}
\n", "
" ], "text/plain": [ " Node extra-NTP_Servers missing-NTP_Servers\n", "0 as1border2 {18.18.18.18} {}\n", "1 as2border1 {18.18.18.18} {}\n", "2 as3border2 {18.18.18.18} {}\n", "3 as1border1 {} {23.23.23.23}\n", "4 as3border1 {18.18.18.18} {}\n", "5 as2border2 {18.18.18.18} {23.23.23.23}" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Find extra and missing servers at each node\n", "ns_extra = node_props[COL_NAME].map(lambda x: set(x) - ref_ntp_servers)\n", "ns_missing = node_props[COL_NAME].map(lambda x: ref_ntp_servers - set(x))\n", "\n", "# Join these columns up with the node columns for a complete view\n", "diff_df = pd.concat([node_props[\"Node\"],\n", " ns_extra.rename('extra-{}'.format(COL_NAME)),\n", " ns_missing.rename('missing-{}'.format(COL_NAME))],\n", " axis=1)\n", "diff_df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Validation scenario 4: Every node has _NTP servers_ that match those in a per-node database.\n", "Every node should match its reference set of _NTP Servers_ which may be stored in an external database. This check enables easy validation of configuration settings that differ acorss nodes.\n", "\n", "We assume data from the database is fetched in the following format, where node names are dictionary keys and specific properties are defined in a property-keyed dictionary per node." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# Mock reference-node-data, presumably taken from an external database\n", "database = {'as1border1': {'NTP_Servers': ['23.23.23.23'],\n", " 'DNS_Servers': ['1.1.1.1']},\n", " 'as1border2': {'NTP_Servers': ['23.23.23.23'],\n", " 'DNS_Servers': ['1.1.1.1']},\n", " 'as2border1': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],\n", " 'DNS_Servers': ['2.2.2.2']},\n", " 'as2border2': {'NTP_Servers': ['18.18.18.18'],\n", " 'DNS_Servers': ['1.1.1.1']},\n", " 'as3border1': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],\n", " 'DNS_Servers': ['2.2.2.2']},\n", " 'as3border2': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],\n", " 'DNS_Servers': ['2.2.2.2']},\n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that there is an extra property in this dictionary that we don't care about comparing right now: `dns-server`. We will filter out this property below, before comparing the data from `Batfish` to that in the database. \n", "\n", "After a little massaging, the database and `Batfish` data can be compared to generate two sets of servers: missing (i.e., present in the database but not in the configurations) and extra (i.e., present in the configurations but not in the database)." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
missing-NTP_Serversextra-NTP_Servers
as1border1{23.23.23.23}{}
as1border2{}{18.18.18.18}
as2border1{}{}
as2border2{}{}
as3border1{}{}
as3border2{}{}
\n", "
" ], "text/plain": [ " missing-NTP_Servers extra-NTP_Servers\n", "as1border1 {23.23.23.23} {}\n", "as1border2 {} {18.18.18.18}\n", "as2border1 {} {}\n", "as2border2 {} {}\n", "as3border1 {} {}\n", "as3border2 {} {}" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Transpose database data so each node has its own row\n", "database_df = pd.DataFrame(data=database).transpose()\n", "\n", "# Index on node for easier comparison\n", "df_node_props = node_props.set_index('Node')\n", "\n", "# Select only columns present in node_props (get rid of the extra dns-servers column)\n", "df_db_node_props = database_df[df_node_props.columns].copy()\n", "\n", "# Convert server lists into sets to support arithmetic below\n", "df_node_props[COL_NAME] = df_node_props[COL_NAME].apply(set)\n", "df_db_node_props[COL_NAME] = df_db_node_props[COL_NAME].apply(set)\n", "\n", "# Figure out what servers are in the configs but not the database and vice versa\n", "missing_servers = (df_db_node_props - df_node_props).rename(\n", " columns={COL_NAME: 'missing-{}'.format(COL_NAME)})\n", "extra_servers = (df_node_props - df_db_node_props).rename(\n", " columns={COL_NAME: 'extra-{}'.format(COL_NAME)})\n", "result = pd.concat([missing_servers, extra_servers], axis=1, sort=False)\n", "result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Continue exploring\n", "\n", "We showed you how to extract the database of configured _NTP servers_ for every node and how to test that the settings are correct for a variety of desired test configurations. The underlying principles can be applied to other network configurations, such as [interfaceProperties](https://pybatfish.readthedocs.io/en/latest/notebooks/configProperties.html#Interface-Properties), [bgpProcessConfiguration](https://pybatfish.readthedocs.io/en/latest/notebooks/configProperties.html#BGP-Process-Configuration), [ospfProcessConfiguration](https://pybatfish.readthedocs.io/en/latest/notebooks/configProperties.html#OSPF-Process-Configuration) etc.\n", "\n", "For example `interfaceProperties()` question can be used to fetch properties like interface MTU using a simple command." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
InterfaceMTU
0as1border1[Ethernet0/0]1500
1as1border2[Ethernet0/0]1500
2as2border1[Ethernet0/0]1500
3as2border2[Ethernet0/0]1500
4as3border1[Ethernet0/0]1500
5as3border2[Ethernet0/0]1500
\n", "
" ], "text/plain": [ " Interface MTU\n", "0 as1border1[Ethernet0/0] 1500\n", "1 as1border2[Ethernet0/0] 1500\n", "2 as2border1[Ethernet0/0] 1500\n", "3 as2border2[Ethernet0/0] 1500\n", "4 as3border1[Ethernet0/0] 1500\n", "5 as3border2[Ethernet0/0] 1500" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Extract interface MTU for Ethernet0/0 interfaces on border routers\n", "interface_mtu = bf.q.interfaceProperties(\n", " interfaces=\"/border/[Ethernet0/0]\",\n", " properties=\"MTU\").answer().frame()\n", "interface_mtu" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "### Get involved with the Batfish community! \n", "Start interacting through [Slack](https://join.slack.com/t/batfish-org/shared_invite/enQtMzA0Nzg2OTAzNzQ1LTcyYzY3M2Q0NWUyYTRhYjdlM2IzYzRhZGU1NWFlNGU2MzlhNDY3OTJmMDIyMjQzYmRlNjhkMTRjNWIwNTUwNTQ) or [GitHub](https://github.com/batfish/batfish) to know more. We would love to talk with you about Batfish or your Network!" ] } ], "metadata": { "hide_input": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 2 }