Welcome to the Batfish documentation!

Setup

Installing Batfish and Pybatfish

Getting started with Batfish is easy. First, pull and run the latest allinone Docker container:

docker pull batfish/allinone
docker run --name batfish -v batfish-data:/data -p 8888:8888 -p 9997:9997 -p 9996:9996 batfish/allinone

Then, install Pybatfish using pip:

python3 -m pip install --upgrade pybatfish

Pybatfish requires python 3 and we recommend that you install it in a virtual environment.

Upgrading Batfish and Pybatfish

In order to upgrade to the latest Docker container, issue these commands on the Batfish server:

docker stop batfish
docker rm batfish
docker pull batfish/allinone
docker run --name batfish -v batfish-data:/data -p 8888:8888 -p 9997:9997 -p 9996:9996 batfish/allinone

To upgrade Pybatfish, use the same command as above:

python3 -m pip install --upgrade pybatfish

We recommend that you upgrade Batfish and Pybatfish together.

Interacting with the Batfish service

Python Imports

In your Python program (or shell) you will need to import Pybatfish modules. The most common imports are shown below. Depending your needs, this list may vary.

[1]:
import pandas as pd
from pybatfish.client.session import Session
from pybatfish.datamodel import *
from pybatfish.datamodel.answer import *
from pybatfish.datamodel.flow import *

Sessions

The Batfish service may be running locally on your machine, or on a remote server. The first step to analyzing your configurations is setting up the connection to the Batfish service.

[4]:
bf = Session(host="localhost")

Uploading configurations

Batfish is designed to analyze a series of snapshots of a network.

A network is a logical grouping of devices – it may mean all of the devices in your network, or a subset (e.g., all devices in a single datacenter.)

A snapshot is a state of the network at a given time. A network may contain many snapshots, allowing you to understand the evolution of your network.

Let’s say we will be working with our example datacenter:

[5]:
bf.set_network('example_dc')
[5]:
'example_dc'

Now you are ready to create your first snapshot. Batfish can ingest a variety of data in order to model your network, so let’s look at how you can package it as a snapshot.

Packaging snapshot data

Batfish expects snapshot data to be organized in a specific folder structure.


  • snapshot [top-level folder]

    • configs [folder with configurations files of network devices]

      • router1.cfg

      • router2.cfg

    • batfish [supplemental information (not device configurations)]

      • isp_config.json


See this snapshot for an example. For illustration, it contains some files that are not used by Batfish, e.g., example-network.png (network diagrams are not needed). It also contains information for host modeling, which need not be provided if you are not modeling hosts.

When you supply the snapshot as a zipped file, the top-level folder (called “snapshot” above) should be part of the zip archive.

Details on the format of configuration files and supplemental information are described here

Initializing a new snapshot

[6]:
SNAPSHOT_DIR = '../../networks/example'
bf.init_snapshot(SNAPSHOT_DIR, name='snapshot-2020-01-01', overwrite=True)
[6]:
'snapshot-2020-01-01'

Analyzing an existing snapshot

If you would like to analyze a previously-initialized snapshot, you do not need to re-initialize it. Simply set the network and snapshot by name:

[7]:
bf.set_network('example_dc')
bf.set_snapshot('snapshot-2020-01-01')
[7]:
'snapshot-2020-01-01'

Running Questions

After initializing (or setting) a snapshot, you can query the Batfish service to retrieve information about the snapshot.

Batfish exposes a series of questions to users. With the help of these questions you can examine data about you network as a whole, or individual devices, in a vendor-agnostic way.

The general pattern for Batfish questions is:

  • bf.q.<question_name>() Creates a question (with parameters, if applicable).

  • bf.q.<question_name>().answer() sends the question to the Batfish service and returns the answer

  • bf.q.<question_name>().answer().frame() converts the answer into a Pandas dataframe for easy data manipulation

This pattern is demonstrated via the initIssues question below.

Initialization issues

While Batfish supports a wide variety of vendors and configuration constructs, it may not fully support your configuration files. We recommend checking the status of the snapshot you just initialized, by runnning bf.q.initIssues:

[8]:
bf.q.initIssues().answer()
[8]:
Nodes Source_Lines Type Details Line_Text Parser_Context
0 ['as1border1'] None Convert warning (redflag) No virtual address set for VRRP on interface: 'GigabitEthernet0/0' None None

Given the answer of a question, you may want to focus on certain rows/columns or ignore certain rows. This is easy via Pandas dataframe manipulation. For instance, if you want to ignore all rows that warn about BGP update source, you may do the following.

[9]:
issues = bf.q.initIssues().answer().frame()
issues[issues['Details'].apply(lambda x: "Could not determine update source for BGP neighbor:" not in x)]
[9]:
Nodes Source_Lines Type Details Line_Text Parser_Context
0 ['as1border1'] None Convert warning (redflag) No virtual address set for VRRP on interface: 'GigabitEthernet0/0' None None

Now that you know the basics of interacting with the Batfish service, you can 1) explore a variety of questions that enable you to analyze your network in great detail; and 2) check out code examples for a range of use cases.

Logging

The server-side logs are accessible via Docker. Assuming your container is named “batfish”, run docker logs batfish to view the logs. See documentation for docker logs command for helpful command line options.

The default client-side logging (by pybatfish) is verbose to inform new users about what is happening. To control logging verbosity, use the following snippet toward the top of your Python script. Replace logging.WARN with your preferred logging level.

[10]:
import logging
logging.getLogger("pybatfish").setLevel(logging.WARN)

Batfish Questions

Batfish builds a model of your network behavior based on configuration and other data you provide. You can query the network model as well parsed configuration settings using several categories of Batfish questions listed below. See here for instructions on running questions.

Configuration Properties

This category of questions enables you to retrieve and process the contents of device configurations in a vendor-agnostic manner (except where the question itself is vendor-specific). Batfish organizes configuration content into several sub-categories.

Node Properties

Returns configuration settings of nodes.

Lists global settings of devices in the network. Settings that are specific to interfaces, routing protocols, etc. are available via other questions.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

properties

Include properties matching this regex.

NodePropertySpec

True

Invocation

[5]:
result = bf.q.nodeProperties().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

AS_Path_Access_Lists

Names of AS path access lists

Set of str

Authentication_Key_Chains

Names of authentication keychains

Set of str

Community_Match_Exprs

Names of expressions for matching a community

Set of str

Community_Set_Exprs

Names of expressions representing a community-set

Set of str

Community_Set_Match_Exprs

Names of expressions for matching a ommunity-set

Set of str

Community_Sets

Names of community-sets

Set of str

Configuration_Format

Configuration format of the node

str

DNS_Servers

Configured DNS servers

Set of str

DNS_Source_Interface

Source interface to use for communicating with DNS servers

str

Default_Cross_Zone_Action

Default action (PERMIT, DENY) for traffic that traverses firewall zones (null for non-firewall nodes)

str

Default_Inbound_Action

Default action (PERMIT, DENY) for traffic destined for this node

str

Domain_Name

Domain name of the node

str

Hostname

Hostname of the node

str

IKE_Phase1_Keys

Names of IKE Phase 1 keys

Set of str

IKE_Phase1_Policies

Names of IKE Phase 1 policies

Set of str

IKE_Phase1_Proposals

Names of IKE Phase 1 proposals

Set of str

IP6_Access_Lists

(Deprecated) Names of IPv6 filters (ACLs, firewall rule sets)

Set of str

IP_Access_Lists

Names of IPv4 filters (ACLs, firewall rule sets)

Set of str

IPsec_Peer_Configs

Names of IPSec peers

Set of str

IPsec_Phase2_Policies

Names of IPSec Phase 2 policies

Set of str

IPsec_Phase2_Proposals

Names of IPSec Phase 2 proposals

Set of str

Interfaces

Names of interfaces

Set of str

Logging_Servers

Configured logging servers

Set of str

Logging_Source_Interface

Source interface for communicating with logging servers

str

NTP_Servers

Configured NTP servers

Set of str

NTP_Source_Interface

Source interface for communicating with NTP servers

str

PBR_Policies

Names of policy-based routing (PBR) policies

Set of str

Route6_Filter_Lists

(Deprecated) Names of structures that filter IPv6 routes (e.g., prefix lists)

Set of str

Route_Filter_Lists

Names of structures that filter IPv4 routes (e.g., prefix lists)

Set of str

Routing_Policies

Names of policies that manipulate routes (e.g., route maps)

Set of str

SNMP_Source_Interface

Source interface to use for communicating with SNMP servers

str

SNMP_Trap_Servers

Configured SNMP trap servers

Set of str

TACACS_Servers

Configured TACACS servers

Set of str

TACACS_Source_Interface

Source interface to use for communicating with TACACS servers

str

VRFs

Names of VRFs present on the node

Set of str

Zones

Names of firewall zones on the node

Set of str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node AS_Path_Access_Lists Authentication_Key_Chains Community_Match_Exprs Community_Set_Exprs Community_Set_Match_Exprs Community_Sets Configuration_Format DNS_Servers DNS_Source_Interface Default_Cross_Zone_Action Default_Inbound_Action Domain_Name Hostname IKE_Phase1_Keys ... Interfaces Logging_Servers Logging_Source_Interface NTP_Servers NTP_Source_Interface PBR_Policies Route6_Filter_Lists Route_Filter_Lists Routing_Policies SNMP_Source_Interface SNMP_Trap_Servers TACACS_Servers TACACS_Source_Interface VRFs Zones
0 as2border2 [] [] ['as1_community', 'as2_community', 'as3_community'] [] ['as1_community', 'as2_community', 'as3_community'] [] CISCO_IOS [] None PERMIT PERMIT lab.local as2border2 [] ... ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] [] None ['18.18.18.18'] None [] [] ['101', '103', 'inbound_route_filter', 'outbound_routes', '~MATCH_SUPPRESSED_SUMMARY_ONLY:default~'] ['as1_to_as2', 'as2_to_as1', 'as2_to_as3', 'as3_to_as2', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:10.23.21.3~', '~BGP_PEER_EXPORT_POLICY:default:2.1.2.1~', '~BGP_PEER_EXPORT_POLICY:default:2.1.2.2~', '~BGP_REDISTRIBUTION_POLICY:default~', '~OSPF_EXPORT_POLICY:default:1~', '~RESOLUTION_POLICY~', '~suppress~rp~summary-only~'] None [] [] None ['default'] []
1 as1border1 [] [] ['as1_community', 'as2_community', 'as3_community'] [] ['as1_community', 'as2_community', 'as3_community'] [] CISCO_IOS [] None PERMIT PERMIT lab.local as1border1 [] ... ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] [] None [] None [] [] ['101', '102', '103', 'default_list', 'inbound_route_filter'] ['as1_to_as2', 'as1_to_as3', 'as2_to_as1', 'as3_to_as1', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:1.10.1.1~', '~BGP_PEER_EXPORT_POLICY:default:10.12.11.2~', '~BGP_PEER_EXPORT_POLICY:default:3.2.2.2~', '~BGP_PEER_EXPORT_POLICY:default:5.6.7.8~', '~BGP_REDISTRIBUTION_POLICY:default~', '~OSPF_EXPORT_POLICY:default:1~', '~RESOLUTION_POLICY~'] None [] [] None ['default'] []
2 as3border2 [] [] ['as1_community', 'as2_community', 'as3_community'] [] ['as1_community', 'as2_community', 'as3_community'] [] CISCO_IOS [] None PERMIT PERMIT lab.local as3border2 [] ... ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] [] None ['18.18.18.18', '23.23.23.23'] None [] [] ['101', '102', '103', 'inbound_route_filter'] ['as1_to_as3', 'as2_to_as3', 'as3_to_as1', 'as3_to_as2', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:10.13.22.1~', '~BGP_PEER_EXPORT_POLICY:default:3.10.1.1~', '~BGP_REDISTRIBUTION_POLICY:default~', '~OSPF_EXPORT_POLICY:default:1~', '~RESOLUTION_POLICY~'] None [] [] None ['default'] []
3 as1border2 [] [] ['as1_community', 'as2_community', 'as3_community', 'as4_community'] [] ['as1_community', 'as2_community', 'as3_community', 'as4_community'] [] CISCO_IOS [] None PERMIT PERMIT lab.local as1border2 [] ... ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] [] None ['18.18.18.18', '23.23.23.23'] None [] [] ['101', '102', '103', 'as4-prefixes', 'inbound_route_filter'] ['as1_to_as2', 'as1_to_as3', 'as1_to_as4', 'as2_to_as1', 'as3_to_as1', 'as4_to_as1', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:1.10.1.1~', '~BGP_PEER_EXPORT_POLICY:default:10.13.22.3~', '~BGP_PEER_EXPORT_POLICY:default:10.14.22.4~', '~BGP_REDISTRIBUTION_POLICY:default~', '~OSPF_EXPORT_POLICY:default:1~', '~RESOLUTION_POLICY~'] None [] [] None ['default'] []
4 as2dept1 [] [] ['as2_community'] [] ['as2_community'] [] CISCO_IOS [] None PERMIT PERMIT lab.local as2dept1 [] ... ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'GigabitEthernet3/0', 'Loopback0'] [] None [] None [] [] ['102'] ['as2_to_dept', 'dept_to_as2', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:2.34.101.3~', '~BGP_PEER_EXPORT_POLICY:default:2.34.201.3~', '~BGP_REDISTRIBUTION_POLICY:default~', '~RESOLUTION_POLICY~'] None [] [] None ['default'] []

5 rows × 37 columns

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Node                                                                                                                                                                                                                                                                                                                                                                                  as2border2
AS_Path_Access_Lists                                                                                                                                                                                                                                                                                                                                                                          []
Authentication_Key_Chains                                                                                                                                                                                                                                                                                                                                                                     []
Community_Match_Exprs                                                                                                                                                                                                                                                                                                                        ['as1_community', 'as2_community', 'as3_community']
Community_Set_Exprs                                                                                                                                                                                                                                                                                                                                                                           []
Community_Set_Match_Exprs                                                                                                                                                                                                                                                                                                                    ['as1_community', 'as2_community', 'as3_community']
Community_Sets                                                                                                                                                                                                                                                                                                                                                                                []
Configuration_Format                                                                                                                                                                                                                                                                                                                                                                   CISCO_IOS
DNS_Servers                                                                                                                                                                                                                                                                                                                                                                                   []
DNS_Source_Interface                                                                                                                                                                                                                                                                                                                                                                        None
Default_Cross_Zone_Action                                                                                                                                                                                                                                                                                                                                                                 PERMIT
Default_Inbound_Action                                                                                                                                                                                                                                                                                                                                                                    PERMIT
Domain_Name                                                                                                                                                                                                                                                                                                                                                                            lab.local
Hostname                                                                                                                                                                                                                                                                                                                                                                              as2border2
IKE_Phase1_Keys                                                                                                                                                                                                                                                                                                                                                                               []
IKE_Phase1_Policies                                                                                                                                                                                                                                                                                                                                                                           []
IKE_Phase1_Proposals                                                                                                                                                                                                                                                                                                                                                                          []
IP6_Access_Lists                                                                                                                                                                                                                                                                                                                                                                              []
IP_Access_Lists                                                                                                                                                                                                                                                                                                                             ['101', '103', 'INSIDE_TO_AS3', 'OUTSIDE_TO_INSIDE']
IPsec_Peer_Configs                                                                                                                                                                                                                                                                                                                                                                            []
IPsec_Phase2_Policies                                                                                                                                                                                                                                                                                                                                                                         []
IPsec_Phase2_Proposals                                                                                                                                                                                                                                                                                                                                                                        []
Interfaces                                                                                                                                                                                                                                                                                        ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0']
Logging_Servers                                                                                                                                                                                                                                                                                                                                                                               []
Logging_Source_Interface                                                                                                                                                                                                                                                                                                                                                                    None
NTP_Servers                                                                                                                                                                                                                                                                                                                                                                      ['18.18.18.18']
NTP_Source_Interface                                                                                                                                                                                                                                                                                                                                                                        None
PBR_Policies                                                                                                                                                                                                                                                                                                                                                                                  []
Route6_Filter_Lists                                                                                                                                                                                                                                                                                                                                                                           []
Route_Filter_Lists                                                                                                                                                                                                                                                                          ['101', '103', 'inbound_route_filter', 'outbound_routes', '~MATCH_SUPPRESSED_SUMMARY_ONLY:default~']
Routing_Policies             ['as1_to_as2', 'as2_to_as1', 'as2_to_as3', 'as3_to_as2', '~BGP_COMMON_EXPORT_POLICY:default~', '~BGP_PEER_EXPORT_POLICY:default:10.23.21.3~', '~BGP_PEER_EXPORT_POLICY:default:2.1.2.1~', '~BGP_PEER_EXPORT_POLICY:default:2.1.2.2~', '~BGP_REDISTRIBUTION_POLICY:default~', '~OSPF_EXPORT_POLICY:default:1~', '~RESOLUTION_POLICY~', '~suppress~rp~summary-only~']
SNMP_Source_Interface                                                                                                                                                                                                                                                                                                                                                                       None
SNMP_Trap_Servers                                                                                                                                                                                                                                                                                                                                                                             []
TACACS_Servers                                                                                                                                                                                                                                                                                                                                                                                []
TACACS_Source_Interface                                                                                                                                                                                                                                                                                                                                                                     None
VRFs                                                                                                                                                                                                                                                                                                                                                                                 ['default']
Zones                                                                                                                                                                                                                                                                                                                                                                                         []
Name: 0, dtype: object

Interface Properties

Returns configuration settings of interfaces.

Lists interface-level settings of interfaces. Settings for routing protocols, VRFs, and zones etc. that are attached to interfaces are available via other questions.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

interfaces

Include interfaces matching this specifier.

InterfaceSpec

True

properties

Include properties matching this specifier.

InterfacePropertySpec

True

excludeShutInterfaces

Exclude interfaces that are shutdown.

bool

True

Invocation

[10]:
result = bf.q.interfaceProperties().answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

Access_VLAN

VLAN number when the switchport mode is access (null otherwise)

int

Active

Whether the interface is active

bool

Admin_Up

Whether the interface is administratively enabled

bool

All_Prefixes

All IPv4 addresses assigned to the interface

List of str

Allowed_VLANs

Allowed VLAN numbers when the switchport mode is trunk

str

Auto_State_VLAN

For VLAN interfaces, whether the operational status depends on member switchports

bool

Bandwidth

Nominal bandwidth in bits/sec, used for protocol cost calculations

float

Blacklisted

Whether the interface is considered down for maintenance

bool

Channel_Group

Name of the aggregated interface (e.g., a port channel) to which this interface belongs

str

Channel_Group_Members

For aggregated interfaces (e.g., a port channel), names of constituent interfaces

List of str

DHCP_Relay_Addresses

IPv4 addresses to which incoming DHCP requests are relayed

List of str

Declared_Names

Any aliases explicitly defined for this interface

List of str

Description

Configured interface description

str

Encapsulation_VLAN

Number for VLAN encapsulation

int

HSRP_Groups

HSRP group identifiers

Set of str

HSRP_Version

HSRP version that will be used

str

Inactive_Reason

Reason why interface is inactive

str

Incoming_Filter_Name

Name of the input IPv4 filter

str

MLAG_ID

MLAG identifier of the interface

int

MTU

Layer3 MTU of the interface

int

Native_VLAN

Native VLAN when switchport mode is trunk

int

Outgoing_Filter_Name

Name of the output IPv4 filter

str

PBR_Policy_Name

Name of policy-based routing (PBR) policy

str

Primary_Address

Primary IPv4 address along with the prefix length

str

Primary_Network

Primary IPv4 subnet, in canonical form

str

Proxy_ARP

Whether proxy ARP is enabled

bool

Rip_Enabled

Whether RIP is enabled

bool

Rip_Passive

Whether interface is in RIP passive mode

bool

Spanning_Tree_Portfast

Whether spanning-tree portfast feature is enabled

bool

Speed

Link speed in bits/sec

float

Switchport

Whether the interface is configured as switchport

bool

Switchport_Mode

Switchport mode (ACCESS, DOT1Q_TUNNEL, DYNAMIC_AUTO, DYNAMIC_DESIRABLE, FEX_FABRIC, MONITOR, NONE, TAP, TOOL, TRUNK) for switchport interfaces

str

Switchport_Trunk_Encapsulation

Encapsulation type (DOT1Q, ISL, NEGOTIATE) for switchport trunk interfaces

str

VRF

Name of the VRF to which the interface belongs

str

VRRP_Groups

All VRRP groups to which the interface belongs

List of int

Zone_Name

Name of the firewall zone to which the interface belongs

str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN ... Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
0 as1border1[Ethernet0/0] None False False [] True 1e+07 False None [] [] ['Ethernet0/0'] None None ... None None None None True False False False 1e+07 False NONE DOT1Q default [] None
1 as1border1[GigabitEthernet0/0] None True True ['1.0.1.1/24'] True 1e+09 False None [] [] ['GigabitEthernet0/0'] None None ... None None 1.0.1.1/24 1.0.1.0/24 True False False False 1e+09 False NONE DOT1Q default [123] None
2 as1border1[GigabitEthernet1/0] None True True ['10.12.11.1/24'] True 1e+09 False None [] [] ['GigabitEthernet1/0'] None None ... None None 10.12.11.1/24 10.12.11.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 as1border1[Loopback0] None True True ['1.1.1.1/32'] True 8e+09 None None [] [] ['Loopback0'] None None ... None None 1.1.1.1/32 1.1.1.1/32 True False False False None False NONE DOT1Q default [] None
4 as1border2[Ethernet0/0] None False False [] True 1e+07 False None [] [] ['Ethernet0/0'] None None ... None None None None True False False False 1e+07 False NONE DOT1Q default [] None

5 rows × 37 columns

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Interface                         as1border1[Ethernet0/0]
Access_VLAN                                          None
Active                                              False
Admin_Up                                            False
All_Prefixes                                           []
Allowed_VLANs
Auto_State_VLAN                                      True
Bandwidth                                           1e+07
Blacklisted                                         False
Channel_Group                                        None
Channel_Group_Members                                  []
DHCP_Relay_Addresses                                   []
Declared_Names                            ['Ethernet0/0']
Description                                          None
Encapsulation_VLAN                                   None
HSRP_Groups                                            []
HSRP_Version                                         None
Inactive_Reason                     Administratively down
Incoming_Filter_Name                                 None
MLAG_ID                                              None
MTU                                                  1500
Native_VLAN                                          None
Outgoing_Filter_Name                                 None
PBR_Policy_Name                                      None
Primary_Address                                      None
Primary_Network                                      None
Proxy_ARP                                            True
Rip_Enabled                                         False
Rip_Passive                                         False
Spanning_Tree_Portfast                              False
Speed                                               1e+07
Switchport                                          False
Switchport_Mode                                      NONE
Switchport_Trunk_Encapsulation                      DOT1Q
VRF                                               default
VRRP_Groups                                            []
Zone_Name                                            None
Name: 0, dtype: object

BGP Process Configuration

Returns configuration settings of BGP processes.

Reports configuration settings for each BGP process on each node and VRF in the network. This question reports only process-wide settings. Peer-specific settings are reported by the bgpPeerConfiguration question.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

properties

Include properties matching this regex.

BgpProcessPropertySpec

True

Invocation

[15]:
result = bf.q.bgpProcessConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF

str

Router_ID

Router ID

str

Confederation_ID

Externally visible autonomous system number for the confederation

int

Confederation_Members

Set of autonomous system numbers visible only within this BGP confederation

Set of int

Multipath_EBGP

Whether multipath routing is enabled for EBGP

bool

Multipath_IBGP

Whether multipath routing is enabled for IBGP

bool

Multipath_Match_Mode

Which AS paths are considered equivalent (EXACT_PATH, FIRST_AS, PATH_LENGTH) when multipath BGP is enabled

str

Neighbors

All peers configured on this process, identified by peer address (for active and dynamic peers) or peer interface (for BGP unnumbered peers)

Set of str

Route_Reflector

Whether any BGP peer in this process is configured as a route reflector client, for ipv4 unicast address family

bool

Tie_Breaker

Tie breaking mode (ARRIVAL_ORDER, CLUSTER_LIST_LENGTH, ROUTER_ID)

str

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node VRF Router_ID Confederation_ID Confederation_Members Multipath_EBGP Multipath_IBGP Multipath_Match_Mode Neighbors Route_Reflector Tie_Breaker
0 as2border2 default 2.1.1.2 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '10.23.21.3'] False ARRIVAL_ORDER
1 as2dist1 default 2.1.3.1 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '2.34.101.4'] False ARRIVAL_ORDER
2 as3core1 default 3.10.1.1 None None True True EXACT_PATH ['3.1.1.1', '3.2.2.2'] True ARRIVAL_ORDER
3 as2border1 default 2.1.1.1 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '10.12.11.1'] False ARRIVAL_ORDER
4 as1core1 default 1.10.1.1 None None True True EXACT_PATH ['1.1.1.1', '1.2.2.2'] True ARRIVAL_ORDER

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node                                               as2border2
VRF                                                   default
Router_ID                                             2.1.1.2
Confederation_ID                                         None
Confederation_Members                                    None
Multipath_EBGP                                           True
Multipath_IBGP                                           True
Multipath_Match_Mode                               EXACT_PATH
Neighbors                ['2.1.2.1', '2.1.2.2', '10.23.21.3']
Route_Reflector                                         False
Tie_Breaker                                     ARRIVAL_ORDER
Name: 0, dtype: object

BGP Peer Configuration

Returns configuration settings for BGP peerings.

Reports configuration settings for each configured BGP peering on each node in the network. This question reports peer-specific settings. Settings that are process-wide are reported by the bgpProcessConfiguration question.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

properties

Include properties matching this regex.

BgpPeerPropertySpec

True

Invocation

[20]:
result = bf.q.bgpPeerConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF

str

Local_AS

Local AS number

int

Local_IP

Local IPv4 address (null for BGP unnumbered peers)

str

Local_Interface

Local Interface

str

Confederation

Confederation AS number

int

Remote_AS

Remote AS numbers with which this peer may establish a session

str

Remote_IP

Remote IP

str

Description

Configured peer description

str

Route_Reflector_Client

Whether this peer is a route reflector client

bool

Cluster_ID

Cluster ID of this peer (null for peers that are not route reflector clients)

str

Peer_Group

Name of the BGP peer group to which this peer belongs

str

Import_Policy

Names of import policies to be applied to routes received by this peer

Set of str

Export_Policy

Names of export policies to be applied to routes exported by this peer

Set of str

Send_Community

Whether this peer propagates communities

bool

Is_Passive

Whether this peer is passive

bool

Print the first 5 rows of the returned Dataframe

[21]:
result.head(5)
[21]:
Node VRF Local_AS Local_IP Local_Interface Confederation Remote_AS Remote_IP Description Route_Reflector_Client Cluster_ID Peer_Group Import_Policy Export_Policy Send_Community Is_Passive
0 as3core1 default 3 3.10.1.1 None None 3 3.1.1.1 None True 3.10.1.1 as3 [] [] True False
1 as2border1 default 2 10.12.11.2 None None 1 10.12.11.1 None False None as1 ['as1_to_as2'] ['as2_to_as1'] True False
2 as2core2 default 2 2.1.2.2 None None 2 2.1.3.2 None True 2.1.2.2 as2 [] [] True False
3 as3border1 default 3 3.1.1.1 None None 3 3.10.1.1 None False None as3 [] [] True False
4 as2core1 default 2 2.1.2.1 None None 2 2.1.3.2 None True 2.1.2.1 as2 [] [] True False

Print the first row of the returned Dataframe

[22]:
result.iloc[0]
[22]:
Node                      as3core1
VRF                        default
Local_AS                         3
Local_IP                  3.10.1.1
Local_Interface               None
Confederation                 None
Remote_AS                        3
Remote_IP                  3.1.1.1
Description                   None
Route_Reflector_Client        True
Cluster_ID                3.10.1.1
Peer_Group                     as3
Import_Policy                   []
Export_Policy                   []
Send_Community                True
Is_Passive                   False
Name: 0, dtype: object

HSRP Properties

Returns configuration settings of HSRP groups.

Lists information about HSRP groups on interfaces.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

interfaces

Include interfaces matching this specifier.

InterfaceSpec

True

virtualAddresses

Include only groups with at least one virtual address matching this specifier.

IpSpec

True

excludeShutInterfaces

Exclude interfaces that are shutdown.

bool

True

Invocation

[25]:
result = bf.q.hsrpProperties().answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

Group_Id

HSRP Group ID

int

Virtual_Addresses

Virtual Addresses

Set of str

Source_Address

Source Address used for HSRP messages

str

Priority

HSRP router priority

int

Preempt

Whether preemption is allowed

bool

Active

Whether the interface is active

bool

Print the first 5 rows of the returned Dataframe

[26]:
result.head(5)
[26]:
Interface Group_Id Virtual_Addresses Source_Address Priority Preempt Active
0 br2[GigabitEthernet0/2] 12 ['192.168.1.254'] 192.168.1.2/24 100 False True
1 br1[GigabitEthernet0/2] 12 ['192.168.1.254'] 192.168.1.1/24 110 False True

Print the first row of the returned Dataframe

[27]:
result.iloc[0]
[27]:
Interface            br2[GigabitEthernet0/2]
Group_Id                                  12
Virtual_Addresses          ['192.168.1.254']
Source_Address                192.168.1.2/24
Priority                                 100
Preempt                                False
Active                                  True
Name: 0, dtype: object

OSPF Process Configuration

Returns configuration parameters for OSPF routing processes.

Returns the values of important properties for all OSPF processes running across the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

properties

Include properties matching this specifier.

OspfProcessPropertySpec

True

Invocation

[30]:
result = bf.q.ospfProcessConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF name

str

Process_ID

Process ID

str

Areas

All OSPF areas for this process

Set of str

Reference_Bandwidth

Reference bandwidth in bits/sec used to calculate interface OSPF cost

float

Router_ID

Router ID of the process

str

Export_Policy_Sources

Names of policies that determine which routes are exported into OSPF

Set of str

Area_Border_Router

Whether this process is at the area border (with at least one interface in Area 0 and one in another area)

bool

Print the first 5 rows of the returned Dataframe

[31]:
result.head(5)
[31]:
Node VRF Process_ID Areas Reference_Bandwidth Router_ID Export_Policy_Sources Area_Border_Router
0 as2border1 default 1 ['1'] 1e+08 2.1.1.1 [] False
1 as2core1 default 1 ['1'] 1e+08 2.1.2.1 [] False
2 as2dist1 default 1 ['1'] 1e+08 2.1.3.1 [] False
3 as2dist2 default 1 ['1'] 1e+08 2.1.3.2 [] False
4 as1border2 default 1 ['1'] 1e+08 1.2.2.2 [] False

Print the first row of the returned Dataframe

[32]:
result.iloc[0]
[32]:
Node                     as2border1
VRF                         default
Process_ID                        1
Areas                         ['1']
Reference_Bandwidth           1e+08
Router_ID                   2.1.1.1
Export_Policy_Sources            []
Area_Border_Router            False
Name: 0, dtype: object

OSPF Interface Configuration

Returns OSPF configuration of interfaces.

Returns the interface level OSPF configuration details for the interfaces in the network which run OSPF.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

properties

Include properties matching this specifier.

OspfInterfacePropertySpec

True

Invocation

[35]:
result = bf.q.ospfInterfaceConfiguration().answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

VRF

VRF name

str

Process_ID

Process ID

str

OSPF_Area_Name

OSPF area to which the interface belongs

int

OSPF_Enabled

Whether OSPF is enabled

bool

OSPF_Passive

Whether interface is in OSPF passive mode

bool

OSPF_Cost

OSPF cost if explicitly configured

int

OSPF_Network_Type

Type of OSPF network associated with the interface

str

OSPF_Hello_Interval

Interval in seconds between sending OSPF hello messages

int

OSPF_Dead_Interval

Interval in seconds before a silent OSPF neighbor is declared dead

int

Print the first 5 rows of the returned Dataframe

[36]:
result.head(5)
[36]:
Interface VRF Process_ID OSPF_Area_Name OSPF_Enabled OSPF_Passive OSPF_Cost OSPF_Network_Type OSPF_Hello_Interval OSPF_Dead_Interval
0 as1core1[GigabitEthernet1/0] default 1 1 True False 1 BROADCAST 10 40
1 as1core1[GigabitEthernet0/0] default 1 1 True False 1 BROADCAST 10 40
2 as2dist1[Loopback0] default 1 1 True False 1 BROADCAST 10 40
3 as3core1[GigabitEthernet0/0] default 1 1 True False 1 BROADCAST 10 40
4 as3core1[GigabitEthernet1/0] default 1 1 True False 1 BROADCAST 10 40

Print the first row of the returned Dataframe

[37]:
result.iloc[0]
[37]:
Interface              as1core1[GigabitEthernet1/0]
VRF                                         default
Process_ID                                        1
OSPF_Area_Name                                    1
OSPF_Enabled                                   True
OSPF_Passive                                  False
OSPF_Cost                                         1
OSPF_Network_Type                         BROADCAST
OSPF_Hello_Interval                              10
OSPF_Dead_Interval                               40
Name: 0, dtype: object

OSPF Area Configuration

Returns configuration parameters of OSPF areas.

Returns information about all OSPF areas defined across the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

Invocation

[40]:
result = bf.q.ospfAreaConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF

str

Process_ID

Process ID

str

Area

Area number

str

Area_Type

Area type

str

Active_Interfaces

Names of active interfaces

Set of str

Passive_Interfaces

Names of passive interfaces

Set of str

Print the first 5 rows of the returned Dataframe

[41]:
result.head(5)
[41]:
Node VRF Process_ID Area Area_Type Active_Interfaces Passive_Interfaces
0 as2dist2 default 1 1 NONE ['GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] []
1 as2border2 default 1 1 NONE ['GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] []
2 as3core1 default 1 1 NONE ['GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] []
3 as2core1 default 1 1 NONE ['GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'GigabitEthernet3/0', 'Loopback0'] []
4 as1border2 default 1 1 NONE ['GigabitEthernet1/0', 'Loopback0'] []

Print the first row of the returned Dataframe

[42]:
result.iloc[0]
[42]:
Node                                                                   as2dist2
VRF                                                                     default
Process_ID                                                                    1
Area                                                                          1
Area_Type                                                                  NONE
Active_Interfaces     ['GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0']
Passive_Interfaces                                                           []
Name: 0, dtype: object

Multi-chassis LAG

Returns MLAG configuration.

Lists the configuration settings for each MLAG domain in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

mlagIds

Include MLAG IDs matching this specifier.

MlagIdSpec

True

Invocation

[45]:
result = bf.q.mlagProperties().answer().frame()

Return Value

Name

Description

Type

Node

Node name

str

MLAG_ID

MLAG domain ID

str

Peer_Address

Peer’s IP address

str

Local_Interface

Local interface used for MLAG peering

Interface

Source_Interface

Local interface used as source-interface for MLAG peering

Interface

Print the first 5 rows of the returned Dataframe

[46]:
result.head(5)
[46]:
Node MLAG_ID Peer_Address Local_Interface Source_Interface
0 dc1-bl1a DC1_BL1 10.255.252.11 dc1-bl1a[Port-Channel3] dc1-bl1a[Vlan4094]
1 dc1-bl1b DC1_BL1 10.255.252.10 dc1-bl1b[Port-Channel3] dc1-bl1b[Vlan4094]
2 dc1-l2leaf5a DC1_L2LEAF5 10.255.252.19 dc1-l2leaf5a[Port-Channel3] dc1-l2leaf5a[Vlan4094]
3 dc1-l2leaf5b DC1_L2LEAF5 10.255.252.18 dc1-l2leaf5b[Port-Channel3] dc1-l2leaf5b[Vlan4094]
4 dc1-l2leaf6a DC1_L2LEAF6 10.255.252.23 dc1-l2leaf6a[Port-Channel3] dc1-l2leaf6a[Vlan4094]

Print the first row of the returned Dataframe

[47]:
result.iloc[0]
[47]:
Node                               dc1-bl1a
MLAG_ID                             DC1_BL1
Peer_Address                  10.255.252.11
Local_Interface     dc1-bl1a[Port-Channel3]
Source_Interface         dc1-bl1a[Vlan4094]
Name: 0, dtype: object

IP Owners

Returns where IP addresses are attached in the network.

For each device, lists the mapping from IPs to corresponding interface(s) and VRF(s).

Inputs

Name

Description

Type

Optional

Default Value

ips

Restrict output to only specified IP addresses.

IpSpec

True

duplicatesOnly

Restrict output to only IP addresses that are duplicated (configured on a different node or VRF) in the snapshot.

bool

False

False

Invocation

[50]:
result = bf.q.ipOwners().answer().frame()

Return Value

Name

Description

Type

Node

Node hostname

str

VRF

VRF name

str

Interface

Interface name

str

IP

IP address

str

Mask

Network mask length

int

Active

Whether the interface is active

bool

Print the first 5 rows of the returned Dataframe

[51]:
result.head(5)
[51]:
Node VRF Interface IP Mask Active
0 as2dist2 default Loopback0 2.1.3.2 32 True
1 as2dist1 default Loopback0 2.1.3.1 32 True
2 as2dept1 default GigabitEthernet1/0 2.34.201.4 24 True
3 as2dept1 default Loopback0 2.1.1.2 32 True
4 as3border2 default GigabitEthernet1/0 3.0.2.1 24 True

Print the first row of the returned Dataframe

[52]:
result.iloc[0]
[52]:
Node          as2dist2
VRF            default
Interface    Loopback0
IP             2.1.3.2
Mask                32
Active            True
Name: 0, dtype: object

Named Structures

Returns named structure definitions.

Return structures defined in the configurations, represented in a vendor-independent JSON format.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

structureTypes

Include structures of this type.

NamedStructureSpec

True

structureNames

Include structures matching this name or regex.

str

True

ignoreGenerated

Whether to ignore auto-generated structures.

bool

True

True

indicatePresence

Output if the structure is present or absent.

bool

True

Invocation

[55]:
result = bf.q.namedStructures().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Structure_Type

Structure type

str

Structure_Name

Structure name

str

Structure_Definition

Structure definition

dict

Print the first 5 rows of the returned Dataframe

[56]:
result.head(5)
[56]:
Node Structure_Type Structure_Name Structure_Definition
0 as1border1 Community_Set_Match_Expr as1_community {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}
1 as1border2 Community_Set_Match_Expr as1_community {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}
2 as2border1 Community_Set_Match_Expr as1_community {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}
3 as2border2 Community_Set_Match_Expr as1_community {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}
4 as3border1 Community_Set_Match_Expr as1_community {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}

Print the first row of the returned Dataframe

[57]:
result.iloc[0]
[57]:
Node                                                                                                                                                                                                                                                as1border1
Structure_Type                                                                                                                                                                                                                        Community_Set_Match_Expr
Structure_Name                                                                                                                                                                                                                                   as1_community
Structure_Definition    {'expr': {'class': 'org.batfish.datamodel.routing_policy.communities.CommunityMatchRegex', 'communityRendering': {'class': 'org.batfish.datamodel.routing_policy.communities.ColonSeparatedRendering'}, 'regex': '(,|\{|\}|^|$| )1:'}}
Name: 0, dtype: object

Defined Structures

Lists the structures defined in the network.

Lists the structures defined in the network, along with the files and line numbers in which they are defined.

Inputs

Name

Description

Type

Optional

Default Value

filename

Include structures defined in the given file.

str

True

nodes

Include files used to generate nodes whose name matches this specifier.

NodeSpec

True

.*

names

Include structures whose name matches this string or regex.

str

True

.*

types

Include structures whose vendor-specific type matches this string or regex.

str

True

.*

Invocation

[60]:
result = bf.q.definedStructures().answer().frame()

Return Value

Name

Description

Type

Structure_Type

Vendor-specific type of the structure

str

Structure_Name

Name of the structure

str

Source_Lines

File and line numbers where the structure is defined

FileLines

Print the first 5 rows of the returned Dataframe

[61]:
result.head(5)
[61]:
Structure_Type Structure_Name Source_Lines
0 extended ipv4 access-list line OUTSIDE_TO_INSIDE: permit ip any any configs/as2border1.cfg:[137]
1 extended ipv4 access-list line blocktelnet: deny tcp any any eq telnet configs/as2core1.cfg:[122]
2 interface GigabitEthernet1/0 configs/as1core1.cfg:[69, 70, 71]
3 route-map-clause as3_to_as2 1 configs/as3border1.cfg:[146, 147, 148, 149]
4 extended ipv4 access-list 101 configs/as2border2.cfg:[140, 141]

Print the first row of the returned Dataframe

[62]:
result.iloc[0]
[62]:
Structure_Type          extended ipv4 access-list line
Structure_Name    OUTSIDE_TO_INSIDE: permit ip any any
Source_Lines              configs/as2border1.cfg:[137]
Name: 0, dtype: object

Referenced Structures

Lists the references in configuration files to vendor-specific structures.

Lists the references in configuration files to vendor-specific structures, along with the line number, the name and the type of the structure referenced, and configuration context in which each reference occurs.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include files used to generate nodes whose name matches this specifier.

NodeSpec

True

names

Include structures whose name matches this string or regex.

str

True

types

Include structures whose vendor-specific type matches this string or regex.

str

True

Invocation

[65]:
result = bf.q.referencedStructures().answer().frame()

Return Value

Name

Description

Type

Structure_Type

Type of structure referenced

str

Structure_Name

The referenced structure

str

Context

Configuration context in which the reference appears

str

Source_Lines

Lines where reference appears

FileLines

Print the first 5 rows of the returned Dataframe

[66]:
result.head(5)
[66]:
Structure_Type Structure_Name Context Source_Lines
0 bgp neighbor 1.10.1.1 (VRF default) bgp neighbor self ref configs/as1border1.cfg:[92, 111]
1 bgp neighbor 10.12.11.2 (VRF default) bgp neighbor self ref configs/as1border1.cfg:[114]
2 bgp neighbor 3.2.2.2 (VRF default) bgp neighbor self ref configs/as1border1.cfg:[112]
3 bgp neighbor 5.6.7.8 (VRF default) bgp neighbor self ref configs/as1border1.cfg:[113]
4 bgp peer-group as1 bgp neighbor peer-group configs/as1border1.cfg:[91]

Print the first row of the returned Dataframe

[67]:
result.iloc[0]
[67]:
Structure_Type                        bgp neighbor
Structure_Name              1.10.1.1 (VRF default)
Context                      bgp neighbor self ref
Source_Lines      configs/as1border1.cfg:[92, 111]
Name: 0, dtype: object

Undefined References

Identifies undefined references in configuration.

Finds configurations that have references to named structures (e.g., ACLs) that are not defined. Such occurrences indicate errors and can have serious consequences in some cases.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Look for undefined references on nodes matching this name or regex.

NodeSpec

True

.*

Invocation

[70]:
result = bf.q.undefinedReferences().answer().frame()

Return Value

Name

Description

Type

File_Name

File containing reference

str

Struct_Type

Type of struct reference is supposed to be

str

Ref_Name

The undefined reference

str

Context

Context of undefined reference

str

Lines

Lines where reference appears

FileLines

Print the first 5 rows of the returned Dataframe

[71]:
result.head(5)
[71]:
File_Name Struct_Type Ref_Name Context Lines
0 configs/as2core2.cfg route-map filter-bogons bgp inbound route-map configs/as2core2.cfg:[110]

Print the first row of the returned Dataframe

[72]:
result.iloc[0]
[72]:
File_Name            configs/as2core2.cfg
Struct_Type                     route-map
Ref_Name                    filter-bogons
Context             bgp inbound route-map
Lines          configs/as2core2.cfg:[110]
Name: 0, dtype: object

Unused Structures

Returns nodes with structures such as ACLs, routemaps, etc. that are defined but not used.

Return nodes with structures such as ACLs, routes, etc. that are defined but not used. This may represent a bug in the configuration, which may have occurred because a final step in a template or MOP was not completed. Or it could be harmless extra configuration generated from a master template that is not meant to be used on those nodes.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Look for unused structures on nodes matching this name or regex.

NodeSpec

True

.*

Invocation

[75]:
result = bf.q.unusedStructures().answer().frame()

Return Value

Name

Description

Type

Structure_Type

Vendor-specific type of the structure

str

Structure_Name

Name of the structure

str

Source_Lines

File and line numbers where the structure is defined

FileLines

Print the first 5 rows of the returned Dataframe

[76]:
result.head(5)
[76]:
Structure_Type Structure_Name Source_Lines
0 bgp peer-group as3 configs/as1border1.cfg:[85]
1 expanded community-list as1_community configs/as1border1.cfg:[121]
2 ipv4 prefix-list inbound_route_filter configs/as1border1.cfg:[131, 132]
3 bgp peer-group as2 configs/as1border2.cfg:[87]
4 expanded community-list as1_community configs/as1border2.cfg:[123]

Print the first row of the returned Dataframe

[77]:
result.iloc[0]
[77]:
Structure_Type                 bgp peer-group
Structure_Name                            as3
Source_Lines      configs/as1border1.cfg:[85]
Name: 0, dtype: object

VLAN Properties

Returns configuration settings of switched VLANs.

Lists information about implicitly and explicitly configured switched VLANs.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

interfaces

Include interfaces matching this specifier.

InterfaceSpec

True

vlans

Include VLANs in this space.

str

True

excludeShutInterfaces

Exclude interfaces that are shutdown.

bool

True

Invocation

[80]:
result = bf.q.switchedVlanProperties().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VLAN_ID

VLAN_ID

int

Interfaces

Switched interfaces carrying traffic for this VLAN

Set of Interface

VXLAN_VNI

VXLAN VNI with which this VLAN is associated

int

Print the first 5 rows of the returned Dataframe

[81]:
result.head(5)
[81]:
Node VLAN_ID Interfaces VXLAN_VNI
0 dc1-bl1b 250 [dc1-bl1b[Port-Channel3], dc1-bl1b[Vlan250]] 20250
1 dc1-leaf1a 210 [dc1-leaf1a[Vlan210]] 20210
2 dc1-leaf1a 211 [dc1-leaf1a[Vlan211]] 20211
3 dc1-leaf2a 3764 [dc1-leaf2a[Port-Channel3]] None
4 dc1-leaf2b 3789 [dc1-leaf2b[Port-Channel3]] None

Print the first row of the returned Dataframe

[82]:
result.iloc[0]
[82]:
Node                                              dc1-bl1b
VLAN_ID                                                250
Interfaces    [dc1-bl1b[Port-Channel3], dc1-bl1b[Vlan250]]
VXLAN_VNI                                            20250
Name: 0, dtype: object

VRRP Properties

Returns configuration settings of VRRP groups.

Lists information about VRRP groups on interfaces.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

interfaces

Include interfaces matching this specifier.

InterfaceSpec

True

virtualAddresses

Include only groups with at least one virtual address matching this specifier.

IpSpec

True

excludeShutInterfaces

Exclude interfaces that are shutdown.

bool

True

Invocation

[85]:
result = bf.q.vrrpProperties().answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

Group_Id

VRRP Group ID

int

Virtual_Addresses

Virtual Addresses

Set of str

Source_Address

Source Address used for VRRP messages

str

Priority

VRRP router priority

int

Preempt

Whether preemption is allowed

bool

Active

Whether the interface is active

bool

Print the first 5 rows of the returned Dataframe

[86]:
result.head(5)
[86]:
Interface Group_Id Virtual_Addresses Source_Address Priority Preempt Active
0 br1[GigabitEthernet0/2] 12 ['192.168.1.254'] 192.168.1.1/24 110 True True
1 br2[GigabitEthernet0/2] 12 ['192.168.1.254'] 192.168.1.2/24 100 True True

Print the first row of the returned Dataframe

[87]:
result.iloc[0]
[87]:
Interface            br1[GigabitEthernet0/2]
Group_Id                                  12
Virtual_Addresses          ['192.168.1.254']
Source_Address                192.168.1.1/24
Priority                                 110
Preempt                                 True
Active                                  True
Name: 0, dtype: object

A10 Virtual Server Configuration

Returns Virtual Server configuration of A10 devices.

Lists all the virtual-server to service-group to server mappings in A10 configurations.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

virtualServerIps

Include virtual servers whose IP match this specifier.

IpSpec

True

Invocation

[90]:
result = bf.q.a10VirtualServerConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Virtual_Server_Name

Virtual Server Name

str

Virtual_Server_Enabled

Virtual Server Enabled

bool

Virtual_Server_IP

Virtual Server IP

str

Virtual_Server_Port

Virtual Server Port

int

Virtual_Server_Port_Enabled

Virtual Server Port Enabled

bool

Virtual_Server_Type

Virtual Server Type

str

Virtual_Server_Port_Type_Name

Virtual Server Port Type Name

str

Service_Group_Name

Service Group Name

str

Service_Group_Type

Service Group Type

str

Servers

List of Servers. Each item is a 4-tuple: Server Name, Port, IP Address, and Active Status.

Set of List of str

Source_NAT_Pool_Name

Source NAT Pool Name

str

Print the first 5 rows of the returned Dataframe

[91]:
result.head(5)
[91]:
Node Virtual_Server_Name Virtual_Server_Enabled Virtual_Server_IP Virtual_Server_Port Virtual_Server_Port_Enabled Virtual_Server_Type Virtual_Server_Port_Type_Name Service_Group_Name Service_Group_Type Servers Source_NAT_Pool_Name
0 lb42 VS_TCP_80 True 10.0.0.1 80 True TCP None SG_TCP_80 TCP [['SERVER1', '80', '10.1.10.11', 'active'], ['SERVER2', '80', '10.1.10.12', 'inactive']] None

Print the first row of the returned Dataframe

[92]:
result.iloc[0]
[92]:
Node                                                                                                                 lb42
Virtual_Server_Name                                                                                             VS_TCP_80
Virtual_Server_Enabled                                                                                               True
Virtual_Server_IP                                                                                                10.0.0.1
Virtual_Server_Port                                                                                                    80
Virtual_Server_Port_Enabled                                                                                          True
Virtual_Server_Type                                                                                                   TCP
Virtual_Server_Port_Type_Name                                                                                        None
Service_Group_Name                                                                                              SG_TCP_80
Service_Group_Type                                                                                                    TCP
Servers                          [['SERVER1', '80', '10.1.10.11', 'active'], ['SERVER2', '80', '10.1.10.12', 'inactive']]
Source_NAT_Pool_Name                                                                                                 None
Name: 0, dtype: object

F5 BIG-IP VIP Configuration

Returns VIP configuration of F5 BIG-IP devices.

Lists all the VIP to server IP mappings contained in F5 BIP-IP configurations.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

Invocation

[95]:
result = bf.q.f5BigipVipConfiguration().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VIP_Name

Virtual Service Name

str

VIP_Endpoint

Virtual Service Endpoint

str

Servers

Servers

Set of str

Description

Description

str

Print the first 5 rows of the returned Dataframe

[96]:
result.head(5)
[96]:
Node VIP_Name VIP_Endpoint Servers Description
0 f5bigip /Common/virtual1 192.0.2.1:80 TCP ['10.0.0.1:80'] virtual1 is cool
1 f5bigip /Common/virtual2 192.0.2.2:80 TCP ['10.0.0.2:80'] pool2 is lame
2 f5bigip /Common/virtual3 192.0.2.3:80 TCP ['10.0.0.4:80', '10.0.0.3:80']

Print the first row of the returned Dataframe

[97]:
result.iloc[0]
[97]:
Node                     f5bigip
VIP_Name        /Common/virtual1
VIP_Endpoint    192.0.2.1:80 TCP
Servers          ['10.0.0.1:80']
Description     virtual1 is cool
Name: 0, dtype: object

Topology

This caterogy of questions is intended to retrieve the network topology used by Batfish. This topology is a combination of information in the snapshot and inference logic (e.g., which interfaces are layer3 neighbors). Currently, Layer 3 topology can be retrieved.

User Provided Layer 1 Topology

Returns normalized Layer 1 edges that were input to Batfish.

Lists Layer 1 edges after potentially normalizing node and interface names. All node names are lower-cased, and for nodes that appear in the snapshot, interface names are canonicalized based on the vendor. All input edges are in the output, including nodes and interfaces that do not appear in the snapshot.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include edges whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include edges whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[5]:
result = bf.q.userProvidedLayer1Edges().answer().frame()

Return Value

Name

Description

Type

Interface

Interface from which the edge originates

Interface

Remote_Interface

Interface at which the edge terminates

Interface

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Interface Remote_Interface
0 dc1-leaf2a[Ethernet1] dc1-spine1[Ethernet2]
1 dc1-leaf2b[Ethernet1] dc1-spine1[Ethernet3]
2 dc1-leaf2b[Ethernet2] dc1-spine2[Ethernet3]
3 dc1-svc3b[Ethernet6] dc1-l2leaf5b[Ethernet2]
4 dc1-leaf2a[Ethernet4] dc1-leaf2b[Ethernet4]

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Interface           dc1-leaf2a[Ethernet1]
Remote_Interface    dc1-spine1[Ethernet2]
Name: 0, dtype: object

Layer 3 Topology

Returns Layer 3 links.

Lists all Layer 3 edges in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include edges whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include edges whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[10]:
result = bf.q.layer3Edges().answer().frame()

Return Value

Name

Description

Type

Interface

Interface from which the edge originates

Interface

IPs

IPs

Set of str

Remote_Interface

Interface at which the edge terminates

Interface

Remote_IPs

Remote IPs

Set of str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Interface IPs Remote_Interface Remote_IPs
0 as1border1[GigabitEthernet0/0] ['1.0.1.1'] as1core1[GigabitEthernet1/0] ['1.0.1.2']
1 as1border1[GigabitEthernet1/0] ['10.12.11.1'] as2border1[GigabitEthernet0/0] ['10.12.11.2']
2 as1border2[GigabitEthernet0/0] ['10.13.22.1'] as3border2[GigabitEthernet0/0] ['10.13.22.3']
3 as1border2[GigabitEthernet1/0] ['1.0.2.1'] as1core1[GigabitEthernet0/0] ['1.0.2.2']
4 as1core1[GigabitEthernet0/0] ['1.0.2.2'] as1border2[GigabitEthernet1/0] ['1.0.2.1']

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Interface           as1border1[GigabitEthernet0/0]
IPs                                    ['1.0.1.1']
Remote_Interface      as1core1[GigabitEthernet1/0]
Remote_IPs                             ['1.0.1.2']
Name: 0, dtype: object

Routing Protocol Sessions and Policies

This category of questions reveals information regarding which routing protocol sessions are compatibly configured and which ones are established. It also allows to you analyze BGP routing policies.

BGP Session Compatibility

Returns the compatibility of configured BGP sessions.

Checks the settings of each configured BGP peering and reports any issue with those settings locally or incompatiblity with its remote counterparts. Each row represents one configured BGP peering on a node and contains information about the session it is meant to establish. For dynamic peers, there is one row per compatible remote peer. Statuses that indicate an independently misconfigured peerings include NO_LOCAL_AS, NO_REMOTE_AS, NO_LOCAL_IP (for eBGP single-hop peerings), LOCAL_IP_UNKNOWN_STATICALLY (for iBGP or eBGP multi-hop peerings), NO_REMOTE_IP (for point-to-point peerings), and NO_REMOTE_PREFIX (for dynamic peerings). INVALID_LOCAL_IP indicates that the peering’s configured local IP does not belong to any active interface on the node; UNKNOWN_REMOTE indicates that the configured remote IP is not present in the network. A locally valid point-to-point peering is deemed HALF_OPEN if it has no compatible remote peers, UNIQUE_MATCH if it has exactly one compatible remote peer, or MULTIPLE_REMOTES if it has multiple compatible remote peers. A locally valid dynamic peering is deemed NO_MATCH_FOUND if it has no compatible remote peers, or DYNAMIC_MATCH if it has at least one compatible remote peer.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include sessions whose first node matches this specifier.

NodeSpec

True

remoteNodes

Include sessions whose second node matches this specifier.

NodeSpec

True

status

Only include sessions for which compatibility status matches this specifier.

BgpSessionCompatStatusSpec

True

type

Only include sessions that match this specifier.

BgpSessionTypeSpec

True

Invocation

[5]:
result = bf.q.bgpSessionCompatibility().answer().frame()

Return Value

Name

Description

Type

Node

The node where this session is configured

str

VRF

The VRF in which this session is configured

str

Local_AS

The local AS of the session

int

Local_Interface

Local interface of the session

Interface

Local_IP

The local IP of the session

str

Remote_AS

The remote AS or list of ASes of the session

str

Remote_Node

Remote node for this session

str

Remote_Interface

Remote interface for this session

Interface

Remote_IP

Remote IP or prefix for this session

str

Address_Families

Address Families participating in this session

Set of str

Session_Type

The type of this session

str

Configured_Status

Configured status

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Configured_Status
0 as1border1 default 1 None 1.1.1.1 1 as1core1 None 1.10.1.1 ['IPV4_UNICAST'] IBGP UNIQUE_MATCH
1 as1border1 default 1 None None 666 None None 3.2.2.2 [] EBGP_SINGLEHOP NO_LOCAL_IP
2 as1border1 default 1 None None 555 None None 5.6.7.8 [] EBGP_SINGLEHOP NO_LOCAL_IP
3 as1border1 default 1 None 10.12.11.1 2 as2border1 None 10.12.11.2 ['IPV4_UNICAST'] EBGP_SINGLEHOP UNIQUE_MATCH
4 as1border2 default 1 None 1.2.2.2 1 as1core1 None 1.10.1.1 ['IPV4_UNICAST'] IBGP UNIQUE_MATCH

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Node                       as1border1
VRF                           default
Local_AS                            1
Local_Interface                  None
Local_IP                      1.1.1.1
Remote_AS                           1
Remote_Node                  as1core1
Remote_Interface                 None
Remote_IP                    1.10.1.1
Address_Families     ['IPV4_UNICAST']
Session_Type                     IBGP
Configured_Status        UNIQUE_MATCH
Name: 0, dtype: object

BGP Session Status

Returns the dynamic status of configured BGP sessions.

Checks whether configured BGP peerings can be established. Each row represents one configured BGP peering and contains information about the session it is configured to establish. For dynamic peerings, one row is shown per compatible remote peer. Possible statuses for each session are NOT_COMPATIBLE, ESTABLISHED, and NOT_ESTABLISHED. NOT_COMPATIBLE sessions are those where one or both peers are misconfigured; the BgpSessionCompatibility question provides further insight into the nature of the configuration error. NOT_ESTABLISHED sessions are those that are configured compatibly but will not come up because peers cannot reach each other (e.g., due to being blocked by an ACL). ESTABLISHED sessions are those that are compatible and are expected to come up.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include sessions whose first node matches this specifier.

NodeSpec

True

remoteNodes

Include sessions whose second node matches this specifier.

NodeSpec

True

status

Only include sessions for which status matches this specifier.

BgpSessionStatusSpec

True

type

Only include sessions that match this specifier.

BgpSessionTypeSpec

True

Invocation

[10]:
result = bf.q.bgpSessionStatus().answer().frame()

Return Value

Name

Description

Type

Node

The node where this session is configured

str

VRF

The VRF in which this session is configured

str

Local_AS

The local AS of the session

int

Local_Interface

Local interface of the session

Interface

Local_IP

The local IP of the session

str

Remote_AS

The remote AS or list of ASes of the session

str

Remote_Node

Remote node for this session

str

Remote_Interface

Remote interface for this session

Interface

Remote_IP

Remote IP or prefix for this session

str

Address_Families

Address Families participating in this session

Set of str

Session_Type

The type of this session

str

Established_Status

Established status

str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
0 as1border1 default 1 None 1.1.1.1 1 as1core1 None 1.10.1.1 ['IPV4_UNICAST'] IBGP ESTABLISHED
1 as1border1 default 1 None None 666 None None 3.2.2.2 [] EBGP_SINGLEHOP NOT_COMPATIBLE
2 as1border1 default 1 None None 555 None None 5.6.7.8 [] EBGP_SINGLEHOP NOT_COMPATIBLE
3 as1border1 default 1 None 10.12.11.1 2 as2border1 None 10.12.11.2 ['IPV4_UNICAST'] EBGP_SINGLEHOP ESTABLISHED
4 as1border2 default 1 None 1.2.2.2 1 as1core1 None 1.10.1.1 ['IPV4_UNICAST'] IBGP ESTABLISHED

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Node                        as1border1
VRF                            default
Local_AS                             1
Local_Interface                   None
Local_IP                       1.1.1.1
Remote_AS                            1
Remote_Node                   as1core1
Remote_Interface                  None
Remote_IP                     1.10.1.1
Address_Families      ['IPV4_UNICAST']
Session_Type                      IBGP
Established_Status         ESTABLISHED
Name: 0, dtype: object

BGP Edges

Returns BGP adjacencies.

Lists all BGP adjacencies in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include adjacencies whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include adjacencies whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[15]:
result = bf.q.bgpEdges().answer().frame()

Return Value

Name

Description

Type

Node

Node from which the edge originates

str

IP

IP at the side of originator

str

Interface

Interface at which the edge originates

str

AS_Number

AS Number at the side of originator

str

Remote_Node

Node at which the edge terminates

str

Remote_IP

IP at the side of the responder

str

Remote_Interface

Interface at which the edge terminates

str

Remote_AS_Number

AS Number at the side of responder

str

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node IP Interface AS_Number Remote_Node Remote_IP Remote_Interface Remote_AS_Number
0 as1border2 1.2.2.2 None 1 as1core1 1.10.1.1 None 1
1 as1core1 1.10.1.1 None 1 as1border1 1.1.1.1 None 1
2 as2dist2 2.1.3.2 None 2 as2core2 2.1.2.2 None 2
3 as3border2 3.2.2.2 None 3 as3core1 3.10.1.1 None 3
4 as3border1 10.23.21.3 None 3 as2border2 10.23.21.2 None 2

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node                as1border2
IP                     1.2.2.2
Interface                 None
AS_Number                    1
Remote_Node           as1core1
Remote_IP             1.10.1.1
Remote_Interface          None
Remote_AS_Number             1
Name: 0, dtype: object

OSPF Session Compatibility

Returns compatible OSPF sessions.

Returns compatible OSPF sessions in the network. A session is compatible if the interfaces involved are not shutdown and do run OSPF, are not OSPF passive and are associated with the same OSPF area.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this name or regex.

NodeSpec

True

remoteNodes

Include remote nodes matching this name or regex.

NodeSpec

True

statuses

Only include sessions matching this status specifier.

OspfSessionStatusSpec

True

Invocation

[20]:
result = bf.q.ospfSessionCompatibility().answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

VRF

VRF

str

IP

Ip

str

Area

Area

int

Remote_Interface

Remote Interface

Interface

Remote_VRF

Remote VRF

str

Remote_IP

Remote IP

str

Remote_Area

Remote Area

int

Session_Status

Status of the OSPF session

str

Print the first 5 rows of the returned Dataframe

[21]:
result.head(5)
[21]:
Interface VRF IP Area Remote_Interface Remote_VRF Remote_IP Remote_Area Session_Status
0 as2core2[GigabitEthernet0/0] default 2.12.22.2 1 as2border2[GigabitEthernet1/0] default 2.12.22.1 1 ESTABLISHED
1 as2core2[GigabitEthernet1/0] default 2.12.12.2 1 as2border1[GigabitEthernet2/0] default 2.12.12.1 1 ESTABLISHED
2 as2border1[GigabitEthernet1/0] default 2.12.11.1 1 as2core1[GigabitEthernet0/0] default 2.12.11.2 1 ESTABLISHED
3 as2border1[GigabitEthernet2/0] default 2.12.12.1 1 as2core2[GigabitEthernet1/0] default 2.12.12.2 1 ESTABLISHED
4 as2core2[GigabitEthernet3/0] default 2.23.21.2 1 as2dist1[GigabitEthernet1/0] default 2.23.21.3 1 ESTABLISHED

Print the first row of the returned Dataframe

[22]:
result.iloc[0]
[22]:
Interface             as2core2[GigabitEthernet0/0]
VRF                                        default
IP                                       2.12.22.2
Area                                             1
Remote_Interface    as2border2[GigabitEthernet1/0]
Remote_VRF                                 default
Remote_IP                                2.12.22.1
Remote_Area                                      1
Session_Status                         ESTABLISHED
Name: 0, dtype: object

OSPF Edges

Returns OSPF adjacencies.

Lists all OSPF adjacencies in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include adjacencies whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include edges whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[25]:
result = bf.q.ospfEdges().answer().frame()

Return Value

Name

Description

Type

Interface

Interface from which the edge originates

Interface

Remote_Interface

Interface at which the edge terminates

Interface

Print the first 5 rows of the returned Dataframe

[26]:
result.head(5)
[26]:
Interface Remote_Interface
0 as1border1[GigabitEthernet0/0] as1core1[GigabitEthernet1/0]
1 as1core1[GigabitEthernet1/0] as1border1[GigabitEthernet0/0]
2 as1border2[GigabitEthernet1/0] as1core1[GigabitEthernet0/0]
3 as1core1[GigabitEthernet0/0] as1border2[GigabitEthernet1/0]
4 as2border1[GigabitEthernet1/0] as2core1[GigabitEthernet0/0]

Print the first row of the returned Dataframe

[27]:
result.iloc[0]
[27]:
Interface           as1border1[GigabitEthernet0/0]
Remote_Interface      as1core1[GigabitEthernet1/0]
Name: 0, dtype: object

Test Route Policies

Evaluates the processing of a route by a given policy.

Find how the specified route is processed through the specified routing policies.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Only examine filters on nodes matching this specifier.

NodeSpec

True

policies

Only consider policies that match this specifier.

RoutingPolicySpec

True

inputRoutes

The BGP route announcements to test the policy on.

List of BgpRoute

False

direction

The direction of the route, with respect to the device (IN/OUT).

str

False

Invocation

[30]:
result = bf.q.testRoutePolicies(policies='/as1_to_/', direction='in', inputRoutes=list([BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[[64512, 64513], [64514]], communities=['64512:42', '64513:21'])])).answer().frame()

Return Value

Name

Description

Type

Node

The node that has the policy

str

Policy_Name

The name of this policy

str

Input_Route

The input route

BgpRoute

Action

The action of the policy on the input route

str

Output_Route

The output route, if any

BgpRoute

Difference

The difference between the input and output routes, if any

BgpRouteDiffs

Trace

Route policy trace that shows which clauses/terms matched the input route. If the trace is empty, either nothing matched or tracing is not yet been implemented for this policy type. This is an experimental feature whose content and format is subject to change.

List of TraceTree

Print the first 5 rows of the returned Dataframe

[31]:
result.head(5)
[31]:
Node Policy_Name Input_Route Action Output_Route Difference Trace
0 as1border1 as1_to_as2 BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) DENY None None
1 as1border1 as1_to_as3 BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) DENY None None
2 as1border2 as1_to_as2 BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) DENY None None
3 as1border2 as1_to_as3 BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) DENY None None
4 as1border2 as1_to_as4 BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) PERMIT BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['1:4', '64512:42', '64513:21'], localPreference=0, metric=50, nextHopIp=None, sourceProtocol=None, tag=0, weight=0) BgpRouteDiffs(diffs=[BgpRouteDiff(fieldName='communities', oldValue='[64512:42, 64513:21]', newValue='[1:4, 64512:42, 64513:21]'), BgpRouteDiff(fieldName='metric', oldValue='0', newValue='50')]) - Matched route-map as1_to_as4 clause 2

Print the first row of the returned Dataframe

[32]:
result.iloc[0]
[32]:
Node                                                                                                                                                                                                                                                                                                                          as1border1
Policy_Name                                                                                                                                                                                                                                                                                                                   as1_to_as2
Input_Route     BgpRoute(network='10.0.0.0/24', originatorIp='4.4.4.4', originType='egp', protocol='bgp', asPath=[{'asns': [64512, 64513], 'confederation': False}, {'asns': [64514], 'confederation': False}], communities=['64512:42', '64513:21'], localPreference=0, metric=0, nextHopIp=None, sourceProtocol=None, tag=0, weight=0)
Action                                                                                                                                                                                                                                                                                                                              DENY
Output_Route                                                                                                                                                                                                                                                                                                                        None
Difference                                                                                                                                                                                                                                                                                                                          None
Trace
Name: 0, dtype: object

Search Route Policies

Finds route announcements for which a route policy has a particular behavior.

This question finds route announcements for which a route policy has a particular behavior. The behaviors can be: that the policy permits the route (permit) or that it denies the route (deny). Constraints can be imposed on the input route announcements of interest and, in the case of a permit action, also on the output route announcements of interest. Route policies are selected using node and policy specifiers, which might match multiple policies. In this case, a (possibly different) answer will be found for each policy. Note: This question currently does not support all of the route policy features that Batfish supports. The question only supports common forms of matching on prefixes, communities, and AS-paths, as well as common forms of setting communities, the local preference, and the metric. The question logs all unsupported features that it encounters as warnings. Due to unsupported features, it is possible for the question to return no answers even for route policies that can in fact exhibit the specified behavior.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Only examine policies on nodes matching this specifier.

NodeSpec

True

policies

Only consider policies that match this specifier.

RoutingPolicySpec

True

inputConstraints

Constraints on the set of input BGP route announcements to consider.

BgpRouteConstraints

True

action

The behavior to be evaluated. Specify exactly one of permit or deny.

str

True

outputConstraints

Constraints on the set of output BGP route announcements to consider.

BgpRouteConstraints

True

perPath

Run the analysis separately for each execution path of a route map.

bool

True

Invocation

[35]:
result = bf.q.searchRoutePolicies(nodes='/^as1/', policies='/as1_to_/', inputConstraints=BgpRouteConstraints(prefix=["10.0.0.0/8:8-32", "172.16.0.0/28:28-32", "192.168.0.0/16:16-32"]), action='permit').answer().frame()

Return Value

Name

Description

Type

Node

The node that has the policy

str

Policy_Name

The name of this policy

str

Input_Route

The input route

BgpRoute

Action

The action of the policy on the input route

str

Output_Route

The output route, if any

BgpRoute

Difference

The difference between the input and output routes, if any

BgpRouteDiffs

Trace

Route policy trace that shows which clauses/terms matched the input route. If the trace is empty, either nothing matched or tracing is not yet been implemented for this policy type. This is an experimental feature whose content and format is subject to change.

List of TraceTree

Print the first 5 rows of the returned Dataframe

[36]:
result.head(5)
[36]:
Node Policy_Name Input_Route Action Output_Route Difference Trace
0 as1border2 as1_to_as4 BgpRoute(network='10.0.0.0/8', originatorIp='0.0.0.0', originType='egp', protocol='bgp', asPath=[], communities=[], localPreference=100, metric=0, nextHopIp='0.0.0.1', sourceProtocol=None, tag=0, weight=0) PERMIT BgpRoute(network='10.0.0.0/8', originatorIp='0.0.0.0', originType='egp', protocol='bgp', asPath=[], communities=['1:4'], localPreference=100, metric=50, nextHopIp='0.0.0.1', sourceProtocol=None, tag=0, weight=0) BgpRouteDiffs(diffs=[BgpRouteDiff(fieldName='communities', oldValue='[]', newValue='[1:4]'), BgpRouteDiff(fieldName='metric', oldValue='0', newValue='50')]) - Matched route-map as1_to_as4 clause 2

Print the first row of the returned Dataframe

[37]:
result.iloc[0]
[37]:
Node                                                                                                                                                                                                                     as1border2
Policy_Name                                                                                                                                                                                                              as1_to_as4
Input_Route           BgpRoute(network='10.0.0.0/8', originatorIp='0.0.0.0', originType='egp', protocol='bgp', asPath=[], communities=[], localPreference=100, metric=0, nextHopIp='0.0.0.1', sourceProtocol=None, tag=0, weight=0)
Action                                                                                                                                                                                                                       PERMIT
Output_Route    BgpRoute(network='10.0.0.0/8', originatorIp='0.0.0.0', originType='egp', protocol='bgp', asPath=[], communities=['1:4'], localPreference=100, metric=50, nextHopIp='0.0.0.1', sourceProtocol=None, tag=0, weight=0)
Difference                                                             BgpRouteDiffs(diffs=[BgpRouteDiff(fieldName='communities', oldValue='[]', newValue='[1:4]'), BgpRouteDiff(fieldName='metric', oldValue='0', newValue='50')])
Trace                                                                                                                                                                                       - Matched route-map as1_to_as4 clause 2
Name: 0, dtype: object

Routing and Forwarding Tables

This category of questions allows you to query the RIBs and FIBs computed by Batfish.

Routes

Returns routing tables.

Shows routes for specified RIB, VRF, and node(s).

Inputs

Name

Description

Type

Optional

Default Value

nodes

Return routes on nodes matching this specifier.

NodeSpec

True

network

Return routes for networks matching this prefix.

str

True

prefixMatchType

Use this prefix matching criterion: EXACT, LONGEST_PREFIX_MATCH, LONGER_PREFIXES, SHORTER_PREFIXES.

str

True

EXACT

protocols

Return routes for protocols matching this specifier.

RoutingProtocolSpec

True

vrfs

Return routes on VRFs matching this name or regex.

str

True

rib

Only return routes from a given protocol RIB.

str

True

Invocation

[5]:
result = bf.q.routes().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF name

str

Network

Network for this route

str

Next_Hop

Route’s Next Hop

NextHop

Next_Hop_IP

Route’s Next Hop IP

str

Next_Hop_Interface

Route’s Next Hop Interface

str

Protocol

Route’s Protocol

str

Metric

Route’s Metric

int

Admin_Distance

Route’s Admin distance

int

Tag

Tag for this route

int

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
0 as1border1 default 1.0.1.0/24 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 connected 0 0 None
1 as1border1 default 1.0.1.1/32 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 local 0 0 None
2 as1border1 default 1.0.2.0/24 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 2 110 None
3 as1border1 default 1.1.1.1/32 interface Loopback0 AUTO/NONE(-1l) Loopback0 connected 0 0 None
4 as1border1 default 1.2.2.2/32 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 3 110 None

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Node                                    as1border1
VRF                                        default
Network                                 1.0.1.0/24
Next_Hop              interface GigabitEthernet0/0
Next_Hop_IP                         AUTO/NONE(-1l)
Next_Hop_Interface              GigabitEthernet0/0
Protocol                                 connected
Metric                                           0
Admin_Distance                                   0
Tag                                           None
Name: 0, dtype: object

BGP RIB

Returns routes in the BGP RIB.

Shows BGP routes for specified VRF and node(s). This question is not available in Batfish containers on dockerhub prior to March 29, 2021.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Examine routes on nodes matching this specifier.

NodeSpec

True

network

Examine routes for networks matching this prefix.

str

True

prefixMatchType

Use this prefix matching criterion: EXACT, LONGEST_PREFIX_MATCH, LONGER_PREFIXES, SHORTER_PREFIXES.

str

True

EXACT

vrfs

Examine routes on VRFs matching this name or regex.

str

True

status

Examine routes whose status matches this specifier.

BgpRouteStatusSpec

True

Invocation

[10]:
result = bf.q.bgpRib().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF name

str

Network

Network for this route

str

Status

Route’s statuses

List of str

Next_Hop

Route’s Next Hop

NextHop

Next_Hop_IP

Route’s Next Hop IP

str

Next_Hop_Interface

Route’s Next Hop Interface

str

Protocol

Route’s Protocol

str

AS_Path

Route’s AS path

str

Metric

Route’s Metric

int

Local_Pref

Route’s Local Preference

int

Communities

Route’s List of communities

List of str

Origin_Protocol

Route’s Origin protocol

str

Origin_Type

Route’s Origin type

str

Originator_Id

Route’s Originator ID

str

Received_From_IP

IP of the neighbor who sent this route

str

Received_Path_Id

Route’s Received Path ID

int

Cluster_List

Route’s Cluster List

List of int

Tunnel_Encapsulation_Attribute

Route’s BGP Tunnel Encapsulation Attribute

str

Weight

Route’s BGP Weight

int

Tag

Tag for this route

int

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Node VRF Network Status Next_Hop Next_Hop_IP Next_Hop_Interface Protocol AS_Path Metric Local_Pref Communities Origin_Protocol Origin_Type Originator_Id Received_From_IP Received_Path_Id Cluster_List Tunnel_Encapsulation_Attribute Weight Tag
0 as1border1 default 1.0.1.0/24 ['BEST'] discard AUTO/NONE(-1l) null_interface bgp 0 100 [] connected igp 1.1.1.1 0.0.0.0 None None None 32768 None
1 as1border1 default 1.0.2.0/24 ['BEST'] ip 1.0.1.2 1.0.1.2 dynamic bgp 2 100 [] ospf igp 1.1.1.1 0.0.0.0 None None None 32768 None
2 as1border1 default 2.128.0.0/16 ['BEST'] ip 10.12.11.2 10.12.11.2 dynamic bgp 2 50 350 ['2:1'] bgp igp 2.1.1.1 10.12.11.2 None None None 0 None
3 as1border1 default 3.0.1.0/24 ['BEST'] ip 10.13.22.3 10.13.22.3 dynamic ibgp 3 50 350 ['3:1'] ibgp igp 1.2.2.2 1.10.1.1 1 [17432833] None 0 None
4 as1border1 default 3.0.2.0/24 ['BEST'] ip 10.13.22.3 10.13.22.3 dynamic ibgp 3 50 350 ['3:1'] ibgp igp 1.2.2.2 1.10.1.1 1 [17432833] None 0 None

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Node                                  as1border1
VRF                                      default
Network                               1.0.1.0/24
Status                                  ['BEST']
Next_Hop                                 discard
Next_Hop_IP                       AUTO/NONE(-1l)
Next_Hop_Interface                null_interface
Protocol                                     bgp
AS_Path
Metric                                         0
Local_Pref                                   100
Communities                                   []
Origin_Protocol                        connected
Origin_Type                                  igp
Originator_Id                            1.1.1.1
Received_From_IP                         0.0.0.0
Received_Path_Id                            None
Cluster_List                                None
Tunnel_Encapsulation_Attribute              None
Weight                                     32768
Tag                                         None
Name: 0, dtype: object

EVPN RIB

Returns routes in the EVPN RIB.

Shows EVPN routes for specified VRF and node(s). This question is not available in Batfish containers on dockerhub prior to March 29, 2021.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Examine routes on nodes matching this specifier.

NodeSpec

True

network

Examine routes for networks matching this prefix.

str

True

prefixMatchType

Use this prefix matching criterion: EXACT, LONGEST_PREFIX_MATCH, LONGER_PREFIXES, SHORTER_PREFIXES.

str

True

EXACT

vrfs

Examine routes on VRFs matching this name or regex.

str

True

Invocation

[15]:
result = bf.q.evpnRib().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF name

str

Network

Network for this route

str

Status

Route’s statuses

List of str

Route_Distinguisher

Route distinguisher

str

Next_Hop

Route’s Next Hop

NextHop

Next_Hop_IP

Route’s Next Hop IP

str

Next_Hop_Interface

Route’s Next Hop Interface

str

Protocol

Route’s Protocol

str

AS_Path

Route’s AS path

str

Metric

Route’s Metric

int

Local_Pref

Route’s Local Preference

int

Communities

Route’s List of communities

List of str

Origin_Protocol

Route’s Origin protocol

str

Origin_Type

Route’s Origin type

str

Originator_Id

Route’s Originator ID

str

Received_Path_Id

Route’s Received Path ID

int

Cluster_List

Route’s Cluster List

List of int

Tunnel_Encapsulation_Attribute

Route’s BGP Tunnel Encapsulation Attribute

str

Weight

Route’s BGP Weight

int

Tag

Tag for this route

int

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node VRF Network Status Route_Distinguisher Next_Hop Next_Hop_IP Next_Hop_Interface Protocol AS_Path Metric Local_Pref Communities Origin_Protocol Origin_Type Originator_Id Received_Path_Id Cluster_List Tunnel_Encapsulation_Attribute Weight Tag
0 dc1-bl1a default 10.1.10.0/24 ['BEST'] 192.168.255.3:15001 vni 15001 vtep 192.168.254.3 AUTO/NONE(-1l) dynamic bgp 65001 65101 0 100 ['2:15001:15001'] bgp igp 192.168.255.1 None None None 0 None
1 dc1-bl1a default 10.1.10.0/24 ['BEST'] 192.168.255.5:15001 vni 15001 vtep 192.168.254.4 AUTO/NONE(-1l) dynamic bgp 65001 65102 0 100 ['2:15001:15001'] bgp igp 192.168.255.1 None None None 0 None
2 dc1-bl1a default 10.1.10.0/24 ['BEST'] 192.168.255.4:15001 vni 15001 vtep 192.168.254.4 AUTO/NONE(-1l) dynamic bgp 65001 65102 0 100 ['2:15001:15001'] bgp igp 192.168.255.1 None None None 0 None
3 dc1-bl1a default 10.1.11.0/24 ['BEST'] 192.168.255.4:15001 vni 15001 vtep 192.168.254.4 AUTO/NONE(-1l) dynamic bgp 65001 65102 0 100 ['2:15001:15001'] bgp igp 192.168.255.1 None None None 0 None
4 dc1-bl1a default 10.1.11.0/24 ['BEST'] 192.168.255.5:15001 vni 15001 vtep 192.168.254.4 AUTO/NONE(-1l) dynamic bgp 65001 65102 0 100 ['2:15001:15001'] bgp igp 192.168.255.1 None None None 0 None

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node                                                  dc1-bl1a
VRF                                                    default
Network                                           10.1.10.0/24
Status                                                ['BEST']
Route_Distinguisher                        192.168.255.3:15001
Next_Hop                          vni 15001 vtep 192.168.254.3
Next_Hop_IP                                     AUTO/NONE(-1l)
Next_Hop_Interface                                     dynamic
Protocol                                                   bgp
AS_Path                                            65001 65101
Metric                                                       0
Local_Pref                                                 100
Communities                                  ['2:15001:15001']
Origin_Protocol                                            bgp
Origin_Type                                                igp
Originator_Id                                    192.168.255.1
Received_Path_Id                                          None
Cluster_List                                              None
Tunnel_Encapsulation_Attribute                            None
Weight                                                       0
Tag                                                       None
Name: 0, dtype: object

Longest Prefix Match

Returns routes that are longest prefix match for a given IP address.

Return longest prefix match routes for a given IP in the RIBs of specified nodes and VRFs.

Inputs

Name

Description

Type

Optional

Default Value

ip

IP address to run LPM on.

str

False

nodes

Examine routes on nodes matching this specifier.

NodeSpec

True

.*

vrfs

Examine routes on VRFs matching this name or regex.

str

True

.*

Invocation

[20]:
result = bf.q.lpmRoutes(ip='2.34.201.10').answer().frame()

Return Value

Name

Description

Type

Node

Node where the route is present

str

VRF

VRF where the route is present

str

Ip

IP that was being matched on

str

Network

The longest-prefix network that matched

str

Num_Routes

Number of routes that matched (in case of ECMP)

int

Print the first 5 rows of the returned Dataframe

[21]:
result.head(5)
[21]:
Node VRF Ip Network Num_Routes
0 as2border1 default 2.34.201.10 2.34.201.0/24 2
1 as2border2 default 2.34.201.10 2.34.201.0/24 2
2 as2core1 default 2.34.201.10 2.34.201.0/24 1
3 as2core2 default 2.34.201.10 2.34.201.0/24 1
4 as2dept1 default 2.34.201.10 2.34.201.0/24 1

Print the first row of the returned Dataframe

[22]:
result.iloc[0]
[22]:
Node             as2border1
VRF                 default
Ip              2.34.201.10
Network       2.34.201.0/24
Num_Routes                2
Name: 0, dtype: object

Packet Forwarding

This category of questions allows you to query how different types of traffic is forwarded by the network and if endpoints are able to communicate. You can analyze these aspects in a few different ways.

Traceroute

Traces the path(s) for the specified flow.

Performs a virtual traceroute in the network from a starting node. A destination IP and ingress (source) node must be specified. Other IP headers are given default values if unspecified. Unlike a real traceroute, this traceroute is directional. That is, for it to succeed, the reverse connectivity is not needed. This feature can help debug connectivity issues by decoupling the two directions.

Inputs

Name

Description

Type

Optional

Default Value

startLocation

Location (node and interface combination) to start tracing from.

LocationSpec

False

headers

Packet header constraints.

HeaderConstraints

False

maxTraces

Limit the number of traces returned.

int

True

ignoreFilters

If set, filters/ACLs encountered along the path are ignored.

bool

True

Invocation

[5]:
result = bf.q.traceroute(startLocation='@enter(as2border1[GigabitEthernet2/0])', headers=HeaderConstraints(dstIps='2.34.201.10', srcIps='8.8.8.8')).answer().frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Traces

The traces for this flow

Set of Trace

TraceCount

The total number traces for this flow

int

Retrieving the flow definition

[6]:
result.Flow
[6]:
0    start=as2border1 interface=GigabitEthernet2/0 [8.8.8.8:49152->2.34.201.10:33434 UDP]
Name: Flow, dtype: object

Retrieving the detailed Trace information

[7]:
len(result.Traces)
[7]:
1
[8]:
result.Traces[0]
[8]:
DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet3/0 ip 2.23.12.3)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0 ip 2.12.12.2)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2core2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0 ip 2.23.22.3)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Evaluating the first Trace

[9]:
result.Traces[0][0]
[9]:
DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet3/0 ip 2.23.12.3)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Retrieving the disposition of the first Trace

[10]:
result.Traces[0][0].disposition
[10]:
'DELIVERED_TO_SUBNET'

Retrieving the first hop of the first Trace

[11]:
result.Traces[0][0][0]
[11]:
node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)

Retrieving the last hop of the first Trace

[12]:
result.Traces[0][0][-1]
[12]:
node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Bi-directional Traceroute

Traces the path(s) for the specified flow, along with path(s) for reverse flows.

This question performs a virtual traceroute in the network from a starting node. A destination IP and ingress (source) node must be specified. Other IP headers are given default values if unspecified. If the trace succeeds, a traceroute is performed in the reverse direction.

Inputs

Name

Description

Type

Optional

Default Value

startLocation

Location (node and interface combination) to start tracing from.

LocationSpec

False

headers

Packet header constraints.

HeaderConstraints

False

maxTraces

Limit the number of traces returned.

int

True

ignoreFilters

If set, filters/ACLs encountered along the path are ignored.

bool

True

Invocation

[15]:
result = bf.q.bidirectionalTraceroute(startLocation='@enter(as2border1[GigabitEthernet2/0])', headers=HeaderConstraints(dstIps='2.34.201.10', srcIps='8.8.8.8')).answer().frame()

Return Value

Name

Description

Type

Forward_Flow

The forward flow.

Flow

Forward_Traces

The forward traces.

List of Trace

New_Sessions

Sessions initialized by the forward trace.

List of str

Reverse_Flow

The reverse flow.

Flow

Reverse_Traces

The reverse traces.

List of Trace

Retrieving the Forward flow definition

[16]:
result.Forward_Flow
[16]:
0    start=as2border1 interface=GigabitEthernet2/0 [8.8.8.8:49152->2.34.201.10:33434 UDP]
Name: Forward_Flow, dtype: object

Retrieving the detailed Forward Trace information

[17]:
len(result.Forward_Traces)
[17]:
1
[18]:
result.Forward_Traces[0]
[18]:
DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet3/0 ip 2.23.12.3)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0 ip 2.12.12.2)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2core2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0 ip 2.23.22.3)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Evaluating the first Forward Trace

[19]:
result.Forward_Traces[0][0]
[19]:
DELIVERED_TO_SUBNET
1. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet3/0 ip 2.23.12.3)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Retrieving the disposition of the first Forward Trace

[20]:
result.Forward_Traces[0][0].disposition
[20]:
'DELIVERED_TO_SUBNET'

Retrieving the first hop of the first Forward Trace

[21]:
result.Forward_Traces[0][0][0]
[21]:
node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospfE2 (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)

Retrieving the last hop of the first Forward Trace

[22]:
result.Forward_Traces[0][0][-1]
[22]:
node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.34.201.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.34.201.10)

Retrieving the Return flow definition

[23]:
result.Reverse_Flow
[23]:
0    start=as2dist2 interface=GigabitEthernet2/0 [2.34.201.10:33434->8.8.8.8:49152 UDP]
Name: Reverse_Flow, dtype: object

Retrieving the detailed Return Trace information

[24]:
len(result.Reverse_Traces)
[24]:
1
[25]:
result.Reverse_Traces[0]
[25]:
NO_ROUTE
1. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  NO_ROUTE(Discarded)

Evaluating the first Reverse Trace

[26]:
result.Reverse_Traces[0][0]
[26]:
NO_ROUTE
1. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  NO_ROUTE(Discarded)

Retrieving the disposition of the first Reverse Trace

[27]:
result.Reverse_Traces[0][0].disposition
[27]:
'NO_ROUTE'

Retrieving the first hop of the first Reverse Trace

[28]:
result.Reverse_Traces[0][0][0]
[28]:
node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  NO_ROUTE(Discarded)

Retrieving the last hop of the first Reverse Trace

[29]:
result.Reverse_Traces[0][0][-1]
[29]:
node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  NO_ROUTE(Discarded)

Reachability

Finds flows that match the specified path and header space conditions.

Searches across all flows that match the specified conditions and returns examples of such flows. This question can be used to ensure that certain services are globally accessible and parts of the network are perfectly isolated from each other.

Inputs

Name

Description

Type

Optional

Default Value

pathConstraints

Constraint the path a flow can take (start/end/transit locations).

PathConstraints

True

headers

Packet header constraints.

HeaderConstraints

True

actions

Only return flows for which the disposition is from this set.

DispositionSpec

True

success

maxTraces

Limit the number of traces returned.

int

True

invertSearch

Search for packet headers outside the specified headerspace, rather than inside the space.

bool

True

ignoreFilters

Do not apply filters/ACLs during analysis.

bool

True

Invocation

[32]:
result = bf.q.reachability(pathConstraints=PathConstraints(startLocation = '/as2/'), headers=HeaderConstraints(dstIps='host1', srcIps='0.0.0.0/0', applications='DNS'), actions='SUCCESS').answer().frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Traces

The traces for this flow

Set of Trace

TraceCount

The total number traces for this flow

int

Retrieving the flow definition

[33]:
result.Flow
[33]:
0    start=as2border1 [10.0.0.0:49152->2.128.0.101:53 UDP]
1    start=as2border2 [10.0.0.0:49152->2.128.0.101:53 UDP]
2      start=as2core1 [10.0.0.0:49152->2.128.0.101:53 UDP]
3      start=as2core2 [10.0.0.0:49152->2.128.0.101:53 UDP]
4      start=as2dept1 [10.0.0.0:49152->2.128.0.101:53 UDP]
5      start=as2dist1 [10.0.0.0:49152->2.128.0.101:53 UDP]
6      start=as2dist2 [10.0.0.0:49152->2.128.0.101:53 UDP]
Name: Flow, dtype: object

Retrieving the detailed Trace information

[34]:
len(result.Traces)
[34]:
7
[35]:
result.Traces[0]
[35]:
ACCEPTED
1. node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
4. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
5. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

ACCEPTED
1. node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet2/0)
4. node: as2dept1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
5. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

ACCEPTED
1. node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2core2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet2/0)
4. node: as2dept1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
5. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

ACCEPTED
1. node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2core2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet3/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
4. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
5. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Evaluating the first Trace

[36]:
result.Traces[0][0]
[36]:
ACCEPTED
1. node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
4. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
5. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Retrieving the disposition of the first Trace

[37]:
result.Traces[0][0].disposition
[37]:
'ACCEPTED'

Retrieving the first hop of the first Trace

[38]:
result.Traces[0][0][0]
[38]:
node: as2border1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
  TRANSMITTED(GigabitEthernet1/0)

Retrieving the last hop of the first Trace

[39]:
result.Traces[0][0][-1]
[39]:
node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Bi-directional Reachability

Searches for successfully delivered flows that can successfully receive a response.

Performs two reachability analyses, first originating from specified sources, then returning back to those sources. After the first (forward) pass, sets up sessions in the network and creates returning flows for each successfully delivered forward flow. The second pass searches for return flows that can be successfully delivered in the presence of the setup sessions.

Inputs

Name

Description

Type

Optional

Default Value

pathConstraints

Constraint the path a flow can take (start/end/transit locations).

PathConstraints

True

headers

Packet header constraints.

HeaderConstraints

False

returnFlowType

Specifies the type of return flows to search.

str

True

SUCCESS

Invocation

[42]:
result = bf.q.bidirectionalReachability(pathConstraints=PathConstraints(startLocation = '/as2dist1/'), headers=HeaderConstraints(dstIps='host1', srcIps='0.0.0.0/0', applications='DNS'), returnFlowType='SUCCESS').answer().frame()

Return Value

Name

Description

Type

Forward_Flow

The forward flow.

Flow

Forward_Traces

The forward traces.

List of Trace

New_Sessions

Sessions initialized by the forward trace.

List of str

Reverse_Flow

The reverse flow.

Flow

Reverse_Traces

The reverse traces.

List of Trace

Retrieving the Forward flow definition

[43]:
result.Forward_Flow
[43]:
0    start=as2dist1 [2.34.101.3:49152->2.128.0.101:53 UDP]
Name: Forward_Flow, dtype: object

Retrieving the detailed Forward Trace information

[44]:
len(result.Forward_Traces)
[44]:
1
[45]:
result.Forward_Traces[0]
[45]:
ACCEPTED
1. node: as2dist1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Evaluating the first Forward Trace

[46]:
result.Forward_Traces[0][0]
[46]:
ACCEPTED
1. node: as2dist1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Retrieving the disposition of the first Forward Trace

[47]:
result.Forward_Traces[0][0].disposition
[47]:
'ACCEPTED'

Retrieving the first hop of the first Forward Trace

[48]:
result.Forward_Traces[0][0][0]
[48]:
node: as2dist1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
  TRANSMITTED(GigabitEthernet2/0)

Retrieving the last hop of the first Forward Trace

[49]:
result.Forward_Traces[0][0][-1]
[49]:
node: host1
  RECEIVED(eth0)
  PERMITTED(filter::INPUT (INGRESS_FILTER))
  ACCEPTED(eth0)

Retrieving the Return flow definition

[50]:
result.Reverse_Flow
[50]:
0    start=host1 [2.128.0.101:53->2.34.101.3:49152 UDP]
Name: Reverse_Flow, dtype: object

Retrieving the detailed Return Trace information

[51]:
len(result.Reverse_Traces)
[51]:
1
[52]:
result.Reverse_Traces[0]
[52]:
ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0, Routes: [connected (Network: 2.34.101.0/24, Next Hop: interface GigabitEthernet0/0)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  ACCEPTED(GigabitEthernet2/0)

Evaluating the first Reverse Trace

[53]:
result.Reverse_Traces[0][0]
[53]:
ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0, Routes: [connected (Network: 2.34.101.0/24, Next Hop: interface GigabitEthernet0/0)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  ACCEPTED(GigabitEthernet2/0)

Retrieving the disposition of the first Reverse Trace

[54]:
result.Reverse_Traces[0][0].disposition
[54]:
'ACCEPTED'

Retrieving the first hop of the first Reverse Trace

[55]:
result.Reverse_Traces[0][0][0]
[55]:
node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)

Retrieving the last hop of the first Reverse Trace

[56]:
result.Reverse_Traces[0][0][-1]
[56]:
node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  ACCEPTED(GigabitEthernet2/0)

Loop detection

Detects forwarding loops.

Searches across all possible flows in the network and returns example flows that will experience forwarding loops.

Inputs

Name

Description

Type

Optional

Default Value

maxTraces

Limit the number of traces returned.

int

True

Invocation

[59]:
result = bf.q.detectLoops().answer().frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Traces

The traces for this flow

Set of Trace

TraceCount

The total number traces for this flow

int

Print the first 5 rows of the returned Dataframe

[60]:
result.head(5)
[60]:
Flow Traces TraceCount

Multipath Consistency for host-subnets

Validates multipath consistency between all pairs of subnets.

Searches across all flows between subnets that are treated differently (i.e., dropped versus forwarded) by different paths in the network and returns example flows.

Inputs

Name

Description

Type

Optional

Default Value

maxTraces

Limit the number of traces returned.

int

True

Invocation

[63]:
result = bf.q.subnetMultipathConsistency().answer().frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Traces

The traces for this flow

Set of Trace

TraceCount

The total number traces for this flow

int

Retrieving the flow definition

[64]:
result.Flow
[64]:
0    start=as2dept1 interface=GigabitEthernet0/0 [2.34.101.1:49152->1.0.1.3:23 TCP (SYN)]
1    start=as2dept1 interface=GigabitEthernet1/0 [2.34.201.1:49152->1.0.1.3:23 TCP (SYN)]
2     start=as2dept1 interface=GigabitEthernet2/0 [2.128.0.2:49152->1.0.1.3:23 TCP (SYN)]
3     start=as2dept1 interface=GigabitEthernet3/0 [2.128.1.2:49152->1.0.1.3:23 TCP (SYN)]
4     start=as2dist1 interface=GigabitEthernet0/0 [2.23.11.1:49152->1.0.1.3:23 TCP (SYN)]
5     start=as2dist1 interface=GigabitEthernet1/0 [2.23.21.1:49152->1.0.1.3:23 TCP (SYN)]
6    start=as2dist1 interface=GigabitEthernet2/0 [2.34.101.1:49152->1.0.1.3:23 TCP (SYN)]
7     start=as2dist2 interface=GigabitEthernet0/0 [2.23.22.1:49152->1.0.1.3:23 TCP (SYN)]
8     start=as2dist2 interface=GigabitEthernet1/0 [2.23.12.1:49152->1.0.1.3:23 TCP (SYN)]
9    start=as2dist2 interface=GigabitEthernet2/0 [2.34.201.1:49152->1.0.1.3:23 TCP (SYN)]
Name: Flow, dtype: object

Retrieving the detailed Trace information

[65]:
len(result.Traces)
[65]:
10
[66]:
result.Traces[0]
[66]:
DENIED_IN
1. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)
2. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.11.2, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2core1
  RECEIVED(GigabitEthernet2/0)
  DENIED(blocktelnet (INGRESS_FILTER))

DELIVERED_TO_SUBNET
1. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)
2. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.21.2, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2core2
  RECEIVED(GigabitEthernet3/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
4. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
5. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0, Routes: [connected (Network: 1.0.1.0/24, Next Hop: interface GigabitEthernet0/0)])
  TRANSMITTED(GigabitEthernet0/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet0/0, Resolved Next Hop IP: 1.0.1.3)

DELIVERED_TO_SUBNET
1. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.34.201.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.201.3)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.22.2, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2core2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
4. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
5. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0, Routes: [connected (Network: 1.0.1.0/24, Next Hop: interface GigabitEthernet0/0)])
  TRANSMITTED(GigabitEthernet0/0)
  DELIVERED_TO_SUBNET(Output Interface: GigabitEthernet0/0, Resolved Next Hop IP: 1.0.1.3)

DENIED_IN
1. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.34.201.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.201.3)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.12.2, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2core1
  RECEIVED(GigabitEthernet3/0)
  DENIED(blocktelnet (INGRESS_FILTER))

Evaluating the first Trace

[67]:
result.Traces[0][0]
[67]:
DENIED_IN
1. node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)
2. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.11.2, Routes: [ibgp (Network: 1.0.1.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2core1
  RECEIVED(GigabitEthernet2/0)
  DENIED(blocktelnet (INGRESS_FILTER))

Retrieving the disposition of the first Trace

[68]:
result.Traces[0][0].disposition
[68]:
'DENIED_IN'

Retrieving the first hop of the first Trace

[69]:
result.Traces[0][0][0]
[69]:
node: as2dept1
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.1.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)

Retrieving the last hop of the first Trace

[70]:
result.Traces[0][0][-1]
[70]:
node: as2core1
  RECEIVED(GigabitEthernet2/0)
  DENIED(blocktelnet (INGRESS_FILTER))

Multipath Consistency for router loopbacks

Validates multipath consistency between all pairs of loopbacks.

Finds flows between loopbacks that are treated differently (i.e., dropped versus forwarded) by different paths in the presence of multipath routing.

Inputs

Name

Description

Type

Optional

Default Value

maxTraces

Limit the number of traces returned.

int

True

Invocation

[73]:
result = bf.q.loopbackMultipathConsistency().answer().frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Traces

The traces for this flow

Set of Trace

TraceCount

The total number traces for this flow

int

Retrieving the flow definition

[74]:
result.Flow
[74]:
0    start=as2core2 [2.1.2.2:49152->2.1.2.1:23 TCP (SYN)]
1    start=as2dist1 [2.1.3.1:49152->2.1.1.1:23 TCP (SYN)]
2    start=as2dist2 [2.1.3.2:49152->2.1.1.1:23 TCP (SYN)]
Name: Flow, dtype: object

Retrieving the detailed Trace information

[75]:
len(result.Traces)
[75]:
3
[76]:
result.Traces[0]
[76]:
ACCEPTED
1. node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.22.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.12.22.1)])
  TRANSMITTED(GigabitEthernet0/0)
2. node: as2border2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet2/0 ip 2.12.21.2)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(Loopback0)

ACCEPTED
1. node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.12.12.1)])
  TRANSMITTED(GigabitEthernet1/0)
2. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2core1
  RECEIVED(GigabitEthernet0/0)
  ACCEPTED(Loopback0)

DENIED_IN
1. node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet2/0 ip 2.23.22.3)])
  TRANSMITTED(GigabitEthernet2/0)
2. node: as2dist2
  RECEIVED(GigabitEthernet0/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.12.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.23.12.2)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2core1
  RECEIVED(GigabitEthernet3/0)
  DENIED(blocktelnet (INGRESS_FILTER))

DENIED_IN
1. node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet3/0 ip 2.23.21.3)])
  TRANSMITTED(GigabitEthernet3/0)
2. node: as2dist1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.11.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.23.11.2)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2core1
  RECEIVED(GigabitEthernet2/0)
  DENIED(blocktelnet (INGRESS_FILTER))

Evaluating the first Trace

[77]:
result.Traces[0][0]
[77]:
ACCEPTED
1. node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.22.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.12.22.1)])
  TRANSMITTED(GigabitEthernet0/0)
2. node: as2border2
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet2/0 ip 2.12.21.2)])
  TRANSMITTED(GigabitEthernet2/0)
3. node: as2core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(Loopback0)

Retrieving the disposition of the first Trace

[78]:
result.Traces[0][0].disposition
[78]:
'ACCEPTED'

Retrieving the first hop of the first Trace

[79]:
result.Traces[0][0][0]
[79]:
node: as2core2
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.22.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.12.22.1)])
  TRANSMITTED(GigabitEthernet0/0)

Retrieving the last hop of the first Trace

[80]:
result.Traces[0][0][-1]
[80]:
node: as2core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(Loopback0)

Access-lists and firewall rules

This category of questions allows you to analyze the behavior of access control lists and firewall rules. It also allows you to comprehensively validate (aka verification) that some traffic is or is not allowed.

Filter Line Reachability

Returns unreachable lines in filters (ACLs and firewall rules).

Finds all lines in the specified filters that will not match any packet, either because of being shadowed by prior lines or because of its match condition being empty.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Examine filters on nodes matching this specifier.

NodeSpec

True

filters

Specifier for filters to test.

FilterSpec

True

ignoreComposites

Whether to ignore filters that are composed of multiple filters defined in the configs.

bool

True

False

Invocation

[5]:
result = bf.q.filterLineReachability().answer().frame()

Return Value

Name

Description

Type

Sources

Filter sources

List of str

Unreachable_Line

Filter line that cannot be matched (i.e., unreachable)

str

Unreachable_Line_Action

Action performed by the unreachable line (e.g., PERMIT or DENY)

str

Blocking_Lines

Lines that, when combined, cover the unreachable line

List of str

Different_Action

Whether unreachable line has an action different from the blocking line(s)

bool

Reason

The reason a line is unreachable

str

Additional_Info

Additional information

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Sources Unreachable_Line Unreachable_Line_Action Blocking_Lines Different_Action Reason Additional_Info
0 ['as2dept1: RESTRICT_HOST_TRAFFIC_OUT'] deny ip 1.128.0.0 0.0.255.255 2.128.0.0 0.0.255.255 DENY ['permit ip any 2.128.0.0 0.0.255.255'] True BLOCKING_LINES None
1 ['as2dept1: RESTRICT_HOST_TRAFFIC_IN'] permit icmp any any PERMIT ['deny ip any any'] True BLOCKING_LINES None

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Sources                                  ['as2dept1: RESTRICT_HOST_TRAFFIC_OUT']
Unreachable_Line           deny   ip 1.128.0.0 0.0.255.255 2.128.0.0 0.0.255.255
Unreachable_Line_Action                                                     DENY
Blocking_Lines                           ['permit ip any 2.128.0.0 0.0.255.255']
Different_Action                                                            True
Reason                                                            BLOCKING_LINES
Additional_Info                                                             None
Name: 0, dtype: object

Search Filters

Finds flows for which a filter takes a particular behavior.

This question searches for flows for which a filter (access control list) has a particular behavior. The behaviors can be: that the filter permits the flow (permit), that it denies the flow (deny), or that the flow is matched by a particular line (matchLine <lineNumber>). Filters are selected using node and filter specifiers, which might match multiple filters. In this case, a (possibly different) flow will be found for each filter.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Only evaluate filters present on nodes matching this specifier.

NodeSpec

True

filters

Only evaluate filters that match this specifier.

FilterSpec

True

headers

Packet header constraints on the flows being searched.

HeaderConstraints

True

action

The behavior that you want evaluated. Specify exactly one of permit, deny, or matchLine <line number>.

str

True

startLocation

Only consider specified locations as possible sources.

LocationSpec

True

invertSearch

Search for packet headers outside the specified headerspace, rather than inside the space.

bool

True

Invocation

[10]:
result = bf.q.searchFilters(headers=HeaderConstraints(srcIps='10.10.10.0/24', dstIps='218.8.104.58', applications = ['dns']), action='deny', filters='acl_in').answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Filter_Name

Filter name

str

Flow

Evaluated flow

Flow

Action

Outcome

str

Line_Content

Line content

str

Trace

ACL trace

List of TraceTree

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Node Filter_Name Flow Action Line_Content Trace
0 rtr-with-acl acl_in start=rtr-with-acl [10.10.10.42:49152->218.8.104.58:53 UDP] DENY 460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain - Matched line 460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Node                                                                    rtr-with-acl
Filter_Name                                                                   acl_in
Flow                     start=rtr-with-acl [10.10.10.42:49152->218.8.104.58:53 UDP]
Action                                                                          DENY
Line_Content                   460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
Trace           - Matched line 460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
Name: 0, dtype: object

Test Filters

Returns how a flow is processed by a filter (ACLs, firewall rules).

Shows how the specified flow is processed through the specified filters, returning its permit/deny status as well as the line(s) it matched.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Only examine filters on nodes matching this specifier.

NodeSpec

True

filters

Only consider filters that match this specifier.

FilterSpec

True

headers

Packet header constraints.

HeaderConstraints

False

startLocation

Location to start tracing from.

LocationSpec

True

Invocation

[15]:
result = bf.q.testFilters(headers=HeaderConstraints(srcIps='10.10.10.1', dstIps='218.8.104.58', applications = ['dns']), nodes='rtr-with-acl', filters='acl_in').answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Filter_Name

Filter name

str

Flow

Evaluated flow

Flow

Action

Outcome

str

Line_Content

Line content

str

Trace

ACL trace

List of TraceTree

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node Filter_Name Flow Action Line_Content Trace
0 rtr-with-acl acl_in start=rtr-with-acl [10.10.10.1:49152->218.8.104.58:53 UDP] PERMIT 660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain - Matched line 660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node                                                                     rtr-with-acl
Filter_Name                                                                    acl_in
Flow                       start=rtr-with-acl [10.10.10.1:49152->218.8.104.58:53 UDP]
Action                                                                         PERMIT
Line_Content                   660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
Trace           - Matched line 660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
Name: 0, dtype: object

Find Matching Filter Lines

Returns lines in filters (ACLs and firewall rules) that match any packet within the specified header constraints.

Finds all lines in the specified filters that match any packet within the specified header constraints.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Examine filters on nodes matching this specifier.

NodeSpec

True

filters

Specifier for filters to check.

FilterSpec

True

headers

Packet header constraints for which to find matching filter lines.

HeaderConstraints

True

action

Show filter lines with this action. By default returns lines with either action.

str

True

ignoreComposites

Whether to ignore filters that are composed of multiple filters defined in the configs.

bool

True

False

Invocation

[20]:
result = bf.q.findMatchingFilterLines(headers=HeaderConstraints(applications='DNS')).answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Filter

Filter name

str

Line

Line text

str

Line_Index

Index of line

int

Action

Action performed by the line (e.g., PERMIT or DENY)

str

Print the first 5 rows of the returned Dataframe

[21]:
result.head(5)
[21]:
Node Filter Line Line_Index Action
0 as1border1 101 permit ip host 1.0.1.0 host 255.255.255.0 0 PERMIT
1 as1border1 101 permit ip host 1.0.2.0 host 255.255.255.0 1 PERMIT
2 as1border1 102 permit ip host 2.0.0.0 host 255.0.0.0 0 PERMIT
3 as1border1 102 permit ip host 2.128.0.0 host 255.255.0.0 1 PERMIT
4 as1border1 103 permit ip host 3.0.1.0 host 255.255.255.0 0 PERMIT

Print the first row of the returned Dataframe

[22]:
result.iloc[0]
[22]:
Node                                         as1border1
Filter                                              101
Line          permit ip host 1.0.1.0 host 255.255.255.0
Line_Index                                            0
Action                                           PERMIT
Name: 0, dtype: object

Snapshot Input

This category of questions allows you to learn how well Batfish understands your network snapshot.

Snapshot Initialization Issues

Returns issues encountered when processing the snapshot.

Reports issues encountered by Batfish, including failure to recognize certain lines in the configuration, lack of support for certain features, and errors when converting to vendor-independent models.

Inputs

Invocation

[5]:
result = bf.q.initIssues().answer().frame()

Return Value

Name

Description

Type

Nodes

The nodes that were converted (if applicable)

List of str

Source_Lines

The files and lines that caused the issues (if applicable)

List of FileLines

Type

The type of issues identified

str

Details

Details about the issues identified

str

Line_Text

The text of the input files that caused the issues (if applicable)

str

Parser_Context

Batfish parser state when issues were encountered (if applicable)

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Nodes Source_Lines Type Details Line_Text Parser_Context
0 ['as1border1'] None Convert warning (redflag) No virtual address set for VRRP on interface: 'GigabitEthernet0/0' None None

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Nodes                                                                 ['as1border1']
Source_Lines                                                                    None
Type                                                       Convert warning (redflag)
Details           No virtual address set for VRRP on interface: 'GigabitEthernet0/0'
Line_Text                                                                       None
Parser_Context                                                                  None
Name: 0, dtype: object

Snapshot Input File Parse Status

Displays file parse status.

For each file in a snapshot, returns the host(s) that were produced by the file and the parse status: pass, fail, partially parsed.

Inputs

Invocation

[10]:
result = bf.q.fileParseStatus().answer().frame()

Return Value

Name

Description

Type

File_Name

The file that was parsed

str

Status

The status of the parsing operation

str

File_Format

The detected format of the file

str

Nodes

Names of nodes produced from this file

List of str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
File_Name Status File_Format Nodes
0 configs/as1border1.cfg PASSED CISCO_IOS ['as1border1']
1 configs/as1border2.cfg PASSED CISCO_IOS ['as1border2']
2 configs/as1core1.cfg PASSED CISCO_IOS ['as1core1']
3 configs/as2border1.cfg PASSED CISCO_IOS ['as2border1']
4 configs/as2border2.cfg PASSED CISCO_IOS ['as2border2']

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
File_Name      configs/as1border1.cfg
Status                         PASSED
File_Format                 CISCO_IOS
Nodes                  ['as1border1']
Name: 0, dtype: object

Snapshot Input File Parse Warnings

Returns warnings that occurred when parsing the snapshot.

Return warnings such as failure to recognize certain lines and lack of support for certain features.

Inputs

Name

Description

Type

Optional

Default Value

aggregateDuplicates

Whether to aggregate duplicate results.

bool

True

Invocation

[15]:
result = bf.q.parseWarning().answer().frame()

Return Value

Name

Description

Type

Filename

The file that was parsed

str

Line

The line number in the input file that caused the warning

int

Text

The text of the input that caused the warning

str

Parser_Context

The context of the Batfish parser when the warning occurred

str

Comment

An optional comment explaining more information about the warning

str

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Filename Line Text Parser_Context Comment

IPSec Tunnels

This category of questions allows you to query IPSec sessions and tunnels.

IPSec Session Status

Returns the status of configured IPSec sessions.

Shows configuration settings and status for each configured IPSec tunnel in the network. The status is IPSEC_SESSION_ESTABLISHED for tunnels that are expected to be established; it is IKE_PHASE1_FAILED if IKE parameters negotiation failed; it is IKE_PHASE1_KEY_MISMATCH if IKE negotiation was successful but IKE keys do not match; it is IPSEC_PHASE2_FAILED if negotiation of IPsec parameters failed; and it is MISSING_END_POINT if the remote endpoint for a configured IPsec tunnel could not be found in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include sessions whose first node matches this specifier.

NodeSpec

True

remoteNodes

Include sessions whose second node matches this specifier.

NodeSpec

True

status

Only include IPSec sessions for which status matches this specifier.

IpsecSessionStatusSpec

True

Invocation

[5]:
result = bf.q.ipsecSessionStatus().answer().frame()

Return Value

Name

Description

Type

Node

IPSec initiator

str

Node_Interface

Initiator Interface

Interface

Node_IP

Initiator IP

str

Remote_Node

IPSec responder

str

Remote_Node_Interface

Responder Interface

Interface

Remote_Node_IP

Responder IP

str

Tunnel_Interfaces

Tunnel interfaces pair used in peering session

str

Status

IPSec session status

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node Node_Interface Node_IP Remote_Node Remote_Node_Interface Remote_Node_IP Tunnel_Interfaces Status
0 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-06b348adabd13452d tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1] 3.19.24.131 Tunnel1 -> vpn-vpn-01c45673532d3e33e-1 IPSEC_SESSION_ESTABLISHED
1 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-06b348adabd13452d tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-2] 52.14.53.162 Tunnel2 -> vpn-vpn-01c45673532d3e33e-2 IPSEC_SESSION_ESTABLISHED
2 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-0888a76c8a371246d tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-1] 34.209.88.227 Tunnel3 -> vpn-vpn-0dc7abdb974ff8a69-1 IPSEC_SESSION_ESTABLISHED
3 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-0888a76c8a371246d tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-2] 44.227.244.7 Tunnel4 -> vpn-vpn-0dc7abdb974ff8a69-2 IPSEC_SESSION_ESTABLISHED
4 tgw-06b348adabd13452d tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1] 3.19.24.131 exitgw exitgw[GigabitEthernet3] 147.75.69.27 vpn-vpn-01c45673532d3e33e-1 -> Tunnel1 IPSEC_SESSION_ESTABLISHED

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Node                                                                      exitgw
Node_Interface                                          exitgw[GigabitEthernet3]
Node_IP                                                             147.75.69.27
Remote_Node                                                tgw-06b348adabd13452d
Remote_Node_Interface    tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1]
Remote_Node_IP                                                       3.19.24.131
Tunnel_Interfaces                         Tunnel1 -> vpn-vpn-01c45673532d3e33e-1
Status                                                 IPSEC_SESSION_ESTABLISHED
Name: 0, dtype: object

IPSec Edges

Returns IPSec tunnels.

Lists all IPSec tunnels in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include tunnels whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include tunnels whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[10]:
result = bf.q.ipsecEdges().answer().frame()

Return Value

Name

Description

Type

Source_Interface

Source interface used in the IPsec session

Interface

Tunnel_Interface

Tunnel interface (if any) used in the IPsec session

Interface

Remote_Source_Interface

Remote source interface used in the IPsec session

Interface

Remote_Tunnel_Interface

Remote tunnel interface (if any) used in the IPsec session

Interface

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Source_Interface Tunnel_Interface Remote_Source_Interface Remote_Tunnel_Interface
0 tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1] tgw-06b348adabd13452d[vpn-vpn-01c45673532d3e33e-1] exitgw[GigabitEthernet3] exitgw[Tunnel1]
1 tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-1] tgw-0888a76c8a371246d[vpn-vpn-0dc7abdb974ff8a69-1] exitgw[GigabitEthernet3] exitgw[Tunnel3]
2 tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-2] tgw-06b348adabd13452d[vpn-vpn-01c45673532d3e33e-2] exitgw[GigabitEthernet3] exitgw[Tunnel2]
3 exitgw[GigabitEthernet3] exitgw[Tunnel4] tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-2] tgw-0888a76c8a371246d[vpn-vpn-0dc7abdb974ff8a69-2]
4 tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-2] tgw-0888a76c8a371246d[vpn-vpn-0dc7abdb974ff8a69-2] exitgw[GigabitEthernet3] exitgw[Tunnel4]

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Source_Interface           tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1]
Tunnel_Interface                tgw-06b348adabd13452d[vpn-vpn-01c45673532d3e33e-1]
Remote_Source_Interface                                   exitgw[GigabitEthernet3]
Remote_Tunnel_Interface                                            exitgw[Tunnel1]
Name: 0, dtype: object

VXLAN and EVPN

This category of questions allows you to query aspects of VXLAN and EVPN configuration and behavior.

VXLAN VNI Properties

Returns configuration settings of VXLANs.

Lists VNI-level network segment settings configured for VXLANs.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

properties

Include properties matching this specifier.

VxlanVniPropertySpec

True

Invocation

[5]:
result = bf.q.vxlanVniProperties().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF

str

VNI

VXLAN Segment ID

int

Local_VTEP_IP

IPv4 address of the local VTEP

str

Multicast_Group

IPv4 address of the multicast group

str

VLAN

VLAN number for the VNI

int

VTEP_Flood_List

All IPv4 addresses in the VTEP flood list

List of str

VXLAN_Port

Destination port number for the VXLAN tunnel

int

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node VRF VNI Local_VTEP_IP Multicast_Group VLAN VTEP_Flood_List VXLAN_Port
0 dc1-svc3a default 10140 192.168.254.6 None 140 ['192.168.254.3', '192.168.254.4', '192.168.254.8'] 4789
1 dc1-svc3b default 10140 192.168.254.6 None 140 ['192.168.254.3', '192.168.254.4', '192.168.254.8'] 4789
2 dc1-leaf2a default 10130 192.168.254.4 None 130 ['192.168.254.3', '192.168.254.6', '192.168.254.8'] 4789
3 dc1-leaf2a default 10160 192.168.254.4 None 160 ['192.168.254.3', '192.168.254.6', '192.168.254.8'] 4789
4 dc1-leaf2b default 10130 192.168.254.4 None 130 ['192.168.254.3', '192.168.254.6', '192.168.254.8'] 4789

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Node                                                         dc1-svc3a
VRF                                                            default
VNI                                                              10140
Local_VTEP_IP                                            192.168.254.6
Multicast_Group                                                   None
VLAN                                                               140
VTEP_Flood_List    ['192.168.254.3', '192.168.254.4', '192.168.254.8']
VXLAN_Port                                                        4789
Name: 0, dtype: object

VXLAN Edges

Returns VXLAN edges.

Lists all VXLAN edges in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include edges whose first node matches this name or regex.

NodeSpec

True

.*

remoteNodes

Include edges whose second node matches this name or regex.

NodeSpec

True

.*

Invocation

[10]:
result = bf.q.vxlanEdges().answer().frame()

Return Value

Name

Description

Type

VNI

VNI of the VXLAN tunnel edge

int

Node

Node from which the edge originates

str

Remote_Node

Node at which the edge terminates

str

VTEP_Address

VTEP IP of node from which the edge originates

str

Remote_VTEP_Address

VTEP IP of node at which the edge terminates

str

VLAN

VLAN associated with VNI on node from which the edge originates

int

Remote_VLAN

VLAN associated with VNI on node at which the edge terminates

int

UDP_Port

UDP port of the VXLAN tunnel transport

int

Multicast_Group

Multicast group of the VXLAN tunnel transport

str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
VNI Node Remote_Node VTEP_Address Remote_VTEP_Address VLAN Remote_VLAN UDP_Port Multicast_Group
0 10130 dc1-leaf2b dc1-svc3a 192.168.254.4 192.168.254.6 130 130 4789 None
1 10140 dc1-leaf2a dc1-svc3a 192.168.254.4 192.168.254.6 140 140 4789 None
2 10130 dc1-svc3a dc1-leaf2a 192.168.254.6 192.168.254.4 130 130 4789 None
3 10111 dc1-leaf1a dc1-leaf2b 192.168.254.3 192.168.254.4 111 111 4789 None
4 10130 dc1-svc3b dc1-leaf2b 192.168.254.6 192.168.254.4 130 130 4789 None

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
VNI                            10130
Node                      dc1-leaf2b
Remote_Node                dc1-svc3a
VTEP_Address           192.168.254.4
Remote_VTEP_Address    192.168.254.6
VLAN                             130
Remote_VLAN                      130
UDP_Port                        4789
Multicast_Group                 None
Name: 0, dtype: object

L3 EVPN VNIs

Returns configuration settings of VXLANs.

Lists VNI-level network segment settings configured for VXLANs.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Include nodes matching this specifier.

NodeSpec

True

Invocation

[15]:
result = bf.q.evpnL3VniProperties().answer().frame()

Return Value

Name

Description

Type

Node

Node

str

VRF

VRF

str

VNI

VXLAN Segment ID

int

Route_Distinguisher

Route distinguisher

str

Import_Route_Target

Import route target

str

Export_Route_Target

Export route target

str

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node VRF VNI Route_Distinguisher Import_Route_Target Export_Route_Target
0 dc1-bl1a Tenant_A_WAN_Zone 15005 192.168.255.8:15005 15005:15005 15005:15005
1 dc1-bl1a Tenant_B_WAN_Zone 25021 192.168.255.8:25021 25021:25021 25021:25021
2 dc1-bl1a Tenant_C_WAN_Zone 35031 192.168.255.8:35031 35031:35031 35031:35031
3 dc1-bl1b Tenant_A_WAN_Zone 15005 192.168.255.9:15005 15005:15005 15005:15005
4 dc1-bl1b Tenant_B_WAN_Zone 25021 192.168.255.9:25021 25021:25021 25021:25021

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node                              dc1-bl1a
VRF                      Tenant_A_WAN_Zone
VNI                                  15005
Route_Distinguisher    192.168.255.8:15005
Import_Route_Target            15005:15005
Export_Route_Target            15005:15005
Name: 0, dtype: object

Resolving Specifiers

Specifier grammars allow you to specify complex inputs for Batfish questions. This category of questions reveals how specifier inputs are resolved by Batfish.

Resolve Location Specifier

Returns the set of locations corresponding to a locationSpec value.

Helper question that shows how specified locationSpec values resolve to the locations in the network.

Inputs

Name

Description

Type

Optional

Default Value

locations

Input to the LocationSpecifier.

LocationSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

Invocation

[5]:
result = bf.q.resolveLocationSpecifier(locations='@enter(as2border1[GigabitEthernet2/0])').answer().frame()

Return Value

Name

Description

Type

Location

Location

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Location
0 InterfaceLinkLocation{nodeName=as2border1, interfaceName=GigabitEthernet2/0}

Print the first row of the returned Dataframe

[7]:
result.iloc[0]
[7]:
Location    InterfaceLinkLocation{nodeName=as2border1, interfaceName=GigabitEthernet2/0}
Name: 0, dtype: object

Resolve Filter Specifier

Returns the set of filters corresponding to a filterSpec value.

Helper question that shows how specified filterSpec values resolve to the filters in the network.

Inputs

Name

Description

Type

Optional

Default Value

filters

Input to the FilterSpecifier.

FilterSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

nodes

Input to the NodeSpecifier that specifies the set of nodes that should be considered.

NodeSpec

True

/.*/

Invocation

[10]:
result = bf.q.resolveFilterSpecifier(filters='@in(as2border1[GigabitEthernet0/0])').answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Filter_Name

Filter name

str

Print the first 5 rows of the returned Dataframe

[11]:
result.head(5)
[11]:
Node Filter_Name
0 as2border1 OUTSIDE_TO_INSIDE

Print the first row of the returned Dataframe

[12]:
result.iloc[0]
[12]:
Node                  as2border1
Filter_Name    OUTSIDE_TO_INSIDE
Name: 0, dtype: object

Resolve Node Specifier

Returns the set of nodes corresponding to a nodeSpec value.

Helper question that shows how specified nodeSpec values resolve to the nodes in the network.

Inputs

Name

Description

Type

Optional

Default Value

nodes

Input to the NodeSpecifier.

NodeSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

Invocation

[15]:
result = bf.q.resolveNodeSpecifier(nodes='/border/').answer().frame()

Return Value

Name

Description

Type

Node

Node

str

Print the first 5 rows of the returned Dataframe

[16]:
result.head(5)
[16]:
Node
0 as1border1
1 as1border2
2 as2border1
3 as2border2
4 as3border1

Print the first row of the returned Dataframe

[17]:
result.iloc[0]
[17]:
Node    as1border1
Name: 0, dtype: object

Resolve Interface Specifier

Returns the set of interfaces corresponding to an interfaceSpec value.

Helper question that shows how specified interfaceSpec values resolve to the interfaces in the network.

Inputs

Name

Description

Type

Optional

Default Value

interfaces

Input to the interfaceSpecifier.

InterfaceSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

nodes

Input to the NodeSpecifier that specifies the set of nodes that should be considered.

NodeSpec

True

/.*/

Invocation

[20]:
result = bf.q.resolveInterfaceSpecifier(interfaces='/border/[.*Ethernet]').answer().frame()

Return Value

Name

Description

Type

Interface

Interface

Interface

Print the first 5 rows of the returned Dataframe

[21]:
result.head(5)
[21]:
Interface
0 as1border1[Ethernet0/0]
1 as1border1[GigabitEthernet0/0]
2 as1border1[GigabitEthernet1/0]
3 as1border2[Ethernet0/0]
4 as1border2[GigabitEthernet0/0]

Print the first row of the returned Dataframe

[22]:
result.iloc[0]
[22]:
Interface    as1border1[Ethernet0/0]
Name: 0, dtype: object

Resolve IPs from Location Specifier

Returns IPs that are auto-assigned to locations.

Helper question that shows IPs that will be assigned to specified locationSpec values by questions are automatically pick IPs based on locations.

Inputs

Name

Description

Type

Optional

Default Value

locations

Input to the LocationSpecifier.

LocationSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

Invocation

[25]:
result = bf.q.resolveIpsOfLocationSpecifier(locations='@enter(as2border1[GigabitEthernet2/0])').answer().frame()

Return Value

Name

Description

Type

Locations

Resolution

str

IP_Space

IP space

str

Print the first 5 rows of the returned Dataframe

[26]:
result.head(5)
[26]:
Locations IP_Space
0 [InterfaceLinkLocation{nodeName=as2border1, interfaceName=GigabitEthernet2/0}] AclIpSpace{lines=[AclIpSpaceLine{action=DENY, ipSpace=AclIpSpace{lines=[AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.2.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.3.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.2.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.0.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.3.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.10.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.10.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.13.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.21.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.12.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.11.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.12.11.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.13.22.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.12.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.22.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.12.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.14.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.23.21.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.12.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.22.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.21.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.11.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.22.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.12.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.23.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.201.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.101.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.201.4}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.101.4}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=90.90.90.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=90.90.90.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.1.101}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.0.101}}]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[2.12.12.0, 2.12.12.255], whitelist=[2.12.12.0/24]}}]}

Print the first row of the returned Dataframe

[27]:
result.iloc[0]
[27]:
Locations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             [InterfaceLinkLocation{nodeName=as2border1, interfaceName=GigabitEthernet2/0}]
IP_Space     AclIpSpace{lines=[AclIpSpaceLine{action=DENY, ipSpace=AclIpSpace{lines=[AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.2.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.3.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.2.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.1.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.0.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.2.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.0.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.0.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.3.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.2.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.1.1.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=3.10.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=1.10.1.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.13.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.21.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.12.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.11.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.12.11.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.13.22.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.12.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.22.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.12.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.12.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.14.22.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.23.21.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.12.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.22.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.21.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.11.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.22.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.12.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.23.11.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=10.23.21.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.201.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.101.3}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.201.4}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.34.101.4}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=90.90.90.2}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=90.90.90.1}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.1.101}}, AclIpSpaceLine{ipSpace=IpIpSpace{ip=2.128.0.101}}]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[2.12.12.0, 2.12.12.255], whitelist=[2.12.12.0/24]}}]}
Name: 0, dtype: object

Resolve IP Specifier

Returns the IP address space corresponding to an ipSpec value.

Helper question that shows how specified ipSpec values resolve to IPs.

Inputs

Name

Description

Type

Optional

Default Value

ips

Input to the IP space specifier.

IpSpec

False

grammarVersion

Version of grammar to use for resolution.

str

True

Invocation

[30]:
result = bf.q.resolveIpSpecifier(ips='/border/[.*Ethernet]').answer().frame()

Return Value

Name

Description

Type

IP_Space

IP space

str

Print the first 5 rows of the returned Dataframe

[31]:
result.head(5)
[31]:
IP_Space
0 AclIpSpace{lines=[AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[1.0.1.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.12.11.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.13.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[1.0.2.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.14.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.12.11.2]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.11.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.12.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.23.21.2]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.21.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[3.0.1.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.23.21.3]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.13.22.3]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[3.0.2.1]}}]}

Print the first row of the returned Dataframe

[32]:
result.iloc[0]
[32]:
IP_Space    AclIpSpace{lines=[AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[1.0.1.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.12.11.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.13.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[1.0.2.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.14.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.12.11.2]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.11.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.12.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.23.21.2]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.22.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[2.12.21.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[3.0.1.1]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.23.21.3]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[10.13.22.3]}}, AclIpSpaceLine{ipSpace=IpWildcardSetIpSpace{blacklist=[], whitelist=[3.0.2.1]}}]}
Name: 0, dtype: object

Differential Questions

Differential questions enable you to discover configuration and behavior differences between two snapshot of the network.

Most of the Batfish questions can be run differentially by using snapshot=<current snapshot> and reference_snapshot=<reference snapshot> parameters in .answer(). For example, to view routing table differences between snapshot1 and snapshot0, run bf.q.routes().answer(snapshot="snapshot1", reference_snapshot="snapshot0").

Batfish also has two questions that are exclusively differential.

Compare Filters

Compares filters with the same name in the current and reference snapshots. Returns pairs of lines, one from each filter, that match the same flow(s) but treat them differently (i.e. one permits and the other denies the flow).

This question can be used to summarize how a filter has changed over time. In particular, it highlights differences that cause flows to be denied when they used to be permitted, or vice versa. The output is a table that includes pairs of lines, one from each version of the filter, that both match at least one common flow, and have different action (permit or deny). This is a differential question and the reference snapshot to compare against must be provided in the call to answer().

Inputs

Name

Description

Type

Optional

Default Value

nodes

Only evaluate filters present on nodes matching this node specifier.

NodeSpec

True

filters

Only evaluate filters that match this filter specifier.

FilterSpec

True

ignoreComposites

Whether to ignore filters that are composed of multiple filters defined in the configs.

bool

True

False

Invocation

[5]:
result = bf.q.compareFilters(nodes='rtr-with-acl').answer(snapshot='filters-change',reference_snapshot='filters').frame()

Return Value

Name

Description

Type

Node

Hostname.

str

Filter_Name

The filter name.

str

Line_Index

The index of the line in the current filter.

str

Line_Content

The current filter line content.

str

Line_Action

The current filter line action.

str

Reference_Line_Index

The index of the line in the reference filter.

str

Reference_Line_Content

The reference filter line content.

str

Print the first 5 rows of the returned Dataframe

[6]:
result.head(5)
[6]:
Node Filter_Name Line_Index Line_Content Line_Action Reference_Line_Index Reference_Line_Content
0 rtr-with-acl acl_in 23 462 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 80 PERMIT 101 2020 deny tcp any any
1 rtr-with-acl acl_in 24 463 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 8080 PERMIT 101 2020 deny tcp any any

Differential Reachability

Returns flows that are successful in one snapshot but not in another.

Searches across all possible flows in the network, with the specified header and path constraints, and returns example flows that are successful in one snapshot and not the other. This is a differential question and the reference snapshot to compare against must be provided in the call to answer().

Inputs

Name

Description

Type

Optional

Default Value

pathConstraints

Constraint the path a flow can take (start/end/transit locations).

PathConstraints

True

headers

Packet header constraints.

HeaderConstraints

True

actions

Only return flows for which the disposition is from this set.

DispositionSpec

True

success

maxTraces

Limit the number of traces returned.

int

True

invertSearch

Search for packet headers outside the specified headerspace, rather than inside the space.

bool

True

ignoreFilters

Do not apply filters/ACLs during analysis.

bool

True

False

Invocation

[9]:
result = bf.q.differentialReachability().answer(snapshot='forwarding-change',reference_snapshot='forwarding').frame()

Return Value

Name

Description

Type

Flow

The flow

Flow

Snapshot_Traces

The traces in the BASE snapshot

Set of Trace

Snapshot_TraceCount

The total number traces in the BASE snapshot

int

Reference_Traces

The traces in the DELTA snapshot

Set of Trace

Reference_TraceCount

The total number traces in the DELTA snapshot

int

Print the first 5 rows of the returned Dataframe

[10]:
result.head(5)
[10]:
Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount
0 start=border1 [10.12.11.2:49152->2.128.1.1:33434 UDP] [((ORIGINATED(default), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])))] 1 [((ORIGINATED(default), FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet1/0)), (RECEIVED(GigabitEthernet0/0), FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)]), PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER)), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(eth0), ACCEPTED(eth0)))] 1
1 start=border1 interface=GigabitEthernet0/0 [10.12.11.1:49152->2.128.1.1:33434 UDP] [((RECEIVED(GigabitEthernet0/0), PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])))] 1 [((RECEIVED(GigabitEthernet0/0), PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet1/0)), (RECEIVED(GigabitEthernet0/0), FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)]), PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER)), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(eth0), ACCEPTED(eth0)))] 1
2 start=border1 interface=GigabitEthernet1/0 [2.12.11.3:49152->2.128.1.1:33434 UDP] [((RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])))] 1 [((RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet1/0)), (RECEIVED(GigabitEthernet0/0), FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)]), PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER)), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(eth0), ACCEPTED(eth0)))] 1
3 start=border1 interface=GigabitEthernet2/0 [2.12.12.3:49152->2.128.1.1:33434 UDP] [((RECEIVED(GigabitEthernet2/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])))] 1 [((RECEIVED(GigabitEthernet2/0), FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet1/0)), (RECEIVED(GigabitEthernet0/0), FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)]), PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER)), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(eth0), ACCEPTED(eth0)))] 1
4 start=border2 [10.23.21.2:49152->2.128.1.1:33434 UDP] [((ORIGINATED(default), FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet1/0)), (RECEIVED(GigabitEthernet0/0), NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])))] 1 [((ORIGINATED(default), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(GigabitEthernet1/0), FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)]), TRANSMITTED(GigabitEthernet2/0)), (RECEIVED(GigabitEthernet1/0), PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER)), FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)]), PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER)), TRANSMITTED(GigabitEthernet3/0)), (RECEIVED(eth0), ACCEPTED(eth0)))] 1

Format of vendor and supplemental data

You can provide two types of data to Batfish: 1) configurations of devices in your network, and 2) supplemental data to enhance the model of your network. This page describes the format of these data.

Device configuration formats

Batfish supports the following vendors. Click on the corresponding link to learn how to provide configuration for a specific vendor.

Except for AWS, all vendor configs files must be placed in the configs folder right below the top-level snapshot folder. It is OK to create sub-folders inside configs; Batfish will recursively read all files. It is also OK to mix files from multiple vendors in the same folder.

Note about vendor detection

Batfish can automatically detect the vendor of a configuration file based on certain tell-tale signs in vendor files. For instance, Arista config files tend to contain lines with “! device: … EOS-4.24” or “boot system flash … swi”. Such lines are almost always present in config files pulled from devices but may not be present in auto-generated files.

If you need to explicitly specify the vendor of your configuration file, include the RANCID content type header. So, for Arista devices, you’d include the following line at the top of the file:

!RANCID-CONTENT-TYPE: arista

The supported vendor type strings are arista, bigip (F5), cisco-nx (NX-OS), cisco-xr (IOS-XR), force10 (Dell), foundry, juniper (all JunOS), mrv, and paloalto.

A10

For each A10 device in the network, create a file in the configs folder. For v4 and higher versions, the content of the file should be the equivalent of the output of show running-config partition-config all command on the device. For v2, use show running-config all-partitions command.

Arista

For each Arista device in the network, create a file in the configs folder. The content of the file should be the equivalent of the output of show running-config command on the device.

AWS

Batfish understands AWS VPC configurations and analyzes it just like physical networks. To use this functionality, place AWS configs in a folder named aws_configs right below the top-level snapshot folder. Each subfolder in this folder should correspond to an AWS account. If you want to analyze only one account, you may skip this level of hierarchy. The subfolders of the account-level folder (or of aws_configs if there is only account) correspond to individual regions. If there is only one region, then this level of hierarchy may be skipped.

The configuration files for a region should be the JSON output of the following API calls:

  • For EC2: describe_addresses, describe_availability_zones, describe_customer_gateways, describe_internet_gateways, describe_network_acls, describe_network_interfaces, describe_instances, describe_route_tables, describe_security_groups, describe_subnets, describe_vpc_endpoints, describe_vpcs, describe_vpn_connections, describe_vpn_gateways

  • For ES: describe_elasticsearch_domains

  • For RDS: describe_db_instances

This output can be collected using the AWS CLI or the boto3 Python SDK. An example script that packages AWS data into a Batfish snapshot is here.

An example snapshot, which includes both physical and AWS configs, is here. It is OK to have only AWS configs in a snapshot (without any physical device configs).

Cisco

Batfish supports Cisco IOS, IOS-XE, IOS-XR, NX-OS, and ASA platforms. For each such device in the network, create a file in the configs folder. The content of the file should be the equivalent of the output of show running-config command on the device.

For NX-OS, we recommend using show run all where possible. This output has additional detail that helps with modeling accuracy.

Check Point

Check Point data is provided in two parts.

  1. Place the output of show configuration from gateway devices into the configs folder. This data contains information about interfaces and routing but not firewall policies, which must be fetched from the Check Point Manager as described next.

  2. Data from Check Point Manager must be placed in a folder called checkpoint_management right under the top-level snapshot folder (parallel to configs folder). The expected hierarchy under this folder is as follows:

  • checkpoint_management

    • Manager1 [Data from Manager1–you may have multiple managers]

      • DomainA [Data from DomainA–the manager may have multiple]

        • show-gateways-and-servers.json

        • show-groups.json

        • show-hosts.json

        • show-networks.json

        • show-package.json

        • show-service-groups.json

        • show-services-icmp.json

        • show-services-other.json

        • show-services-tcp.json

        • show-services-udp.json

        • Package1 [Data for Package1–the domain may have multiple packages]

          • show-access-rulebase.json

          • show-nat-rulebase.json

          • show-package.json

The json files contain the output of corresponding Check Point Management API calls.

An example script that fetches this data and organizes it as above is here.

Cumulus Linux

Cumulus devices can be configured by editing individual files, such as /etc/network/interfaces, /etc/cumulus/ports.conf, and /etc/frr/frr.conf or invoking the Network Command-Line Utility (NCLU)

Batfish supports both the NCLU format and individual Cumulus configuration files. We recommend using the configuration files because Batfish can extract more data from them than from the NCLU output.

Note

If you are using the BGP Unnumbered feature on Cumulus devices, you will need to supply Layer-1 topology.

F5 BIG-IP

F5 BIG-IP configuration spans multiple individual files, typically the following 3 configuration files:

  • Base

  • LTM

  • Routing [There are 2 formats for this routing configuration. Legacy imish and the newer structured format]

These files are typically in a folder named “config” as:

  • Base: bigip_base.conf

  • LTM: bigip.conf

  • Routing: bigip_routing.conf

For Batfish to correctly process the F5 configuration, the 3 files need to be combined into a single file in a specific order.

As an example, let’s say you have these 3 files:

  • site1-f5-a-base.conf

  • site1-f5-a.conf

  • site1-f5-a-routing.conf

On any unix machine, simply run:

cat site1-f5-a-base.conf site1-f5-a.conf site1-f5-a-routing.conf > site1-f5-a-concat.cfg

Add site1-f5-a-concat.cfg to the configs folder with the rest of the devices.

Note

The routing configuration MUST be the last thing copied into the file, otherwise Batfish will not be able to correctly parse the file.

Fortinet

For each FortiOS firewall, create a file in the configs folder. The content of the file should be the equivalent of the output of show command on the device.

Juniper

Batfish supports all JunOS-based platforms, including EX, MX, PTX, QFX, SRX, and the T-series. For each such device in the network, create a file in the configs folder. The content of the file should be the equivalent of the output of show configuration | display set command on the device.

Batfish will also accept JunOS configuration in the hierarchical format and will internally pre-process files in this format to the “set” format.

Palo Alto Networks

Batfish supports Palo Alto Networks devices with or without Panorama.

From Panorama (preferred)

For devices managed through Panorama, with no configuration done directly on the device, you can pull configurations from Panorama without having to run any commands on the managed-devices themselves. If configuration is also done directly on the managed-device, you should follow the instructions for pulling configuration From individual devices instead.

To pull these configurations through Panorama, use the pan-python module as follows; note: replace <IP_ADDR> and <LOGIN_USERNAME> with the IP address and username for the Panorama device:

# Generate an API key and save in .panrc so subsequent commands don't require logging in again
# This only needs to be done once
panxapi.py -t panorama_tag -h <IP_ADDR> -l <LOGIN_USERNAME> -k >> .panrc
# Pull the XML Panorama config
panxapi.py -t panorama_tag -sxr > panorama_config.xml
# Convert the XML Panorama config into set-format
panconf.py --config pan-panorama_config.xml  --set  > panorama_config.set.txt

This will generate a single configuration (panorama_config.set.txt) representing all firewalls managed by the specified Panorama device, and this config file is what Batfish needs to model the managed firewalls. Place this file in the configs folder.

From individual devices

For each device, concatenate the following show commands into one file and put the file in the configs folder.

set cli config-output-format set
set cli pager off
show config pushed-shared-policy
show config pushed-shared-policy vsys <value> // run for each vsys
show config merged

The first two commands may not be available on all PAN-OS versions; just make sure that the output is NOT in XML format (first command) and that definitions are not truncated (second command).

SONiC

Configuration files for SONiC devices must be placed under a folder called sonic_configs right under the top-level snapshot folder. For each SONiC device in the network, create a folder under the sonic_configs folder, and put its files in this folder. Currently, config_db.json, frr.conf, resolv.conf, and snmp.yml files are supported. The last two files are optional. File names must end with these respective strings. So, “config_db.json” and “rt123_config_db.json” are valid names for config DB configuration, but “config_db_rt123.json” is not. The device folder must have only these files.

See this example snapshot with SONiC configs.

Supplemental data formats

You can provide additional data to Batfish to enhance the model of your network and to model parts of the network whose configuration is not available. The following types of data is supported.

Modeling hosts

You can model end hosts in the network by adding host files with information about their names, a pointer to their iptables configuration file, and their interfaces. An example host file is:

{
  "hostname" : "host1",
  "iptablesFile" : "iptables/host1.iptables",
  "hostInterfaces" : {
    "eth0" : {
      "name": "eth0",
      "prefix" : "2.128.0.101/24"
    }
  }
}

iptables/host1.iptables is the path relative to the snapshot where this host’s iptables configuration can be found. iptables configuration files should be in the format that is generated by iptables-save.

There should be one such file per host that you want to model and these files should be placed in a folder called hosts right below the top-level snapshot folder.

See this example snapshot with host files.

Layer-1 topology

Batfish can infer Layer-3 interface adjacencies based on IP address configuration on interfaces. For instance, if there are two interfaces in the network with IP assignments 192.168.1.1/24 and 192.128.1.2/24, Batfish will infer that these interfaces are adjacent.

Such inference does not work if the network re-uses IP address space or has link-local addresses. In those situations, you must provide a Layer-1 topology file that has cabling information. Then, Layer-3 adjacencies will be computed by combining the supplied Layer-1 adjacencies with Layer-2 and Layer-3 configuration to get a more accurate model.

The expected Layer-1 topology file is a JSON file that has a list of edge records, where each edge record has node and interface names of the two ends. See this file for an example.

The name of your Layer-1 topology file must be layer1_topology.json and it must be placed in a folder called batfish right below the top-level snapshot folder.

Modeling ISPs

Batfish can model ISPs (and Internet connectivity) for a network based on a file that specifies which ISPs to model and how to configure them. The ISP modeling file has a few different sections that control different aspects of the modeling.

  • BorderInterfaces: Specify the interfaces in the snapshot that connect to the ISP you want to model. Each interface must be a layer-3 interface connected directly to the ISP and establish an eBGP single-hop session to the ISP.

{
  "borderInterfaces": [
    {
      "borderInterface": {
        "hostname": "as2border1",
        "interface": "GigabitEthernet3/0"
      }
    }, 
    ....
  ]
}
  • BgpPeers: Specify the BGP peers defined on border routers in the snapshot. You can also optionally specify how the ISP attaches to the snapshot, by specifying the interface in the snapshot where it establishes physical connectivity. In the absence of this attachment specification, Batfish will assume that the ISP is attached at the interface of the specified peering.

{
  "bgpPeers": [
    {
      "hostname": "as2border1",
      "peerAddress": "10.10.10.10",
      "vrf": "internet",   // Optional; default VRF is assumed if left unspecified
      "ispAttachment": {   // This section is optional. 
          "hostname": "borderSwitch", // Optional; the host of the peering ('as2border1') is assumed if left unspecified
          "interface": "GigabitEthernet3/0",  // mandatory
          "vlanTag": 86 // Specifies the Dot1q encapsulation that the modeled ISP should use for the peering. 
                        // Optional; no tagging is assumed if left unspecified
      }
    }, 
    ....
  ]
}
  • Filter: This section allows you restrict ISPs that are modeled, by specifying the ASNs (onlyRemoteAsns) and IPs (onlyRemoteIps) of the ISPs. Without this section, all ISPs specified by BorderInterfaces and BgpPeers sections.

{
  "filter": {
    "onlyRemoteAsns": [65432],
    "onlyRemoteIps": ["10.10.10.10"]
  }
}
  • IspNodeInfos: This section lets you configure certain aspects of the modeled ISP nodes. One aspect of this configuration is specifying the role of the ISP—whether it is a transit provider (TRANSIT) or a private backbone (PRIVATE_BACKBONE) such as a carrier MPLS or your own wide-area network that connects multiple sites. Transit ISPs connects to the modeled Internet node, do not propagate any communities and block reserved address space traffic. Private backbones do not connect to the Internet, propagate (standard and extended) communities and do not filter any traffic.

{
  "ispNodeInfo": [
    {
      "asn": 65432,
      "name": "ATT", // Specifies the friendly name (not hostname) to use for the ISP
      "role": "PRIVATE_BACKBONE"  // Optional; default is "TRANSIT"
    },
    ...
   ]
}
  • IspPeerings: This section lets you configure eBGP peering between modeled ISP nodes. When not present, none of the ISPs connect to each other. The specification is:

{
   "ispPeerings": [
     {
        "peer1": {
           "asn": 65432,
        },
        "peer2": {
           "asn": 64325,
        }
     },
     ...
   ]
}

All sections that you need must be included in a file called isp_config.json, placed in a folder called batfish right below the top-level snapshot folder. An example snapshot with ISP modeling file is here.

The hostname of the modeled ISP nodes will be of the form isp_<ASN> (e.g., ‘isp_65432’). These nodes can be queried and analyzed just like any other node in the snapshot. Only one node per ASN is generated even if you have multiple peerings to the same ASN.

A node representing the Internet is also created if any of the ISPs is “TRANSIT”. This node announces the default route (0.0.0.0/0) to connected ISPs. Its hostname is ‘internet’ and it too can be queried and analyzed like any other node in the snapshot.

Runtime interface information

Batfish infers interface attributes from static configuration files. All relevant information, however, may not be present in the configuration files, including whether an interface is line down and speed/bandwidth for some vendors. You can enhance Batfish’s inference of such properties by providing runtime data.

The format of this data is

{
  "runtimeData": {
    "router1": {
      "Ethernet1/1": {
         "bandwidth": 100000000000,
         "lineUp": false,
         "speed": 100000000000
      }
    },
    "router2": {
      "Ethernet21/1": {
         "lineUp": true
      }
    }    
  }
}

The name of this must be runtime_data.json and it must be placed in a folder called batfish right below the top-level snapshot folder. The same file should contain information for all devices and interfaces, and only a subset of the properties may be specified for an interface.

The runtime data specification also allows you to analyze the impact of failing certain interfaces in the network—just mark them as line down in the data.

External BGP announcements

To help analyze connectivity to specific external prefixes or analyze the treatment of external routes by your network, you can specify the external routes coming into the network, as follows.

{
  "Announcements": [
      {
          "type" : "ebgp_sent", // whether the attributes of this route are what was sent by the external peer, 
                                // or they represent attributes after processing by import routing policy ('ebgp_received')
          "network" : "4.0.0.0/8",
          "nextHopIp" : "10.14.22.4",
          "srcIp" : "10.14.22.4",
          "dstNode" : "as1border2",
          "dstIp" : "10.14.22.1",
          "srcProtocol" : "AGGREGATE",
          "originType" : "egp",
          "localPreference" : 0,
          "med" : 20,
          "originatorIp" : "0.0.0.0",
          "asPath" : [ [ 1239 ], [7018, 7019]],
          "communities" : [ 262145 ],
          "dstVrf":"default",
          "clusterList":[]
      },
      ...
   ]
}

These announcements must be placed in the file called external_bgp_announcements.json and placed inside the top-level snapshot folder.

Advanced Concepts

Specifier grammars

Batfish questions support parameters with rich specifications for nodes, interfaces etc. The grammar for parameter types is described below. Before reading those grammars, we recommend reading the general notes.

For many parameters types, there is a “resolver” question that may be used to learn what a given specification expands to. For instance, resolveNodeSpecifier is the resolver for nodeSpec, and bfq.resolveNodeSpecifier(nodes="/bor/") (Pybatfish syntax) will return the set of nodes that match /bor/.

General notes on the grammar

  • Set operations: Specifiers denote sets of entities (e.g., nodeSpec resolves to a set of nodes). In many cases, the grammar allows for union, intersection, and difference of such sets, respectively, using ,, &, and \. Thus, (node1, node2)\node1 will resolve to node1.

  • Escaping names: Names of entities such as nodes and interfaces must be double-quoted if they begin with a digit (0-9), double quote (‘”’), or slash (‘/’), or they contain a space or one of ,&()[]@!#$%^;?<>={}. Thus, the following names are legal:

    • as1border1 (no quotes)

    • as1-border1

    • "as1border1" (quotes unnecessary, but OK)

    • "1startsWithADigit" (quotes needed)

    • "has space"

    • "has["

  • Regexes: Regular expressions must be enclosed by /s like /abc/. Batfish uses Java’s syntax and semantics for regular expressions. For simple expressions, this language is similar to others. For example:

    • /abc/, /^abc/, /abc$/ match strings strings containing, beginning with, and ending with ‘abc’

    • /ab[c-d]/ and /ab(c|d)/ match strings ‘abc’ and ‘abd’.

  • Case-insensitive names: All names and regexes use case-insensitive matching. Thus, AS1BORDER1 is same as as1border1 and Ethernet0/0 is same as ethernet0/0.

Set of enums or names

Many types such as applicationSpec or mlagIdSpec are simply sets of values. Such parameters share a common grammar, but with different base values. Example expressions for this grammar are:

  • val1 specifies a singleton set with that value.

  • /val.*/ specifies a set whose values all match regex val.*.

  • val1, val2 specifies a set with exactly those two values.

  • ! val1 specifies all values other than val1.

  • /val.*/, ! val1 specifies all values that match regex val.* other than val1.

The full specification of this grammar is:

enumSetSpec :=
   enumSetTerm [, enumSetTerm]

enumSetTerm :=
   <enum-value>
   | !<enum-value>
   | /<regex-over-enum-values>/
   | !/<regex-over-enum-values>/

Application Specifier

A specification for IP traffic that includes information about protocols (ICMP, TCP, UDP) and about destination ports for TCP and UDP and type, code for ICMP.

  • HTTP specifies TCP traffic to port 80.

  • tcp/80 also specifies TCP traffic to port 80.

  • tcp/80,3000-3030 specifies TCP traffic to port 80 and ports between 3000 and 3030.

  • tcp specifies TCP traffic to all ports.

  • icmp also specifies ICMP traffic of all types.

  • icmp/0/0 specifies ICMP traffic of type 0 and code 0.

  • HTTP, udp/53 specifies TCP traffic to port 80 and UDP traffic to port 53.

Application Specifier Grammar
applicationSpec :=
    applicationTerm [, applicationTerm]

applicationTerm :=
    tcp[/portSpec]
    | udp[/portSpec]
    | icmp[/<icmp-type>[/<icmp-code>]]
    | <application-name>

portSpec :=
    portTerm [, portTerm]

portTerm := 
    <port-number>
    | <from-port>-<to-port>

Application name is one of DNS (means udp/53), ECHO-REPLY (icmp/0/0), ECHO-REQUEST (icmp/8/0), HTTP (tcp/80), HTTPS (tcp/443), MYSQL (tcp/3306), SNMP (udp/161), SSH (tcp/22), TELNET (tcp/23).

BGP Peer Property Specifier

A specification for a set of BGP peer properties (e.g., those returned by the bgpPeerConfiguration question).

A BGP peer property property specifier follows the enum set grammar over the following values: Local_AS, Local_IP, Is_Passive, Remote_AS, Route_Reflector_Client, Cluster_ID, Peer_Group, Import_Policy, Export_Policy, Send_Community.

BGP Process Property Specifier

A specification for a set of BGP process properties (e.g., those returned by the bgpProcessConfiguration question).

A BGP process property specifier follows the enum set grammar over the following values: Multipath_Match_Mode, Multipath_EBGP, Multipath_IBGP, Neighbors, Route_Reflector, Tie_Breaker.

BGP Route Status Specifier

A specification for a set of BGP route statuses.

A BGP route status specifier follows the enum set grammar over the following values:

  • BEST - a route that is the unique best route for an NLRI (e.g. an IP prefix for IPv4 unicast) in the BGP RIB, or a route that is equivalent to the unique best route for the purpose of ECMP routing. A BEST route may be installed in the main RIB.

  • BACKUP - a route that is inferior to all BEST routes in the BGP RIB for the same NLRI. Such a route will not be installed in the main RIB. However, it may be advertised on a BGP ADD-PATH-enabled session and the route matches the add-path policy.

BGP Session Compat Status Specifier

A specification for a set of BGP session compatibility statuses.

A BGP session compat status specifier follows the enum set grammar over the following values:

  • LOCAL_IP_UNKNOWN_STATICALLY — local IP address for an iBGP or multihop eBGP session is not configured

  • NO_LOCAL_IP— local IP address for a singlehop eBGP session is not configured

  • NO_LOCAL_AS— local AS for the session is not configured

  • NO_REMOTE_IP — remote IP address for a point-to-point peer is not configured

  • NO_REMOTE_PREFIX — remote prefix for a dynamic peer is not configured

  • NO_REMOTE_AS — remote AS for the session is not configured

  • INVALID_LOCAL_IP — configured local IP address does not belong to any active interface

  • UNKNOWN_REMOTE — configured remote IP is not present in the network snapshot

  • HALF_OPEN — no compatible match found in the network snapshot for a point-to-point peer

  • MULTIPLE_REMOTES — multiple compatible matches found for a point-to-point peer

  • UNIQUE_MATCH — exactly one match found for a point-to-point peer

  • DYNAMIC_MATCH — at least one compatible match found for a dynamic peer

  • NO_MATCH_FOUND — no compatible match found for a dynamic peer

BGP Session Status Specifier

A specification for a set of BGP session statuses.

A BGP session status specifier follows the enum set grammar over the following values:

  • NOT_COMPATIBLE — the BGP session is not compatibly configured

  • NOT_ESTABLISHED — the BGP session configuration is compatible but the session was not established

  • ESTABLISHED — the BGP session is established

BGP Session Type Specifier

A specification for a set of BGP session types.

A BGP session type specifier follows the enum set grammar over the following values: IBGP, EBGP_SINGLEHOP, EBGP_MULTIHOP, EBGP_UNNUMBERED, IBGP_UNNUMBERED, UNSET.

Disposition Specifier

Flow dispositions are used in questions like reachability to identify flow outcomes. The disposition specifier takes as input a comma-separated list of disposition values, which are interpreted using logical OR.

There are two coarse-grained flow dispositions:

  • Success: a flow has been successfully delivered

  • Failure: a flow has been dropped somewhere in the network

The following fine-grained disposition values are also supported:

  • Success dispositions:

    • Accepted: a flow has been accepted by a device in the snapshot

    • Delivered_to_subnet: a flow has been delivered to a host subnet

    • Exits_network: a flow has been successfully forwarded to a device currently outside of the snapshot

  • Failure dispositions:

    • Denied_in: a flow was denied by an input filter (an ACL or a firewall rule) on an interface

    • Denied_out: a flow was denied by an output filter on an interface

    • No_route: a flow was dropped because no matching route exists on device

    • Null_routed: a flow was dropped because it matched a null route

    • Neighbor_unreachable: a flow was dropped because it could not reach the next hop (e.g., an ARP failure)

    • Loop: the flow encountered a forwarding loop

    • Insufficient_info: Batfish does not have enough information to make a determination with certainty (e.g., some device configs are missing)

Filter Specifier

A specification for filters (ACLs or firewall rules) in the network.

  • filter1 includes filters on all nodes with that name.

  • /^acl/ includes all filters (on all nodes) whose names name regex ‘^acl’, i.e., begin with ‘acl’.

  • nodeTerm[filterWithoutNode] indicates filters that match the filterWithoutNode specification on nodes that match the nodeTerm specification. A simple example is as1border1[filter1] which refers to the filter filter1 on as1border1.

  • @in(interfaceSpec) refers to filters that get applied when packets enter the specified interfaces. For example, @in(Ethernet0/0) includes filters for incoming packets on interfaces named Ethernet0/0 on all nodes.

  • @out(interfaceSpec) is similar except that it indicates filters that get applied when packets exit the specified interfaces.

Filter Specifier Grammar
filterSpec :=
    filterTerm [(&|,|\) filterTerm]

filterTerm :=
    filterWithNode
    | filterWithoutNode
    | (filterSpec)

filterWithNode :=
    nodeTerm[filterWithoutNode]

filterWithoutNode :=
    filterWithoutNodeTerm [(&|,|\) filterWithoutNodeTerm]

filterWithoutNodeTerm :=
    <filter-name>
    | /<filter-name-regex>/
    | @in(interfaceSpec)
    | @out(interfaceSpec)
    | (filterWithoutNode)


Filter Specifier Resolver
  • resolveFilterSpecifier shows the set of filters represented by the given input.

Interface Property Specifier

A specification for a set of interface-level properties (e.g., those returned by the interfaceProperties question).

An interface property specifier follows the enum set grammar over the following values: Access_VLAN, Active, Allowed_VLANs, All_Prefixes, Auto_State_VLAN, Bandwidth, Blacklisted, Channel_Group, Channel_Group_Members, Declared_Names, Description, DHCP_Relay_Addresses, Encapsulation_VLAN, HSRP_Groups, HSRP_Version, Incoming_Filter_Name, MLAG_ID, MTU, Native_VLAN, Outgoing_Filter_Name, PBR_Policy_Name, Primary_Address, Primary_Network, Proxy_ARP, Rip_Enabled, Rip_Passive, Spanning_Tree_Portfast, Speed, Switchport, Switchport_Mode, Switchport_Trunk_Encapsulation, VRF, VRRP_Groups, Zone_Name.

Interface Specifier

A specification for interfaces in the network.

  • Ethernet0/1 indicates interfaces on all nodes with that name.

  • /^Eth/ indicates all interfaces (on all nodes) whose names match the regex ‘^Eth’, i.e., start with ‘Eth’.

  • nodeTerm[interfaceWithoutNode] indicates interfaces that match the interfaceWithoutNode specification on nodes that match the nodeTerm specification. A simple example is as1border1[Ethernet0/1] which refers to the interface Ethernet0/1 on as1border1.

  • @connectedTo(ipSpec) indicates all interfaces with configured IPv4 networks that overlap with specified IPs (see ipSpec)

  • @interfaceGroup(book, group) looks in the configured reference library for an interface group with name ‘group’ in book with name ‘book’.

  • @vrf(vrf1) indicates all interfaces configured to be in the VRF with name ‘vrf1’.

  • @zone(zone3) indicates all interfaces configured to be in the zone with name ‘zone3’.

Interface Specifier Grammar
interfaceSpec :=
    interfaceTerm [(&|,|\) interfaceTerm]

interfaceTerm :=
    interfaceWithNode
    | interfaceWithoutNode
    | (interfaceSpec)

interfaceWithNode :=
    nodeTerm[interfaceWithoutNode]

interfaceWithoutNode :=
    interfaceWithoutNodeTerm [(&|,|\) interfaceWithoutNodeTerm]

interfaceWithoutNodeTerm :=
    <interface-name>
    | /<interface-name-regex>/
    | interfaceFunc
    | (interfaceWithoutNode)

interfaceFunc :=
    @connectedTo(ipSpec)
    | @interfaceGroup(<reference-book-name>, <<interface-group-name>)
    | @vrf(<vrf-name>)
    | @zone(<zone-name>)
Interface Specifier Resolver
  • resolveInterfaceSpecifier shows the set of interfaces represented by the given input.

IP Protocol Specifier

A specification for a set of IP protocols.

  • IP protocol names from the list below, such as TCP, may be used.

  • IP protocol numbers between 0 and 255 (inclusive), such as 6 to denote TCP, may be used.

  • A negation operator ! may be used to denote all IP protocols other than the one specified. The semantics of negation is:

    • !TCP refers to all IP protocols other than TCP

    • !TCP, !UDP refers to all IP protocols other than TCP and UDP

    • TCP, !UDP refers to TCP

IP Protocol Specifier Grammar
ipProtocolSpec :=
    ipProtocolTerm [, ipProtocolTerm]

ipProtocolTerm :=
    ipProtocol
    | !ipProtocol

ipProtocol :=
    <ip-protocol-name>
    | <ip-protocol-number>
IP Protocol Names

Batfish understands the following protocol names (with corresponding numbers in parenthesis): AHP (51), AN (107), ANY_0_HOP_PROTOCOL (114), ANY_DISTRIBUTED_FILE_SYSTEM (68), ANY_HOST_INTERNAL_PROTOCOL (61), ANY_LOCAL_NETWORK (63), ANY_PRIVATE_ENCRYPTION_SCHEME (99), ARGUS (13), ARIS (104), AX25 (93), BBN_RCC_MON (10), BNA (49), BR_SAT_MON (76), CBT (7), CFTP (62), CHAOS (16), COMPAQ_PEER (110), CPHB (73), CPNX (72), CRTP (126), CRUDP (127), DCCP (33), DCN_MEAS (19), DDP (37), DDX (116), DGP (86), EGP (8), EIGRP (88), EMCON (14), ENCAP (98), ESP (50), ETHERIP (97), FC (133), FIRE (125), GGP (3), GMTP (100), GRE (47), HIP (139), HMP (20), HOPOPT (0), I_NLSP (52), IATP (117), IPV6_ROUTE (43), IPX_IN_IP (111), IRTP (28), ISIS (124), ISO_IP (80), ISO_TP4 (29), KRYPTOLAN (65), L2TP (115), LARP (91), LEAF1 (25), LEAF2 (26), MANAET (138), MERIT_INP (32), MFE_NSP (31), MHRP (48), MICP (95), MOBILE (55), MOBILITY (135), MPLS_IN_IP (137), MTP (92), MUX (18), NARP (54), NETBLT (30), NSFNET_IGP (85), NVPII (11), OSPF (89), PGM (113), PIM (103), PIPE (131), PNNI (102), PRM (21), PTP (123), PUP (12), PVP (75), QNX (106), RDP (27), ROHC (142), RSVP (46), RSVP_E2E_IGNORE (134), RVD (66), SAT_EXPAK (64), SAT_MON (69), SCC_SP (96), SCPS (105), SCTP (132), SDRP (42), SECURE_VMTP (82), SHIM6 (140), SKIP (57), SM (122), SMP (121), SNP (109), SPRITE_RPC (90), SPS (130), SRP (119), SSCOPMCE (128), ST (5), STP (118), SUN_ND (77), SWIPE (53), TCF (87), TCP (6), THREE_PC (34), TLSP (56), TPPLUSPLUS (39), TRUNK1 (23), TRUNK2 (24), TTP (84), UDP (17), UDP_LITE (136), UTI (120), VINES (83), VISA (70), VMTP (81), VRRP (112), WB_EXPAK (79), WB_MON (78), WESP (141), WSN (74), XNET (15), XNS_IDP (22), XTP (36).

IP Specifier

A specification for a set of IPv4 addresses.

  • Constant values that denote addresses (e.g., 1.2.3.4), prefixes (e.g., 1.2.3.0/24), address ranges (e.g., 1.2.3.4 - 1.2.3.7), and wildcards (e.g., 1.2.3.4:255.255.255.0) may be used.

  • @addressGroup(book, group) looks in the configured reference library for an address group name ‘group’ in book name ‘book’.

  • locationSpec can be used to denote addresses corresponding to the specified location (see locationSpec). For example, as1border1[Ethernet0/0] includes all IPv4 addresses configured on as1border1 interface Ethernet0/0.

IP Specifier Grammar
ipSpec :=
    ipTerm [(&|,|\) ipTerm]

ipTerm :=
    <ip-address>
    | <ip-prefix>
    | <ip-address-low> - <ip-address-high>
    | <ip wildcard>
    | @addressGroup(<reference-book-name>, <address-group-name>)
    | locationSpec
IP Specifier Resolver
  • resolveIpSpecifier shows the set of IP addresses represented by the given input.

IPSec Session Status Specifier

An IPSec session status specifier follows the enum set grammar over the following values: IPSEC_SESSION_ESTABLISHED, IKE_PHASE1_FAILED, IKE_PHASE1_KEY_MISMATCH, IPSEC_PHASE2_FAILED, MISSING_END_POINT.

Location Specifier

A specification for locations of packets, including where they start or terminate.

There are two types of locations:

  • InterfaceLocation: at the interface, used to model packets that originate or terminate at the interface

  • InterfaceLinkLocation: on the link connected to the interface, used to model packets before they enter the interface or after they exit

Unless expilcitly specified, questions like traceroute and reachability will automatically assign IP addresses to packets based on their location. For InterfaceLocation, the set of assigned addresses is the interface address(es). This set is empty for interfaces that do not have an assigned address. For InterfaceLinkLocation, the set of assigned addresses corresponds to what (hypothetical) hosts attached to that interface can have, which includes all addresses in the subnet except for the address of the interface and the first and last addresses of the subnet. This set is empty for interface subnets that are /30 or longer (e.g., loopback interfaces).

Locations for which Batfish cannot automatically assign a viable IP are ignored. To force their consideration, explicit source IPs must be specified.

Some examples:

  • as1border1 specifies the InterfaceLocation for all interfaces on node as1border1. Any nodeTerm (see node specifier grammar) can be used as a location specifier.

  • as1border1[Ethernet0/0] specifies the InterfaceLocation for Ethernet0/0 on node as1border1. Any valid interfaceWithNode expression can be used as a location specifier.

  • @vrf(vrf1) specifies the InterfaceLocation for any interface in vrf1 on all nodes. Any interfaceFunc can be used as a location specifier.

  • @enter(as1border1[Ethernet0/0]) specifies the InterfaceLinkLocation for packets entering Ethernet0/0 on as1border1.

Location Specifier Grammar
locationSpec :=
    locationTerm [(&|,|\) locationTerm]

locationTerm :=
    locationInterface
    | @enter(locationInterface)
    | (locationSpec)

locationInterface :=
    nodeTerm
    | interfaceFunc
    | interfaceWithNode
Location Specifier Resolver
  • resolveLocationSpecifier shows the set of locations represented by the given input.

  • resolveIpsOfLocationSpecifier shows the mapping from locations to IPs that will be used in traceroute and reachability questions when IPs are not explicitly specified.

MLAG ID Specifier

A specification for a set of MLAG domain identifiers.

An MLAG ID specifier follows the enum set grammar over the domain ID values that appear in the snapshot.

Named Structure Specifier

A specification for a set of structure types in Batfish’s vendor independent model.

A named structure specifier follows the enum set grammar over the following values: AS_PATH_ACCESS_LIST, AUTHENTICATION_KEY_CHAIN, COMMUNITY_MATCH_EXPRS, COMMUNITY_SET_EXPRS, COMMUNITY_SET_MATCH_EXPRS, COMMUNITY_SETS, IKE_PHASE1_KEYS, IKE_PHASE1_POLICIES, IKE_PHASE1_PROPOSALS, IP_ACCESS_LIST, IP_6_ACCESS_LIST, IPSEC_PEER_CONFIGS, IPSEC_PHASE2_POLICIES, IPSEC_PHASE2_PROPOSALS, PBR_POLICY, ROUTE_FILTER_LIST, ROUTE_6_FILTER_LIST, ROUTING_POLICY, VRF, ZONE.

Node Property Specifier

A specification for a set of node-level properties (e.g., those returned by the nodeProperties question).

A node property specifier follows the enum set grammar over the following values: AS_Path_Access_Lists, Authentication_Key_Chains, Canonical_IP, Community_Match_Exprs, Community_Set_Exprs, Community_Set_Match_Exprs, Community_Sets, Configuration_Format, Default_Cross_Zone_Action, Default_Inbound_Action, DNS_Servers, DNS_Source_Interface, Domain_Name, Hostname, IKE_Phase1_Keys, IKE_Phase1_Policies, IKE_Phase1_Proposals, Interfaces, IP_Access_Lists, IP_Spaces, IP6_Access_Lists, IPsec_Peer_Configs, IPsec_Phase2_Policies, IPsec_Phase2_Proposals, IPSec_Vpns, Logging_Servers, Logging_Source_Interface, NTP_Servers, NTP_Source_Interface, PBR_Policies, Route_Filter_Lists, Route6_Filter_Lists, Routing_Policies, SNMP_Source_Interface, SNMP_Trap_Servers, TACACS_Servers, TACACS_Source_Interface, VRFs, Zones.

Node Specifier

A specification for nodes in the network.

  • as1border1 indicates a node with that name.

  • /^as1/ indicates all nodes whose names match the regex ^as1, i.e., start with ‘as1’.

  • @role(dim, role) indicates all nodes with role name ‘role’ in dimension name ‘dim’.

Node Specifier Grammar
nodeSpec :=
    nodeTerm [(&|,|\) nodeTerm]

nodeTerm :=
    <node-name>
    | /<node-name-regex>/
    | nodeFunc
    | (nodeSpec)

nodeFunc :=
    @role(<dimension-name>, <role-name>)
Node Specifier Resolver
  • resolveNodeSpecifier shows the set of nodes represented by the given input.

OSPF Interface Property Specifier

A specification for a set of OSPF interface properties.

An OSPF interface property specifier follows the enum set grammar over the following values: OSPF_AREA_NAME, OSPF_COST, OSPF_ENABLED, OSPF_PASSIVE, OSPF_NETWORK_TYPE.

OSPF Process Property Specifier

A specification for a set of OSPF process properties.

An OSPF process property specifier follows the enum set grammar over the following values: AREA_BORDER_ROUTER, AREAS, EXPORT_POLICY_SOURCES, REFERENCE_BANDWIDTH, RFC_1583_COMPATIBLE, ROUTER_ID.

OSPF Session Status Specifier

A specification for a set of OSPF session statuses.

An OSPF session status specifier follows the enum set grammar over the following values: AREA_INVALID, AREA_MISMATCH, AREA_TYPE_MISMATCH, DEAD_INTERVAL_MISMATCH, DUPLICATE_ROUTER_ID, ESTABLISHED, HELLO_INTERVAL_MISMATCH, MTU_MISMATCH, NETWORK_TYPE_MISMATCH, NO_SESSION, PASSIVE_MISMATCH, PROCESS_INVALID, UNKNOWN_COMPATIBILITY_ISSUE.

Routing Protocol Specifier

A specification for a set of routing protocols.

The routing protocol specifier grammar follows the enum set grammar over protocol names. The set of names include most-specific protocols such as OSPF-INTRA and logical names that denote multiple specific protocols. The logical name ALL denotes all protocols. The full hierarchy of names is:

ALL

  • IGP

    • OSPF

      • OSPF-INT

        • OSPF-INTRA

        • OSPF-INTER

      • OSPF-EXT

        • OSPF-EXT1

        • OSPF-EXT2

    • ISIS

      • ISIS-L1

      • ISIS-L2

    • EIGRP

      • EIGRP-INT

      • EIGRP-EXT

    • RIP

  • BGP

    • EBGP

    • IBGP

  • AGGREGATE

  • STATIC

  • LOCAL

  • CONNECTED

Routing Policy Specifier

A specification for routing policies in the network.

  • routingPolicy1 includes routing policies on all nodes with that name.

  • /^rtpol/ includes all routing policies (on all nodes) whose names match the regex ‘^rtpol’, i.e., start wtih ‘rtpol’.

Routing Policy Grammar
routingPolicySpec :=
    routingPolicyTerm [(&|,|\) routingPolicyTerm]

routingPolicyTerm :=
    <routing-policy-name>
    | /<routing-policy-name-regex>/
    | (routingPolicySpec)

VXLAN VNI Property Specifier

A specification for a set of VXLAN VNI properties.

A VXLAN VNI property specifier follows the enum set grammar over the following values: LOCAL_VTEP_IP, MULTICAST_GROUP, VLAN, VNI, VTEP_FLOOD_LIST, VXLAN_PORT.

Assertion helpers

Utility assert functions for writing network tests (or policies).

All assert_* methods will raise an BatfishAssertException if the assertion fails.

pybatfish.client.asserts.assert_filter_has_no_unreachable_lines(filters, soft=False, snapshot=None, session=None, df_format='table')[source]

Check that a filter (e.g. an ACL) has no unreachable lines.

A filter line is considered unreachable if it will never match a packet, e.g., because its match condition is empty or covered completely by those of prior lines.”

Parameters
  • filters – the specification for the filter (filterSpec) to check

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • snapshot – the snapshot on which to check the assertion

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

Returns

True if the assertion passes

pybatfish.client.asserts.assert_filter_denies(filters, headers, startLocation=None, soft=False, snapshot=None, session=None, df_format='table')[source]

Check if a filter (e.g., ACL) denies a specified set of flows.

Parameters
  • filters – the specification for the filter (filterSpec) to check

  • headersHeaderConstraints

  • startLocation – LocationSpec indicating where a flow starts

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • snapshot – the snapshot on which to check the assertion

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

Returns

True if the assertion passes

pybatfish.client.asserts.assert_filter_permits(filters, headers, startLocation=None, soft=False, snapshot=None, session=None, df_format='table')[source]

Check if a filter (e.g., ACL) permits a specified set of flows.

Parameters
  • filters – the specification for the filter (filterSpec) to check

  • headersHeaderConstraints

  • startLocation – LocationSpec indicating where a flow starts

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • snapshot – the snapshot on which to check the assertion

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

Returns

True if the assertion passes

pybatfish.client.asserts.assert_flows_fail(startLocation, headers, soft=False, snapshot=None, session=None, df_format='table')[source]

Check if the specified set of flows, denoted by starting locations and headers, fail.

Parameters
  • startLocation – LocationSpec indicating where the flow starts

  • headersHeaderConstraints

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • snapshot – the snapshot on which to check the assertion

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

Returns

True if the assertion passes

pybatfish.client.asserts.assert_flows_succeed(startLocation, headers, soft=False, snapshot=None, session=None, df_format='table')[source]

Check if the specified set of flows, denoted by starting locations and headers, succeed.

Parameters
  • startLocation – LocationSpec indicating where the flow starts

  • headersHeaderConstraints

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • snapshot – the snapshot on which to check the assertion

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

Returns

True if the assertion passes

pybatfish.client.asserts.assert_has_no_route(routes: Union[pandas.core.frame.DataFrame, Dict[str, Dict[str, List[Dict[str, Any]]]]], expected_route: Dict[str, Any], node: str, vrf: str = 'default', soft: bool = False) → bool[source]

Assert that a particular route is NOT present.

Note

If a node or VRF is missing in the route answer the assertion will NOT fail, but a warning will be generated.

Parameters
  • routes (Union[DataFrame, Dict[str, Dict[str, List[Dict[str, any]]]]]) – Routes returned by the Batfish routes or ribs questions, either as a Pandas DataFrame or a multilevel dictionary from hostname to VRF name to list of routes, where each route is a dictionary.

  • expected_route (Dict[str, any]) – A dictionary describing route to match.

  • node (str) – node hostname on which to look for a route.

  • vrf – VRF name where the route should be present. Default is default.

  • soft (bool) – whether this assertion is soft (i.e., generates a warning but not a failure)

pybatfish.client.asserts.assert_has_route(routes: Union[pandas.core.frame.DataFrame, Dict[str, Dict[str, List[Dict[str, Any]]]]], expected_route: Dict[str, Any], node: str, vrf: str = 'default', soft: bool = False) → bool[source]

Assert that a particular route is present.

Parameters
  • routes (Union[DataFrame, Dict[str, Dict[str, List[Dict[str, any]]]]]) – Routes returned by the Batfish routes or ribs questions, either as a Pandas DataFrame or a multilevel dictionary from hostname to VRF name to list of routes, where each route is a dictionary.

  • expected_route (Dict[str, any]) – A dictionary describing route to match.

  • node (str) – node hostname on which to look for a route.

  • vrf – VRF name where the route should be present. Default is default.

  • soft (bool) – whether this assertion is soft (i.e., generates a warning but not a failure)

pybatfish.client.asserts.assert_no_duplicate_router_ids(snapshot: Optional[str] = None, nodes: Optional[str] = None, protocols: Optional[List[str]] = None, soft: bool = False, session: Optional[Session] = None, df_format: str = 'table', ignore_same_node: bool = False) → bool[source]

Assert that there are no duplicate router IDs present in the snapshot.

Parameters
  • snapshot – the snapshot on which to check the assertion

  • protocols – the protocols on which to run the assertion, currently BGP and OSPF are supported

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

  • ignore_same_node – whether to ignore duplicate router-ids on the same node

pybatfish.client.asserts.assert_no_forwarding_loops(snapshot=None, soft=False, session=None, df_format='table')[source]

Assert that there are no forwarding loops in the snapshot.

Parameters
  • snapshot – the snapshot on which to check the assertion

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

pybatfish.client.asserts.assert_no_incompatible_bgp_sessions(nodes=None, remote_nodes=None, status=None, snapshot=None, soft=False, session=None, df_format='table')[source]

Assert that there are no incompatible BGP sessions present in the snapshot.

Parameters
  • nodes – search sessions with specified nodes on one side of the sessions.

  • remote_nodes – search sessions with specified remote_nodes on other side of the sessions.

  • status – select sessions matching the specified BGP session status specifier, if none is specified then all statuses other than UNIQUE_MATCH, DYNAMIC_MATCH, and UNKNOWN_REMOTE are selected.

  • snapshot – the snapshot on which to check the assertion

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

pybatfish.client.asserts.assert_no_incompatible_ospf_sessions(nodes=None, remote_nodes=None, snapshot=None, soft=False, session=None, df_format='table')[source]

Assert that there are no incompatible or unestablished OSPF sessions present in the snapshot.

Parameters
  • nodes – search sessions with specified nodes on one side of the sessions.

  • remote_nodes – search sessions with specified remote_nodes on other side of the sessions.

  • snapshot – the snapshot on which to check the assertion

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

pybatfish.client.asserts.assert_no_unestablished_bgp_sessions(nodes=None, remote_nodes=None, snapshot=None, soft=False, session=None, df_format='table')[source]

Assert that there are no BGP sessions that are compatible but not established.

This assertion is run (only) for sessions that are compatible based on configuration settings and it will fail if any such session is not established because of routing or forwarding problems. To find sessions that are incompatible you may run the assert_no_incompatible_bgp_sessions assertion.

Parameters
  • nodes – search sessions with specified nodes on one side of the sessions.

  • remote_nodes – search sessions with specified remote_nodes on other side of the sessions.

  • snapshot – the snapshot on which to check the assertion

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

pybatfish.client.asserts.assert_no_undefined_references(snapshot=None, soft=False, session=None, df_format='table')[source]

Assert that there are no undefined references present in the snapshot.

Parameters
  • snapshot – the snapshot on which to check the assertion

  • soft – whether this assertion is soft (i.e., generates a warning but not a failure)

  • session – Batfish session to use for the assertion

  • df_format – How to format the Dataframe content in the output message. Valid options are ‘table’ and ‘records’ (each row is a key-value pairs).

pybatfish.client.asserts.assert_num_results(answer, num, soft=False)[source]

Assert an exact number of results were returned.

Parameters
  • answer – Batfish answer or DataFrame

  • num (int) – expected number of results

  • soft (bool) – whether this assertion is soft (i.e., generates a warning but not a failure)

pybatfish.client.asserts.assert_zero_results(answer, soft=False)[source]

Assert no results were returned.

Parameters
  • answer – Batfish answer or DataFrame

  • soft (bool) – whether this assertion is soft (i.e., generates a warning but not a failure)

Using Reference Books

Reference book allows users to create groups of information that can be used in Batfish queries. They can contain:

Name

Description

Type

name

Name of the reference book

str

addressGroup

A list of addressGroups

List of addressGroup

interfaceGroup

A list of interfaceGroups

List of interfaceGroup

Example usage

Create a reference book for information about border routers and add an InterfaceGroup for the as2 border interfaces

[4]:
interfacegroup = [InterfaceGroup('as2-border',
                                 interfaces = [Interface('as2border1','GigabitEthernet2/0'),
                                               Interface('as2border2', 'GigabitEthernet2/0')])]
refbook = ReferenceBook(name='border', interfaceGroups= interfacegroup)

bf.put_reference_book(refbook)

Example usage

Add an AddressGroup for the IP addresses of all as2 border interfaces to the ReferenceBook('border')

[5]:
refbook = bf.get_reference_book('border')
addressGroup = [
    AddressGroup(name='as2-border', addresses = ['2.12.21.1', '2.12.12.1'])
]
[6]:
refbook.addressGroups.extend(addressGroup)
bf.put_reference_book(refbook)
[7]:
bf.get_reference_book('border')
[7]:
ReferenceBook(name='border', addressGroups=[AddressGroup(name='as2-border', addresses=['2.12.12.1', '2.12.21.1'], childGroupNames=[])], interfaceGroups=[InterfaceGroup(name='as2-border', interfaces=[Interface(hostname='as2border1', interface='GigabitEthernet2/0'), Interface(hostname='as2border2', interface='GigabitEthernet2/0')])])

Configuration file annotation

Batfish may ignore certain lines, or portions of a line, in your configuration files. Such lines fall in one of these categories:

  1. Batfish does not understand the content of the line (unrecognized syntax)

  2. Batfish does not support the feature mentioned in the line

  3. Batfish deems that the line is irrelevant to its network model

The first two categories of ignored lines are reported in the initIssues question; the third category is not.

If you wanted a full account of what is ignored, we have created a tool that annotates each line in the config file. The annotation is one of the following, respectively, corresponding to the categories above

  1. UNRECOGNIZED SYNTAX

  2. PARTIALLY UNSUPPORTED

  3. SILENTLY IGNORED

In each case, the annotation is followed by the part of the line to which the annotation applies.

Using this tool requires building Batfish from source (i.e., not pre-built Docker images). If you are using Bazel, you can run this tool as

bazel run //tools:annotate <input dir> <output dir>

It will read all files in the input dir and output an annotated version in the output dir. The annotations mention which lines are being fully or partially ignored by the analysis. No annotation is added for lines that are fully supported.

Example output for a file:

!
! SILENTLY IGNORED: version 15.2
version 15.2
!
hostname as1border1
!
! SILENTLY IGNORED: boot-start-marker
boot-start-marker
! SILENTLY IGNORED: boot-end-marker
boot-end-marker
!
no aaa new-model
! SILENTLY IGNORED: no ip icmp rate-limit unreachable
no ip icmp rate-limit unreachable
! SILENTLY IGNORED: ip cef
ip cef
!
!
no ip domain lookup
ip domain name lab.local
!
...
...

Batfish Commands

Here we describe the non-question related Batfish functions

Networks

pybatfish.client.session.Session.set_network(self, name: Optional[str] = None, prefix: str = 'pcp') → str

Configure the network used for analysis.

Parameters
  • name (str) – name of the network to set. If None, a name will be generated

  • prefix – prefix to prepend to auto-generated network names if name is empty

Returns

name of the configured network

Return type

str

Raises

BatfishException – if configuration fails

pybatfish.client.session.Session.list_networks(self)

List networks the session’s API key can access.

Returns

network names

Return type

list

pybatfish.client.session.Session.delete_network(self, name)

Delete network by name.

Parameters

name (str) – name of the network to delete

Snapshots

pybatfish.client.session.Session.init_snapshot(self, upload, name=None, overwrite=False, extra_args=None)

Initialize a new snapshot.

Parameters
  • upload (str) – path to the snapshot zip or directory

  • name (str) – name of the snapshot to initialize

  • overwrite (bool) – whether or not to overwrite an existing snapshot with the same name

  • extra_args (dict) – extra arguments to control snapshot processing: 1) “ignoremanagementinterfaces” (bool) – whether to shut management interfaces (default is True); 2) “parsereuse” (bool) – whether to reuse parsing work from prior snapshots when file content is identical (default is True)

Returns

name of initialized snapshot

Return type

str

pybatfish.client.session.Session.set_snapshot(self, name=None, index=None)

Set the current snapshot by name or index.

Parameters
  • name (str) – name of the snapshot to set as the current snapshot

  • index (int) – set the current snapshot to the index-th most recent snapshot

Returns

the name of the successfully set snapshot

Return type

str

pybatfish.client.session.Session.list_snapshots(self, verbose=False)

List snapshots for the current network.

Parameters

verbose (bool) – If true, return the full output of Batfish, including snapshot metadata.

Returns

snapshot names or the full JSON response containing snapshots and metadata (if verbose=True)

Return type

list

pybatfish.client.session.Session.delete_snapshot(self, name)

Delete specified snapshot from current network.

Parameters

name (str) – name of the snapshot to delete

pybatfish.client.session.Session.fork_snapshot(self, base_name, name=None, overwrite=False, deactivate_interfaces=None, deactivate_nodes=None, restore_interfaces=None, restore_nodes=None, add_files=None, extra_args=None)

Copy an existing snapshot and deactivate or reactivate specified interfaces, nodes, and links on the copy.

Parameters
  • base_name (str) – name of the snapshot to copy

  • name (str) – name of the snapshot to initialize

  • overwrite (bool) – whether or not to overwrite an existing snapshot with the same name

  • deactivate_interfaces (list[Interface]) – list of interfaces to deactivate in new snapshot

  • deactivate_nodes (list[str]) – list of names of nodes to deactivate in new snapshot

  • restore_interfaces (list[Interface]) – list of interfaces to reactivate

  • restore_nodes (list[str]) – list of names of nodes to reactivate

  • add_files (str) – path to zip file or directory containing files to add

  • extra_args (dict) – extra arguments to control snapshot processing: 1) “ignoremanagementinterfaces” (bool) – whether to shut management interfaces (default is True); 2) “parsereuse” (bool) – whether to reuse parsing work from prior snapshots when file content is identical (default is True)

Returns

name of initialized snapshot or None if the call fails

Return type

Optional[str]

Reference Library

pybatfish.client.session.Session.get_reference_library(self)

Returns the reference library for the active network.

pybatfish.client.session.Session.get_reference_book(self, name)

Returns the specified reference book for the active network.

Parameters

name (str) – name of the reference book to fetch

pybatfish.client.session.Session.put_reference_book(self, book)

Put a reference book in the active network.

If a book with the same name exists, it is overwritten.

Parameters

book (ReferenceBook) – The ReferenceBook object to add

pybatfish.client.session.Session.delete_reference_book(self, name)

Deletes the reference book with the specified name for the active network.

Parameters

name (str) – name of the reference book to delete

Diagnostics

pybatfish.client.session.Session.upload_diagnostics(self, dry_run: bool = True, netconan_config: Optional[str] = None, contact_info: Optional[str] = None, proxy: Optional[str] = None) → str

Fetch, anonymize, and optionally upload snapshot diagnostics information.

This runs a series of diagnostic questions on the current snapshot (including collecting parsing and conversion information).

The information collected is anonymized with Netconan which either anonymizes passwords and IP addresses (default) or uses the settings in the provided netconan_config.

The anonymous information is then either saved locally (if dry_run is True) or uploaded to Batfish developers (if dry_run is False). The uploaded information will be accessible only to Batfish developers and will be used to help diagnose any issues you encounter.

If contact_info is supplied (e.g. email address), Batfish developers may contact you if they have follow-up questions or to update you when the issues you encountered are resolved.

Parameters
  • dry_run (bool) – if True, upload is skipped and the anonymized files will be stored locally for review. If False, anonymized files will be uploaded to the Batfish developers

  • netconan_config (str) – path to Netconan configuration file

  • contact_info (str) – optional contact info associated with this upload

  • proxy – a proxy URL to use when uploading data.

Returns

location of anonymized files (local directory if doing dry run, otherwise upload ID)

Return type

str

Datamodel classes

Here we describe classes used in answers and their attributes, which may help you filter your answers as desired.

Base Types

class pybatfish.datamodel.primitives.Edge(node1: str, node1interface, node2: str, node2interface)[source]

A network edge (i.e., a link between two node/interface pairs).

Variables
  • node1 – First node name

  • node1interface – First node’s interface name

  • node2 – Second node name

  • node2interface – Second node’s interface name

class pybatfish.datamodel.primitives.Interface(hostname: str, interface: str)[source]

A network interface — a combination of node and interface names.

Variables
  • hostname – Node hostname to which this interface belongs

  • interface – Interface name

ACL Traces

class pybatfish.datamodel.acl.AclTrace(events: List[pybatfish.datamodel.acl.AclTraceEvent] = NOTHING)[source]

The trace of a packet’s life through an ACL.

Variables

events – A list of AclTraceEvent

class pybatfish.datamodel.acl.AclTraceEvent(description: Optional[str] = None)[source]

One event corresponding to a packet’s life through an ACL.

Variables

description – The description of the event

class pybatfish.datamodel.acl.Fragment[source]

An element in TraceElement.fragments, can be one of TextFragment or LinkFragment.

class pybatfish.datamodel.acl.LinkFragment(text: str, vendorStructureId: pybatfish.datamodel.acl.VendorStructureId)[source]

Represents a Fragment that links to a vendor structure.

Variables
  • text – Text content of the fragment

  • vendorStructureId – Link of the fragment

class pybatfish.datamodel.acl.TextFragment(text: str)[source]

Represents a plain-text Fragment.

Variables

text – Text content of the fragment

class pybatfish.datamodel.acl.TraceElement(fragments: List[pybatfish.datamodel.acl.Fragment])[source]

Metadata used to create human-readable traces.

Variables

fragments – A list of Fragment which describes an element of a trace

class pybatfish.datamodel.acl.TraceTree(traceElement: pybatfish.datamodel.acl.TraceElement, children: List[TraceTree])[source]

Represents a filter trace tree.

Variables
  • traceElement – Metadata and description of the node

  • children – A list of sub-traces, i.e. children of the node

class pybatfish.datamodel.acl.VendorStructureId(filename: str, structureType: str, structureName: str)[source]

Identifies a vendor structure in a configuration file.

Variables
  • filename – Filename of the configuration file

  • structureType – Type of the vendor structure

  • structureName – Name of the vendor structure

Flows and Packets

class pybatfish.datamodel.flow.ArpErrorStepDetail(outputInterface: Optional[str], resolvedNexthopIp: Optional[str])[source]

Details of a step representing the arp error of a flow when sending out of a Hop.

Variables
  • outputInterface – Interface of the Hop from which the flow exits

  • resolvedNexthopIp – Resolve next hop Ip address

class pybatfish.datamodel.flow.DelegatedToNextVrf(nextVrf: str, type: str = 'DelegatedToNextVrf')[source]

A flow being delegated to a different VRF for further processing.

class pybatfish.datamodel.flow.DeliveredStepDetail(outputInterface: Optional[str], resolvedNexthopIp: Optional[str])[source]

Details of a step representing the flow is delivered or exiting the network.

Variables
  • outputInterface – Interface of the Hop from which the flow exits

  • resolvedNexthopIp – Resolve next hop Ip address

class pybatfish.datamodel.flow.Discarded(type: str = 'Discarded')[source]

A flow being discarded.

class pybatfish.datamodel.flow.EnterInputIfaceStepDetail(inputInterface: str, inputVrf: Optional[str])[source]

Details of a step representing the entering of a flow into a Hop.

Variables
  • inputInterface – Interface of the Hop on which this flow enters

  • inputVrf – VRF associated with the input interface

class pybatfish.datamodel.flow.ExitOutputIfaceStepDetail(outputInterface: str, transformedFlow: Optional[str])[source]

Details of a step representing the exiting of a flow out of a Hop.

Variables
  • outputInterface – Interface of the Hop from which the flow exits

  • transformedFlow – Transformed Flow if a source NAT was applied on the Flow

class pybatfish.datamodel.flow.FilterStepDetail(filter: str, filterType: str, inputInterface: str, flow: Optional[pybatfish.datamodel.flow.Flow])[source]

Details of a step representing a filter step.

Variables
  • filter – filter name

  • type – filter type

  • inputInterface – input interface of the flow

  • flow – current flow

class pybatfish.datamodel.flow.ForwardedIntoVxlanTunnel(vni: int, vtep: str, type: str = 'ForwardedIntoVxlanTunnel')[source]

A flow being forwarded into a VXLAN tunnel.

class pybatfish.datamodel.flow.ForwardedOutInterface(outputInterface: str, resolvedNextHopIp: Optional[str] = None, type: str = 'ForwardedOutInterface')[source]

A flow being forwarded out an interface.

If there is no resolved next-hop IP and this is the final step on this node, the destination IP of the flow will be used as the next gateway IP.

class pybatfish.datamodel.flow.Flow(dscp, dstIp, dstPort, ecn, fragmentOffset, icmpCode, icmpVar, ingressInterface: Optional[str], ingressNode: Optional[str], ingressVrf: Optional[str], ipProtocol: str, packetLength: str, srcIp, srcPort, tcpFlagsAck, tcpFlagsCwr, tcpFlagsEce, tcpFlagsFin, tcpFlagsPsh, tcpFlagsRst, tcpFlagsSyn, tcpFlagsUrg)[source]

A concrete IPv4 flow.

Noteworthy attributes for flow inspection/filtering:

Variables
  • srcIP – Source IP of the flow

  • dstIP – Destination IP of the flow

  • srcPort – Source port of the flow

  • dstPort – Destination port of the flow

  • ipProtocol – the IP protocol of the flow either as its name (e.g., TCP) for well-known protocols or a string like UNNAMED_168

  • ingressNode – the node where the flow started (or entered the network)

  • ingressInterface – the interface name where the flow started (or entered the network)

  • ingressVrf – the VRF name where the flow started (or entered the network)

get_flag_str()[source]

Returns a print friendly version of all set TCP flags.

get_ip_protocol_str()[source]

Returns a print-friendly version of IP protocol and any protocol-specific information (e.g., flags for TCP, type/code for ICMP.

class pybatfish.datamodel.flow.HeaderConstraints(srcIps: Optional[str] = None, dstIps: Optional[str] = None, srcPorts=None, dstPorts=None, ipProtocols=None, applications=None, icmpCodes=None, icmpTypes=None, ecns=None, dscps=None, packetLengths=None, fragmentOffsets=None, tcpFlags=None)[source]

Constraints on an IPv4 packet header space.

Specify constraints on packet headers by specifying lists of allowed values in each field of IP packet.

Variables
  • srcIps (str) – Source location/IP

  • dstIps (str) – Destination location/IP

  • srcPorts – Source ports as list of ranges (e.g., "22,53-99")

  • dstPorts – Destination ports as list of ranges, (e.g., "22,53-99")

  • applications – Shorthands for application protocols (e.g., SSH, DNS, SNMP)

  • ipProtocols – List of well-known IP protocols (e.g., TCP, UDP, ICMP)

  • icmpCodes – List of integer ICMP codes

  • icmpTypes – List of integer ICMP types

  • dscps – List of allowed DSCP value ranges

  • ecns – List of allowed ECN values ranges

  • packetLengths – List of allowed packet length value ranges

  • fragmentOffsets – List of allowed fragmentOffset value ranges

  • tcpFlags – List of MatchTcpFlags – conditions on which TCP flags to match

Lists of values in each fields are subject to a logical “OR”:

>>> HeaderConstraints(ipProtocols=["TCP", "UDP"])
HeaderConstraints(srcIps=None, dstIps=None, srcPorts=None, dstPorts=None, ipProtocols=['TCP', 'UDP'], applications=None,
icmpCodes=None, icmpTypes=None, ecns=None, dscps=None, packetLengths=None, fragmentOffsets=None, tcpFlags=None)

means allow TCP OR UDP.

Different fields are ANDed together:

>>> HeaderConstraints(srcIps="1.1.1.1", dstIps="2.2.2.2", applications=["SSH"])
HeaderConstraints(srcIps='1.1.1.1', dstIps='2.2.2.2', srcPorts=None, dstPorts=None, ipProtocols=None, applications=['SSH'],
icmpCodes=None, icmpTypes=None, ecns=None, dscps=None, packetLengths=None, fragmentOffsets=None, tcpFlags=None)

means an SSH connection originating at 1.1.1.1 and going to 2.2.2.2

Any None values will be treated as unconstrained.

classmethod of(flow)[source]

Create header constraints from an existing flow.

class pybatfish.datamodel.flow.Hop(node: str, steps: List[pybatfish.datamodel.flow.Step])[source]

A single hop in a flow trace.

Variables
  • node – Name of node considered as the Hop

  • steps – List of steps taken at this Hop

class pybatfish.datamodel.flow.InboundStepDetail(interface: str)[source]

Details of a step representing the receiving (acceptance) of a flow into a Hop.

Variables

interface – interface that owns the destination IP

class pybatfish.datamodel.flow.MatchSessionStepDetail(sessionScope: pybatfish.datamodel.flow.SessionScope, sessionAction: pybatfish.datamodel.flow.SessionAction, matchCriteria: pybatfish.datamodel.flow.SessionMatchExpr, transformation: Optional[List[pybatfish.datamodel.flow.FlowDiff]] = NOTHING)[source]

Details of a step for when a flow matches a firewall session.

Variables
  • sessionScope – Scope of flows session can match (incoming interfaces or originating VRF)

  • sessionAction – A SessionAction that the firewall takes for a matching session

  • matchCriteria – A SessionMatchExpr that describes the match criteria of the session

  • transformation – List of FlowDiffs that will be applied after session match

class pybatfish.datamodel.flow.MatchTcpFlags(tcpFlags: pybatfish.datamodel.flow.TcpFlags, useAck: bool = True, useCwr: bool = True, useEce: bool = True, useFin: bool = True, usePsh: bool = True, useRst: bool = True, useSyn: bool = True, useUrg: bool = True)[source]

Match given TcpFlags.

For each bit in the TCP flags, a useX must be set to true, otherwise the bit is treated as “don’t care”.

Variables
  • tcpFlags – tcp flags to match

  • useAck

  • useCwr

  • useEce

  • useFin

  • usePsh

  • useRst

  • useSyn

  • useUrg

static match_ack()[source]

Return match conditions checking that ACK bit is set.

Other bits may take any value.

static match_established()[source]

Return a list of match conditions matching an established flow (ACK or RST bit set).

Other bits may take any value.

static match_not_established()[source]

Return a list of match conditions matching a non-established flow.

Meaning both ACK and RST bits are unset. Other bits may take any value.

static match_rst()[source]

Return match conditions checking that RST bit is set.

Other bits may take any value.

static match_syn()[source]

Return match conditions checking that the SYN bit is set.

Other bits may take any value.

static match_synack()[source]

Return match conditions checking that both the SYN and ACK bits are set.

Other bits may take any value.

class pybatfish.datamodel.flow.OriginateStepDetail(originatingVrf: str)[source]

Details of a step representing the originating of a flow in a Hop.

Variables

originatingVrf – VRF from which the Flow originates

class pybatfish.datamodel.flow.RoutingStepDetail(routes: List[pybatfish.datamodel.flow.RouteInfo], forwardingDetail: Optional[pybatfish.datamodel.flow.ForwardingDetail], arpIp: Optional[str], outputInterface: Optional[str])[source]

Details of a step representing the routing from input interface to output interface.

Variables

routes – List of routes which were considered to select the forwarding action

class pybatfish.datamodel.flow.SetupSessionStepDetail(sessionScope: pybatfish.datamodel.flow.SessionScope, sessionAction: pybatfish.datamodel.flow.SessionAction, matchCriteria: pybatfish.datamodel.flow.SessionMatchExpr, transformation: Optional[List[pybatfish.datamodel.flow.FlowDiff]] = NOTHING)[source]

Details of a step for when a firewall session is created.

Variables
  • sessionScope – Scope of flows session can match (incoming interfaces or originating VRF)

  • sessionAction – A SessionAction that the firewall takes for a return traffic matching the session

  • matchCriteria – A SessionMatchExpr that describes the match criteria of the session

  • transformation – List of FlowDiffs that will be applied on the return traffic matching the session

class pybatfish.datamodel.flow.PathConstraints(startLocation: Optional[str] = None, endLocation: Optional[str] = None, transitLocations: Optional[str] = None, forbiddenLocations: Optional[str] = None)[source]

Constraints on the path of a flow.

Variables
  • startLocation – Location specification for where a flow is allowed to start

  • endLocation – Node specification for where a flow is allowed to terminate

  • transitLocations – Node specification for where a flow must transit

  • forbiddenLocations – Node specification for where a flow is not allowed to transit

class pybatfish.datamodel.flow.TcpFlags(ack: bool = False, cwr: bool = False, ece: bool = False, fin: bool = False, psh: bool = False, rst: bool = False, syn: bool = False, urg: bool = False)[source]

Represents a set of TCP flags in a packet.

Variables
  • ack

  • cwr

  • ece

  • fin

  • psh

  • rst

  • syn

  • urg

class pybatfish.datamodel.flow.Trace(disposition: str, hops: List[pybatfish.datamodel.flow.Hop])[source]

A trace of a flow through the network.

A Trace is a combination of hops and flow fate (i.e., disposition).

Variables
  • disposition – Flow disposition

  • hops – A list of hops (Hop) the flow took

class pybatfish.datamodel.flow.TransformationStepDetail(transformationType: str, flowDiffs: List[pybatfish.datamodel.flow.FlowDiff])[source]

Details of a step representation a packet transformation.

Variables
  • transformationType – The type of the transformation

  • flowDiffs – Set of changed flow fields

Reference Library

class pybatfish.datamodel.referencelibrary.AddressGroup(name: str, addresses=NOTHING, childGroupNames=NOTHING)[source]

Information about an address group.

Variables
  • name – The name of the group

  • addresses – a list of ‘addresses’ where each element is a string that represents an IP address (e.g., “1.1.1.1”), prefix (e.g., 1.1.1.0/24), or an address:mask (e.g., “1.1.1.1:0.0.0.8”).

  • childGroupNames – a list of names of child groups in this address group. The child groups must exist in the same reference book. Circular descendant relationships between address groups are allowed. The address group is considered to contain all addresses that are directly in it or in any of its descendants.

class pybatfish.datamodel.referencelibrary.InterfaceGroup(name: str, interfaces=NOTHING)[source]

Information about an interface group.

Variables
  • name – The name of the group

  • interfaces – a list of interfaces, of type Interface.

class pybatfish.datamodel.referencelibrary.NodeRole(name: str, regex: str)[source]

Information about a node role.

Variables
  • name – Name of the node role.

  • regex – A regular expression over node names to describe nodes that belong to this role. The regular expression must be a valid Java regex.

class pybatfish.datamodel.referencelibrary.NodeRoleDimension(name: str, roles=NOTHING, roleDimensionMappings=NOTHING)[source]

Information about a node role dimension.

Variables
  • name – Name of the node role dimension.

  • roles – The list of NodeRole objects in this dimension (deprecated).

  • roleDimensionMappings – The list of RoleDimensionMapping objects in this dimension.

class pybatfish.datamodel.referencelibrary.NodeRolesData(defaultDimension: Optional[str] = None, roleDimensionOrder=NOTHING, roleMappings=NOTHING)[source]

Information about a node roles data.

:ivar defaultDimension :ivar roleDimensionOrder: The precedence order of role dimensions. :ivar roleMappings: A list of RoleMapping objects

class pybatfish.datamodel.referencelibrary.ReferenceBook(name: str, addressGroups=NOTHING, interfaceGroups=NOTHING)[source]

Information about a reference book.

Variables
  • name – Name of the reference book.

  • addressGroups – A list of groups, of type AddressGroup.

  • interfaceGroups – A list of groups, of type InterfaceGroup.

class pybatfish.datamodel.referencelibrary.ReferenceLibrary(books=NOTHING)[source]

Information about a reference library.

Variables

books – A list of books of type ReferenceBook.

class pybatfish.datamodel.referencelibrary.RoleDimensionMapping(regex: str, groups: List[int] = [1], canonicalRoleNames: Dict[str, str] = {})[source]

Information about a role dimension mapping.

Variables
  • regex – A regular expression over node names to describe nodes that belong to this role. The regular expression must be a valid Java regex.

  • groups – A list of group numbers (integers) that identify the role name for a given node name (default value is [1]).

  • canonicalRoleNames – A map from Java regexes over role names determined from the groups to a canonical set of role names for this dimension (default value is {}).

class pybatfish.datamodel.referencelibrary.RoleMapping(name: str, regex: str, roleDimensionGroups: Dict[str, int], canonicalRoleNames: Dict[str, Dict[str, str]] = NOTHING)[source]

A mapping from node name to role dimensions.

Variables
  • name – (Optional) the name of the role mapping

  • regex – A java regex over hostnames, with groups to extract role data

  • roleDimensionGroups – a map from each role dimension name to the list of regex groups that signify the role name for that dimension.

  • canonicalRoleNames – for each role dimension, a map from the default role name that was obtained from the node name to a canonical role name

Routes

class pybatfish.datamodel.route.BgpRoute(network: str, originatorIp: str, originType: str, protocol: str, asPath: list = [], communities: list = [], localPreference: int = 0, metric: int = 0, nextHopIp: str = None, sourceProtocol: str = None, tag: int = 0, weight: int = 0)[source]

A BGP routing advertisement.

Variables
  • network – The network prefix advertised by the route.

  • asPath – The AS path of the route.

  • communities – The communities of the route.

  • localPreference – The local preference of the route.

  • metric – The metric of the route.

  • nextHopIp – The next hop IP of the route.

  • protocol – The protocol of the route.

  • originatorIp – The IP address of the originator of the route.

  • originType – The origin type of the route.

  • sourceProtocol – The source protocol of the route.

  • tag – The tag of the route.

  • weight – The weight of the route.

class pybatfish.datamodel.route.BgpRouteConstraints(prefix=None, complementPrefix: Optional[bool] = None, localPreference=None, med=None, communities=None, asPath=None)[source]

Constraints on a BGP route announcement.

Specify constraints on route announcements by specifying allowed values in each field of the announcement.

Variables
  • prefix – Allowed prefixes as a list of prefix ranges (e.g., “0.0.0.0/0:0-32”)

  • complementPrefix – A flag indicating that all prefixes except the ones in prefix are allowed

  • localPreference – List of allowed local preference integer ranges, as a string

  • med – List of allowed MED integer ranges, as a string

  • communities – List of allowed and disallowed community regexes

  • asPath – List of allowed and disallowed AS-path regexes

class pybatfish.datamodel.route.BgpRouteDiff(fieldName: str, oldValue: str, newValue: str)[source]

A difference between two BGP routes.

Variables
  • fieldName – A Flow field name that has changed.

  • oldValue – The old value of the field.

  • newValue – The new value of the field.

class pybatfish.datamodel.route.BgpRouteDiffs(diffs: List[pybatfish.datamodel.route.BgpRouteDiff])[source]

A set of differences between two BGP routes.

Variables

diffs – The set of BgpRouteDiff objects.

class pybatfish.datamodel.route.NextHop[source]

A next-hop of a route

class pybatfish.datamodel.route.NextHopDiscard(type: str = 'discard')[source]

Indicates the packet should be dropped

class pybatfish.datamodel.route.NextHopInterface(interface: str, ip: Optional[str] = None, type: str = 'interface')[source]

A next-hop of a route with a fixed output interface and optional next gateway IP.

If there is no IP, the destination IP of the packet will be used as the next gateway IP.

class pybatfish.datamodel.route.NextHopIp(ip: str, type: str = 'ip')[source]

A next-hop of a route including the next gateway IP

class pybatfish.datamodel.route.NextHopVrf(vrf: str, type: str = 'vrf')[source]

A next-hop of a route indicating the destination IP should be resolved in another VRF

class pybatfish.datamodel.route.NextHopVtep(vni: int, vtep: str, type: str = 'vtep')[source]

A next-hop of a route indicating the packet should be routed through a VXLAN tunnel

Misc question outputs

class pybatfish.datamodel.primitives.FileLines(filename: str, lines: List[int] = NOTHING)[source]

A class that represents a set of lines in a file.

Variables
  • filename – The filename referenced

  • lines – A list of lines referenced

System Requirements for running Batfish

Batfish can be run on any operating system that supports Docker. The containers are actively tested on Mac OS X and Ubuntu 18.04 LTS.

To get started with the example Jupyter notebooks, all you need is a reasonably capable laptop:

  • Dual core CPU

  • 8 GB RAM

  • 256 GB hard-drive

When you transition to running Batfish on your own network, we recommend a server that at least has:

  • Quad-core CPU with 2 threads per CPU

  • 32 GB RAM

  • 256 GB hard-drive

Supported Network Device and Operating System List

Batfish supports configurations for a large and growing set of (physical and virtual) devices, from vendors including:

  • A10 Networks

  • Arista

  • Amazon Web Services (AWS) constructs

    • Internet Gateways

    • NAT Gateways

    • Network ACLs

    • Security Groups

    • Virtual Private Clouds (VPCs)

    • VPN Gateways

  • Check Point

  • Cisco

    • ASA

    • IOS

    • IOS-XE

    • IOS-XR

    • NX-OS

  • Cumulus

  • F5 BIG-IP

  • Fortinet

  • Free-Range Routing (FRR)

  • iptables (on hosts)

  • Juniper (All JunOS platforms)

    • EX

    • MX

    • PTX

    • QFX

    • SRX

    • T-series

  • Palo Alto Networks

  • SONiC

Batfish has limited support for the following platforms:

  • Aruba

  • Dell Force10

  • Foundry

If you’d like support for additional vendors or currently-unsupported configuration features, let us know via Slack or GitHub issue. We’ll try to add support. Or, you can — we welcome pull requests! :)

Code Examples

Complementing the documentation here, there is a library of Jupyter notebooks (also available directly here) that show you how to use Batfish to analyze your network.

Basics

Getting Started with Batfish

This notebook uses pybatfish, a Python-based SDK for Batfish, to analyze a sample network. It shows how to submit your configurations and other network data for analysis and how to query its vendor-neutral network model. Other notebooks show how to use Batfish for different types of network validation tasks.

Check out a video demo of an earlier version of this notebook here.

Initializing a Network and Snapshot

A network is a logical group of routers and links. It can be your entire network or a subset of it. A snapshot is a collection of information (configuration files, routing data, up/down status of nodes and links) that represent the network state. Snapshots can contain the actual state of the network or candidate states (e.g, those corresponding to a planned change) that you want to analyze.

[1]:
# Import packages
%run startup.py
bf = Session(host="localhost")

SNAPSHOT_PATH below can be updated to point to a custom snapshot directory, see the Batfish instructions for how to package data for analysis. More example networks are available in the networks folder of the Batfish repository.

[2]:
# Assign a friendly name to your network and snapshot
NETWORK_NAME = "example_network"
SNAPSHOT_NAME = "example_snapshot"

SNAPSHOT_PATH = "networks/example"

# Now create the network and initialize the snapshot
bf.set_network(NETWORK_NAME)
bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
[2]:
'example_snapshot'

If you used the example we provided, the network you initialized above is illustrated below. You can download/view devices’ configuration files here.

example-network


Querying the Network Model

Batfish creates a comprehensive vendor-neutral device and network model which can be queried for information about devices, interfaces, VRFs, routes, etc. It offers a set of questions to query this model.

[3]:
# You can also use tab-completion on the Batfish question module - bf.q. -> press TAB key,
# uncomment and try on the following line
# bf.q.

# In IPython and Jupyter you can use the "?" shorthand to get help on a question
?bf.q.nodeProperties

# help(bf.q.nodeProperties) # in standard Python console

Getting status of parsed files

Batfish may ignore certain lines in the configuration. To retrieve the parsing status of snapshot files, use the fileParseStatus() question.

[4]:
parse_status = bf.q.fileParseStatus().answer().frame()

answer() runs the question and returns the answer in a JSON format.

frame() wraps the answer as pandas dataframe.

[5]:
# View the parse status results
parse_status
[5]:
File_Name Status File_Format Nodes
0 configs/as1border1.cfg PASSED CISCO_IOS ['as1border1']
1 configs/as1border2.cfg PASSED CISCO_IOS ['as1border2']
2 configs/as1core1.cfg PASSED CISCO_IOS ['as1core1']
3 configs/as2border1.cfg PASSED CISCO_IOS ['as2border1']
4 configs/as2border2.cfg PASSED CISCO_IOS ['as2border2']
5 configs/as2core1.cfg PASSED CISCO_IOS ['as2core1']
6 configs/as2core2.cfg PASSED CISCO_IOS ['as2core2']
7 configs/as2dept1.cfg PASSED CISCO_IOS ['as2dept1']
8 configs/as2dist1.cfg PASSED CISCO_IOS ['as2dist1']
9 configs/as2dist2.cfg PASSED CISCO_IOS ['as2dist2']
10 configs/as3border1.cfg PASSED CISCO_IOS ['as3border1']
11 configs/as3border2.cfg PASSED CISCO_IOS ['as3border2']
12 configs/as3core1.cfg PASSED CISCO_IOS ['as3core1']
13 hosts/host1.json PASSED HOST ['host1']
14 hosts/host2.json PASSED HOST ['host2']
15 iptables/host1.iptables PASSED IPTABLES ['iptables/host1.iptables']
16 iptables/host2.iptables PASSED IPTABLES ['iptables/host2.iptables']

Additional post-processing can be done on this data, like filtering for values in one or multiple columns, reducing the number of columns, etc. using pandas. We show a few examples of Pandas filtering below, some more filtering examples for Batfish answers are here, and a general tutorial is here.

[6]:
# An example: use a filter on the returned dataframe to see which files failed to parse completely
parse_status[parse_status['Status'] != 'PASSED']  # change '!=' to '==' to get the files which passed
[6]:
File_Name Status File_Format Nodes
[7]:
# View details if some of the files were not parsed completely
bf.q.parseWarning().answer().frame()
[7]:
Filename Line Text Parser_Context Comment

Extracting properties of network entities

Entities in the network refer to things like nodes, interfaces, routing processes, and VRFs. Batfish makes it trivial to extract configured properties of such entities in a vendor neutral manner.

Node properties

The nodeProperties question extracts information on nodes in the snapshot.

[8]:
# Extract the properties of all nodes whose names contain 'border'
node_properties = bf.q.nodeProperties(nodes="/border/").answer().frame()
[9]:
# View what columns (properties) are present in the answer
node_properties.columns
[9]:
Index(['Node', 'AS_Path_Access_Lists', 'Authentication_Key_Chains',
       'Community_Match_Exprs', 'Community_Set_Exprs',
       'Community_Set_Match_Exprs', 'Community_Sets', 'Configuration_Format',
       'DNS_Servers', 'DNS_Source_Interface', 'Default_Cross_Zone_Action',
       'Default_Inbound_Action', 'Domain_Name', 'Hostname', 'IKE_Phase1_Keys',
       'IKE_Phase1_Policies', 'IKE_Phase1_Proposals', 'IP6_Access_Lists',
       'IP_Access_Lists', 'IPsec_Peer_Configs', 'IPsec_Phase2_Policies',
       'IPsec_Phase2_Proposals', 'Interfaces', 'Logging_Servers',
       'Logging_Source_Interface', 'NTP_Servers', 'NTP_Source_Interface',
       'PBR_Policies', 'Route6_Filter_Lists', 'Route_Filter_Lists',
       'Routing_Policies', 'SNMP_Source_Interface', 'SNMP_Trap_Servers',
       'TACACS_Servers', 'TACACS_Source_Interface', 'VRFs', 'Zones'],
      dtype='object')
[10]:
# To extract only a subset of properties, use the properties parameter
bf.q.nodeProperties(nodes="/border/", properties="Domain_Name,NTP_Servers,Interfaces").answer().frame()
[10]:
Node Domain_Name Interfaces NTP_Servers
0 as2border2 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] ['18.18.18.18']
1 as3border1 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] ['18.18.18.18', '23.23.23.23']
2 as3border2 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] ['18.18.18.18', '23.23.23.23']
3 as1border2 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] ['18.18.18.18', '23.23.23.23']
4 as2border1 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'GigabitEthernet2/0', 'Loopback0'] ['18.18.18.18', '23.23.23.23']
5 as1border1 lab.local ['Ethernet0/0', 'GigabitEthernet0/0', 'GigabitEthernet1/0', 'Loopback0'] []
Interface properties

To retrieve information about interfaces and the properties of them, use the interfaceProperties question

[11]:
# Fetch specific properties of Loopback interfaces
bf.q.interfaceProperties(interfaces="/loopback/", properties="Bandwidth,VRF,Primary_Address").answer().frame()
[11]:
Interface Bandwidth Primary_Address VRF
0 as1border1[Loopback0] 8e+09 1.1.1.1/32 default
1 as1border2[Loopback0] 8e+09 1.2.2.2/32 default
2 as1core1[Loopback0] 8e+09 1.10.1.1/32 default
3 as2border1[Loopback0] 8e+09 2.1.1.1/32 default
4 as2border2[Loopback0] 8e+09 2.1.1.2/32 default
5 as2core1[Loopback0] 8e+09 2.1.2.1/32 default
6 as2core2[Loopback0] 8e+09 2.1.2.2/32 default
7 as2dept1[Loopback0] 8e+09 2.1.1.2/32 default
8 as2dist1[Loopback0] 8e+09 2.1.3.1/32 default
9 as2dist2[Loopback0] 8e+09 2.1.3.2/32 default
10 as3border1[Loopback0] 8e+09 3.1.1.1/32 default
11 as3border2[Loopback0] 8e+09 3.2.2.2/32 default
12 as3core1[Loopback0] 8e+09 3.10.1.1/32 default

Similar questions extract properties of other entities (e.g., bgpProcessConfiguration() extracts properties of BGP processes).


Inspecting referential integrity of configuration structures

Network configuratons define and reference named structures like route maps, access control lists (ACLs), prefix lists, etc. Two common indicators of buggy configurations include references to structures that are not defined anywhere (which can lead to disastrous consequences on some platforms) or defined structures that are not referenced anywhere. Batfish makes it easy to flag such instances because it understand the underlying semantics of configuration.

[12]:
# List references to undefined structures
bf.q.undefinedReferences().answer().frame()
[12]:
File_Name Struct_Type Ref_Name Context Lines
0 configs/as2core2.cfg route-map filter-bogons bgp inbound route-map configs/as2core2.cfg:[110]

The question for listing any unused structures is unusedStructures().


Inspecting topologies

Nodes in a network form multiple types of topologies that are defined by edges at layer 3 (IP layer) or by routing protocols such as BGP or OSPF. Batfish has questions that return such edges. These questions take nodes and remoteNodes parameters that can limit the output to a subset of the nodes.

[13]:
# Get layer 3 edges
bf.q.layer3Edges(nodes="as1border1").answer().frame()
[13]:
Interface IPs Remote_Interface Remote_IPs
0 as1border1[GigabitEthernet0/0] ['1.0.1.1'] as1core1[GigabitEthernet1/0] ['1.0.1.2']
1 as1border1[GigabitEthernet1/0] ['10.12.11.1'] as2border1[GigabitEthernet0/0] ['10.12.11.2']
[14]:
# Get BGP edges
bf.q.bgpEdges(nodes="as1border1").answer().frame()
[14]:
Node IP Interface AS_Number Remote_Node Remote_IP Remote_Interface Remote_AS_Number
0 as1border1 10.12.11.1 None 1 as2border1 10.12.11.2 None 2
1 as1border1 1.1.1.1 None 1 as1core1 1.10.1.1 None 1

Exploring Routing and Forwarding

Batfish computes routing and forwarding tables (aka RIBs and FIBs) of the network from snapshot data itself. These tables can be examined to understand the routing and forwarding behavior of the network.

One way to examine this behavior is using a virtual traceroute. Unlike the live-network traceroute, Batfish shows all possible flow paths in the network and identifies routing entries that cause each hop to be taken.

[15]:
# Do a traceroute from host1 to 1.0.2.2
tr_frame = bf.q.traceroute(startLocation="host1", headers=HeaderConstraints(dstIps="1.0.2.2")).answer().frame()

# Display results using customizations to handle large string values

show(tr_frame)
Flow Traces TraceCount
0 Start Location: host1
Src IP: 2.128.0.101
Src Port: 49152
Dst IP: 1.0.2.2
Dst Port: 33434
IP Protocol: UDP
ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.11.2, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
4. node: as2core1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(blocktelnet (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.11.1, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
5. node: as2border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
6. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.1.2, Routes: [ospf (Network: 1.0.2.0/24, Next Hop: interface GigabitEthernet0/0 ip 1.0.1.2)])
  TRANSMITTED(GigabitEthernet0/0)
7. node: as1core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(GigabitEthernet0/0)

ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.34.101.3, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 2.34.101.3)])
  TRANSMITTED(GigabitEthernet0/0)
3. node: as2dist1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.21.2, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
4. node: as2core2
  RECEIVED(GigabitEthernet3/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
5. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
6. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.1.2, Routes: [ospf (Network: 1.0.2.0/24, Next Hop: interface GigabitEthernet0/0 ip 1.0.1.2)])
  TRANSMITTED(GigabitEthernet0/0)
7. node: as1core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(GigabitEthernet0/0)

ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.34.201.3, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 2.34.201.3)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.22.2, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
4. node: as2core2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
5. node: as2border1
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
6. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.1.2, Routes: [ospf (Network: 1.0.2.0/24, Next Hop: interface GigabitEthernet0/0 ip 1.0.1.2)])
  TRANSMITTED(GigabitEthernet0/0)
7. node: as1core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(GigabitEthernet0/0)

ACCEPTED
1. node: host1
  ORIGINATED(default)
  FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 2.128.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 2.128.0.1)])
  PERMITTED(filter::OUTPUT (EGRESS_FILTER))
  TRANSMITTED(eth0)
2. node: as2dept1
  RECEIVED(GigabitEthernet2/0)
  PERMITTED(RESTRICT_HOST_TRAFFIC_IN (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.34.201.3, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 2.34.201.3)])
  TRANSMITTED(GigabitEthernet1/0)
3. node: as2dist2
  RECEIVED(GigabitEthernet2/0)
  FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.12.2, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet1/0)
4. node: as2core1
  RECEIVED(GigabitEthernet3/0)
  PERMITTED(blocktelnet (INGRESS_FILTER))
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.11.1, Routes: [ibgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  TRANSMITTED(GigabitEthernet0/0)
5. node: as2border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.12.11.1, Routes: [bgp (Network: 1.0.2.0/24, Next Hop: ip 10.12.11.1)])
  PERMITTED(INSIDE_TO_AS1 (EGRESS_FILTER))
  TRANSMITTED(GigabitEthernet0/0)
6. node: as1border1
  RECEIVED(GigabitEthernet1/0)
  FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.1.2, Routes: [ospf (Network: 1.0.2.0/24, Next Hop: interface GigabitEthernet0/0 ip 1.0.1.2)])
  TRANSMITTED(GigabitEthernet0/0)
7. node: as1core1
  RECEIVED(GigabitEthernet1/0)
  ACCEPTED(GigabitEthernet0/0)
4

Another way to understand the routing behavior in detail is to examine the routing tables directly.

[16]:
# Fetch the routing table of all VRFs on all nodes in the snapshot
routes_all = bf.q.routes().answer().frame()

(For a large network, the first time you run a question that needs the dataplane, fetching the answer can take a few minutes. Subsequent questions are quick as the generated dataplane is saved by Batfish.)

As used above, the routes() question can generate a lot of results. You may restrict the output using parameters to the question—to restrict the results to core routers, use nodes = “/core/”, and to restrict results to the prefix 90.90.90.0/24, use **network=90.90.90.0/24”.

[17]:
# Get all routes for the network 90.90.90.0/24 on core routers
bf.q.routes(nodes="/core/", network="90.90.90.0/24").answer().frame()
[17]:
Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
0 as3core1 default 90.90.90.0/24 interface GigabitEthernet2/0 AUTO/NONE(-1l) GigabitEthernet2/0 connected 0 0 None
1 as3core1 default 90.90.90.0/24 interface GigabitEthernet3/0 AUTO/NONE(-1l) GigabitEthernet3/0 connected 0 0 None

That’s it for now! Feel free to explore further by adding cells and running other questions, or play with other notebooks.


Get involved with the Batfish community

Join our community on Slack and GitHub.

Pandas Examples

Batfish questions can return a huge amount of data, which you may want to filter in various ways based on your task. While most Batfish questions support basic filtering, they may not support your desired filtering criteria. Further, for performance, you may want to fetch the answer once and filter it using multiple different criteria. These scenarios are where Pandas-based filtering can help.

Batfish answers can be easily turned into a Pandas DataFrame (using .frame()), after which you can use the full power of Pandas to filter and manipulate data. This notebook provides a few examples of common manipulations for Batfish. It is not intended as a complete guide of Pandas data manipulation.

Let’s first initialize a snapshot that we will use in our examples.

[1]:
# Import packages
%run startup.py
bf = Session(host="localhost")

# Initialize a network and a snapshot
bf.set_network("pandas-example")

SNAPSHOT_NAME = "snapshot"
SNAPSHOT_PATH = "networks/hybrid-cloud/"
bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
Your snapshot was successfully initialized but Batfish failed to fully recognized some lines in one or more input files. Some unrecognized configuration lines are not uncommon for new networks, and it is often fine to proceed with further analysis. You can help the Batfish developers improve support for your network by running:

    bf.upload_diagnostics(dry_run=False, contact_info='<optional email address>')

to share private, anonymized information. For more information, see the documentation with:

    help(bf.upload_diagnostics)
[1]:
'snapshot'
Filtering initIssues

After initializing the snapshot, you often want to look at the initIssues answer. If there are too many issues, you may want to ignore a particular class of issues. We show below how to do that.

[2]:
# Lets get the initIssues for our snapshot
issues = bf.q.initIssues().answer().frame()
issues
[2]:
Nodes Source_Lines Type Details Line_Text Parser_Context
0 ['leaf1'] None Convert warning (redflag) Interface Ethernet12 has an undefined channel group Port-Channel20 None None
1 None [configs/Leaf2.cfg:[6], configs/Leaf4.cfg:[6], configs/Leaf1.cfg:[6], configs/Leaf3.cfg:[6], configs/Spine1.cfg:[6], configs/Spine2.cfg:[6]] Parse warning This syntax is unrecognized transceiver qsfp default-mode 4x10G [arista_configuration]
2 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-west-2/VpcEndpointServices.json None None
3 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-east-2/VpcEndpointServices.json None None
[3]:
# Ignore all issues whose Line_Text contain one of these as a substring
line_texts_to_ignore = ["transceiver"]


def has_substring(text: Optional[str], substrings: List[str]) -> bool:
    """Returns True if 'text' is not None and contains one of the 'substrings'"""
    return text is not None and any(substr in text for substr in substrings)


issues[
    issues.apply(
        lambda issue: not has_substring(issue["Line_Text"], line_texts_to_ignore),
        axis=1,
    )
]
[3]:
Nodes Source_Lines Type Details Line_Text Parser_Context
0 ['leaf1'] None Convert warning (redflag) Interface Ethernet12 has an undefined channel group Port-Channel20 None None
2 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-west-2/VpcEndpointServices.json None None
3 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-east-2/VpcEndpointServices.json None None

In the code above, we are using the Pandas method apply to map issues to a binary array based on whether the issue has one of the substrings in line_texts_to_ignore. Passing axis=1 makes apply iterate over rows instead of columns. The helper method has_substring makes this determination. It returns True if text is not None and has any of the substrings. The Python method any returns True if any element of the input iterable is True. Using the binary array as a filter for issues produces rows that match our criterion.

Instead of ignoring some issues, you may want to focus on issues that match a certain criteria. That too can be easily accomplished, as follows.

[4]:
# Only show issues whose details match these substrings
focus_details = ["Unrecognized element 'ServiceDetails' in AWS"]

issues[
    issues.apply(lambda issue: has_substring(issue["Details"], focus_details), axis=1)
]
[4]:
Nodes Source_Lines Type Details Line_Text Parser_Context
2 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-west-2/VpcEndpointServices.json None None
3 None [aws_configs:[]] Parse warning (unimplemented) Unrecognized element 'ServiceDetails' in AWS file aws_configs/us-east-2/VpcEndpointServices.json None None

The code above is similar to the one we used earlier, with the only differences being that we use the focus_details list as the argument to the has_substrings helper and we do not invert its result.

Filtering objects
`Line_Text` and `Details` columns above have string values, but many Batfish answers contain other data types as well. We generalize the approach above to filter other data types and to filter based on multiple columns. We use the `interfaceProperties` question for this demonstrate.
[5]:
# Fetch interface properties and display its first five rows
interfaces = bf.q.interfaceProperties().answer().frame()
interfaces.head(5)
[5]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN HSRP_Groups HSRP_Version Inactive_Reason Incoming_Filter_Name MLAG_ID MTU Native_VLAN Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
0 __aws-services-gateway__[aws-services] None True True [] True 1e+12 False None [] [] [] To AWS services None [] None None None 1500 None None None link-local:169.254.0.1 None False False False False None False NONE DOT1Q default [] None
1 __aws-services-gateway__[backbone] None True True [] True 1e+12 False None [] [] [] To AWS backbone None [] None None None 1500 None None None link-local:169.254.0.1 None False False False False None False NONE DOT1Q default [] None
2 exitgw[GigabitEthernet1] None True True ['10.10.100.2/24'] True 1e+09 False None [] [] ['GigabitEthernet1'] None None [] None None None 1500 None None None 10.10.100.2/24 10.10.100.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 exitgw[GigabitEthernet2] None True True ['10.10.101.2/24'] True 1e+09 False None [] [] ['GigabitEthernet2'] None None [] None None None 1500 None None None 10.10.101.2/24 10.10.101.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
4 exitgw[GigabitEthernet3] None True True ['147.75.69.27/31'] True 1e+09 False None [] [] ['GigabitEthernet3'] None None [] None None None 1500 None None None 147.75.69.27/31 147.75.69.26/31 True False False False 1e+09 False NONE DOT1Q default [] None

To filter based on a column, we need to know its data type. We can learn that in the Batfish documentation or by inspecting the answer we got from Batfish (e.g., using Python’s type() method).

We show three examples of filtering based on the Interface and Active columns, which are of type pybatfish.datamodel.primitives.Interface and bool, respectively. The former has hostname and interface properties (which are strings).

[6]:
# Display all interfaces on node 'exitgw'
interfaces[interfaces.apply(lambda row: row["Interface"].hostname == "exitgw", axis=1)]
[6]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN HSRP_Groups HSRP_Version Inactive_Reason Incoming_Filter_Name MLAG_ID MTU Native_VLAN Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
2 exitgw[GigabitEthernet1] None True True ['10.10.100.2/24'] True 1e+09 False None [] [] ['GigabitEthernet1'] None None [] None None None 1500 None None None 10.10.100.2/24 10.10.100.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 exitgw[GigabitEthernet2] None True True ['10.10.101.2/24'] True 1e+09 False None [] [] ['GigabitEthernet2'] None None [] None None None 1500 None None None 10.10.101.2/24 10.10.101.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
4 exitgw[GigabitEthernet3] None True True ['147.75.69.27/31'] True 1e+09 False None [] [] ['GigabitEthernet3'] None None [] None None None 1500 None None None 147.75.69.27/31 147.75.69.26/31 True False False False 1e+09 False NONE DOT1Q default [] None
5 exitgw[GigabitEthernet4] None False False [] True 1e+09 False None [] [] ['GigabitEthernet4'] None None [] None Administratively down None None 1500 None None None None None True False False False 1e+09 False NONE DOT1Q default [] None
6 exitgw[Loopback0] None True True ['2.2.2.2/32'] True 8e+09 None None [] [] ['Loopback0'] None None [] None None None 1500 None None None 2.2.2.2/32 2.2.2.2/32 True False False False None False NONE DOT1Q default [] None
7 exitgw[Loopback123] None True True ['192.168.123.7/32'] True 8e+09 None None [] [] ['Loopback123'] None None [] None None None 1500 None None None 192.168.123.7/32 192.168.123.7/32 True False False False None False NONE DOT1Q default [] None
8 exitgw[Tunnel1] None True True ['169.254.25.162/30'] True 100000 None None [] [] ['Tunnel1'] None None [] None None None 1500 None None None 169.254.25.162/30 169.254.25.160/30 True False False False None False NONE DOT1Q default [] None
9 exitgw[Tunnel2] None True True ['169.254.172.2/30'] True 100000 None None [] [] ['Tunnel2'] None None [] None None None 1500 None None None 169.254.172.2/30 169.254.172.0/30 True False False False None False NONE DOT1Q default [] None
10 exitgw[Tunnel3] None True True ['169.254.252.78/30'] True 100000 None None [] [] ['Tunnel3'] None None [] None None None 1500 None None None 169.254.252.78/30 169.254.252.76/30 True False False False None False NONE DOT1Q default [] None
11 exitgw[Tunnel4] None True True ['169.254.215.82/30'] True 100000 None None [] [] ['Tunnel4'] None None [] None None None 1500 None None None 169.254.215.82/30 169.254.215.80/30 True False False False None False NONE DOT1Q default [] None
[7]:
# Display all GigabitEthernet interfaces on node 'exitgw'
interfaces[
    interfaces.apply(
        lambda row: row["Interface"].hostname == "exitgw"
        and row["Interface"].interface.startswith("GigabitEthernet"),
        axis=1,
    )
]
[7]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN HSRP_Groups HSRP_Version Inactive_Reason Incoming_Filter_Name MLAG_ID MTU Native_VLAN Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
2 exitgw[GigabitEthernet1] None True True ['10.10.100.2/24'] True 1e+09 False None [] [] ['GigabitEthernet1'] None None [] None None None 1500 None None None 10.10.100.2/24 10.10.100.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 exitgw[GigabitEthernet2] None True True ['10.10.101.2/24'] True 1e+09 False None [] [] ['GigabitEthernet2'] None None [] None None None 1500 None None None 10.10.101.2/24 10.10.101.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
4 exitgw[GigabitEthernet3] None True True ['147.75.69.27/31'] True 1e+09 False None [] [] ['GigabitEthernet3'] None None [] None None None 1500 None None None 147.75.69.27/31 147.75.69.26/31 True False False False 1e+09 False NONE DOT1Q default [] None
5 exitgw[GigabitEthernet4] None False False [] True 1e+09 False None [] [] ['GigabitEthernet4'] None None [] None Administratively down None None 1500 None None None None None True False False False 1e+09 False NONE DOT1Q default [] None
[8]:
# Display all active GigabitEthernet interfaces on node 'exitgw'
interfaces[
    interfaces.apply(
        lambda row: row["Interface"].hostname == "exitgw"
        and row["Interface"].interface.startswith("GigabitEthernet")
        and row["Active"],
        axis=1,
    )
]
[8]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN HSRP_Groups HSRP_Version Inactive_Reason Incoming_Filter_Name MLAG_ID MTU Native_VLAN Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
2 exitgw[GigabitEthernet1] None True True ['10.10.100.2/24'] True 1e+09 False None [] [] ['GigabitEthernet1'] None None [] None None None 1500 None None None 10.10.100.2/24 10.10.100.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 exitgw[GigabitEthernet2] None True True ['10.10.101.2/24'] True 1e+09 False None [] [] ['GigabitEthernet2'] None None [] None None None 1500 None None None 10.10.101.2/24 10.10.101.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
4 exitgw[GigabitEthernet3] None True True ['147.75.69.27/31'] True 1e+09 False None [] [] ['GigabitEthernet3'] None None [] None None None 1500 None None None 147.75.69.27/31 147.75.69.26/31 True False False False 1e+09 False NONE DOT1Q default [] None
Filtering columns

When viewing Batfish answers, you may want to view only some of the columns. Pandas makes that easy for both original answers and answers where some rows have been filtered, as both of them are just DataFrames.

[9]:
# Filter interfaces to all active GigabitEthernet interfaces on node exitgw
exitgw_gige_active_interfaces = interfaces[
    interfaces.apply(
        lambda row: row["Interface"].hostname == "exitgw"
        and row["Interface"].interface.startswith("GigabitEthernet")
        and row["Active"],
        axis=1,
    )
]
# Display only the Interface and All_Prefixes columns of the filtered DataFrame
exitgw_gige_active_interfaces[["Interface", "All_Prefixes"]]
[9]:
Interface All_Prefixes
2 exitgw[GigabitEthernet1] ['10.10.100.2/24']
3 exitgw[GigabitEthernet2] ['10.10.101.2/24']
4 exitgw[GigabitEthernet3] ['147.75.69.27/31']
Counting rows

Often, you would be interested in counting the number of rows in the filtered answer. This is super easy because Python’s len() method, which we use for iterables, can be used on DataFrames as well.

[10]:
# Show the number of rows in the filtered DataFrame that we obtained above
len(exitgw_gige_active_interfaces)
[10]:
3
Grouping rows

For more advanced operations than filtering rows and columns, chances are that you will find Pandas groupyby pretty handy. This method enables you to group rows using a custom criteria and analyze those groups. For instance, if you wanted to group interfaces by nodes, you may do the following:

[11]:
# Get interfaces grouped by node name
intefaces_by_hostname = interfaces.groupby(
    lambda index: interfaces.loc[index]["Interface"].hostname
)

We obtained a Pandas DataFrameGroupBy object above. The groupby method iterates over row indexes (apply iterated over rows), calls the lambda over each, and groups rows whose indices yield the same value. In our example, the lambda first gets the row using interfaces.loc[index], then gets the interface (which is of type pybatfish.datamodel.primitives.Interface), and finally the hostname.

DataFrameGroupBy objects offer many functions that are useful for analysis. We demonstrate two of them below.

[12]:
# Display the rows corresponding to node 'exitgw' group
intefaces_by_hostname.get_group("exitgw")
[12]:
Interface Access_VLAN Active Admin_Up All_Prefixes Allowed_VLANs Auto_State_VLAN Bandwidth Blacklisted Channel_Group Channel_Group_Members DHCP_Relay_Addresses Declared_Names Description Encapsulation_VLAN HSRP_Groups HSRP_Version Inactive_Reason Incoming_Filter_Name MLAG_ID MTU Native_VLAN Outgoing_Filter_Name PBR_Policy_Name Primary_Address Primary_Network Proxy_ARP Rip_Enabled Rip_Passive Spanning_Tree_Portfast Speed Switchport Switchport_Mode Switchport_Trunk_Encapsulation VRF VRRP_Groups Zone_Name
2 exitgw[GigabitEthernet1] None True True ['10.10.100.2/24'] True 1e+09 False None [] [] ['GigabitEthernet1'] None None [] None None None 1500 None None None 10.10.100.2/24 10.10.100.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
3 exitgw[GigabitEthernet2] None True True ['10.10.101.2/24'] True 1e+09 False None [] [] ['GigabitEthernet2'] None None [] None None None 1500 None None None 10.10.101.2/24 10.10.101.0/24 True False False False 1e+09 False NONE DOT1Q default [] None
4 exitgw[GigabitEthernet3] None True True ['147.75.69.27/31'] True 1e+09 False None [] [] ['GigabitEthernet3'] None None [] None None None 1500 None None None 147.75.69.27/31 147.75.69.26/31 True False False False 1e+09 False NONE DOT1Q default [] None
5 exitgw[GigabitEthernet4] None False False [] True 1e+09 False None [] [] ['GigabitEthernet4'] None None [] None Administratively down None None 1500 None None None None None True False False False 1e+09 False NONE DOT1Q default [] None
6 exitgw[Loopback0] None True True ['2.2.2.2/32'] True 8e+09 None None [] [] ['Loopback0'] None None [] None None None 1500 None None None 2.2.2.2/32 2.2.2.2/32 True False False False None False NONE DOT1Q default [] None
7 exitgw[Loopback123] None True True ['192.168.123.7/32'] True 8e+09 None None [] [] ['Loopback123'] None None [] None None None 1500 None None None 192.168.123.7/32 192.168.123.7/32 True False False False None False NONE DOT1Q default [] None
8 exitgw[Tunnel1] None True True ['169.254.25.162/30'] True 100000 None None [] [] ['Tunnel1'] None None [] None None None 1500 None None None 169.254.25.162/30 169.254.25.160/30 True False False False None False NONE DOT1Q default [] None
9 exitgw[Tunnel2] None True True ['169.254.172.2/30'] True 100000 None None [] [] ['Tunnel2'] None None [] None None None 1500 None None None 169.254.172.2/30 169.254.172.0/30 True False False False None False NONE DOT1Q default [] None
10 exitgw[Tunnel3] None True True ['169.254.252.78/30'] True 100000 None None [] [] ['Tunnel3'] None None [] None None None 1500 None None None 169.254.252.78/30 169.254.252.76/30 True False False False None False NONE DOT1Q default [] None
11 exitgw[Tunnel4] None True True ['169.254.215.82/30'] True 100000 None None [] [] ['Tunnel4'] None None [] None None None 1500 None None None 169.254.215.82/30 169.254.215.80/30 True False False False None False NONE DOT1Q default [] None

Here, we used the get_group method to get all information for ‘exitgw’, thus viewing all interfaces for that node. This is possible using row filtering as well, but we can do other things that are not, such as:

[13]:
# Display the number of interfaces per node
intefaces_by_hostname.count()[["Interface"]]
[13]:
Interface
__aws-services-gateway__ 2
exitgw 10
i-01602d9efaed4409a 1
i-02cae6eaa9edeed70 1
i-04cd3db5124a05ee6 1
i-0a5d64b8b58c6dd09 1
igw-02fd68f94367a67c7 2
igw-0a8309f3192e7cea3 2
internet 3
isp_16509 6
isp_65200 2
leaf1 15
leaf2 13
leaf3 13
leaf4 14
spine1 18
spine2 18
srv-101 1
subnet-009d57c7f13813630 4
subnet-0333a0749ea4ce3df 4
subnet-03acae3b9a534fff9 3
subnet-06005943afe32f714 4
subnet-06a692ed4ef84368d 4
subnet-09b389def558a9c7d 3
subnet-0cb5f4c094bee5214 3
subnet-0f84a4be105f7aaef 3
tgw-06b348adabd13452d 8
tgw-0888a76c8a371246d 8
vpc-00157b5941bfd4959 5
vpc-00b65e98077106059 8
vpc-0276455718806058a 5
vpc-0574d08f8d05917e4 8

In this example, we used the count method, which counts non-null entries for each column in the group. We then filtered by the Interface column to see interfaces per node.

Summary

In this notebook, we showed how you can use Pandas methods to manipulate Batfish answers, including filtering rows, filtering columns, and grouping rows. Hopefully, these examples help you get started with your analyses. Find us on Slack (link below) if you have questions.


Get involved with the Batfish community

Join our community on Slack and GitHub.

Validating Configuration Settings with Batfish

Network engineers routinely need to validate configuration settings of various devices in their network. In a multi-vendor network, this validation can be hard and few tools exist today to enable this basic task. However, the vendor-independent models of Batfish and its querying mechanisms make such validation almost trivial.

In this notebook, we show how to validate configuration settings with Batfish. More specifically, we examine how the configuration of NTP servers can be validated. The same validation scenarios can be performed for other configuration settings of nodes (such as dns servers, tacacs servers, snmp communities, VRFs, etc.) interfaces (such as MTU, bandwidth, input and output access lists, state, etc.), VRFs, BGP and OSPF sessions, and more.

Check out a video demo of this notebook here.

Initializing our Network and Snapshot

SNAPSHOT_PATH below can be updated to point to a custom snapshot directory, see the Batfish instructions for how to package data for analysis. More example networks are available in the networks folder of the Batfish repository.

[1]:
# Import packages
%run startup.py
bf = Session(host="localhost")

# Initialize a network and snapshot
NETWORK_NAME = "example_network"
SNAPSHOT_NAME = "example_snapshot"

SNAPSHOT_PATH = "networks/example"

bf.set_network(NETWORK_NAME)
bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
[1]:
'example_snapshot'

The network snapshot that we initialized above is illustrated below. You can download/view devices’ configuration files here. We will focus on the validation for the six border routers.

example-network

Extracting configured NTP servers

This can be done using the nodeProperties() question.

[2]:
# Set the property that we want to extract
COL_NAME = "NTP_Servers"

# Extract NTP servers for all routers with 'border' in their name
node_props = bf.q.nodeProperties(
    nodes="/border/",
    properties=COL_NAME).answer().frame()
node_props
[2]:
Node NTP_Servers
0 as1border2 ['18.18.18.18', '23.23.23.23']
1 as2border1 ['18.18.18.18', '23.23.23.23']
2 as3border2 ['18.18.18.18', '23.23.23.23']
3 as1border1 []
4 as3border1 ['18.18.18.18', '23.23.23.23']
5 as2border2 ['18.18.18.18']

The .frame() function call above returns a Pandas data frame that contains the answer.

Validating NTP Servers Configuration

Depending on the network’s policy, there are several possible validation scenarios for NTP-servers configuration: 1. Every node has at least one NTP server configured. 2. Every node has at least one NTP server configured from the reference set. 3. Every node has the reference set of NTP servers configured. 4. Every node has NTP servers that match those in a per-node database.

We demonstrate each scenario below.

Validation scenario 1: Every node has at least one NTP server configured

Now that we have the list of NTP servers, let’s check if at least one server is configured on the border routers. We accomplish that by using (lambda expressions) to identify nodes where the list is empty.

[3]:
# Find nodes that have no NTP servers configured
ns_violators = node_props[node_props[COL_NAME].apply(
    lambda x: len(x) == 0)]
ns_violators
[3]:
Node NTP_Servers
3 as1border1 []
Validation scenario 2: Every node has at least one NTP server configured from the reference set.

Now if we want to validate that configured NTP servers should contain at least one NTP server from a reference set, we can use the command below. It identifies any node whose configured set of NTP servers does not overlap with the reference set at all.

[4]:
# Define the reference set of NTP servers
ref_ntp_servers = set(["23.23.23.23"])

# Find nodes that have no NTP server in common with the reference set
ns_violators = node_props[node_props[COL_NAME].apply(
    lambda x: len(ref_ntp_servers.intersection(set(x))) == 0)]
ns_violators
[4]:
Node NTP_Servers
3 as1border1 []
5 as2border2 ['18.18.18.18']

Because as1border1 has no configured NTP servers, it clearly violates our assertion, and so does as2border2 which has a configured server but not one that is present in the reference set.

Validation scenario 3: Every node has the reference set of NTP servers configured

A common use case for validating NTP servers involves checking that the set of NTP servers exactly matches a desired reference set. Such validation is quite straightforward as well.

[5]:
# Find violating nodes whose configured NTP servers do not match the reference set
ns_violators = node_props[node_props[COL_NAME].apply(
    lambda x: ref_ntp_servers != set(x))]
ns_violators
[5]:
Node NTP_Servers
0 as1border2 ['18.18.18.18', '23.23.23.23']
1 as2border1 ['18.18.18.18', '23.23.23.23']
2 as3border2 ['18.18.18.18', '23.23.23.23']
3 as1border1 []
4 as3border1 ['18.18.18.18', '23.23.23.23']
5 as2border2 ['18.18.18.18']

As we can see, all border nodes violate this condition.

A slightly advanced version of pandas filtering can also show us which configured NTP servers are missing or extra (compared to the reference set) at each node.

[6]:
# Find extra and missing servers at each node
ns_extra = node_props[COL_NAME].map(lambda x: set(x) - ref_ntp_servers)
ns_missing = node_props[COL_NAME].map(lambda x: ref_ntp_servers - set(x))

# Join these columns up with the node columns for a complete view
diff_df = pd.concat([node_props["Node"],
                     ns_extra.rename('extra-{}'.format(COL_NAME)),
                     ns_missing.rename('missing-{}'.format(COL_NAME))],
                    axis=1)
diff_df
[6]:
Node extra-NTP_Servers missing-NTP_Servers
0 as1border2 {18.18.18.18} {}
1 as2border1 {18.18.18.18} {}
2 as3border2 {18.18.18.18} {}
3 as1border1 {} {23.23.23.23}
4 as3border1 {18.18.18.18} {}
5 as2border2 {18.18.18.18} {23.23.23.23}
Validation scenario 4: Every node has NTP servers that match those in a per-node database.

Every node should match its reference set of NTP Servers which may be stored in an external database. This check enables easy validation of configuration settings that differ acorss nodes.

We assume data from the database is fetched in the following format, where node names are dictionary keys and specific properties are defined in a property-keyed dictionary per node.

[7]:
# Mock reference-node-data, presumably taken from an external database
database = {'as1border1': {'NTP_Servers': ['23.23.23.23'],
                           'DNS_Servers': ['1.1.1.1']},
            'as1border2': {'NTP_Servers': ['23.23.23.23'],
                           'DNS_Servers': ['1.1.1.1']},
            'as2border1': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],
                           'DNS_Servers': ['2.2.2.2']},
            'as2border2': {'NTP_Servers': ['18.18.18.18'],
                           'DNS_Servers': ['1.1.1.1']},
            'as3border1': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],
                           'DNS_Servers': ['2.2.2.2']},
            'as3border2': {'NTP_Servers': ['18.18.18.18', '23.23.23.23'],
                           'DNS_Servers': ['2.2.2.2']},
            }

Note that there is an extra property in this dictionary that we don’t care about comparing right now: dns-server. We will filter out this property below, before comparing the data from Batfish to that in the database.

After a little massaging, the database and Batfish data can be compared to generate two sets of servers: missing (i.e., present in the database but not in the configurations) and extra (i.e., present in the configurations but not in the database).

[8]:
# Transpose database data so each node has its own row
database_df = pd.DataFrame(data=database).transpose()

# Index on node for easier comparison
df_node_props = node_props.set_index('Node')

# Select only columns present in node_props (get rid of the extra dns-servers column)
df_db_node_props = database_df[df_node_props.columns].copy()

# Convert server lists into sets to support arithmetic below
df_node_props[COL_NAME] = df_node_props[COL_NAME].apply(set)
df_db_node_props[COL_NAME] = df_db_node_props[COL_NAME].apply(set)

# Figure out what servers are in the configs but not the database and vice versa
missing_servers = (df_db_node_props - df_node_props).rename(
    columns={COL_NAME: 'missing-{}'.format(COL_NAME)})
extra_servers = (df_node_props - df_db_node_props).rename(
    columns={COL_NAME: 'extra-{}'.format(COL_NAME)})
result = pd.concat([missing_servers, extra_servers], axis=1, sort=False)
result
[8]:
missing-NTP_Servers extra-NTP_Servers
as1border1 {23.23.23.23} {}
as1border2 {} {18.18.18.18}
as2border1 {} {}
as2border2 {} {}
as3border1 {} {}
as3border2 {} {}
Continue exploring

We showed you how to extract the database of configured NTP servers for every node and how to test that the settings are correct for a variety of desired test configurations. The underlying principles can be applied to other network configurations, such as interfaceProperties, bgpProcessConfiguration, ospfProcessConfiguration etc.

For example interfaceProperties() question can be used to fetch properties like interface MTU using a simple command.

[9]:
# Extract interface MTU for Ethernet0/0 interfaces on border routers
interface_mtu = bf.q.interfaceProperties(
    interfaces="/border/[Ethernet0/0]",
    properties="MTU").answer().frame()
interface_mtu
[9]:
Interface MTU
0 as1border1[Ethernet0/0] 1500
1 as1border2[Ethernet0/0] 1500
2 as2border1[Ethernet0/0] 1500
3 as2border2[Ethernet0/0] 1500
4 as3border1[Ethernet0/0] 1500
5 as3border2[Ethernet0/0] 1500

Get involved with the Batfish community!

Start interacting through Slack or GitHub to know more. We would love to talk with you about Batfish or your Network!

Uncovering Configuration and Behavior Drift

When debugging network issues, it is important to understand how the network is different today compared to yesterday or to the desired golden state. A text diff of device configs is one way to do this, but it tends to be too noisy. It will show differences that you may not care about (e.g., changes in whitespace or timestamps), and it is hard to control what is reported. More importantly, text diffs also do not tell you about the impact of change on network behavior, such as if new traffic will be permitted or if some BGP edges will go down.

Batfish parses and builds a vendor-neutral model of device configs and behavior. This model enables you to learn how two snapshots of the network differ exactly along the aspects you care about. The behavior modeling of Batfish also lets you understand the full impact of these changes. This notebook illustrates this capability.

We focus on the following differences across three categories.

  1. Configuration settings

    1. Node-level properties

    2. Interface-level properties

    3. Properties of BGP peers

  2. Structures and references

    1. Structures defined in device configs

    2. Undefined references

  3. Network behavior

    1. BGP adjacencies

    2. ACL lines with treat flows differently

These are examples of different types of changes that you can analyze using Batfish. You may be interested in a different aspects of your network, and you should be able to adapt the code below to suit your needs.

Text diff will help with the configuration settings category at best. The other two categories require understanding the structure of the config and the network behavior it induces. To illustrate this point, the text diff of example configs that we use in this notebook is below.

[1]:
# Use recursive diff, followed by some pretty printing hacks
!diff -ur networks/drift/reference networks/drift/snapshot | sed -e 's;diff.*snapshot/\(configs.*cfg\);^-----------\1---------;g' | tr '^' '\n' | grep -v networks/drift

-----------configs/as1border1.cfg---------
@@ -21,7 +21,7 @@
 !
 !
 no ip domain lookup
-ip domain name lab.local
+ip domain name lab.localp
 no ipv6 cef
 !
 !

-----------configs/as1border2.cfg---------
@@ -11,7 +11,7 @@
 !
 !
 ntp server 18.18.18.18
-ntp server 23.23.23.23
+ntp server 18.18.18.19
 !
 !
 no aaa new-model

-----------configs/as2border2.cfg---------
@@ -59,13 +59,14 @@
  duplex auto
 !
 interface GigabitEthernet0/0
- ip address 10.23.21.2 255.255.255.0
- ip access-group OUTSIDE_TO_INSIDE in
- ip access-group INSIDE_TO_AS3 out
- media-type gbic
- speed 1000
- duplex full
- negotiation auto
+ shutdown
+! ip address 10.23.21.2 255.255.255.0
+! ip access-group OUTSIDE_TO_INSIDE in
+! ip access-group INSIDE_TO_AS3 out
+! media-type gbic
+! speed 1000
+! duplex full
+! negotiation auto
 !
 interface GigabitEthernet1/0
  ip address 2.12.22.1 255.255.255.0

-----------configs/as2core1.cfg---------
@@ -60,6 +60,7 @@
  duplex auto
 !
 interface GigabitEthernet0/0
+ description "To as2border1 GigabitEthernet1/0"
  ip address 2.12.11.2 255.255.255.0
  media-type gbic
  speed 1000
@@ -67,6 +68,7 @@
  negotiation auto
 !
 interface GigabitEthernet1/0
+ description "To as2border2 GigabitEthernet2/0"
  ip address 2.12.21.2 255.255.255.0
  negotiation auto
 !

-----------configs/as2dept1.cfg---------
@@ -84,6 +84,7 @@
  neighbor as2 remote-as 2
  neighbor 2.34.101.3 peer-group as2
  neighbor 2.34.201.3 peer-group as2
+ neighbor 2.34.209.3 peer-group as2
  !
  address-family ipv4
   bgp dampening
@@ -96,7 +97,6 @@
   neighbor as2 route-map dept_to_as2 out
   neighbor 2.34.101.3 activate
   neighbor 2.34.201.3 activate
-  maximum-paths eibgp 5
  exit-address-family
 !
 ip forward-protocol nd

-----------configs/as2dist1.cfg---------
@@ -82,13 +82,13 @@
  bgp log-neighbor-changes
  neighbor as2 peer-group
  neighbor as2 remote-as 2
- neighbor dept peer-group
- neighbor dept remote-as 65001
+ neighbor dept2 peer-group
+ neighbor dept2 remote-as 65001
  neighbor 2.1.2.1 peer-group as2
  neighbor 2.1.2.1 update-source Loopback0
  neighbor 2.1.2.2 peer-group as2
  neighbor 2.1.2.2 update-source Loopback0
- neighbor 2.34.101.4 peer-group dept
+ neighbor 2.34.101.4 peer-group dept2
  !
  address-family ipv4
   bgp dampening
@@ -113,6 +113,7 @@
 no ip http server
 no ip http secure-server
 !
+access-list 102 permit tcp host 2.128.0.0 host 255.255.0.0
 access-list 102 permit ip host 2.128.0.0 host 255.255.0.0
 access-list 105 permit ip host 1.0.1.0 host 255.255.255.0
 access-list 105 permit ip host 1.0.2.0 host 255.255.255.0
@@ -128,6 +129,9 @@
  match community dept_community
  set local-preference 350
 !
+route-map dept_to_as2dist permit 200
+ match community dept_community_new
+ set local-preference 350
 !
 !
 control-plane

-----------configs/as2dist2.cfg---------
@@ -118,6 +118,7 @@
 access-list 105 permit ip host 1.0.2.0 host 255.255.255.0
 access-list 105 permit ip host 3.0.1.0 host 255.255.255.0
 access-list 105 permit ip host 3.0.2.0 host 255.255.255.0
+access-list 105 permit ip host 3.0.3.0 host 255.255.255.0
 !
 route-map as2dist_to_dept permit 100
  match ip address 105

-----------configs/as3border1.cfg---------
@@ -120,6 +120,10 @@
 !
 ip prefix-list default_list seq 5 permit 0.0.0.0/0
 !
+ip prefix-list bogons seq 5 permit 10.0.0.0/8
+ip prefix-list bogons seq 10 permit 172.16.0.0/16
+ip prefix-list bogons seq 15 permit 192.168.0.0/16
+!
 ip prefix-list inbound_route_filter seq 5 deny 3.0.0.0/8 le 32
 ip prefix-list inbound_route_filter seq 10 permit 0.0.0.0/0 le 32
 access-list 101 permit ip host 1.0.1.0 host 255.255.255.0

-----------configs/as3core1.cfg---------
@@ -51,9 +51,6 @@
 !
 !
 !
-interface Loopback0
- ip address 3.10.1.1 255.255.255.255
-!
 interface Ethernet0/0
  no ip address
  shutdown
@@ -78,6 +75,9 @@
  ip address 90.90.90.2 255.255.255.0
  negotiation auto
 !
+interface Loopback0
+ ip address 3.10.1.1 255.255.255.255
+!
 router ospf 1
  network 3.0.0.0 0.255.255.255 area 1
 !

As we can see, it is difficult to grasp the nature and impact of the change from this output, not to mention that it is impossible to build automation on top of it (e.g., to alert on certain types of differences). We show next how Batfish offers a meaningful view of these differences and their impact on network behavior.

[2]:
# Import packages, helpers, and load questions
%run startup.py
from drift_helper import diff_frames, diff_properties
bf = Session(host="localhost")

# Initialize both the snapshot and the reference that we want to use
NETWORK_NAME = "my_network"
SNAPSHOT_PATH = "networks/drift/snapshot"
REFERENCE_PATH = "networks/drift/reference"

bf.set_network(NETWORK_NAME)
bf.init_snapshot(SNAPSHOT_PATH, name="snapshot", overwrite=True)
bf.init_snapshot(REFERENCE_PATH, name="reference", overwrite=True)
[2]:
'reference'
1. Configuration settings

Let first uncover differences in configuration settings, starting with node-level properties.

1A. Node-level properties

We focus on three example properties: 1) NTP servers, 2) Domain name, and 3) VRFs that exist on the device. The complete list of node properties extracted by Batfish is here.

We will compute the property differences between across snapshots using Batfish questions. Batfish makes its models available via a set of questions. When questions are run in differential mode, it outputs how the answer differ across two snapshots.

[3]:
# Properties of interest
NODE_PROPERTIES = ["NTP_Servers" , "Domain_Name", "VRFs"]

# Compute the difference across two snapshots and return a Pandas DataFrame
node_diff = bf.q.nodeProperties(
                properties=",".join(NODE_PROPERTIES)
            ).answer(
                snapshot="snapshot",
                reference_snapshot="reference"
            ).frame()

# Print the DataFrame
show(node_diff.head())
Node KeyPresence Snapshot_Domain_Name Reference_Domain_Name Snapshot_NTP_Servers Reference_NTP_Servers Snapshot_VRFs Reference_VRFs
0 as1border1 In both lab.localp lab.local default default
1 as1border2 In both lab.local lab.local 18.18.18.19

18.18.18.18
23.23.23.23

18.18.18.18
default default

The output above shows all property differences for all nodes. There is a row per node. We see that on as1border1 the domain name has changed, and on as1border2 the set of NTP servers has changes. There is no other difference for any other node for the chosen properties.

This structured output can be transformed and fed into any type of automation, e.g., to alert you when an important property has changed. We can also generate readable drift reports using the helper function we defined above.

[4]:
# Print readable messages on the differences
diff_properties(node_diff, "Node", ["Node"], NODE_PROPERTIES)

Differences for Node=as1border1
    Domain_Name: lab.local -> lab.localp

Differences for Node=as1border2
    NTP_Servers: ['23.23.23.23', '18.18.18.18'] -> ['18.18.18.19', '18.18.18.18']
1B. Interface-level properties

We next check if any interface-level properties have changed. We again focus on three example settings: 1) whether the interface is active, 2) description, and 3) primary IP address. The complete list of interface settings extracted by Batfish are here.

[5]:
# Properties of interest
INTERFACE_PROPERTIES = ['Active', 'Description', 'Primary_Address']

# Compute the difference across two snapshots and return a Pandas DataFrame
interface_diff = bf.q.interfaceProperties(
                    properties=",".join(INTERFACE_PROPERTIES)
                ).answer(
                    snapshot="snapshot",
                    reference_snapshot="reference"
                ).frame()

# Print a readable version of the differences
diff_properties(interface_diff, "Interface", ["Interface"], INTERFACE_PROPERTIES)

Differences for Interface=as2border2[GigabitEthernet0/0]
    Active: True -> False
    Primary_Address: 10.23.21.2/24 -> None

Differences for Interface=as2core1[GigabitEthernet0/0]
    Description: None -> "To as2border1 GigabitEthernet1/0"

Differences for Interface=as2core1[GigabitEthernet1/0]
    Description: None -> "To as2border2 GigabitEthernet2/0"

We see that the interface GigabitEthernet0/0 on as2border2 has been shutdown and its address assignment has been eliminated. We also see that the description has been added for two interfaces on as2core1.

1C. BGP peer properties

We next check properties of BGP peers, focusing on four example properties: 1) description, 2) peer group, 3) Import policies applied to the peer, and 4) Export policies applied to the peer. The complete list of BGP peers properties is here.

[6]:
# Properties of interest
BGP_PEER_PROPERTIES = ['Remote_AS', 'Description', 'Peer_Group', 'Import_Policy', 'Export_Policy']

# Compute the difference across two snapshots and return a Pandas DataFrame
bgp_peer_diff = bf.q.bgpPeerConfiguration(
                    properties=",".join(BGP_PEER_PROPERTIES)
                ).answer(
                    snapshot="snapshot",
                    reference_snapshot="reference"
                ).frame()

#Print readable messages on the differences
diff_properties(bgp_peer_diff, "BgpPeer", ["Node", "VRF", "Local_Interface", "Remote_IP"], BGP_PEER_PROPERTIES)

BgpPeers only in snapshot
    Node=as2dept1, VRF=default, Local_Interface=None, Remote_IP=2.34.209.3

Differences for Node=as2dist1, VRF=default, Local_Interface=None, Remote_IP=2.34.101.4
    Peer_Group: dept -> dept2
    Import_Policy: ['dept_to_as2dist'] -> []
    Export_Policy: ['as2dist_to_dept'] -> []

The output shows that a new peer has been defined on as2dept1 with remote IP address 2.34.209.3; and the peer group has changed for an an existing peer on as2dist1, which then also led to its import and export policies changing. This correlated change in import/export policies are invisible in the text diff.

2. Structures and references

Batfish models include all structures defined in device configs (e.g., ACLs, prefix-lists) and how they are referenced in other parts of the config. You can use these models to learn if structures have been defined or deleted, which represents a major change in the configuration.

2A. Structures defined in configs

The definedStructures question is the basis for learning about structures defined in the config.

[7]:
# Extract defined structures from both snapshots as a Pandas DataFrame
snapshot_structures = bf.q.definedStructures().answer(snapshot="snapshot").frame()
reference_structures = bf.q.definedStructures().answer(snapshot="reference").frame()

# Show me what the information looks like by printing the first few rows
show(snapshot_structures.head())
Structure_Type Structure_Name Source_Lines
0 extended ipv4 access-list line OUTSIDE_TO_INSIDE: permit ip any any FileLines(filename='configs/as2border1.cfg', lines=[137])
1 bgp peer-group as2 FileLines(filename='configs/as1border1.cfg', lines=[81])
2 extended ipv4 access-list line blocktelnet: deny tcp any any eq telnet FileLines(filename='configs/as2core1.cfg', lines=[124])
3 interface GigabitEthernet1/0 FileLines(filename='configs/as1core1.cfg', lines=[69, 70, 71])
4 extended ipv4 access-list OUTSIDE_TO_INSIDE FileLines(filename='configs/as2border2.cfg', lines=[132, 133, 134])

The output snippet shows how Batfish captures the exact lines in each file where each structure is defined. We can process this information from the two snapshots to produce a report on all differences.

[8]:
# Remove the line numbers but keep the filename. We don't care about where in the file structure are defined.
snapshot_structures_without_lines = snapshot_structures[['Structure_Type', 'Structure_Name']].assign(
    File_Name=snapshot_structures["Source_Lines"].map(lambda x: x.filename))
reference_structures_without_lines = reference_structures[['Structure_Type', 'Structure_Name']].assign(
    File_Name=reference_structures["Source_Lines"].map(lambda x: x.filename))

# Print a readable message on the differences
diff_frames(snapshot_structures_without_lines,
            reference_structures_without_lines,
            "DefinedStructure")

DefinedStructures only in snapshot
     File_Name=configs/as3border1.cfg, Structure_Name=bogons, Structure_Type=ipv4 prefix-list
     File_Name=configs/as2dist1.cfg, Structure_Name=dept2, Structure_Type=bgp peer-group
     File_Name=configs/as2dist1.cfg, Structure_Name=dept_to_as2dist 200, Structure_Type=route-map-clause
     File_Name=configs/as2dist2.cfg, Structure_Name=105: permit ip host 3.0.3.0 host 255.255.255.0, Structure_Type=extended ipv4 access-list line
     File_Name=configs/as2dist1.cfg, Structure_Name=102: permit tcp host 2.128.0.0 host 255.255.0.0, Structure_Type=extended ipv4 access-list line

DefinedStructures only in reference
     File_Name=configs/as2dist1.cfg, Structure_Name=dept, Structure_Type=bgp peer-group

We can easily see in this output that a BGP peer group named dept2 was newly defined on as2dist1 and a prefix-list named bogons was defined on as2border1. We also see that the peer group named dept was removed from as2dist1. The peer group change is related to what we saw earlier with a peer property changing. This view shows that the entire structure has been removed and defined.

2B. Undefined structure references

References to undefined structures are symptoms of configuration errors. Using the undefinedReferences question, Batfish can help you understand if new undefined references have been introduced or old ones have been cleared.

[9]:
# Extract undefined references from both snapshots as a Pandas DataFrame
snapshot_undefined_references=bf.q.undefinedReferences().answer(snapshot="snapshot").frame()
reference_undefined_references= bf.q.undefinedReferences().answer(snapshot="reference").frame()

# Show me all undefined references in the snapshot
show(snapshot_undefined_references)
File_Name Struct_Type Ref_Name Context Lines
0 configs/as2core2.cfg route-map filter-bogons bgp inbound route-map FileLines(filename='configs/as2core2.cfg', lines=[110])
1 configs/as2dist1.cfg community-list dept_community_new route-map match community-list FileLines(filename='configs/as2dist1.cfg', lines=[133])
2 configs/as2dist1.cfg undeclared bgp peer-group dept bgp peer-group referenced before defined FileLines(filename='configs/as2dist1.cfg', lines=[99, 100, 101])

The output shows that there are three undefined references in the snapshot. Let us find out which ones were newly introduced relative to the reference.

[10]:
# Remove Lines since we don't care about where it was referenced
snapshot_undefined_references_without_lines = snapshot_undefined_references.drop(columns=['Lines'])
reference_undefined_references_without_lines = reference_undefined_references.drop(columns=['Lines'])

# Print a readable message on the differences
diff_frames(snapshot_undefined_references_without_lines,
            reference_undefined_references_without_lines,
            "UndefinedRefeference")

UndefinedRefeferences only in snapshot
     Struct_Type=community-list, Ref_Name=dept_community_new, File_Name=configs/as2dist1.cfg, Context=route-map match community-list
     Struct_Type=undeclared bgp peer-group, Ref_Name=dept, File_Name=configs/as2dist1.cfg, Context=bgp peer-group referenced before defined

We thus see that, of the three undefined references that we saw earlier, two were newly introduced and one exists in both snapshots.

3. Network behavior

We now turn our attention to behavioral differences between network snapshots, starting with changes in BGP adjacencies.

3A. BGP adjacencies

The bgpEdges question of Batfish enables you to learn about all BGP adjacencines in the network, as follows.

[11]:
# Get the edges from both snapshots as Pandas DataFrames
snapshot_bgp_edges = bf.q.bgpEdges().answer(snapshot="snapshot").frame()
reference_bgp_edges = bf.q.bgpEdges().answer(snapshot="reference").frame()

# Show me the schema by printing the first few rows
show(snapshot_bgp_edges.head())
Node IP Interface AS_Number Remote_Node Remote_IP Remote_Interface Remote_AS_Number
0 as1border2 1.2.2.2 None 1 as1core1 1.10.1.1 None 1
1 as1core1 1.10.1.1 None 1 as1border1 1.1.1.1 None 1
2 as2dist2 2.1.3.2 None 2 as2core2 2.1.2.2 None 2
3 as3border2 3.2.2.2 None 3 as3core1 3.10.1.1 None 3
4 as2dist2 2.34.201.3 None 2 as2dept1 2.34.201.4 None 65001

We see that Batfish knows which BGP edges in the snapshot come up and shows key information about them. We can use the answer to this question to learn which edges exist only in the snapshot or only in the refrence.

[12]:
# Retain only columns we care about for this analysis
snapshot_bgp_edges_nodes = snapshot_bgp_edges[['Node', 'Remote_Node']]
reference_bgp_edges_nodes = reference_bgp_edges[['Node', 'Remote_Node']]

# DataFrames contain one edge per direction; keep only one direction
snapshot_bgp_bidir_edges_nodes = snapshot_bgp_edges_nodes[
                                    snapshot_bgp_edges_nodes['Node'] < snapshot_bgp_edges_nodes['Remote_Node']
                                  ]
reference_bgp_bidir_edges_nodes = reference_bgp_edges_nodes[
                                    reference_bgp_edges_nodes['Node'] < reference_bgp_edges_nodes['Remote_Node']
                                  ]

# Print a readable message on the differences
diff_frames(snapshot_bgp_bidir_edges_nodes,
            reference_bgp_bidir_edges_nodes,
            "BgpEdge")

BgpEdges only in reference
     Remote_Node=as3border1, Node=as2border2

One BGP edge exists only in the reference, that is, it disappeared in the snapshot. We can find more about this edge, like so:

[13]:
# Find the matching edge in the reference edges answer from before
missing_snapshot_edge = reference_bgp_edges[
                           (reference_bgp_edges['Node']=="as2border2")
                           & (reference_bgp_edges['Remote_Node']=="as3border1")
                         ]

# Print the edge information
show(missing_snapshot_edge)
Node IP Interface AS_Number Remote_Node Remote_IP Remote_Interface Remote_AS_Number
20 as2border2 10.23.21.2 None 2 as3border1 10.23.21.3 None 3

Do you recall the interface on as2border2 that was shut earlier? This BGP edge was removed because of that interface shutdown (which you confirm using IP of the interface—10.23.21.2/24).

3B. ACL behavior

To compute the behavior differences between ACLs, we use the compare filters question. It returns pairs of lines, one from the filter definition in each snapshot, that match the same flow(s) but treat them differently (i.e. one permits and the other denies the flow).

[14]:
# compute behavior differences between ACLs
compare_filters = bf.q.compareFilters().answer(
                                            snapshot='snapshot',
                                            reference_snapshot='reference'
                                        ).frame()

# print the result
show(compare_filters)
Node Filter_Name Line_Index Line_Content Line_Action Reference_Line_Index Reference_Line_Content
0 as2dist2 105 4 permit ip host 3.0.3.0 host 255.255.255.0 PERMIT End of ACL

We see that the only difference in the ACL behaviors of the two snapshots is for ACL 105 on as2dist. Line permit ip host 3.0.3.0 host 255.255.255.0 in the snapshot permits some flows that were being denied in the reference snapshhot because of the implicit deny at the end of the ACL. Thus, we have permitted flows that were not being permitted before.

If you were paying attention to the text diff above, the result above may surprise you. The text diff (relevant snippet repeated below) showed that ACL 102 on as2dist1 changed as well.

[15]:
!diff -ur networks/drift/reference/configs/as2dist1.cfg networks/drift/snapshot/configs/as2dist1.cfg | grep -A 7 '@@ -113,6 +113,7 @@'
@@ -113,6 +113,7 @@
 no ip http server
 no ip http secure-server
 !
+access-list 102 permit tcp host 2.128.0.0 host 255.255.0.0
 access-list 102 permit ip host 2.128.0.0 host 255.255.0.0
 access-list 105 permit ip host 1.0.1.0 host 255.255.255.0
 access-list 105 permit ip host 1.0.2.0 host 255.255.255.0

You may have expected a behahvior diff corresponding to this change, but Batfish analysis reveals that that didn’t happen. The added line is permitting TCP traffic between two hosts for which IP traffic was already permitted, so no new traffic was permitted. So, either this change was unnecessary or someone mistyped the host addresses.

Summary

Batfish enables you to easily understand how your device configs differ from a historial reference or golden versions. It provides structured information about not only changes to settings in configs but also about changes in network behavior. This information provides important context beyond simple text diffs and can be inserted into an automated pipeline that alerts on important changes.


Get involved with the Batfish community

Join our community on Slack and GitHub.

ACLs and firewalls

Analyzing ACLs and firewall rules with Batfish

Network and security engineers are responsible for ensuring that the ACLs and firewall rules in their networks are permitting and denying traffic as intended. This task usually requires manual checking or loading rulesets onto a lab device in order to test behavior on example packets of interest. These methods are not only time consuming but also error-prone.

Batfish makes it easy to deeply analyze ACLs and firewall rules, which we generally call filters. It can show what a filter will do with a given packet, right down to the line of the filter that matches the packet; it can provide guarantees on how a filter treats a large space of packets; and it can sanity check a filter to ensure that every line in it matches at least some packets.

In this notebook, we demonstrate these capabilities. In our “Provably Safe ACL and Firewall Changes” notebook, we show how Batfish can guarantee that filter changes are safe.

Check out a video demo of this notebook here.

Initialization

We use an example network with two devices, a router with ACLs and a firewall. The device configurations can be seen here.

[1]:
# Import packages
%run startup.py
bf = Session(host="localhost")

# Initialize a network and a snapshot
bf.set_network("network-example-filters")

SNAPSHOT_NAME = "current"
SNAPSHOT_PATH = "networks/example-filters/current"
bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
[1]:
'current'
testFilters: Testing how filters treat a flow

The testFilters question shows what filters do with a particular flow and why. It takes as input the details of the flow and a set of filters to test. The answer provides a detailed view of how the flow is treated by each filter in the set.

Flows are specified using source and destination IP addresses and, optionally, other fields such as IP protocols, ports, and TCP flags. You may also specify the ingress interface which is factored in when the behavior of a filter depends on it. See documentation for details. The question will fill in any unspecified fields with reasonable defaults.

The set of filters to examine can be narrowed using various criteria, including regexes for node and filter names. Left unspecified, testFilters will give results for every filter in the network.

Example 1: Test if hosts in a subnet can reach the DNS server

Suppose we wanted to test if our ACL acl_in in router rtr-with-acl allows hosts in a subnet to reach our DNS server. This check is easily expressed. If the subnet is 10.10.10.0/24 and the DNS server is at the IP address 218.8.104.58, then the query will be as shown below, where we have picked 10.10.10.1 as a representative source IP for the subnet.

[2]:
# Check if a representative host can reach the DNS server
dns_flow = HeaderConstraints(srcIps="10.10.10.1",
                             dstIps="218.8.104.58",
                             applications=["dns"])
answer = bf.q.testFilters(headers=dns_flow,
                         nodes="rtr-with-acl",
                         filters="acl_in").answer()
show(answer.frame())
Node Filter_Name Flow Action Line_Content Trace
0 rtr-with-acl acl_in Start Location: rtr-with-acl
Src IP: 10.10.10.1
Src Port: 49152
Dst IP: 218.8.104.58
Dst Port: 53
IP Protocol: UDP
PERMIT 660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
  • Matched line 660 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain

The result above shows that the flow we tested is indeed permitted by the filter acl_in (in Filter_Name column) on device rtr-with-acl (in Node column). The Flow column shows the exact flow tested, including the default values chosen by Batfish for unspecified parameters. The remaining columns show how the flow is treated by the filter. The Action column shows the final outcome, Line_Content shows which line(s) led to that action, and Trace shows all the lines the packet matched. In this case, the trace is not deep, but it can be quite deep for filters that reference other structures such as policy maps and object groups.

Example 2: Test that HTTP flows cannot go from one zone to another

Now suppose we want to test that our firewall blocks HTTP flows that cross the boundary from one zone to another. A query like the one below can help us do that. It is testing how output filters on interfaces in the second zone treat a packet with the specified header fields when it enters the firewall through interfaces in the first zone.

The arguments to startLocation and filters below use Batfish specifiers that enable easy specification of various concepts. enter(firewall[Ethernet0/0/2]) specifies that the packet we want to test enters ‘firewall’ via ‘Ethernet0/0/2’ (the interface in the first zone) and outFilterOf(Ethernet0/0/3) specifies that the filter we want to test is the out filter on ‘Ethernet0/0/3’ (the interface in the second zone).

[3]:
# Test if a host can reach the DNS server
http_flow = HeaderConstraints(srcIps="10.114.64.1",
                              dstIps="10.114.60.10",
                              applications=["http"])
answer = bf.q.testFilters(headers=http_flow,
                         startLocation="@enter(firewall[GigabitEthernet0/0/2])",
                         filters="@out(GigabitEthernet0/0/3)").answer()
show(answer.frame())
Node Filter_Name Flow Action Line_Content Trace
0 firewall ~ZONE_OUTGOING_ACL~zone-z3~ Start Location: firewall interface=GigabitEthernet0/0/2
Src IP: 10.114.64.1
Src Port: 49152
Dst IP: 10.114.60.10
Dst Port: 80
IP Protocol: TCP (SYN)
DENY no-match

    As we can see the HTTP flow is indeed denied, and the columns in the table show the reason for that denial.

    searchFilters: Verifying how filters treat a (very large) space of flows

    While testFilters reasons about individual flows, the searchFilters question provides comprehensive guarantees by reasoning about (potentially very large) spaces of flows. The space of flows is specified using source and destination prefixes (where the default prefix 0.0.0.0/0 denotes all 4,294,967,296 possibilities), a list of protocols, a range of ports, and so on. Given a flow space and a match condition, which can be ‘permit’, ‘deny’, or ‘matchLine ’, searchFilters returns flows within this space that match the condition. If no flow is returned, it is guaranteed that no flow in the entire space satisfies the condition.

    Example 1: Verify that all hosts in a subnet can reach the DNS server

    Earlier we checked that a subnet could reach the DNS server using testFilters with a representative source IP address. Now, let’s use searchfilter to ascertain that the entire subnet can reach the server.

    The query below is asking if there is any flow from 10.10.10.0/24 to 218.8.104.58 that is denied by the filter acl_in. That is, we’re asking for violations of the desired policy. An empty result means the policy is correctly implemented. Any flow returned by the query demonstrates that the policy is not correctly implemented.

    [4]:
    
    # Check if the subnet can reach the DNS server
    dns_traffic = HeaderConstraints(srcIps="10.10.10.0/24",
                                    dstIps="218.8.104.58",
                                    applications=["dns"])
    answer = bf.q.searchFilters(headers=dns_traffic,
                               action="deny",
                               filters="acl_in").answer()
    show(answer.frame())
    
    Node Filter_Name Flow Action Line_Content Trace
    0 rtr-with-acl acl_in Start Location: rtr-with-acl
    Src IP: 10.10.10.42
    Src Port: 49152
    Dst IP: 218.8.104.58
    Dst Port: 53
    IP Protocol: UDP
    DENY 460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
    • Matched line 460 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain

    As we can see, we did get a flow that matches the search condition and thus violates our desired guarantee of the entire subnet being able to reach the DNS server. The columns carry the same information as those for testFilters and provide insight into the violation. In particular, we see that a flow with source IP 10.10.10.42 is denied by an earlier line in the ACL. Such needles in the haystack are impossible to find with tools like pen testing.

    Example 2: Verify that the only TCP traffic that can go from one zone to another is NFS traffic

    In the second example for testFilters we used an example flow to test if firewall denied HTTP flows from one zone to another. Now, suppose that we wanted a stricter condition—that only TCP-based NFS flows are allowed from one zone to another. With searchFilters we can verify this condition by searching for violations in the large space of non-NFS flows, as below.

    [5]:
    
    # Check if any non-NFS, TCP flow can go from one zone to another
    non_nfs_tcp_traffic = HeaderConstraints(srcIps="101.164.101.231",
                                            dstIps="101.164.9.0/24",
                                            ipProtocols=["tcp"],
                                            dstPorts="!2049")  # exclude NFS port (2049)
    answer = bf.q.searchFilters(headers=non_nfs_tcp_traffic,
                               startLocation="@enter(firewall[GigabitEthernet0/0/2])",
                               filters="@out(GigabitEthernet0/0/3)",
                               action="permit").answer()
    show(answer.frame())
    
    Node Filter_Name Flow Action Line_Content Trace

    Since we got back an empty answer, we can be certain that no non-NFS flow can go from one zone to another.

    Such strong guarantees are impossible with any other tool today.

    filterLineReachability: Analyzing reachability of filter lines

    When debugging or editing filters, it can be useful to confirm that every line is reachable—that is, each line can match some packets that do not match earlier lines. Unreachable filter lines are often symptomatic of bugs and past edits that did not achieve policy intent. Even when they do not represent bugs, they represent opportunities to reduce the length of the filter.

    The filterLineReachability question identifies unreachable filter lines. Given no parameters, it will check every filter in the network, but its scope can be narrowed using filters and nodes parameters (see documentation).

    [6]:
    
    # Find unreachable lines in filters of rtr-with-acl
    aclAns = bf.q.filterLineReachability(nodes="rtr-with-acl").answer()
    show(aclAns.frame().sort_values(by="Unreachable_Line"))
    
    Sources Unreachable_Line Unreachable_Line_Action Blocking_Lines Different_Action Reason Additional_Info
    1 rtr-with-acl: acl_in 670 permit ip 166.146.58.184 any PERMIT 540 deny ip 166.144.0.0/12 any True BLOCKING_LINES None
    0 rtr-with-acl: acl_in 790 deny ip 54.203.159.1/32 any DENY 500 deny ip 54.0.0.0/8 any False BLOCKING_LINES None

    Each line in the answer above identifies an unreachable line in a filter. Let’s take a closer look at the first one. It shows that the line 670 permit ip 166.146.58.184 any is unreachable because it is blocked by the line 540 shown in the Blocking_Lines column. The Different_Action column indicates that the blocking line 540 has the opposite action as the blocked line 670, a more worrisome situation than if actions were the same.

    The Reason column shows that the line is unreachable because of it has other lines that block it, Lines can also be independently unreachable (i.e., no packet will ever match it) or may be unreachable because of circular references. The filterLineReachability question identifies such cases as well, and provides more information about them in the Additional_Info column.

    Summary

    In this notebook, we showed how you can use Batfish to: 1. Test how filters in the network treat a flow (testFilters) 2. Verify how filters treat a large space of flows (searchFilters) 3. Find filter lines that will never match any packet (filterLineReachability)

    If you found these capabilities useful, check out our “Provably Safe ACL and Firewall Changes” notebook that shows how to change filters in a provably safe manner.


    Get involved with the Batfish community

    Join our community on Slack and GitHub.

    Provably safe ACL and firewall rule changes

    Changing ACLs or firewall rules is one of the riskiest updates to a network. Even a small error can block connectivity for a large set of critical services or open up sensitive resources to the world at large.

    This notebook shows a 3-step process that uses Batfish to make provably safe and correct changes to ACLs and firewall rules, which we generally call filters. For a broader view of Batfish’s support for analyzing filters, check out the “Analyzing ACLs and Firewall Rules” notebook.

    Check out a video demo of this notebook here.

    We will primarily use the searchFilters question of Batfish in this process. This question searches within large spaces of flows (specified using packet headers) for flows that match the specified action (‘permit’ or ‘deny’). See here for its documentation.

    Change scenario

    Our goal is to update an ACL on one of our routers to permit HTTP traffic (ports 80 and 8080) from one subnet (10.10.10.0/24) to another (18.18.18.0/27). We will implement this by adding rules to permit this traffic to our ACLs, and we will then use Batfish to check if the implementation was correct.

    Initialization

    We start by initializing the pre-change snapshot and variables that describe the change. Our example snapshot contains two devices, and we’ll change the ACL acl_in on rtr-with-acl.

    [1]:
    
    # Import packages
    %run startup.py
    bf = Session(host="localhost")
    
    # Initialize a network and snapshot
    CURRENT_SNAPSHOT_NAME = "current"
    CURRENT_SNAPSHOT_PATH = "networks/example-filters/current"
    bf.set_network("network-example-filters")
    bf.init_snapshot(CURRENT_SNAPSHOT_PATH, name=CURRENT_SNAPSHOT_NAME, overwrite=True)
    
    [1]:
    
    'current'
    
    [2]:
    
    node_name = "rtr-with-acl"  # The router to change
    filter_name = "acl_in"      # Name of the ACL to change
    
    # The traffic to allow
    change_traffic = HeaderConstraints(srcIps="10.10.10.0/24",
                                       dstIps="18.18.18.0/27",
                                       ipProtocols=["tcp"],
                                       dstPorts="80, 8080")
    
    Step 1: Ensure that the intended traffic is not already permitted

    Before we make the change to allow the intended traffic, we verify that that traffic is not already permitted — because if it is, we do not need to change anything. We accomplish this using the searchFilters question. Given a space of flows, specified using header fields such as source and destination addresses and ports, and a matching condition (e.g., permit, deny) as input, this question finds flows that satisfy the condition. If it reports no flows, then it is guaranteed that no flow within the space satisfies the condition.

    [3]:
    
    # Check if the intended traffic is already permitted in the current snapshot
    answer = bf.q.searchFilters(headers=change_traffic,
                               filters=filter_name,
                               nodes=node_name,
                               action="permit").answer(
                                   snapshot=CURRENT_SNAPSHOT_NAME)
    show(answer.frame())
    
    Node Filter_Name Flow Action Line_Content Trace

    Since the query above did not find any results, we know with certainty that no flow within the specified space is already permitted. We can now proceed. If some flow is returned as part of the query, we may want to delete the filter line(s) that permits that flow before we update the filter.

    Step 2: Ensure that the intended traffic is permitted in the candidate change

    Assume that we implemented a candidate change, shown as the diff below.

    diff -r networks/example-filters/current/configs/rtr-with-acl.cfg \
            networks/example-filters/candidate1/configs/rtr-with-acl.cfg
    39a40,41
    >   462 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 80
    >   463 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 8080
    

    We can load the snapshot with this change into Batfish and ensure that all flows within the intended traffic are permitted. We will do that by asking the same searchFilters question as before, except now searching for flows that are denied instead of permitted. If it produces no results, then we have the guarantee that all possible flows in the intended space are allowed.

    [4]:
    
    # Load the candidate1 change
    CANDIDATE1_SNAPSHOT_NAME = "candidate1"
    CANDIDATE1_SNAPSHOT_PATH = "networks/example-filters/candidate1"
    bf.init_snapshot(CANDIDATE1_SNAPSHOT_PATH, name=CANDIDATE1_SNAPSHOT_NAME, overwrite=True)
    
    # Check if any flow in the intended traffic is denied in candidate1
    answer = bf.q.searchFilters(headers=change_traffic,
                               filters=filter_name,
                               nodes=node_name,
                               action="deny").answer(
                                   snapshot=CANDIDATE1_SNAPSHOT_NAME)
    show(answer.frame())
    
    Node Filter_Name Flow Action Line_Content Trace

    Since we got no results, we can be confident that our candidate change permits all traffic that we intended to permit. If there were any flow in the desired space that was not permitted by the change, the query above would have found it.

    Step 3: Ensure that no collateral damage has occurred

    Typically, engineers will stop change validation after checking that the intended traffic has been successfully permitted by the change. However, for safety and correctness, we must also check that no traffic outside of the intended space has been impacted — that is, our change has not caused collateral damage.

    We can verify this using a “differential” version of the searchFilters question that compares two snapshots. The query below compares the candidate1 and initial snapshots, and is asking Batfish if there is any flow outside of the intended traffic that the two snapshots treat differently (i.e., one of them permits and the other rejects, or vice versa). To search traffic outside the specified flow space, we use the invertSearch flag. If this query returns no result, then combined with the result above, we have ensured that the change is completely correct.

    [5]:
    
    # Check if traffic other than the intended traffic has been impacted
    answer = bf.q.searchFilters(headers=change_traffic,
                               invertSearch=True,
                               filters=filter_name,
                               nodes=node_name).answer(snapshot=CANDIDATE1_SNAPSHOT_NAME,
                                                       reference_snapshot=CURRENT_SNAPSHOT_NAME)
    show(answer.frame())
    
    Node Filter_Name Flow KeyPresence Snapshot_Action Reference_Action Snapshot_Line_Content Reference_Line_Content Snapshot_Trace Reference_Trace
    0 rtr-with-acl acl_in Start Location: rtr-with-acl
    Src IP: 10.10.10.0
    Src Port: 49152
    Dst IP: 18.18.18.32
    Dst Port: 80
    IP Protocol: TCP (SYN)
    In both PERMIT DENY 462 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 80 2020 deny tcp any any
    • Matched line 462 permit tcp 10.10.10.0/24 18.18.18.0/26 eq 80
    • Matched line 2020 deny tcp any any

    Unfortunately, we do get a result, indicating that at least one flow outside of the intended space will be treated differently than before. The column Flow shows a flow that the two snapshots treat differently. In particular, this flow has destination IP address 18.18.18.32, which is outside of the address range 18.18.18.0/27 that we wanted to permit. The columns that start with Base_ show how CANDIDATE1_SNAPSHOT treats that flow, and those that start with Delta_ show how CURRENT_SNAPSHOT treats the flow. As shown, the candidate snapshot permits the flow while the current snapshot denies it. That means we’ve accidentally opened up more space than we intended.

    The root cause of the problem is apparent if we look at the diff above more carefully. The updated ACL permits destination prefix 18.18.18.0/26 rather than the intended 18.18.18.0/27. We need to fix this.

    Step 2 (again): Ensure that the intended traffic is permitted in the candidate change

    Assume that we implemented another candidate change, shown by the diff below.

    diff -r networks/example-filters/current/configs/rtr-with-acl.cfg \
            networks/example-filters/candidate2/configs/rtr-with-acl.cfg
    39a40,41
    >   462 permit tcp 10.10.10.0/24 18.18.18.0/27 eq 80
    >   463 permit tcp 10.10.10.0/24 18.18.18.0/27 eq 8080
    

    We will now load this change and repeat the same validation steps that we ran on the prior candidate change.

    [6]:
    
    # Load (another) candidate change
    CANDIDATE2_SNAPSHOT_NAME = "candidate2"
    CANDIDATE2_SNAPSHOT_PATH = "networks/example-filters/candidate2"
    bf.init_snapshot(CANDIDATE2_SNAPSHOT_PATH, name=CANDIDATE2_SNAPSHOT_NAME, overwrite=True)
    
    # Check if any part of the intended traffic is denied in candidate2
    answer = bf.q.searchFilters(headers=change_traffic,
                               filters=filter_name,
                               nodes=node_name,
                               action="deny").answer(snapshot=CANDIDATE2_SNAPSHOT_NAME)
    show(answer.frame())
    
    Node Filter_Name Flow Action Line_Content Trace

    As before, we get no results, which means that no flow in the intended space is being denied; we correctly permitted all intended traffic.

    Step 3 (again): Ensure that no collateral damage has occurred

    Now, let’s also check again that no other traffic is impacted.

    [7]:
    
    # Check if traffic other than the intended traffic has been impacted
    answer = bf.q.searchFilters(headers=change_traffic,
                               filters=filter_name,
                               nodes=node_name,
                               invertSearch=True).answer(snapshot=CANDIDATE2_SNAPSHOT_NAME,
                                                         reference_snapshot=CURRENT_SNAPSHOT_NAME)
    show(answer.frame())
    
    Node Filter_Name Flow KeyPresence Snapshot_Action Reference_Action Snapshot_Line_Content Reference_Line_Content Snapshot_Trace Reference_Trace

    This time, we got no collateral damage results! That implies this change is completely correct: It allows all traffic that we meant to allow and has no impact on other traffic. Therefore we can apply it with full confidence that it will have the exact desired behavior.

    Summary

    In this notebook, we showed how you can use Batfish to ensure that changes to filters are correct and permit or deny only the intended traffic.

    The steps for provably safe ACL and firewall changes are: 1. Check that the intended traffic does not already match the desired action (permit or deny) 2. Check that the intended traffic is treated correctly in the candidate change 3. Check that nothing but the intended traffic is impacted by the candidate change

    For additional ways to analyze filter using Batfish, see the “Analyzing ACLs and Firewall Rules” notebook.


    Get involved with the Batfish community

    Join our community on Slack and GitHub.

    Safely refactoring ACLs and firewall rules

    Changing ACLs or firewall rules (or filters) is one of the riskiest updates to a network. Even a small error can block connectivity for a large set of critical services or open up sensitive resources to the world at large. Earlier notebooks showed how to analyze filters for what they do and do not allow and how to make specific changes in a provably safe manner.

    This notebook shows how to refactor complex filters in a way that the full impact of refactoring can be understood and analyzed for correctness before refactored filters are pushed to the network.

    Original ACL

    We will use the following ACL as a running example in this notebook. The ACL can be read as a few separate sections:

    • Line 10: Deny ICMP redirects

    • Lines 20, 23: Permit BFD traffic on certain blocks

    • Lines 40-80: Permit BGP traffic

    • Lines 90-100: Permit DNS traffic a /24 subnet while denying it from a /32 within that

    • Lines 110-500: Permit or deny IP traffic from certain subnets

    • Line 510: Permit ICMP echo reply

    • Lines 520-840: Deny IP traffic to certain subnets

    • Lines 850-880: Deny all other types of traffic

    (The IP address space in the ACL appears all over the place because it has been anonymized via Netconan. Netconan preserves the super- and sub-prefix relationships when anonymizing IP addresses and prefixes.)

    [1]:
    
    # The ACL before refactoring
    original_acl = """
    ip access-list acl
      10 deny icmp any any redirect
      20 permit udp 117.186.185.0/24 range 49152 65535 117.186.185.0/24 eq 3784
      30 permit udp 117.186.185.0/24 range 49152 65535 117.186.185.0/24 eq 3785
      40 permit tcp 11.36.216.170/32 11.36.216.169/32 eq bgp
      50 permit tcp 11.36.216.176/32 11.36.216.179/32 eq bgp
      60 permit tcp 204.150.33.175/32 204.150.33.83/32 eq bgp
      70 permit tcp 205.248.59.64/32 205.248.59.67/32 eq bgp
      80 permit tcp 205.248.58.190/32 205.248.58.188/32 eq bgp
      90 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
      100 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
      110 deny ip 54.0.0.0/8 any
      120 deny ip 163.157.0.0/16 any
      130 deny ip 166.144.0.0/12 any
      140 deny ip 198.170.50.0/24 any
      150 deny ip 198.120.0.0/16 any
      160 deny ip 11.36.192.0/19 any
      170 deny ip 11.125.64.0/19 any
      180 permit ip 166.146.58.184/32 any
      190 deny ip 218.66.57.0/24 any
      200 deny ip 218.66.56.0/24 any
      210 deny ip 218.67.71.0/24 any
      220 deny ip 218.67.72.0/24 any
      230 deny ip 218.67.96.0/22 any
      240 deny ip 8.89.120.0/22 any
      250 deny ip 54.203.159.1/32 any
      260 permit ip 218.8.104.0/25 any
      270 permit ip 218.8.104.128/25 any
      280 permit ip 218.8.103.0/24 any
      290 deny ip 144.49.45.40/32 any
      300 deny ip 163.255.18.63/32 any
      310 deny ip 202.45.130.141/32 any
      320 deny ip 212.26.132.18/32 any
      330 deny ip 218.111.16.132/32 any
      340 deny ip 218.246.165.90/32 any
      350 deny ip 29.228.179.210/32 any
      360 deny ip 194.181.135.214/32 any
      370 deny ip 10.64.90.249/32 any
      380 deny ip 207.70.46.217/32 any
      390 deny ip 219.185.241.117/32 any
      400 deny ip 2.80.3.219/32 any
      410 deny ip 27.212.145.150/32 any
      420 deny ip 131.159.53.215/32 any
      430 deny ip 214.220.213.107/32 any
      440 deny ip 196.64.84.239/32 any
      450 deny ip 28.69.250.136/32 any
      460 deny ip 200.45.87.238/32 any
      470 deny ip any 11.125.89.32/30
      480 deny ip any 11.125.89.36/30
      490 deny ip any 11.125.89.40/30
      500 deny ip any 11.125.89.44/30
      510 permit icmp any any echo-reply
      520 deny ip any 11.36.199.216/30
      530 deny ip any 11.36.199.36/30
      540 deny ip any 11.36.199.2/30
      550 deny ip any 11.36.199.52/30
      560 deny ip any 11.36.199.20/30
      570 deny ip any 11.125.82.216/30
      580 deny ip any 11.125.82.220/32
      590 deny ip any 11.125.82.36/30
      600 deny ip any 11.125.82.12/30
      610 deny ip any 11.125.80.136/30
      620 deny ip any 11.125.80.141/32
      630 deny ip any 11.125.87.48/30
      640 deny ip any 11.125.87.168/30
      650 deny ip any 11.125.87.173/32
      660 deny ip any 11.125.90.56/30
      670 deny ip any 11.125.90.240/30
      680 deny ip any 11.125.74.224/30
      690 deny ip any 11.125.91.132/30
      700 deny ip any 11.125.89.132/30
      710 deny ip any 11.125.89.12/30
      720 deny ip any 11.125.92.108/30
      730 deny ip any 11.125.92.104/32
      740 deny ip any 11.125.92.28/30
      750 deny ip any 11.125.92.27/32
      760 deny ip any 11.125.92.160/30
      770 deny ip any 11.125.92.164/32
      780 deny ip any 11.125.92.204/30
      790 deny ip any 11.125.92.202/32
      800 deny ip any 11.125.93.192/29
      810 deny ip any 11.125.95.204/30
      820 deny ip any 11.125.95.224/30
      830 deny ip any 11.125.95.180/30
      840 deny ip any 11.125.95.156/30
      850 deny tcp any any
      860 deny icmp any any
      870 deny udp any any
      880 deny ip any any
    """
    
    Compressed ACL

    Now, assume that we want to compress this ACL to make it more manageable. We do the following operations:

    • Merge the two BFD permit statements on lines 20-30 into one statement using the range directive.

    • Remove the BGP session on line 80 because it has been decommissioned

    • Remove lines 180 and 250 because they are shadowed by earlier lines and will never match a packet. Such lines can be found via the filterLineReachability question, as shown here.

    • Merge pairs of lines (190, 200), (210, 220), and (260, 270) by combining their prefixes into a less specific prefix.

    • Remove all deny statements on lines 520-870. They are not needed given the final deny on line 880.

    The result of these actions, which halve the ACL size, is shown below. To enable easy observation of changes, we have preserved the line numbers.

    [2]:
    
    compressed_acl = """
    ip access-list acl
      10 deny icmp any any redirect
      20 permit udp 117.186.185.0/24 range 49152 65535 117.186.185.0/24 range 3784 3785
    ! 30 MERGED WITH LINE ABOVE
      40 permit tcp 11.36.216.170/32 11.36.216.169/32 eq bgp
      50 permit tcp 11.36.216.176/32 11.36.216.179/32 eq bgp
      60 permit tcp 204.150.33.175/32 204.150.33.83/32 eq bgp
      70 permit tcp 205.248.59.64/32 205.248.59.67/32 eq bgp
    ! 80 DECOMMISSIONED BGP SESSION
      90 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
      100 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
      110 deny ip 54.0.0.0/8 any
      120 deny ip 163.157.0.0/16 any
      130 deny ip 166.144.0.0/12 any
      140 deny ip 198.170.50.0/24 any
      150 deny ip 198.120.0.0/16 any
      160 deny ip 11.36.192.0/19 any
      170 deny ip 11.125.64.0/19 any
    ! 180 REMOVED UNREACHABLE LINE
      190 deny ip 218.66.56.0/23 any
    ! 200 MERGED WITH LINE ABOVE
      210 deny ip 218.67.71.0/23 any
    ! 220 MERGED WITH LINE ABOVE
      230 deny ip 218.67.96.0/22 any
      240 deny ip 8.89.120.0/22 any
    ! 250 REMOVED UNREACHABLE LINE
      260 permit ip 218.8.104.0/24 any
    ! 270 MERGED WITH LINE ABOVE
      280 permit ip 218.8.103.0/24 any
      290 deny ip 144.49.45.40/32 any
      300 deny ip 163.255.18.63/32 any
      310 deny ip 202.45.130.141/32 any
      320 deny ip 212.26.132.18/32 any
      330 deny ip 218.111.16.132/32 any
      340 deny ip 218.246.165.90/32 any
      350 deny ip 29.228.179.210/32 any
      360 deny ip 194.181.135.214/32 any
      370 deny ip 10.64.90.249/32 any
      380 deny ip 207.70.46.217/32 any
      390 deny ip 219.185.241.117/32 any
      400 deny ip 2.80.3.219/32 any
      410 deny ip 27.212.145.150/32 any
      420 deny ip 131.159.53.215/32 any
      430 deny ip 214.220.213.107/32 any
      440 deny ip 196.64.84.239/32 any
      450 deny ip 28.69.250.136/32 any
      460 deny ip 200.45.87.238/32 any
      470 deny ip any 11.125.89.32/28
      510 permit icmp any any echo-reply
    ! 520-870 REMOVED UNNECESSARY DENIES
      880 deny ip any any
    """
    

    The challenge for us is to find out if and how this compressed ACL differs from the original. That is, is there is traffic that is treated differently by the two ACLs, and if so, which lines are responsible for the difference.

    This task is difficult to get right through manual reasoning alone, which is why we developed the compareFilters question in Batfish.

    Comparing filters

    We can compare the two ACLs above as follows. To initialize snapshots, we will use Batfish’s init_snapshot_from_text function which creates a snapshot with a single device who configuration is the provided text. The analysis shown below can be done even when the filters are embedded within bigger device configurations.

    [3]:
    
    # Import packages
    %run startup.py
    bf = Session(host="localhost")
    
    # Initialize a snapshot with the original ACL
    original_snapshot = bf.init_snapshot_from_text(
        original_acl,
        platform="cisco-nx",
        snapshot_name="original",
        overwrite=True)
    
    # Initialize a snapshot with the compressed ACL
    compressed_snapshot = bf.init_snapshot_from_text(
        compressed_acl,
        platform="cisco-nx",
        snapshot_name="compressed",
        overwrite=True)
    
    # Now, compare the two ACLs in the two snapshots
    answer = bf.q.compareFilters().answer(snapshot=compressed_snapshot, reference_snapshot=original_snapshot)
    show(answer.frame())
    
    Node Filter_Name Line_Index Line_Content Line_Action Reference_Line_Index Reference_Line_Content
    0 config acl 16 210 deny ip 218.67.71.0/23 any DENY 50 510 permit icmp any any echo-reply
    1 config acl 40 510 permit icmp any any echo-reply PERMIT 21 220 deny ip 218.67.72.0/24 any
    2 config acl 41 880 deny ip any any DENY 7 80 permit tcp 205.248.58.190/32 205.248.58.188/32 eq bgp

    The compareFilters question compares two filters and returns pairs of lines, one from each filter, that match the same flow(s) but treat them differently. If it reports no output, the filters are guaranteed to be identical. The analysis is exhaustive and considers all possible flows.

    As we can see from the output above, our compressed ACL is not the same as the original one. In particular, line 210 of the compressed ACL will deny some flows that were being permitted by line 510 of the original; and line 510 of the compressed ACL will permit some flows that were being denied by line 220 of the original ACL. Because the permit statements correspond to ICMP traffic, we can tell that the traffic treated by the two filters is ICMP. To narrow learn specific source and destination IPs that are impacted, one may run the searchFilters question, as shown here.

    By looking at the output above, we can immediately understand the difference:

    • The first line is showing that the compressed ACL is denying some traffic on line 210 (with index 16) that the original ACL was permitting via line 510, and the compressed ACL is permitting some traffic on line 510 that the original ACL was denying via line 220.

      It turns out that the address space merger we did for lines 210 and 220 in the original ACL, where we combined 218.67.72.0/24 and 218.67.71.0/24 into 218.67.71.0/23, was not correct. The other similar mergers of 218.66.57.0/24 and 218.66.56.0/24 into 218.66.56.0/23 and of 218.8.104.0/25 and 218.8.104.128/25 into 218.8.104.0/24 were correct.

    • The third line is showing that the compressed ACL is denying some traffic at the end of the ACL that the original ACL was permitting via line 80. This is an expected change of decommissioning the BGP session on line 80.

      It is not always the case that refactoring is semantics preserving. Where compareFilters helps is succinctly enumerating all differences. Engineers can look at the differences and decide if the refactored filter meets their intent.

    Splitting ACLs

    Compressing large ACLs is one type of refactoring engineers do; another one is splitting a large ACL into multiple smaller ACLs and composing them on the same device or spreading across multiple devices in the network. Smaller ACLs are easier to maintain and evolve. However, the split operation is risky. We may forget to include in the smaller ACLs some protections that exist in the original ACL. We show how such splits can be safely done using Batfish.

    Suppose we want to split the compressed ACL above into multiple smaller ACLs that handle different concerns. So, we should have different ACLs for different types of traffic and different ACLs for different logical groups of nodes in the network. The result of such splitting is shown below. For ease of exposition, we have retained the line numbers from the original ACL and mimic a scenario in which all ACLs live on the same device.

    [4]:
    
    smaller_acls = """
    ip access-list deny-icmp-redirect
      10 deny icmp any any redirect
    
    ip access-list permit-bfd
      20 permit udp 117.186.185.0/24 range 49152 65535 117.186.185.0/24 range 3784 3785
    
    ip access-list permit-bgp-session
      40 permit tcp 11.36.216.170/32 11.36.216.169/32 eq bgp
      50 permit tcp 11.36.216.176/32 11.36.216.179/32 eq bgp
      60 permit tcp 204.150.33.175/32 204.150.33.83/32 eq bgp
      70 permit tcp 205.248.59.64/32 205.248.59.67/32 eq bgp
    
    ip access-list acl-dns
      90 deny udp 10.10.10.42/32 218.8.104.58/32 eq domain
      100 permit udp 10.10.10.0/24 218.8.104.58/32 eq domain
    
    ip access-list deny-untrusted-sources-group1
      110 deny ip 54.0.0.0/8 any
      120 deny ip 163.157.0.0/16 any
      130 deny ip 166.144.0.0/12 any
      140 deny ip 198.170.50.0/24 any
      150 deny ip 198.120.0.0/16 any
      160 deny ip 11.36.192.0/19 any
    
    ip access-list deny-untrusted-sources-group2
      160 deny ip 11.36.192.0/20 any
      190 deny ip 218.66.56.0/23 any
      210 deny ip 218.67.71.0/23 any
      230 deny ip 218.67.96.0/22 any
      240 deny ip 8.89.120.0/22 any
    
    ip access-list permit-trusted-sources
      260 permit ip 218.8.104.0/24 any
      280 permit ip 218.8.103.0/24 any
    
    ip access-list deny-untrusted-sources-group3
      290 deny ip 144.49.45.40/32 any
      300 deny ip 163.255.18.63/32 any
      310 deny ip 202.45.130.141/32 any
      320 deny ip 212.26.132.18/32 any
      300 deny ip 218.111.16.132/32 any
      340 deny ip 218.246.165.90/32 any
      350 deny ip 29.228.179.210/32 any
      360 deny ip 194.181.135.214/32 any
      370 deny ip 10.64.90.249/32 any
      380 deny ip 207.70.46.217/32 any
      390 deny ip 219.185.241.117/32 any
    
    ip access-list deny-untrusted-sources-group4
      400 deny ip 2.80.3.219/32 any
      410 deny ip 27.212.145.150/32 any
      420 deny ip 131.159.53.215/32 any
      430 deny ip 214.220.213.107/32 any
      440 deny ip 196.64.84.239/32 any
      450 deny ip 28.69.250.136/32 any
      460 deny ip 200.45.87.238/32 any
    
    ip access-list acl-tail
      470 deny ip any 11.125.89.32/28
      510 permit icmp any any echo-reply
      880 deny ip any any
    """
    

    Given the split ACLs above, one analysis may be to figure out if each untrusted source subnet was included in a smaller ACL. Otherwise, we have lost protection that was present in the original ACL. We can accomplish this analysis via the findMatchingFilterLines question, as shown below.

    Once we are satisfied with analysis of filters, for an end-to-end safety guarantee, we should also analyze if there are new flows that the network will allow (or disallow) after the change. Such an analysis can be done via the differentialReachability question, as shown here.

    [5]:
    
    # Initialize a snapshot with the smaller ACLs
    smaller_snapshot = bf.init_snapshot_from_text(
        smaller_acls,
        platform="cisco-nx",
        snapshot_name="smaller",
        overwrite=True)
    
    # All untrusted subnets
    untrusted_source_subnets = ["54.0.0.0/8",
                                "163.157.0.0/16",
                                "166.144.0.0/12",
                                "198.170.50.0/24",
                                "198.120.0.0/16",
                                "11.36.192.0/19",
                                "11.125.64.0/19",
                                "218.66.56.0/24",
                                "218.66.57.0/24",
                                "218.67.71.0/23",
                                "218.67.96.0/22",
                                "8.89.120.0/22"
                               ]
    
    for subnet in untrusted_source_subnets:
        # Find which ACLs match traffic from this source subnet
        answer = bf.q.findMatchingFilterLines(
            headers=HeaderConstraints(srcIps=subnet),
            filters="/deny-untrusted/").answer(snapshot=smaller_snapshot)
    
        # Each source subnet should match exactly one ACL
        af = answer.frame()
        if len(af) == 1:
            print("{} .... OK".format(subnet))
        elif len(af) == 0:
            print("{} .... ABSENT".format(subnet))
        else:
            print("{} .... Multiply present".format(subnet))
            show(af)
    
    54.0.0.0/8 .... OK
    163.157.0.0/16 .... OK
    166.144.0.0/12 .... OK
    198.170.50.0/24 .... OK
    198.120.0.0/16 .... OK
    11.36.192.0/19 .... Multiply present
    
    Node Filter Line Line_Index Action
    0 config deny-untrusted-sources-group1 160 deny ip 11.36.192.0/19 any 5 DENY
    1 config deny-untrusted-sources-group2 160 deny ip 11.36.192.0/20 any 0 DENY
    11.125.64.0/19 .... ABSENT
    218.66.56.0/24 .... OK
    218.66.57.0/24 .... OK
    218.67.71.0/23 .... OK
    218.67.96.0/22 .... OK
    8.89.120.0/22 .... OK
    

    In the code above, we first enumerate all untrusted subnets in the network. The granularity of this specification need not be the same as that in the ACL. For instance, we enumerate 218.66.56.0/24 and 218.66.57.0/24 as untrusted subnets but the ACL has a less specific prefix 218.66.56.0/23. Batfish understands such relationships and provides an accurate analysis that is not possible with simple string matching.

    The for loop above uses the findMatchingFilterLines question to find out which lines across all ACLs whose names contain “deny-untrusted” will match packets starting the the specified subnet. Our expectation is that each subnet should match exactly one line in exactly one ACL, and the output shows “OK” against such subnets. It shows “Absent” for subnets that do not match any line and shows the multiple matching lines for subnets where that happens.

    We see that during the split above, we ended up matching the subnet 11.36.192.0/19 twice, once as a /19 in ACL deny-untrusted-sources-group1 and then as /20 in ACL deny-untrusted-sources-group2. More dangerously, we completely forgot to match the 11.125.64.0/19, which will open a security hole in the network if these smaller ACLs were applied.

    Summary

    In this notebook, we showed how to use the compareFilters and findMatchingFilterLines questions of Batfish to safely refactor complex filters.

    • compareFilters analyzes the original and revised filter to enumerate all cases that will treat any flow differently.

    • findMatchingFilterLines enumerates all lines across all specified filters that match the given space of flows.

    For additional ways to analyze filter using Batfish, see the “Analyzing ACLs and Firewall Rules” and the “Provably Safe ACL and Firewall Changes” notebooks.


    Get involved with the Batfish community

    Join our community on Slack and GitHub.

    Routing analysis

    Introduction to Route Computation and Analysis using Batfish

    Network engineers routinely need to validate routing and forwarding in the network. They often do that by connecting to multiple network devices and executing a series of show route commands. This distributed debugging is highly complex even in a moderately-sized network. Batfish makes this task extremely simple by providing an easy-to-query, centralized view of routing tables in the network.

    In this notebook, we will look at how you can extract routing information from Batfish.

    Check out a video demo of this notebook here.

    [1]:
    
    # Import packages
    %run startup.py
    bf = Session(host="localhost")
    
    Initializing the Network and Snapshot

    SNAPSHOT_PATH below can be updated to point to a custom snapshot directory, see the Batfish instructions for how to package data for analysis. More example networks are available in the networks folder of the Batfish repository.

    [2]:
    
    # Initialize a network and snapshot
    NETWORK_NAME = "example_network"
    SNAPSHOT_NAME = "example_snapshot"
    
    SNAPSHOT_PATH = "networks/example"
    
    bf.set_network(NETWORK_NAME)
    bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
    
    [2]:
    
    'example_snapshot'
    

    The network snapshot that we initialized above is illustrated below. You can download/view devices’ configuration files here.

    example-network

    All of the information we will show you in this notebook is dynamically computed by Batfish based on the configuration files for the network devices.

    View Routing Tables for ALL devices and ALL VRFs

    Batfish makes all routing tables in the network easily accessible. Let’s take a look at how you can retrieve the specific information you want.

    [3]:
    
    # Get routing tables for all nodes and VRFs
    routes_all = bf.q.routes().answer().frame()
    

    We are not going to print this table as it has a large number of entries.

    View Routing Tables for default VRF on AS1 border routers

    There are 2 ways that we can get the desired subset of data:

    Option 1) Only request that information from Batfish by passing in parameters into the routes() question. This is useful to do when you need to reduce the amount of data being returned, but is limited to regex filtering based on VRF, Node, Protocol and Network.

    Option 2) Filter the output of the routes() question using the Pandas APIs.

    [4]:
    
    ?bf.q.routes
    
    [5]:
    
    # Get the routing table for the 'default' VRF on border routers of as1
    # using BF parameters
    routes_as1border = bf.q.routes(nodes="/as1border/", vrfs="default").answer().frame()
    
    [6]:
    
    # Get the routing table for the 'default' VRF on border routers of as1
    # using Pandas filtering
    routes_as1border = routes_all[(routes_all['Node'].str.contains('as1border')) & (routes_all['VRF'] == 'default')]
    routes_as1border
    
    [6]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    0 as1border1 default 1.0.1.0/24 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 connected 0 0 None
    1 as1border1 default 1.0.1.1/32 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 local 0 0 None
    2 as1border1 default 1.0.2.0/24 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 2 110 None
    3 as1border1 default 1.1.1.1/32 interface Loopback0 AUTO/NONE(-1l) Loopback0 connected 0 0 None
    4 as1border1 default 1.2.2.2/32 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 3 110 None
    5 as1border1 default 1.10.1.1/32 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 2 110 None
    6 as1border1 default 2.128.0.0/16 ip 10.12.11.2 10.12.11.2 dynamic bgp 50 20 None
    7 as1border1 default 3.0.1.0/24 ip 10.13.22.3 10.13.22.3 dynamic ibgp 50 200 None
    8 as1border1 default 3.0.2.0/24 ip 10.13.22.3 10.13.22.3 dynamic ibgp 50 200 None
    9 as1border1 default 10.12.11.0/24 interface GigabitEthernet1/0 AUTO/NONE(-1l) GigabitEthernet1/0 connected 0 0 None
    10 as1border1 default 10.12.11.1/32 interface GigabitEthernet1/0 AUTO/NONE(-1l) GigabitEthernet1/0 local 0 0 None
    11 as1border1 default 10.13.22.0/24 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospfE2 20 110 None
    12 as1border1 default 10.14.22.0/24 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospfE2 20 110 None
    13 as1border2 default 1.0.1.0/24 interface GigabitEthernet1/0 ip 1.0.2.2 1.0.2.2 GigabitEthernet1/0 ospf 2 110 None
    14 as1border2 default 1.0.2.0/24 interface GigabitEthernet1/0 AUTO/NONE(-1l) GigabitEthernet1/0 connected 0 0 None
    15 as1border2 default 1.0.2.1/32 interface GigabitEthernet1/0 AUTO/NONE(-1l) GigabitEthernet1/0 local 0 0 None
    16 as1border2 default 1.1.1.1/32 interface GigabitEthernet1/0 ip 1.0.2.2 1.0.2.2 GigabitEthernet1/0 ospf 3 110 None
    17 as1border2 default 1.2.2.2/32 interface Loopback0 AUTO/NONE(-1l) Loopback0 connected 0 0 None
    18 as1border2 default 1.10.1.1/32 interface GigabitEthernet1/0 ip 1.0.2.2 1.0.2.2 GigabitEthernet1/0 ospf 2 110 None
    19 as1border2 default 2.128.0.0/16 ip 10.12.11.2 10.12.11.2 dynamic ibgp 50 200 None
    20 as1border2 default 3.0.1.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    21 as1border2 default 3.0.2.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    22 as1border2 default 10.12.11.0/24 interface GigabitEthernet1/0 ip 1.0.2.2 1.0.2.2 GigabitEthernet1/0 ospfE2 20 110 None
    23 as1border2 default 10.13.22.0/24 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 connected 0 0 None
    24 as1border2 default 10.13.22.1/32 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 local 0 0 None
    25 as1border2 default 10.14.22.0/24 interface GigabitEthernet2/0 AUTO/NONE(-1l) GigabitEthernet2/0 connected 0 0 None
    26 as1border2 default 10.14.22.1/32 interface GigabitEthernet2/0 AUTO/NONE(-1l) GigabitEthernet2/0 local 0 0 None
    View BGP learnt routes for default VRF on AS1 border routers
    [7]:
    
    # Getting BGP routes in the routing table for the 'default' VRF on border routers of as1
    # using BF parameters
    routes_as1border_bgp = bf.q.routes(nodes="/as1border/", vrfs="default", protocols="bgp").answer().frame()
    
    [8]:
    
    # Geting BGP routes in the routing table for the 'default' VRF on border routers of as1
    # using Pandas filtering
    routes_as1border_bgp = routes_all[(routes_all['Node'].str.contains('as1border')) & (routes_all['VRF'] == 'default') & (routes_all['Protocol'] == 'bgp')]
    routes_as1border_bgp
    
    [8]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    6 as1border1 default 2.128.0.0/16 ip 10.12.11.2 10.12.11.2 dynamic bgp 50 20 None
    20 as1border2 default 3.0.1.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    21 as1border2 default 3.0.2.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    View BGP learnt routes for ALL VRFs on ALL routers with Metric >=50

    We cannot pass in metric as a parameter to Batfish, so this task is best handled with the Pandas API.

    [9]:
    
    routes_filtered = routes_all[(routes_all['Protocol'] == 'bgp') & (routes_all['Metric'] >= 50)]
    routes_filtered
    
    [9]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    6 as1border1 default 2.128.0.0/16 ip 10.12.11.2 10.12.11.2 dynamic bgp 50 20 None
    20 as1border2 default 3.0.1.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    21 as1border2 default 3.0.2.0/24 ip 10.13.22.3 10.13.22.3 dynamic bgp 50 20 None
    40 as2border1 default 1.0.1.0/24 ip 10.12.11.1 10.12.11.1 dynamic bgp 50 20 None
    41 as2border1 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic bgp 50 20 None
    106 as2border2 default 3.0.1.0/24 ip 10.23.21.3 10.23.21.3 dynamic bgp 50 20 None
    107 as2border2 default 3.0.2.0/24 ip 10.23.21.3 10.23.21.3 dynamic bgp 50 20 None
    182 as2dept1 default 1.0.1.0/24 ip 2.34.101.3 2.34.101.3 dynamic bgp 50 20 None
    183 as2dept1 default 1.0.1.0/24 ip 2.34.201.3 2.34.201.3 dynamic bgp 50 20 None
    184 as2dept1 default 1.0.2.0/24 ip 2.34.101.3 2.34.101.3 dynamic bgp 50 20 None
    185 as2dept1 default 1.0.2.0/24 ip 2.34.201.3 2.34.201.3 dynamic bgp 50 20 None
    195 as2dept1 default 3.0.1.0/24 ip 2.34.101.3 2.34.101.3 dynamic bgp 50 20 None
    196 as2dept1 default 3.0.1.0/24 ip 2.34.201.3 2.34.201.3 dynamic bgp 50 20 None
    197 as2dept1 default 3.0.2.0/24 ip 2.34.101.3 2.34.101.3 dynamic bgp 50 20 None
    198 as2dept1 default 3.0.2.0/24 ip 2.34.201.3 2.34.201.3 dynamic bgp 50 20 None
    226 as2dist1 default 2.128.0.0/24 ip 2.34.101.4 2.34.101.4 dynamic bgp 50 20 None
    227 as2dist1 default 2.128.1.0/24 ip 2.34.101.4 2.34.101.4 dynamic bgp 50 20 None
    261 as2dist2 default 2.128.0.0/24 ip 2.34.201.4 2.34.201.4 dynamic bgp 50 20 None
    262 as2dist2 default 2.128.1.0/24 ip 2.34.201.4 2.34.201.4 dynamic bgp 50 20 None
    271 as3border1 default 2.128.0.0/16 ip 10.23.21.2 10.23.21.2 dynamic bgp 50 20 None
    281 as3border2 default 1.0.1.0/24 ip 10.13.22.1 10.13.22.1 dynamic bgp 50 20 None
    282 as3border2 default 1.0.2.0/24 ip 10.13.22.1 10.13.22.1 dynamic bgp 50 20 None
    View the routing entries for network 1.0.2.0/24 on ALL routers in ALL VRFs
    [10]:
    
    # grab the route table entry for network 1.0.2.0/24 from all routers in all VRFs
    # using BF parameters
    routes_filtered = bf.q.routes(network="1.0.2.0/24").answer().frame()
    
    [11]:
    
    # grab the route table entry for network 1.0.2.0/24 from all routers in all VRFs
    # using Pandas filtering
    routes_filtered = routes_all[routes_all['Network'] == "1.0.2.0/24"]
    routes_filtered
    
    [11]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    2 as1border1 default 1.0.2.0/24 interface GigabitEthernet0/0 ip 1.0.1.2 1.0.1.2 GigabitEthernet0/0 ospf 2 110 None
    14 as1border2 default 1.0.2.0/24 interface GigabitEthernet1/0 AUTO/NONE(-1l) GigabitEthernet1/0 connected 0 0 None
    29 as1core1 default 1.0.2.0/24 interface GigabitEthernet0/0 AUTO/NONE(-1l) GigabitEthernet0/0 connected 0 0 None
    41 as2border1 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic bgp 50 20 None
    77 as2border2 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic ibgp 50 200 None
    113 as2core1 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic ibgp 50 200 None
    148 as2core2 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic ibgp 50 200 None
    184 as2dept1 default 1.0.2.0/24 ip 2.34.101.3 2.34.101.3 dynamic bgp 50 20 None
    185 as2dept1 default 1.0.2.0/24 ip 2.34.201.3 2.34.201.3 dynamic bgp 50 20 None
    200 as2dist1 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic ibgp 50 200 None
    235 as2dist2 default 1.0.2.0/24 ip 10.12.11.1 10.12.11.1 dynamic ibgp 50 200 None
    270 as3border1 default 1.0.2.0/24 ip 10.13.22.1 10.13.22.1 dynamic ibgp 50 200 None
    282 as3border2 default 1.0.2.0/24 ip 10.13.22.1 10.13.22.1 dynamic bgp 50 20 None
    294 as3core1 default 1.0.2.0/24 ip 10.13.22.1 10.13.22.1 dynamic ibgp 50 200 None

    Using Panda’s filtering it is easy to retrieve the list of nodes which have the network in the routing table for at least 1 VRF. This type of processing should always be done using the Pandas APIs.

    [12]:
    
    # Get the list of nodes that have the network 1.0.2.0/24 in at least 1 VRF
    # the .unique function removes duplicate entries that would have been returned if the network was in multiple VRFs on a node or there were
    # multiple route entries for the network (ECMP)
    print(sorted(routes_filtered["Node"].unique()))
    
    ['as1border1', 'as1border2', 'as1core1', 'as2border1', 'as2border2', 'as2core1', 'as2core2', 'as2dept1', 'as2dist1', 'as2dist2', 'as3border1', 'as3border2', 'as3core1']
    

    Now we will retrieve the list of nodes that do NOT have this prefix in their routing table. This is easy to do with the Pandas groupby and filter functions.

    [13]:
    
    # Group all routes by Node and filter for those that don't have '1.0.2.0/24'
    routes_filtered = routes_all.groupby('Node').filter(lambda x: all(x['Network'] != '1.0.2.0/24'))
    
    # Get the unique node names and sort the list
    print(sorted(routes_filtered["Node"].unique()))
    
    ['host1', 'host2']
    

    The only devices that do not have a route to 1.0.2.0/24 are the 2 hosts in the snapshot. This is expected, as they should just have a default route. Let’s verify that.

    [14]:
    
    routes_all[routes_all['Node'].str.contains('host')]
    
    [14]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    309 host1 default 0.0.0.0/0 interface eth0 ip 2.128.0.1 2.128.0.1 eth0 static 0 1 None
    310 host1 default 2.128.0.0/24 interface eth0 AUTO/NONE(-1l) eth0 connected 0 0 None
    311 host1 default 2.128.0.101/32 interface eth0 AUTO/NONE(-1l) eth0 local 0 0 None
    312 host2 default 0.0.0.0/0 interface eth0 ip 2.128.1.1 2.128.1.1 eth0 static 0 1 None
    313 host2 default 2.128.1.0/24 interface eth0 AUTO/NONE(-1l) eth0 connected 0 0 None
    314 host2 default 2.128.1.101/32 interface eth0 AUTO/NONE(-1l) eth0 local 0 0 None

    With Batfish and Pandas you can easily retrieve the exact information you are looking for from the routing tables on ANY or ALL network devices for ANY or ALL VRFs.

    This concludes the notebook. To recap, in this notebook we covered the foundational tasks for route analysis:

    1. How to get routes at all nodes in the network or only at a subset of them

    2. How to retrieve routing entries that match a specific protocol or metric

    3. How to find which nodes have an entry for a prefix or which ones do not

    We hope you found this notebook useful and informative. Future notebooks will dive into more advanced topics like path analysis, debugging ACLs and firewall rules, validating routing policy, etc.. so stay tuned!

    Want to know more?

    Reach out to us through Slack or GitHub to learn more, or send feedback.

    Introduction to BGP Analysis using Batfish

    Network engineers routinely need to validate BGP configuration and session status in the network. They often do that by connecting to multiple network devices and executing a series of show ip bgp commands. This distributed debugging is highly complex even in a moderately-sized network. And it is reactive, the configuration changes are already in the network.

    Batfish allows network engineers to proactively validate BGP configuration to ensure sessions are compatibly configured and will be established, thereby avoiding potential network outages.

    In this notebook, we will look at how you can extract BGP configuration and session status information from Batfish.

    [1]:
    
    # Import packages
    %run startup.py
    bf = Session(host="localhost")
    
    Initializing the Network and Snapshot

    SNAPSHOT_PATH below can be updated to point to a custom snapshot directory, see the Batfish instructions for how to package data for analysis. More example networks are available in the networks folder of the Batfish repository.

    [2]:
    
    # Initialize a network and snapshot
    NETWORK_NAME = "example_network"
    SNAPSHOT_NAME = "example_snapshot"
    
    SNAPSHOT_PATH = "networks/example-bgp"
    
    bf.set_network(NETWORK_NAME)
    bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
    
    [2]:
    
    'example_snapshot'
    

    The network snapshot that we initialized above is illustrated below. You can download/view devices’ configuration files here.

    example-bgp-network

    All of the information we will show you in this notebook is dynamically computed by Batfish based on the configuration files for the network devices.

    View BGP Configuration for ALL devices

    Batfish makes BGP Configuration settings in the network easily accessible. Let’s take a look at how you can retrieve the specific information you want. Let’s start with configuration attributes of the BGP process on all devices running BGP by using the question bf.q.bgpProcessConfiguration.

    [3]:
    
    # Get BGP process configuration information for ALL devices
    bgp_config = bf.q.bgpProcessConfiguration().answer().frame()
    bgp_config
    
    [3]:
    
    Node VRF Router_ID Confederation_ID Confederation_Members Multipath_EBGP Multipath_IBGP Multipath_Match_Mode Neighbors Route_Reflector Tie_Breaker
    0 as2border2 default 2.1.1.2 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '10.23.21.3'] False ARRIVAL_ORDER
    1 as3core1 default 3.10.1.1 None None True True EXACT_PATH ['3.1.1.1', '3.2.2.2'] True ARRIVAL_ORDER
    2 as2border1 default 2.1.1.1 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '10.12.11.1'] False ARRIVAL_ORDER
    3 as1core1 default 1.10.1.1 None None True True EXACT_PATH ['1.1.1.1', '1.2.2.2'] True ARRIVAL_ORDER
    4 as1border1 default 1.1.1.1 None None True True EXACT_PATH ['1.10.1.1', '3.2.2.2', '5.6.7.8', '10.12.11.2'] False ARRIVAL_ORDER
    5 as2core1 default 2.1.2.1 None None True True EXACT_PATH ['2.1.1.1', '2.1.1.2', '2.1.3.1', '2.1.3.2'] True ARRIVAL_ORDER
    6 as3border2 default 3.2.2.2 None None True True EXACT_PATH ['3.10.1.1', '10.13.22.1'] False ARRIVAL_ORDER
    7 as2core2 default 2.1.2.2 None None True True EXACT_PATH ['2.1.1.1', '2.1.1.2', '2.1.3.1', '2.1.3.2'] True ARRIVAL_ORDER
    8 as1border2 default 1.2.2.2 None None True True EXACT_PATH ['1.10.1.1', '10.13.22.3', '10.14.22.4'] False ARRIVAL_ORDER
    9 as2dept1 default 2.1.4.1 None None True True EXACT_PATH ['2.34.101.3', '2.34.201.3'] False ARRIVAL_ORDER
    10 as3border1 default 3.1.1.1 None None True True EXACT_PATH ['3.10.1.1', '10.23.21.2'] False ARRIVAL_ORDER
    11 as2dist2 default 2.1.3.2 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '2.34.0.0/16'] False ARRIVAL_ORDER
    12 as2dist1 default 2.1.3.1 None None True True EXACT_PATH ['2.1.2.1', '2.1.2.2', '2.34.0.0/16'] False ARRIVAL_ORDER

    Now let’s drill into the configuration of a specific BGP session. Let’s look at the sessions on as2dept1. To do this, we will use the bf.q.bgpPeerConfiguration question.

    [4]:
    
    # Get all of the BGP peer configuration for as2dept1 devices
    bgp_peer_config = bf.q.bgpPeerConfiguration(nodes='as2dept1').answer().frame()
    bgp_peer_config
    
    [4]:
    
    Node VRF Local_AS Local_IP Local_Interface Confederation Remote_AS Remote_IP Description Route_Reflector_Client Cluster_ID Peer_Group Import_Policy Export_Policy Send_Community Is_Passive
    0 as2dept1 default 65001 2.34.101.4 None None 2 2.34.101.3 None False None as2 ['as2_to_dept'] ['dept_to_as2'] True False
    1 as2dept1 default 65001 2.34.201.4 None None 2 2.34.201.3 None False None as2 ['as2_to_dept'] ['dept_to_as2'] True False
    View BGP Session Status

    Now that we have seen the configuration for each peer on as2dept1, let’s ensure that their configuration is compatible with their peers

    The bgpSessionCompatibility question allows you to ensure that BGP sessions are compatibly configured, so that if there is IP reachability between the peers the sessions will be established. Compatiblity checks that the remote-as matches up on both ends, the correct update source is specified, peer-ip addresses match-up, etc…

    [5]:
    
    # Check if the bgp Sessions on as2dept1 are properly configured
    bgpSessCompat = bf.q.bgpSessionCompatibility(nodes='as2dept1').answer().frame()
    bgpSessCompat
    
    [5]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Configured_Status
    0 as2dept1 default 65001 None 2.34.101.4 2 as2dist1 None 2.34.101.3 ['IPV4_UNICAST'] EBGP_SINGLEHOP UNIQUE_MATCH
    1 as2dept1 default 65001 None 2.34.201.4 2 as2dist2 None 2.34.201.3 ['IPV4_UNICAST'] EBGP_SINGLEHOP UNIQUE_MATCH

    Both of the configured BGP peers on as2dept1 are compatible. We know that since the Configured_Status is UNIQUE_MATCH. So now let’s check if they are established.

    [6]:
    
    # Check if the bgp Sessions on as2dept1 are ESTABLISHED
    bgpSessStat = bf.q.bgpSessionStatus(nodes='as2dept1').answer().frame()
    bgpSessStat
    
    [6]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
    0 as2dept1 default 65001 None 2.34.101.4 2 as2dist1 None 2.34.101.3 ['IPV4_UNICAST'] EBGP_SINGLEHOP ESTABLISHED
    1 as2dept1 default 65001 None 2.34.201.4 2 as2dist2 None 2.34.201.3 ['IPV4_UNICAST'] EBGP_SINGLEHOP ESTABLISHED

    Both sessions are established. Let’s see if there are any configured BGP sessions, on any other device in the network, that are not established

    [7]:
    
    # Find any BGP sessions in the network that are NOT ESTABLISHED
    bgpSessStat = bf.q.bgpSessionStatus().answer().frame()
    bgpSessStat[bgpSessStat['Established_Status'] != 'ESTABLISHED']
    
    [7]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
    1 as1border1 default 1 None None 666 None None 3.2.2.2 [] EBGP_SINGLEHOP NOT_COMPATIBLE
    2 as1border1 default 1 None None 555 None None 5.6.7.8 [] EBGP_SINGLEHOP NOT_COMPATIBLE
    6 as1border2 default 1 None 10.14.22.1 4 None None 10.14.22.4 [] EBGP_SINGLEHOP NOT_COMPATIBLE
    9 as2border1 default 2 None 2.1.1.1 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    12 as2border2 default 2 None 2.1.1.2 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    15 as2core1 default 2 None 2.1.2.1 2 as2border1 None 2.1.1.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    16 as2core1 default 2 None 2.1.2.1 2 as2border2 None 2.1.1.2 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    17 as2core1 default 2 None 2.1.2.1 2 as2dist1 None 2.1.3.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    18 as2core1 default 2 None 2.1.2.1 2 as2dist2 None 2.1.3.2 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    25 as2dist1 default 2 None 2.1.3.1 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    28 as2dist2 default 2 None 2.1.3.2 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED

    Looking at the Established_Status column we see that there are a lot of sessions that are configured, but not established. Let’s dig into these issues.

    First, let’s find all of the sessions that are not compatibly configured. For that we are looking for sessions which are not either a UNIQUE_MATCH or DYNAMIC_MATCH. The latter is for dynamic peers (these are peers configured to use a listen-range).

    Debug NOT_COMPATIBLE sessions
    [8]:
    
    # Find BGP sessions that are not compatibly configured - i.e Batfish does not identify them as being a UNIQUE_MATCH or a DYNAMIC_MATCH
    bgpSessCompat = bf.q.bgpSessionCompatibility().answer().frame()
    bgpSessCompat[~bgpSessCompat['Configured_Status'].isin(["UNIQUE_MATCH", "DYNAMIC_MATCH"])]
    
    [8]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Configured_Status
    1 as1border1 default 1 None None 666 None None 3.2.2.2 [] EBGP_SINGLEHOP NO_LOCAL_IP
    2 as1border1 default 1 None None 555 None None 5.6.7.8 [] EBGP_SINGLEHOP NO_LOCAL_IP
    6 as1border2 default 1 None 10.14.22.1 4 None None 10.14.22.4 [] EBGP_SINGLEHOP UNKNOWN_REMOTE
    13 as2border2 default 2 None None 2 None None 2.1.2.2 [] IBGP LOCAL_IP_UNKNOWN_STATICALLY

    We see 4 entries in this table even though we only saw 3 BGP session with thes status NOT_COMPATIBLE. This means that despite not having a Configured_Status of UNIQUE_MATCH or DYNAMIC_MATCH one of these sessions was indeed established. This typically occurs if the mis-configuration is such that the session can ONLY be established when initiated from one side, but not the other. The likely candidate in this output is the as2border2 session to 2.1.2.2 that has status of LOCAL_IP_UNKNOWN_STATICALLY.

    Debug UNKNOWN_REMOTE peer on as1border2

    Batfish deems the BGP peer 10.14.22.4 is not compatible because it cannot find a device in the snapshot that has that IP address configured on any interface. That is why the status is UNKNOWN_REMOTE. This will occur in most networks, since you will not have the configurations of the devices for your external peers (ISPs, content partners, etc…). We can easily verify this by checking the output of bf.q.ipOwners

    [9]:
    
    # check if there is a node in the network that has the `10.14.22.4` on an interface
    ipOwn = bf.q.ipOwners().answer().frame()
    ipOwn[ipOwn['IP']=='10.14.22.4']
    
    [9]:
    
    Node VRF Interface IP Mask Active
    Debug LOCAL_IP_UNKNOWN_STATICALLY on as2border2

    Now, let’s dig into the sessions that are LOCAL_IP_UNKNOWN_STATICALLY.

    An iBGP session will have the status LOCAL_IP_UNKNOWN_STATICALLY if you are missing the update-source command.

    So, in this case, it is likely that the issue is that as2border2 is missing the update-source command for the BGP session. This is needed for iBGP sessions to ensure the peers pick the correct IP address to use when trying to establish the TCP session. We can find out the target remote node by looking at the bf.q.ipOwners output

    [10]:
    
    #find the device(s) that own these ip addresses
    ipOwn[ipOwn['IP'].isin(['2.1.2.2'])]
    
    [10]:
    
    Node VRF Interface IP Mask Active
    22 as2core2 default Loopback0 2.1.2.2 32 True

    So this session is supposed to be between as2border2 and as2core2. Let’s see if this session is established.

    [11]:
    
    # check if either direction of the session can be established
    bgpSessStat[((bgpSessStat['Local_IP']=='2.1.1.2') & (bgpSessStat['Remote_IP']=='2.1.2.2')) | ((bgpSessStat['Local_IP']=='2.1.2.2') & (bgpSessStat['Remote_IP']=='2.1.1.2'))]
    
    [11]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
    20 as2core2 default 2 None 2.1.2.2 2 as2border2 None 2.1.1.2 ['IPV4_UNICAST'] IBGP ESTABLISHED

    This confirms that our theory. We are missing the update-source command on as2border2 which prevents it from initiating the BGP session. But since as2core2 is properly configured, the session is established.

    Debug NO_LOCAL_IP on as1border1
    [12]:
    
    # Identify the BGP sessions on as1border1 that are not compatible with Configured_Status of NO_LOCAL_IP
    bgpSessCompat[(bgpSessCompat['Configured_Status'] == 'NO_LOCAL_IP') & (bgpSessCompat['Node']=='as1border1')]
    
    [12]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Configured_Status
    1 as1border1 default 1 None None 666 None None 3.2.2.2 [] EBGP_SINGLEHOP NO_LOCAL_IP
    2 as1border1 default 1 None None 555 None None 5.6.7.8 [] EBGP_SINGLEHOP NO_LOCAL_IP

    For an EBGP_SINGLEHOP session to have CONFIGURED_STATUS of NO_LOCAL_IP this typically means that no interface exists on the box which is in the same subnet as the BGP peer’s IP address. This can easily be checked by looking at the IP addresses configued on as1border1.

    [13]:
    
    # Identify the IP addresses that are owned by as1border1
    ipOwn[ipOwn['Node']=='as1border1']
    
    [13]:
    
    Node VRF Interface IP Mask Active
    7 as1border1 default GigabitEthernet1/0 10.12.11.1 24 True
    14 as1border1 default GigabitEthernet0/0 1.0.1.1 24 True
    19 as1border1 default Loopback0 1.1.1.1 32 True

    As you can see there are no interfaces with addresses that would be in the same subnet as 3.2.2.2 and 5.6.7.8. There could be many explanations for this:

    1. BGP peers were configured before the physical connection to neighboring routers was up.

    2. The user simply configured the wrong BGP peer address.

    3. The interfaces used to exist but were decommissioned, but the BGP config was not cleaned up at the same time.

    4. The session is meant to be be an eBGP multi-hop session, but the user didn’t add the ebgp multihop configuration option and specify an update-source.

    Batfish determines the Local_IP for each BGP session, either based on explicit configuration with updates-source foo (or equivalent non-IOS command) for iBGP sessions or eBGP multi-hop sessions, or by determining the interface the router will use to send packets towards the BGP peer. The latter method requires the route to the peer to be known.

    So, if there is no route to the configured peer, or the configured peer does not exist in the snapshot, you will see this status. We can check the output of bf.q.routes and bf.q.ipOwners questions to dig into this

    [14]:
    
    # Find owner of IP addresses for the incompatible BGP sessions on as1border`
    bad_bgp_peer = ['3.2.2.2', '5.6.7.8']
    ipOwn[ipOwn['IP'].isin(bad_bgp_peer)]
    
    [14]:
    
    Node VRF Interface IP Mask Active
    10 as3border2 default Loopback0 3.2.2.2 32 True

    So we can see that 5.6.7.8 does not exist in the network. This either means there is a mis-configuration, or the device is just not expected to be in the snapshot.

    Now let’s dig into the peer 3.2.2.2, which we know is the Loopback interface on as3border2

    [15]:
    
    # retrieve the routing table entry for as3border2 loopback0 - 3.2.2.2/32 on as1border1
    routes = bf.q.routes(network='3.2.2.2/32').answer().frame()
    routes[routes['Node']=='as1border1']
    
    [15]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag

    The specific /32 is not present on as1border1. What about other routers?

    [16]:
    
    # retrieve the routing table entry for as3border2 loopback0 - 3.2.2.2/32 on ALL routers in the network
    routes
    
    [16]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    0 as3border1 default 3.2.2.2/32 interface GigabitEthernet0/0 ip 3.0.1.2 3.0.1.2 GigabitEthernet0/0 ospf 3 110 None
    1 as3border2 default 3.2.2.2/32 interface Loopback0 AUTO/NONE(-1l) Loopback0 connected 0 0 None
    2 as3core1 default 3.2.2.2/32 interface GigabitEthernet0/0 ip 3.0.2.1 3.0.2.1 GigabitEthernet0/0 ospf 2 110 None

    We can see that this route does not leave as3, which is why as1border1 is unable to established the configured BGP session. This session should have been configured as an eBGP multi-hop session with static routes pointing to the appropriate interface and next-hop

    Debugging BGP sessions that are NOT_ESTABLISHED

    We have root-caused the NOT_COMPATIBLE sessions, now let’s dig into the ones that are NOT_ESTABLISHED.

    [17]:
    
    # Find all BGP sessions in the network that were compatible but NOT_ESTABLISHED
    bgpSessStat[bgpSessStat['Established_Status'] == 'NOT_ESTABLISHED']
    
    [17]:
    
    Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
    9 as2border1 default 2 None 2.1.1.1 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    12 as2border2 default 2 None 2.1.1.2 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    15 as2core1 default 2 None 2.1.2.1 2 as2border1 None 2.1.1.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    16 as2core1 default 2 None 2.1.2.1 2 as2border2 None 2.1.1.2 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    17 as2core1 default 2 None 2.1.2.1 2 as2dist1 None 2.1.3.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    18 as2core1 default 2 None 2.1.2.1 2 as2dist2 None 2.1.3.2 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    25 as2dist1 default 2 None 2.1.3.1 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED
    28 as2dist2 default 2 None 2.1.3.2 2 as2core1 None 2.1.2.1 ['IPV4_UNICAST'] IBGP NOT_ESTABLISHED

    The reasons for a session that is compatiable (UNIQUE_MATCH or DYNAMIC_MATCH) to not get established would be 1) missing routes or 2) some ACL in the path blocking traffic. Let’s check the routing tables on as2core1

    [18]:
    
    # retrieve routing table for as2core1 and check the route to the BGP peer 2.1.1.1 - as2border1
    routes = bf.q.routes(nodes='as2core1').answer().frame()
    routes[routes['Network']=='2.1.1.1/32']
    
    [18]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag
    0 as2core1 default 2.1.1.1/32 interface GigabitEthernet0/0 ip 2.12.11.1 2.12.11.1 GigabitEthernet0/0 ospf 2 110 None

    As we can see as2core1 has a route to the configured neighbor 2.1.1.1. What about the other routers?

    [19]:
    
    # retrieve routing table for as2border1 to check if it has a route to as2core1 loopback0 - 2.1.2.1/32
    routes=bf.q.routes(nodes='as2border1').answer().frame()
    routes[routes['Network']=='2.1.2.1/32']
    
    [19]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag

    as2border1 does not have a route to the loopback address 2.1.2.1 of as2core1. Let’s also check as2border2.

    [20]:
    
    # retrieve routing table for as2border2 to check if it has a route to as2core1 loopback0 - 2.1.2.1/32
    routes=bf.q.routes(nodes='as2border2').answer().frame()
    routes[routes['Network']=='2.1.2.1/32']
    
    [20]:
    
    Node VRF Network Next_Hop Next_Hop_IP Next_Hop_Interface Protocol Metric Admin_Distance Tag

    Neither border router has a route to the loopback of as2core1. Since the loopback is supposed to be distributed via OSPF, next step is to look at the OSPF configuration on as2core1

    From the snippet of the configuration of as2core1, we can see that the Loopback address isn’t part of the OSPF process:

    interface Loopback0
     ip address 2.1.2.1 255.255.255.255
    
    router ospf 1
     router-id 2.1.2.1
     !network 2.0.0.0 0.255.255.255 area 1
     network 2.12.0.0 0.0.255.255 area 1
     network 2.23.0.0 0.0.255.255 area 1
    

    This explains why the BGP session wasn’t established.

    With that we have root-caused all of the BGP sessions that were NOT_ESTABLISHED or NOT_COMPATIBLE.

    Summary

    Batfish allows you to easily retrieve information about BGP configuration of all devices and peers, as well as status of each peer. With Batfish you can ensure that no change is pushed to the network that would cause a BGP session to not come up.

    We hope you found this notebook useful and informative. Future notebooks will dive into more advanced topics like validating routing policy. Stay tuned!

    Want to learn more? Come find us on Slack and GitHub

    Analyzing BGP Route Policies

    Route policies for BGP are complex and error prone, which is why some of the biggest outages in the Internet involve misconfigured route policies that end up leaking routes or accepting routes they shouldn’t (e.g., BGP Leak Causing Internet Outages in Japan and Beyond, How a Tiny Error Shut Off the Internet for Parts of the US, Telia engineer error to blame for massive net outage). While it is often clear to network engineers what the route policy should or should not do (e.g., see MANRS guidelines), ensuring that the route policy implementation is correct is notoriously hard.

    In this notebook we show how you can use Batfish to validate your route policies. Batfish’s testRoutePolicies question provides an easy way to test route-policy behavior—given a route, it shows how it is transformed (or denied) by the route policy. Batfish’s searchRoutePolicies actively searches for routes that cause a policy to violate its intent.

    To illustrate these capabilities, we’ll use an example network with two border routers, named border1 and border2. Each router has a BGP session with a customer network and a BGP session with a provider network. Our goal in this notebook is to validate the in-bound route policy from the customer, called from_customer, and the out-bound route policy to the provider, called to_provider.

    The intent of the from_customer route policy is:

    • filter private addresses

    • only permit routes to known prefixes if they have the correct origin AS

    • tag permitted routes with an appropriate community, and update the local preference

    The intent of the to_provider route policy is:

    • advertise all prefixes that we own

    • advertise all customer routes

    • don’t advertise anything else

    We’ll start, as usual, by initializing the example network that we will use in this notebook.

    [1]:
    
    # Import packages
    %run startup.py
    from pybatfish.datamodel.route import BgpRouteConstraints
    bf = Session(host="localhost")
    
    # Initialize a network and snapshot
    NETWORK_NAME = "example_network"
    SNAPSHOT_NAME = "example_snapshot"
    
    SNAPSHOT_PATH = "networks/route-analysis"
    
    bf.set_network(NETWORK_NAME)
    bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
    
    [1]:
    
    'example_snapshot'
    
    Example 1: Filter private addresses in-bound

    When peering with external entities, an almost universally-desired policy is to filter out all announcements of the private IP address space. For our network, we’d like to ensure that the two from_customer route policies properly filter such announcements.

    Traditionally you might validate this policy through some form testing that involves the production or a lab device. With Batfish’s testRoutePolicies question we can easily test a route policy’s behavior without access to a device.

    [2]:
    
    # Create an example route to use for testing
    inRoute1 = BgpRoute(network="10.0.0.0/24",
                        originatorIp="4.4.4.4",
                        originType="egp",
                        protocol="bgp")
    
    # Test how our policy treats this route
    result = bf.q.testRoutePolicies(policies="from_customer",
                                 direction="in",
                                 inputRoutes=[inRoute1]).answer().frame()
    # Pretty print the result
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace
    0 border1 from_customer Network: 10.0.0.0/24
    AS Path: []
    Communities: []
    Local Preference: 0
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    DENY None None
    • Matched route-map from_customer clause 100
    1 border2 from_customer Network: 10.0.0.0/24
    AS Path: []
    Communities: []
    Local Preference: 0
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    DENY None None
    • Matched route-map from_customer clause 100

    The first line of code above creates a BgpRoute object that specifies the input route announcement to use for testing, which in this case announces the prefix 10.0.0.0/24, has an originator IP that we arbitrarily chose, and has default values for other parts of the announcement. The second line uses testRoutePolicies to test the behavior of the two from_customer route policies on this announcement.

    The output of the question shows the results of the test. As we see in the Action column, in both border routers, the from_customer route policy properly denies this private address. The Trace column tell us that this happened because the input route matched clause 100 of the route map.

    That result gives us some confidence in our route policies, but it is just a single test. We can run testRoutePolicies on more private addresses to ensure they are denied.

    However, how can we be sure that all private addresses are denied by the two in-bound route maps? For that, we will use the searchRoutePolicies question and change our perspective a bit. Instead of testing individual routes, we will ask Batfish to search for a route-policy behavior that violates our intent. If we get one or more results, then we’ve found a bug. If we get no results, then we can be sure that our configurations satisfy the intent, since Batfish explores all possible route-policy behaviors.

    [3]:
    
    # Define the space of private addresses
    privateIps = ["10.0.0.0/8:8-32",
                  "172.16.0.0/12:12-32",
                  "192.168.0.0/16:16-32"]
    
    # Specify all route announcements for the private space
    inRoutes1 = BgpRouteConstraints(prefix=privateIps)
    
    # Verify that no such announcement is permitted by our policy
    result = bf.q.searchRoutePolicies(policies="from_customer",
                                     inputConstraints=inRoutes1,
                                     action="permit").answer().frame()
    # Pretty print the result
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace
    0 border2 from_customer Network: 192.168.0.0/32
    AS Path: []
    Communities: []
    Local Preference: 100
    Metric: 0
    Next Hop IP: 0.0.0.1
    Originator IP: 0.0.0.0
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    PERMIT Network: 192.168.0.0/32
    AS Path: []
    Communities: [20:30]
    Local Preference: 300
    Metric: 0
    Next Hop IP: 0.0.0.1
    Originator IP: 0.0.0.0
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    Communities: [] --> [20:30]
    Local Preference: 100 --> 300
    • Matched route-map from_customer clause 400

    The first line above specifies the space of all private IP prefixes. The second line creates a BgpRouteConstraints object, which is like the BgpRoute object we saw earlier but represents a set of announcements rather than a single one. In this case, we are interested in all announcements that announce a prefix in privateIps. Finally, the third line of code uses searchRoutePolicies to search for an announcement in the set inRoutes that is permitted by the from_customer route policy.

    There are no results for border1, which means that its from_customer route policy properly filters all private addresses. However, the result for border2 shows that its version of from_customer permits an announcement for the prefix 192.168.0.0/32. The table also shows the route announcement that will be produced by from_customer in this case, along with a “diff” of the input and output announcements.

    Inspecting the configurations, we see that both routers deny all announcements for prefixes in the prefix list private-ips. However, the definition of private-ips on border2 accidentally omitted the ge /16 clause, so only applied to /16 prefixes. Relevant parts of the config at border2 are:

    ip prefix-list private-ips seq 15 permit 192.168.0.0/16  // <-- missing ge /16
    
    ...
    
    route-map from_customer deny 100
     match ip address prefix-list private-ips
    !
    ....
    
    route-map from_customer permit 400
     set community 20:30
     set local-preference 300
    !
    

    Batfish is able to correctly model the semantics of route maps and prefix lists and deduce that some prefix with private IPs will get past our policy.

    Example 2: Filter based on origin AS in-bound

    Another common BGP policy is to make sure that announcements for certain prefixes (e.g., customer-owned prefixes) are acccepted only if they have a specific origin AS.

    For our example, we assume that announcements for routes with any prefix in the range 5.5.5.0/24:24-32 should originate from the AS 44. We will use searchRoutePolicies to ask: Is there any permitted announcement for a prefix in the range 5.5.5.0/24:24-32 that does not originate from AS 44?

    [4]:
    
    # Define expected prefixes
    knownPrefixes = "5.5.5.0/24:24-32"
    
    # Define invalid AS-path -- all those that do not have 44 as the origin AS
    badOrigin = "!/( |^)44$/"
    
    # Specify the route announcements we must not permit
    inRoutes2 = BgpRouteConstraints(prefix=knownPrefixes, asPath=badOrigin)
    
    # Verify that our policy does not permit any such announcement
    result = bf.q.searchRoutePolicies(policies="from_customer",
                                     inputConstraints=inRoutes2,
                                     action="permit").answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace

    The first line above defines the known prefixes. The second line specifies that we are interested in AS-paths that do not end in 44, using the same syntax that Batfish uses for regular-expression specifiers. Specifically, a regular expression is surrounded by / characters, and the leading ! indicates that we are interested in AS-paths that do not match this regular expression. Since there are no results, we can be sure that the our intent is satisfied.

    Example 3: Set attributes in-bound

    As the third and final policy for inbound routes, let’s make sure that our from_customer route policies tag each permitted route with the community 20:30 and set the local preference to 300. To start, we can use testRoutePolicies to test this property on a specific route announcement.

    [5]:
    
    # Define a test route and test what the policy does to it
    inRoute3 = BgpRoute(network="2.0.0.0/8",
                        originatorIp="4.4.4.4",
                        originType="egp",
                        protocol="bgp")
    result = bf.q.testRoutePolicies(policies="from_customer",
                                   direction="in",
                                   inputRoutes=[inRoute3]).answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace
    0 border1 from_customer Network: 2.0.0.0/8
    AS Path: []
    Communities: []
    Local Preference: 0
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    PERMIT Network: 2.0.0.0/8
    AS Path: []
    Communities: [20:30]
    Local Preference: 0
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    Communities: [] --> [20:30]
    • Matched route-map from_customer clause 400
    1 border2 from_customer Network: 2.0.0.0/8
    AS Path: []
    Communities: []
    Local Preference: 0
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    PERMIT Network: 2.0.0.0/8
    AS Path: []
    Communities: [20:30]
    Local Preference: 300
    Metric: 0
    Next Hop IP: None
    Originator IP: 4.4.4.4
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    Communities: [] --> [20:30]
    Local Preference: 0 --> 300
    • Matched route-map from_customer clause 400

    The results show what each router does to the test route. This information can be seen in the Output_Route column, which shows the full output route announcement, as well as the Difference column, which shows the differences between the input and output route announcements. We see that there is an error in border1’s configuration: the permitted route is tagged with community 20:30, but its local preference is not set to 300. However, border2 is doing the right thing. A look at the configuration for border1 reveals that the set local-preference line is accidentally omitted from one clause of the policy.

    This example shows why testing is so important. However, we’d like to make sure that there aren’t any other lurking bugs. We can use searchRoutePolicies for this purpose. First we’ll check that all permitted routes are tagged with the community 20:30. To check this property, we will leverage the ability of searchRoutePolicies not only to search for particular input announcements, but also to search for particular output announcements. In this case, we will ask: Is there a permitted route whose output announcement is not tagged with community 20:30?

    [6]:
    
    # Define invalid communities -- those that do not contain 20:30
    outRoutes3a = BgpRouteConstraints(communities="!20:30")
    # Verify that our policy does not output routes with such communities
    result = bf.q.searchRoutePolicies(policies="from_customer",
                                     action="permit",
                                     outputConstraints=outRoutes3a).answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace

    There are no results, so we know that our intent is satisfied on both routers.

    Now let’s do a similar thing to check that the local preference is properly set. We’ve already seen that border1 is not properly setting the local preference, so we’ll just check that border2’s configuration is correct.

    [7]:
    
    # Verify that all permitted routes have the expected local preference
    outRoutes3b = BgpRouteConstraints(localPreference="!300")
    result = bf.q.searchRoutePolicies(nodes="border2",
                                     policies="from_customer",
                                     action="permit",
                                     outputConstraints=outRoutes3b).answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace

    There are no results for border2’s from_customer policy, so we have the strong assurance that it is properly setting the local preference on all permitted routes.

    Example 4: Announce your own addresses out-bound

    Ok, now let’s validate the to_provider route policies. The first thing we want to ensure is that they allow all our addresses to be advertised. Lets assume that these are addresses in the ranges 1.2.3.0/24:24-32 and 1.2.4.0/24:24-32. We can use searchRoutePolicies to validate this property. Specifically, we ask: Is there an announcement for an address that we own that is denied by ``to_provider``?

    [8]:
    
    # Verify that no route for our address space is ever denied by to_provider policies
    ownedSpace=["1.2.3.0/24:24-32", "1.2.4.0/24:24-32"]
    inRoutes4 = BgpRouteConstraints(prefix=ownedSpace)
    result = bf.q.searchRoutePolicies(policies="to_provider",
                                     inputConstraints=inRoutes4,
                                     action="deny").answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace

    Since there are no results, this implies that no such announcement exists. Hurray!

    Example 5: Announce your customers’ routes out-bound

    Next we will check that the to_provider route policies are properly announcing our customers’ routes. These are identified by announcements that are tagged with the community 20:30, as we saw earlier.

    [9]:
    
    # Verify that no customer routes (i.e., those tagged with the community 20:30) is ever denied
    customerCommunities = "20:30"
    inRoutes5 = BgpRouteConstraints(communities=customerCommunities)
    result = bf.q.searchRoutePolicies(policies="to_provider",
                                     inputConstraints=inRoutes5,
                                     action="deny").answer().frame()
    show(result)
    
    Node Policy_Name Input_Route Action Output_Route Difference Trace
    0 border2 to_provider Network: 10.0.0.0/8
    AS Path: []
    Communities: [20:30]
    Local Preference: 100
    Metric: 0
    Next Hop IP: 0.0.0.1
    Originator IP: 0.0.0.0
    Origin Type: egp
    Protocol: bgp
    Source Protocol: None
    Tag: 0
    Weight: 0
    DENY None None

      There are no results for border1, so it permits announcement of all customer routes. But border2 has a bug since it denies some customer routes, a concrete example of which is shown in the Input_Route column.

      A look at the configuration reveals that someone has fat-fingered the definition of the community list:

      ip community-list cust_community permit 2:30
      

      Such mistakes are difficult to find with any other tool.

      Example 6: Don’t advertise anything else out-bound

      Last, we want to make sure that our to_provider route policies don’t announce any routes other than the ones we own and the ones that our customers own. We will use searchRoutePolicies to ask: Is there a permitted route whose prefix is not one we own and which is not tagged with the community 20:30?

      [10]:
      
      # Set of routes that are neither in our owned space nor have the customer community (20:30)
      inRoutes6 = BgpRouteConstraints(prefix=ownedSpace,
                                      complementPrefix=True,
                                      communities="!20:30")
      
      # Verify that no such route is permitted
      result = bf.q.searchRoutePolicies(policies="to_provider",
                                       inputConstraints=inRoutes6,
                                       action="permit").answer().frame()
      show(result)
      
      Node Policy_Name Input_Route Action Output_Route Difference Trace
      0 border2 to_provider Network: 10.0.0.0/8
      AS Path: []
      Communities: [2:30]
      Local Preference: 100
      Metric: 0
      Next Hop IP: 0.0.0.1
      Originator IP: 0.0.0.0
      Origin Type: egp
      Protocol: bgp
      Source Protocol: None
      Tag: 0
      Weight: 0
      PERMIT Network: 10.0.0.0/8
      AS Path: []
      Communities: [2:30]
      Local Preference: 100
      Metric: 0
      Next Hop IP: 0.0.0.1
      Originator IP: 0.0.0.0
      Origin Type: egp
      Protocol: bgp
      Source Protocol: None
      Tag: 0
      Weight: 0
      • Matched route-map to_provider clause 200

      The complementPrefix parameter above is used to indicate that we are interested in routes whose prefix is not in ownedSpace.

      Since there are no results for border1 we can be sure that it is not advertising any routes that it shouldn’t be. We already saw in the previous example that border2 accidentally advertises routes tagged with 2:30, and that error shows up again here.

      Current Status

      The testRoutePolicies question supports all of the vendors and route-policy features that are supported by Batfish.

      The searchRoutePolicies question has been (at the time of this writing) newly added to Batfish. It supports all of the vendors that are supported by Batfish. The question supports a host of common route policy behaviors and intents, as shown above, but it does not currently support all routing constructs. See its documentation for details, and feel free to reach out to us with questions or specific needs. We’ll continue to enhance its coverage.

      Summary
      In this notebook we showed you two ways to use Batfish to check whether your route policies meet your intent:
      1. The testRoutePolicies question allows you to easily test the behavior of a route policy offline, without access to the live network.
      2. The searchRoutePolicies question allows you to search for violations of intent, identifying concrete errors if they exist and providing strong correctness guarantees if not.

      Get involved with the Batfish community

      Join our community on Slack and GitHub.

      Forwarding analysis

      Introduction to Forwarding Analysis using Batfish

      Analyzing how the network forwards packets is one of the most common tasks for network engineers. Typically, it is performed by running traceroute between multiple sources and destinations. This process is highly complex even in a moderately-sized network. It also fails to provide strong assurance as only some of the source-destination pairs and some of the packets can be feasibly tested.

      Batfish makes forwarding analysis extremely simple by providing 1) easy-to-use queries over a centralized view of the network; and 2) ability to reason comprehensively about entire spaces of flows. Further, it can perform this analysis proactively, that is, analyze the impact of configuration changes before they are pushed to the network.

      In this notebook, we will show you how to perform forwarding analysis with Batfish.

      Check out a video demo of this notebook here.

      [1]:
      
      # Import packages
      %run startup.py
      bf = Session(host="localhost")
      
      Setup: Initializing the Network and Snapshot

      SNAPSHOT_PATH below can be updated to point to a custom snapshot directory. See the Batfish instructions for how to package data for analysis.

      More example networks are available in the networks folder of the Batfish repository.

      [2]:
      
      NETWORK_NAME = "example_network"
      SNAPSHOT_NAME = "example_snapshot"
      
      SNAPSHOT_PATH = "networks/example"
      
      bf.set_network(NETWORK_NAME)
      bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
      
      [2]:
      
      'example_snapshot'
      

      The network snapshot that we initialized above is illustrated below. You can view or download the devices’ configuration files here.

      example-network

      All of the information we will show you in this notebook is dynamically computed by Batfish based on the configuration files for the network devices.


      Batfish Smart Traceroute: Detailed analysis of path(s) of a flow

      In this section, we will use the traceroute question to find the path taken by AS3 core routers to reach the DNS Server (host1) in AS2. Traceroute has three (composite) parameters that you can specify, allowing for a variety of queries. We will focus on the two main ones:

      • startLocation - where in the network the flow starts

      • headers - packet headers for the flow you are interested in tracing. This is not just limited to UDP or ICMP.

      We want the trace to start from the Loopback0 interface on as3core1, and we want to use the IP address of that interface as the source address. For this we set the startLocation to as3core1[Loopback0]. Batfish automatically chooses the IP address of Loopback0 as the source IP.

      We set the destination IP address of our virtual packet by specifying dstIps='ofLocation(host1)'. Batfish will automatically pick one of the IP addresses for host1 as the destination IP address. See the ofLocation function (see documentation for more detail).

      To run the query:

      [3]:
      
      # start the traceroute from the Loopback0 interface of as3core1 to host1
      headers = HeaderConstraints(dstIps='host1')
      tracert = bf.q.traceroute(startLocation="as3core1[Loopback0]", headers=headers).answer().frame()
      

      To pretty-print the traces in HTML use the display_html function. We will show you how to extract more detailed information below.

      [4]:
      
      show(tracert)
      
      Flow Traces TraceCount
      0 Start Location: as3core1
      Src IP: 3.10.1.1
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 33434
      IP Protocol: UDP
      DENIED_IN
      1. node: as3core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as3border1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: host1
        RECEIVED(eth0)
        DENIED(filter::INPUT (INGRESS_FILTER))

      DENIED_IN
      1. node: as3core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as3border1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: host1
        RECEIVED(eth0)
        DENIED(filter::INPUT (INGRESS_FILTER))

      DENIED_IN
      1. node: as3core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as3border1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: host1
        RECEIVED(eth0)
        DENIED(filter::INPUT (INGRESS_FILTER))

      DENIED_IN
      1. node: as3core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as3border1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: host1
        RECEIVED(eth0)
        DENIED(filter::INPUT (INGRESS_FILTER))
      4

      The Flow column describes the packet being traced: it starts at as3core1 using source IP 3.10.1.1 (of Loopback0) and and destination IP 2.128.0.101. By default, bf.q.traceroute uses the standard UDP traceroute to destination port 33434, and Batfish arbitrarily picks the lowest ephemeral source port of 49152.

      [5]:
      
      tracert['Flow'][0]
      
      [5]:
      
      Start Location: as3core1
      Src IP: 3.10.1.1
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 33434
      IP Protocol: UDP

      The Trace column contains the detailed information provided by Batfish about the paths through the network for each flow. Let’s look in detail on the first path:

      [6]:
      
      tracert['Traces'][0][0]  # Get the trace for the first path of the first flow
      
      [6]:
      
      DENIED_IN
      1. node: as3core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as3border1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: host1
        RECEIVED(eth0)
        DENIED(filter::INPUT (INGRESS_FILTER))

      This flow starts at as3core1 and crosses from AS3 into AS2 via the border routers as3border1 and as2border2; on as2border2, the flow is permitted by the inbound ACL OUTSIDE_TO_INSIDE. Once inside AS2, the flow is forwarded through AS2’s core and distribution servers to the department router. The flow does reach host1, but is blocked by that server iptables rule filter::INPUT on eth0.

      The TraceCount column reports the total number of paths for each flow. In this example, the count 4 matches the four paths we saw in the Traces column. These are all the best-cost paths inside AS2 – the flow can go through either as2core1 or as2core2 and either as2dist1 or as2dist2.

      Detail: TraceCount may not always match the TracesColumn: in networks with high ECMP, there may be hundreds or even thousands of traces even for a single flow, in which case the Traces column will produce fewer results.

      [7]:
      
      tracert['TraceCount'][0]
      
      [7]:
      
      4
      

      To programmatically get the detailed information about the final hop of the first trace in pure Python form:

      [8]:
      
      last_hop = tracert['Traces'][0][0].hops[-1]
      repr(last_hop)
      
      [8]:
      
      "Hop(node='host1', steps=[Step(detail=EnterInputIfaceStepDetail(inputInterface='eth0', inputVrf='default'), action='RECEIVED'), Step(detail=FilterStepDetail(filter='filter::INPUT', filterType='INGRESS_FILTER', inputInterface='eth0', flow=Flow(dscp=0, dstIp='2.128.0.101', dstPort=33434, ecn=0, fragmentOffset=0, icmpCode=None, icmpVar=None, ingressInterface=None, ingressNode='as3core1', ingressVrf='default', ipProtocol='UDP', packetLength=512, srcIp='3.10.1.1', srcPort=49152, tcpFlagsAck=0, tcpFlagsCwr=0, tcpFlagsEce=0, tcpFlagsFin=0, tcpFlagsPsh=0, tcpFlagsRst=0, tcpFlagsSyn=0, tcpFlagsUrg=0)), action='DENIED')])"
      

      Note that compared to running traceroute on a router, Batfish is able to provide much more detail about the trace:

      1. All active parallel paths between the source and destination

      2. The reason why each hop in a path is taken (the specific routing entry that was matched)

      3. All processing steps inside each hop on the path

      4. All interfaces visited and filters encountered during the trace

      5. The disposition of the flow for each path

      Batfish Reachability: Search for forwarding behaviors in (large) spaces of flows

      Batfish’s smart traceroute provides detailed information about all paths taken by a specified flow through the network, which is a powerful capability for exploring and testing network behavior. However, network engineers often need information about some type of flow, without being able to specify a particular flow of that type. For example, we may want to know what TCP flows can (or cannot) reach a particular host. In other words, we want to search within the (huge!) space of TCP flows addressed to that host. Batfish’s reachability question is an easy and efficient way to perform exactly this kind of search.

      Example 1: Search for DNS flows that reach the DNS server

      Continuing with our running example, let’s search for DNS flows within AS2 that reach our DNS server host1. The parameter srcIps='0.0.0.0/0' tells Batfish to search within the entire space of source IP addresses.

      Detail: In traceroute ofLocation(host1) specified a single IP address of host1 (chosen arbitrarily). In reachability, it specifies to search over all IP addresses of host1.

      [9]:
      
      path = PathConstraints(startLocation="/as2/")
      headers = HeaderConstraints(srcIps="0.0.0.0/0", dstIps="host1", applications="DNS")
      reach = bf.q.reachability(pathConstraints=path, headers=headers, actions="success").answer().frame()
      show(reach)
      
      Flow Traces TraceCount
      0 Start Location: as2border1
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2border1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      4
      1 Start Location: as2border2
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2border2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      4
      2 Start Location: as2core1
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2core1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      2. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      2
      3 Start Location: as2core2
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      2. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      2
      4 Start Location: as2dept1
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2dept1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      1
      5 Start Location: as2dist1
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2dist1
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      1
      6 Start Location: as2dist2
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2dist2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      1

      As you can see, this query found some DNS flow entering the network at each as2... node destined for host1 that would be delivered. This guarantees that DNS is at least partially available (some authorized nodes can reach host1).

      Using Batfish Reachability as a verification tool

      Batfish’s reachability analysis performs an exhaustive search, considering every possible flow. This makes it a very powerful tool for finding bugs, and when no bugs are found, it provides strong guarantees about the network behavior. It allows you to verify essential network properties like availability or security (i.e., presence or absence of reachability). In the following examples shows we’ll use reachability to verify the intended availability and security properties of our DNS server.

      Example 2: Verify that the DNS server is available everywhere inside AS2

      To verify DNS is available inside of AS2, we search for flows that would demonstrate a lack of availability – i.e. DNS flows to host1 that would fail to be delivered. Our intent in this case is that no such flows should exist in the network, and this is where we see the power of exhaustive search. If Batfish does not find any dropped DNS flows to host1, we have verified availability.

      [10]:
      
      path = PathConstraints(startLocation="/as2/")
      headers = HeaderConstraints(dstIps="host1", applications="DNS")
      reach = bf.q.reachability(pathConstraints=path, headers=headers, actions="failure").answer().frame()
      show(reach)
      
      Flow Traces TraceCount

      The fact that Batfish returned 0 flows guarantees that host1 is reachable via DNS from everywhere within AS2. This guarantees that DNS is available to all authorized nodes.

      Example 3: No UDP traffic except DNS is accessible on host1

      Next, let’s verify a security property: that no UDP traffic other than DNS will reach host1. We do this by searching for any UDP flows accepted by host1 that are not DNS:

      [11]:
      
      path = PathConstraints(startLocation="/as2/")
      headers = HeaderConstraints(srcIps="0.0.0.0/0", dstIps="host1", ipProtocols="UDP", dstPorts="!53")
      reach = bf.q.reachability(pathConstraints=path, headers=headers, actions="accepted").answer().frame()
      show(reach)
      
      Flow Traces TraceCount

      Success! Since Batfish returned 0 flows, we are guaranteed that no unauthorized UDP flows will reach host1.

      Example 4: Verify that the DNS server is not reachable from anywhere outside AS2

      Next, let’s verify that no DNS traffic from outside of AS2 can reach host1. We can do this by searching for flows starting from the border interfaces (GigabitEthernet0/0 on AS2 border routers).

      Details: We use the enter function to model that traffic is received on the interface rather than starting from a border router. If we did not relax source IP to 0.0.0.0/0, only source IP addresses within the connected subnet of the border interfaces would be included in the search.

      [12]:
      
      path = PathConstraints(startLocation="@enter(/as2border/[GigabitEthernet0/0])")
      headers = HeaderConstraints(srcIps="0.0.0.0/0", dstIps="host1", applications="DNS")
      reach = bf.q.reachability(pathConstraints=path, headers=headers, actions="success").answer().frame()
      show(reach)
      
      Flow Traces TraceCount
      0 Start Location: as2border1 interface=GigabitEthernet0/0
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      4
      1 Start Location: as2border2 interface=GigabitEthernet0/0
      Src IP: 10.0.0.0
      Src Port: 49152
      Dst IP: 2.128.0.101
      Dst Port: 53
      IP Protocol: UDP
      ACCEPTED
      1. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2core2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2dist1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)

      ACCEPTED
      1. node: as2border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: as2dist2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: as2dept1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host1
        RECEIVED(eth0)
        PERMITTED(filter::INPUT (INGRESS_FILTER))
        ACCEPTED(eth0)
      4

      We found that the DNS server is not secure: external DNS traffic can reach host1! However, we did find where to look: in all likelihood, the OUTSIDE_TO_INSIDE ACL on the border router should be blocking more DNS traffic.

      Multipath Consistency: Verify consistent treatment of a flow across all paths

      Finally, we will demonstrate an experimental feature to detect an important class of reachability bugs in any network with no user input: the multipathconsistency check. This question will report find flows with multipath routing where some paths reach the destination and some paths fail. Multipath inconsistencies are almost always bugs, and may be a sign that the network is not robust to failures.

      In this example network there are mutiple multipath consistencies; for conciseness we show only the first result.

      [13]:
      
      multipath = bf.q.multipathConsistency().answer().frame()
      first_result = multipath.head(1)  # this check returns many results, just show 1
      show(first_result)
      
      Flow Traces TraceCount
      0 Start Location: as2core2
      Src IP: 2.1.2.2
      Src Port: 49152
      Dst IP: 2.1.2.1
      Dst Port: 23
      IP Protocol: TCP (SYN)
      ACCEPTED
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.12.22.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.12.22.1)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: as2border2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet2/0 ip 2.12.21.2)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: as2core1
        RECEIVED(GigabitEthernet1/0)
        ACCEPTED(Loopback0)

      ACCEPTED
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.12.1, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.12.12.1)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: as2border1
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2core1
        RECEIVED(GigabitEthernet0/0)
        ACCEPTED(Loopback0)

      DENIED_IN
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet2/0 ip 2.23.22.3)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: as2dist2
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.23.12.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet1/0 ip 2.23.12.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: as2core1
        RECEIVED(GigabitEthernet3/0)
        DENIED(blocktelnet (INGRESS_FILTER))

      DENIED_IN
      1. node: as2core2
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet3/0 ip 2.23.21.3)])
        TRANSMITTED(GigabitEthernet3/0)
      2. node: as2dist1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 2.23.11.2, Routes: [ospf (Network: 2.1.2.1/32, Next Hop: interface GigabitEthernet0/0 ip 2.23.11.2)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: as2core1
        RECEIVED(GigabitEthernet2/0)
        DENIED(blocktelnet (INGRESS_FILTER))
      4

      The above trace shows that traffic from as2core2 to as2core1 can take four paths: through either of the two border routers or through either distribution router. However, telnet traffic will be blocked for only two of these four paths: the ones that traverse the distribution layer.

      Wrap-up

      This concludes the notebook. To recap, we covered the foundational tasks for path analysis:

      1. We performed a traceroute to check connectivity to host1

      2. Analyzed detailed path & hop information for the traceroute

      3. Explored a space of flows with the reachablity question and found an ACL bug that allows some external clients to reach the DNS server

      4. Perfomed a security check that ensures that only SSH and DNS traffic can reach host1

      5. Found multipath inconsistency in the network, for which only some paths result in successful communication

      We hope you found this notebook useful and informative. Future notebooks will dive into more advanced topics ensuring planned configuration changes do not have unintended consequences. Stay tuned!

      Want to learn more?

      Reach out to us through Slack or GitHub to learn more, or send feedback.

      Introduction to Forwarding Change Validation

      Network engineers frequently have to make changes to network that can impact forwarding behavior: add new routes, open or close flows, route traffic through different devices, etc. These changes are often hard to get right and hard to validate.

      This notebook will show how Batfish can help validate changes to network forwarding before you deploy them. We will do this using Batfish’s reachability and differentialReachability questions that can provide guarantees that our changes are correct and have no unintended side-effects. As we will see, these anaylses are a powerful way to understand, test, and validate changes to the network.

      Check out a video demo of this notebook here.

      [1]:
      
      # Import packages
      %run startup.py
      bf = Session(host="localhost")
      

      In this notebook we will use the network shown in the diagram below. You can view and download the device configuration files here.

      example-network

      Change Scenario 1: Costing out a core router

      The network is overprovisioned with failover redundancy for the core routers. All traffic is normally routed through core1 but will automatically switch to core2 in case of a failure or during maintenance. In this scenario, we want to service core1 and thus want to shift traffic to core2. We’ll implement a change to cost out core1, and verify that it does not affect end-to-end reachability. In general, we care about three classes of end-to-end traffic: external-to-host, host-to-external, and host-to-host. For simplicity, we focus on the external-to-host traffic in this notebook but similar queries can cover other classes.

      Step 1: Test current behavior

      Before beginning, let’s check that the network is working as expected (i.e., routing through core1). First we load our snapshot into Batfish.

      [2]:
      
      NETWORK_NAME = "forwarding-change-validation"
      BASE_NAME = "base"
      BASE_PATH = "networks/forwarding-change-validation/base"
      
      bf.set_network(NETWORK_NAME)
      bf.init_snapshot(BASE_PATH, name=BASE_NAME, overwrite=True)
      
      [2]:
      
      'base'
      

      Batfish will automatically compute the RIBs and FIBs from the configuration files in the snapshot, allowing us to test the forwarding behavior offline. Let’s do that now, by using the traceroute question to see how external-to-host traffic is routed. The parameter startLocation="@enter(/border/[GigabitEthernet0/0])" says to start the trace entering the external border router interfaces. The parameter dstIps="/host/)" indicates that the flow should be addressed to one of the internal hosts. These two parameters are using specifier grammar.

      [3]:
      
      answer = bf.q.traceroute(
          startLocation="@enter(/border/[GigabitEthernet0/0])",
          headers=HeaderConstraints(dstIps="/host/")
      ).answer(snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount
      0 Start Location: border1 interface=GigabitEthernet0/0
      Src IP: 10.12.11.1
      Src Port: 49152
      Dst IP: 2.128.0.1
      Dst Port: 33434
      IP Protocol: UDP
      ACCEPTED
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/30, Next Hop: interface GigabitEthernet2/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host-db
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1
      1 Start Location: border2 interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Src Port: 49152
      Dst IP: 2.128.0.1
      Dst Port: 33434
      IP Protocol: UDP
      ACCEPTED
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/30, Next Hop: interface GigabitEthernet2/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host-db
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1

      The traceroute results include a flow from each border router, and all possible paths of each flow. As we can see in the Traces column, both flows are routed through core1. For more detail on traceroute question, see the notebook Introduction to Forwarding Analysis.

      Next, we’ll cost out core1 and cause all traffic to start being routed through core2. Below you can see the configuration changes we’re going to make. We add the command ip ospf cost 500 to each interface on core1, increasing its OSPF cost from the previous value of 1. This will cause the lower-cost routes through core2 to be preferred.

      $ diff -r base/ change1/
      diff -r base/configs/core1.cfg change1/configs/core1.cfg
      68c68
      <  ip ospf cost 1
      ---
      >  ip ospf cost 500
      73c73
      <  ip ospf cost 1
      ---
      >  ip ospf cost 500
      78c78
      <  ip ospf cost 1
      ---
      >  ip ospf cost 500
      83c83
      <  ip ospf cost 1
      ---
      >  ip ospf cost 500
      

      We implemented this change offline in a new snapshot, and will validate that the change doesn’t affect reachability. Having done so, we will be able to push the change to the network with complete confidence.

      We’ll validate the change using a two-step process, verifying that it has the intended effect, and that it causes no collateral damage. More specifically, the change must: 1. Ensure that no traffic is routed through core1. 1. Have no effect on external-to-host traffic.

      Step 2: Ensure that no traffic is routed through core1

      The following commands will load our change snapshot into batfish:

      [4]:
      
      CHANGE1_NAME = "change"
      CHANGE1_PATH = "networks/forwarding-change-validation/change1"
      
      bf.init_snapshot(CHANGE1_PATH, name=CHANGE1_NAME, overwrite=True)
      
      [4]:
      
      'change'
      

      To verify that no outside-to-host traffic is routed through core1, we need to search for counterexamples: outside-to-host traffic that is routed through core1. If no counterexamples are found, we have proven that core1 is never used. We do this by running the reachability question with the transitLocations parameter to search for flows that transit core1. We set the actions parameter to SUCCESS,FAILURE to include dropped flows as well as those that are successfully delivered.

      [5]:
      
      # Search for any traffic routed through core1
      answer = bf.q.reachability(
          pathConstraints=PathConstraints(
              startLocation="@enter(/border/[GigabitEthernet0/0])",
              transitLocations="core1"),
          headers=HeaderConstraints(dstIps="/host/"),
          actions="SUCCESS,FAILURE"
      ).answer(snapshot=CHANGE1_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount

      Good! Since we found no counter-examples, we are guaranteed that no outside-to-host traffic from will be routed through core1. This verifies the first requirement of the change. Having done so, let’s check our second requirement – that end-to-end reachability is completely unchanged.

      Step 3: Outside-to-host traffic is unaffected.

      In this step, we’ll compare the forwarding behavior of the candidate change snapshot against the original using the differentialReachability question. In particular, we’ll use the question to search for flows that are successfully delivered in one snapshot but not the other. If the change is correct, no such flows will be found, because costing out core1 should have no effect on end-to-end reachability.

      [6]:
      
      answer = bf.q.differentialReachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(dstIps="/host/")
      ).answer(
          snapshot=CHANGE1_NAME,
          reference_snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount
      0 Start Location: border1 interface=GigabitEthernet0/0
      Src IP: 10.12.11.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 33434
      IP Protocol: UDP
      NULL_ROUTED
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.12.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: core2
        RECEIVED(GigabitEthernet1/0)
        NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])
      1 ACCEPTED
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet3/0)
      5. node: host-www
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1
      1 Start Location: border2 interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 33434
      IP Protocol: UDP
      NULL_ROUTED
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: core2
        RECEIVED(GigabitEthernet0/0)
        NULL_ROUTED(Discarded, Routes: [static (Network: 2.128.1.1/32, Next Hop: discard)])
      1 ACCEPTED
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet3/0)
      5. node: host-www
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1

      As we can see, moving traffic from core1 to core2 does affect reachability: some traffic that was being delivered in the reference snapshot (before the change) is being null routed in the change snapshot (after the change). This means if we deploy the change now, there will be a loss of connectivity. Fortunately the differentialReachability question was able to identify that bug before we deployed the change.

      The results include an example flow from each start location that has traffic affected by the change. Each flow comes with detailed traces of all the paths it can take through the network, which helps us diagnose the problem: core2 has a rogue static route for 2.180.0.0/24 that should have been removed. A similar problem could occur with rogue ACLs along the backup path (which Batfish will find as well)

      Step 2 (again): Ensure that no traffic is routed through core1

      We remove the bad static route and load the updated change snapshot into batfish. Then we perform the same validation steps again.

      [7]:
      
      CHANGE1_FIXED_NAME = "change-fixed"
      CHANGE1_FIXED_PATH = "networks/forwarding-change-validation/change1-fixed"
      bf.init_snapshot(CHANGE1_FIXED_PATH, name=CHANGE1_FIXED_NAME, overwrite=True)
      
      [7]:
      
      'change-fixed'
      
      [8]:
      
      # Requirement 1: No traffic is routed through core1.
      answer = bf.q.reachability(
          pathConstraints=PathConstraints(
              startLocation="@enter(/border/[GigabitEthernet0/0])",
              transitLocations="core1"),
          headers=HeaderConstraints(dstIps="/host/"),
          actions="SUCCESS,FAILURE"
      ).answer(snapshot=CHANGE1_FIXED_NAME)
      show(answer.frame())
      
      
      Flow Traces TraceCount

      Again, we find no traffic being routed through core1, so it is still correctly costed-out.

      Step 3 (again): Outside-to-host traffic is unaffected.

      We now move on to check that after removing the bad null route, costing out core1 has no impact on the reachability matrix:

      [9]:
      
      # Requirement 2: Outside-to-host traffic is unaffected.
      answer = bf.q.differentialReachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(dstIps="/host/")
      ).answer(
          snapshot=CHANGE1_FIXED_NAME,
          reference_snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount

      Success! We have now verified that our change will correctly cost-out core1 without affecting reachability. We are ready to deploy the change and do the maintenance work for core1 with complete confidence.

      Summary

      Let’s recap the steps we took to verify this change: 1. First, we verified that the primary intent of the change is achieved: traffic is moved from core1 to core2. We used the reachability query to search all outside-to-host flows in the network and verify that none will transit core1 after the change. 1. Second, we verified that moving the traffic did not affect reachability. For this, we used the differentialReachability query to compare the forwarding behavior of two snapshots. This verified that no flow is affected by the change.

      Change Scenario 2: Validating the end-to-end impact of an ACL change

      In this second part of this notebook, we’ll validate another change to the same network. Unlike the previous scenario, this time we do want to alter end-to-end reachability, and we will verify that our change has the intended effect. As before, we will also verify that it has no unintended effects.

      In this scenario, we have developed and tested a new web service on host host-www, and are now ready to open it to HTTP traffic from the outside world. The service is running on the hosts behind leaf1, which has an ACL in place that filters traffic to each host. The change we’ll make and validate will open traffic to the host subnet in the border router ACLs that filter traffic entering the network.

      Step 1: Test current behavior

      We start by using the traceroute question to verify that host-www is not accessible via HTTP from outside the network. The parameter dstIps=ofLocation(host-www) tells traceroute to pick any IP belonging to host-www as the destination IP.

      [10]:
      
      answer = bf.q.traceroute(
          startLocation="@enter(/border/[GigabitEthernet0/0])",
          headers=HeaderConstraints(dstIps="host-www", applications="HTTP")
      ).answer(snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount
      0 Start Location: border1 interface=GigabitEthernet0/0
      Src IP: 10.12.11.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      DENIED_IN
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        DENIED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
      1
      1 Start Location: border2 interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      DENIED_IN
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        DENIED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
      1

      As you can see, the flow is dropped by the ingress ACL OUTSIDE_TO_INSIDE on each border router. This is where we’ll make our change. The following snippet shows the original ACL definition:

      ip access-list extended OUTSIDE_TO_INSIDE
       permit tcp any 2.128.0.0 0.0.1.255 eq ssh
       permit udp any 2.0.0.0 0.255.255.255
       deny ip any any
      

      The first line permits SSH traffic to host subnet. We’ll create a similar rule for HTTP, since the leaf1 already does the required per-host filtering. Here’s the updated version of the ACL:

      ip access-list extended OUTSIDE_TO_INSIDE
       permit tcp any 2.128.0.0 0.0.1.255 eq ssh
       permit tcp any 2.128.0.0 0.0.1.255 eq www
       permit udp any 2.0.0.0 0.255.255.255
       deny ip any any
      

      Next we load the snapshot with our change into batfish so we can validate it before deployment.

      [11]:
      
      CHANGE2_NAME = "change2"
      CHANGE2_PATH = "networks/forwarding-change-validation/change2"
      bf.init_snapshot(CHANGE2_PATH, name=CHANGE2_NAME, overwrite=True)
      
      [11]:
      
      'change2'
      

      We can test our change by running the above traceroute command on the change snapshot:

      [12]:
      
      answer = bf.q.traceroute(
          startLocation="@enter(/border/[GigabitEthernet0/0])",
          headers=HeaderConstraints(dstIps="host-www", applications="HTTP")
      ).answer(snapshot=CHANGE2_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount
      0 Start Location: border1 interface=GigabitEthernet0/0
      Src IP: 10.12.11.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      ACCEPTED
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet3/0)
      5. node: host-www
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1
      1 Start Location: border2 interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Src Port: 49152
      Dst IP: 2.128.1.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      ACCEPTED
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.1.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet3/0, Routes: [connected (Network: 2.128.1.0/30, Next Hop: interface GigabitEthernet3/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet3/0)
      5. node: host-www
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1

      Good. We now see that HTTP traffic can reach host-www from outside the network. We may be tempted to call it good and ship the change. However, batfish gives us the ability to do much more to ensure complete correctness.

      Following the steps outlined in the Provably Safe ACL and Firewall Changes notebook, we can independently validate the change to each border router ACL. We omit those steps from this notebook, and proceed to validating the end-to-end network behavior.

      As before, end-to-end validation has two requirements: 1. The change has the intended effect: HTTP traffic from outside the network can reach host-www. 1. The change has no unintended effects: No other traffic is affected.

      Step 2: External HTTP traffic can now reach host-www

      The traceroute results above show that some HTTP traffic can now reach host-www from outside the network. However, this doesn’t ensure that all such traffic can reach host-www. For that, we use the reachability query to search for counterexamples of the requirement: HTTP flows from the outside that cannot reach host-www.

      [13]:
      
      answer = bf.q.reachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(
              dstIps="host-www",
              srcIps="0.0.0.0/0",
              applications="HTTP"),
          actions="FAILURE"
      ).answer(snapshot=CHANGE2_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount

      Good! Since batfish’s comprehensive search found no counterexamples, we are guaranteed that none exist. In other words, the requirement is met.

      Step 3: No unintended consequences

      Next, we check the second requirement – that the change has no unintended effects. As before, we’ll use the differentialReachability question to compare the reachability of our change snapshot against the original network. We search all flows entering the border routers that are not HTTP traffic addressed to host-www. The invertSearch=True parameter causes batfish to search outside the specified header space instead of within it.

      [14]:
      
      answer = bf.q.differentialReachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(dstIps="host-www", applications="HTTP"),
          invertSearch=True
      ).answer(snapshot=CHANGE2_NAME, reference_snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount
      0 Start Location: border1 interface=GigabitEthernet0/0
      Src IP: 10.12.11.1
      Src Port: 49152
      Dst IP: 2.128.0.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      ACCEPTED
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.11.2, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: core1
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/30, Next Hop: interface GigabitEthernet2/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host-db
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1 DENIED_IN
      1. node: border1
        RECEIVED(GigabitEthernet0/0)
        DENIED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
      1
      1 Start Location: border2 interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Src Port: 49152
      Dst IP: 2.128.0.1
      Dst Port: 80
      IP Protocol: TCP (SYN)
      ACCEPTED
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        PERMITTED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: core1
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: spine2
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/30, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: leaf1
        RECEIVED(GigabitEthernet1/0)
        PERMITTED(RESTRICT_NETWORK_TRAFFIC_IN (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/30, Next Hop: interface GigabitEthernet2/0)])
        PERMITTED(RESTRICT_HOST_TRAFFIC_OUT (EGRESS_FILTER))
        TRANSMITTED(GigabitEthernet2/0)
      5. node: host-db
        RECEIVED(eth0)
        ACCEPTED(eth0)
      1 DENIED_IN
      1. node: border2
        RECEIVED(GigabitEthernet0/0)
        DENIED(OUTSIDE_TO_INSIDE (INGRESS_FILTER))
      1

      Unfortunately, our change had a broader impact than we intended. It turns out that leaf1 was not properly filtering traffic to host-db: it permits HTTP to both hosts, rather than just host-www.

      Step 2 (again): Verify HTTP traffic can now reach host-www

      We fix the buggy ACL on leaf1, load the fixed change snapshot into batfish and begin the validation process again. Here is the difference relative the first change attempt:

      $ diff -r change2/ change2-fixed/
      diff -r change2/configs/leaf1.cfg change2-fixed/configs/leaf1.cfg
      119c119
      <  permit tcp any 2.128.0.0 0.0.255.255 eq www
      ---
      >  permit tcp any 2.128.1.0 0.0.0.255 eq www
      
      [15]:
      
      CHANGE2_FIXED_NAME = "change2-fixed"
      CHANGE2_FIXED_PATH = "networks/forwarding-change-validation/change2-fixed"
      bf.init_snapshot(CHANGE2_FIXED_PATH, name=CHANGE2_FIXED_NAME, overwrite=True)
      
      [15]:
      
      'change2-fixed'
      
      [16]:
      
      answer = bf.q.reachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(dstIps="host-www", applications="HTTP"),
          actions="FAILURE"
      ).answer(snapshot=CHANGE2_FIXED_NAME)
      show(answer.frame())
      
      Flow Traces TraceCount

      As before, the requirement is met: since we did not find any dropped HTTP flows to host-www, we are guaranteed that all will be delivered successfully. Our first requirement is still met.

      Step 3 (again): No unintended consequences
      [17]:
      
      answer = bf.q.differentialReachability(
          pathConstraints=PathConstraints(startLocation="@enter(/border/[GigabitEthernet0/0])"),
          headers=HeaderConstraints(dstIps="host-www", applications="HTTP"),
          invertSearch=True
      ).answer(snapshot=CHANGE2_FIXED_NAME, reference_snapshot=BASE_NAME)
      show(answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount

      Success! This time differentialReachability returns no results, which means no traffic was affected by the changes other than external HTTP traffic to host-www. Our second requirement is now met.

      Summary

      Let’s recap the steps we took to verify this change: 1. First, we verified that the primary intent of the change is achieved: it allows external HTTP traffic to reach host-www. 1. Second, we verified that there were no other changes to external-to-host reachability. We used the differentialReachability query to compare the forwarding behavior of two snapshots. This verified that only external HTTP traffic to host-www is affected by the change.

      Conclusion

      In this notebook, you saw how batfish can help you validate changes to forwarding behavior before you deploy them to the network. Using batfish’s differential analysis of forwarding behavior, you can guarantee that your change does exactly what you intend – no more and no less.

      Want to learn more? Come find us on Slack and GitHub.

      Failure-impact analysis

      Analyzing the Impact of Failures (and letting loose a Chaos Monkey)

      Planned (maintenance) and unplanned failure of nodes and interfaces in the network is a frequent occurrence. While most networks are designed to be tolerant to such failures, gaining confidence that they are in fact tolerant is difficult. Network engineers often reason about network behavior under failures manually, which is a complex and error-prone task. Consequently, the network could be one link failure away from an outage that leads to a massive loss of revenue and reputation.

      Fortunately, based just on device configurations, Batfish makes it easy to proactively analyze the network behavior under failures and offer guarantees on its tolerance to a range of failure scenarios.

      In this notebook, we will show how to use Batfish to analyze network behavior under failures. Specifically, we will describe how to simulate a specific network failure scenario, how to check forwarding changes for all flows in that scenario, and finally how to identify vulnerabilities using Chaos Monkey style testing.

      Check out a video demo of this notebook here.

      Initialization

      We will use the example network shown below with three autonomous systems (ASes) spread that connect via eBGP. Within each AS iBGP and OSPF is used. The configurations of these devices are available here.

      example-network


      [1]:
      
      # Import packages
      %run startup.py
      bf = Session(host="localhost")
      
      # Initialize the example network and snapshot
      NETWORK_NAME = "example_network"
      BASE_SNAPSHOT_NAME = "base"
      
      SNAPSHOT_PATH = "networks/failure-analysis"
      
      bf.set_network(NETWORK_NAME)
      bf.init_snapshot(SNAPSHOT_PATH, name=BASE_SNAPSHOT_NAME, overwrite=True)
      
      [1]:
      
      'base'
      
      bf.fork_snapshot: Simulating network failures

      To simulate network failures, Batfish offers a simple API bf.fork_snapshot that clones the original snapshot to a new one with the specified failure scenarios.

      Suppose we want to analyze the scenario where node London fails. We can use bf.fork_snapshot to simulate this failure as shown below.

      [2]:
      
      # Fork a new snapshot with London deactivated
      FAIL_LONDON_SNAPSHOT_NAME = "fail_london"
      bf.fork_snapshot(BASE_SNAPSHOT_NAME, FAIL_LONDON_SNAPSHOT_NAME, deactivate_nodes=["london"], overwrite=True)
      
      [2]:
      
      'fail_london'
      

      In the code, bf.fork_snapshot accepts four parameters: BASE_SNAPSHOT_NAME indicates the original snapshot name, FAIL_LONDON_SNAPSHOT_NAME is the name of the new snapshot, deactivate_nodes is a list of nodes that we wish to fail, and overwrite=True indicates that we want to reinitialize the snapshot if it already exists.

      In addition to deactivate_nodes, bf.fork_snapshot can also take deactivate_interfaces as a parameter to simulate interface failures. Combining these functions, Batfish allows us to simulate complicated failure scenarios involving interfaces and nodes, for example: bf.fork_snapshot(BASE_SNAPSHOT_NAME, FAIL_SNAPSHOT_NAME, deactivate_nodes=FAIL_NODES, deactivate_interfaces=FAIL_INTERFACES, overwrite=True)).

      To understand network behavior under the simulated failure, we can run any Batfish question on the newly created snapshot. As an example, to ensure that the flows from Paris would still reach PoP even if London failed, we can run the traceroute question on the snapshot in which London has failed, as shown below. (See the Introduction to Forwarding Analysis using Batfish notebook for more forwarding analysis questions).

      [3]:
      
      # Get the answer of a traceroute question from Paris to the PoP's prefix
      pop_prefix = "2.128.0.0/24"
      tr_answer = bf.q.traceroute(
          startLocation="paris",
          headers=HeaderConstraints(dstIps=pop_prefix),
          maxTraces=1
      ).answer(FAIL_LONDON_SNAPSHOT_NAME)
      
      # Display the result in a pretty form
      show(tr_answer.frame())
      
      Flow Traces TraceCount
      0 Start Location: paris
      Src IP: 1.0.1.2
      Src Port: 49152
      Dst IP: 2.128.0.0
      Dst Port: 33434
      IP Protocol: UDP
      EXITS_NETWORK
      1. node: paris
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.2.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      7. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      9. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      1 Start Location: paris
      Src IP: 1.0.2.2
      Src Port: 49152
      Dst IP: 2.128.0.0
      Dst Port: 33434
      IP Protocol: UDP
      EXITS_NETWORK
      1. node: paris
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.2.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      7. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      9. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      2 Start Location: paris
      Src IP: 1.10.1.1
      Src Port: 49152
      Dst IP: 2.128.0.0
      Dst Port: 33434
      IP Protocol: UDP
      EXITS_NETWORK
      1. node: paris
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.2.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      7. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      9. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      3 Start Location: paris
      Src IP: 90.90.90.2
      Src Port: 49152
      Dst IP: 2.128.0.0
      Dst Port: 33434
      IP Protocol: UDP
      EXITS_NETWORK
      1. node: paris
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.2.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      7. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      9. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4

      Great! We have confirmed that Paris can still reach PoP via Asia even when London has failed.

      differentialReachability: Checking changes of forwarding behavior for all flows

      Above, we saw how Batfish can create new snapshots that simulate failure scenarios and run analysis on them. This capability is useful to test the forwarding behavior under interesting failure scenarios. In some cases, we may also want to verify that certain network failures have no impact to the network, i.e., the forwarding behavior of all flows would not be changed by those failures.

      We now show a powerful question differentialReachability of Batfish, which allows us to analyze changes of any flow between two snapshots. This question will report any flow that was successfully delivered in the base snapshot but will not be delivered in failure snapshot or the other way around—not delivered in the base snapshot but delivered in the failure snapshot.

      Let us revisit the scenario where London fails. To understand if this failure impacts any flow to the PoP in the US, we can run the differential reachability question as below, by scoping the search to flows destined to the PoP(from anywhere) and comparing FAIL_LONDON_SNAPSHOT_NAME with BASE_SNAPSHOT_NAME as the reference. Leaving the headers field unscoped would search across flows to all possible destinations.

      [4]:
      
      # Get the answer to the differential reachability question given two snapshots
      diff_reachability_answer = bf.q.differentialReachability(
          headers=HeaderConstraints(dstIps=pop_prefix), maxTraces=1).answer(
          snapshot=FAIL_LONDON_SNAPSHOT_NAME,
          reference_snapshot=BASE_SNAPSHOT_NAME)
      
      # Display the results
      show(diff_reachability_answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount
      0 Start Location: milan
      Src IP: 1.0.2.1
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      EXITS_NETWORK
      1. node: milan
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4 DENIED_IN
      1. node: milan
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 1.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: paris
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 1.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: london
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.12.11.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: newyork
        RECEIVED(GigabitEthernet0/0)
        DENIED(AS1_TO_AS2 (INGRESS_FILTER))
      1
      1 Start Location: milan interface=GigabitEthernet1/0
      Src IP: 1.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      EXITS_NETWORK
      1. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4 DENIED_IN
      1. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 1.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: paris
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 1.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: london
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.12.11.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: newyork
        RECEIVED(GigabitEthernet0/0)
        DENIED(AS1_TO_AS2 (INGRESS_FILTER))
      1
      2 Start Location: paris interface=GigabitEthernet0/0
      Src IP: 1.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      EXITS_NETWORK
      1. node: paris
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 1.0.2.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      2. node: milan
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet0/0 with resolved next-hop IP: 10.13.22.3, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.13.22.3)])
        TRANSMITTED(GigabitEthernet0/0)
      3. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      6. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      7. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      8. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      9. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4 DENIED_IN
      1. node: paris
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 1.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: london
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.12.11.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.12.11.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: newyork
        RECEIVED(GigabitEthernet0/0)
        DENIED(AS1_TO_AS2 (INGRESS_FILTER))
      1

      We see from the result that the failures of London would in fact permit a flow that was originally being blocked by the AS1_TO_AS2 ACL on New York. This difference reveals a potential security vulnerability! Luckily, Batfish allows us to catch and fix it before something bad happens in production. Similarly, if there were flows that were carried in BASE_SNAPSHOT_NAME but dropped in FAIL_LONDON_SNAPSHOT_NAME (an availability issue), Batfish would have caught it.

      Check out our Introduction to Forwarding Change Validation notebook for more use cases of differential reachability queries.

      Chaos Monkey Testing

      Chaos Monkey style testing is a common method to to build highly reliable software systems. In it, different components of the system are randomly failed to see what impact it has on the service performance. Such testing is known to be highly effective but is not possible to do in the networking context. Until now.

      Batfish can easily enable Chaos Monkey testing for networks. Using the basic functions shown above, we can compose more complicated functions that randomly fail links and identify potential vulnerabilities in the network.

      Suppose we wanted our network to be robust to any possible 2-link failures. The example below shows how to perform Chaos Monkey testing to identify 2-link-failures that can cause an outage. Specifically, we will fail a pair of links picked at random and check whether the forwarding behavior would be changed by the failure using the differentialReachability question.

      Next, we run Chaos Monkey testing, shown as below.

      [5]:
      
      # Fix for demonstration purpose
      random.seed(0)
      
      max_iterations = 5
      
      # Get all links in the network
      links = bf.q.edges().answer(BASE_SNAPSHOT_NAME).frame()
      
      for i in range(max_iterations):
          # Get two links at random
          failed_link1_index = random.randint(0, len(links) - 1)
          failed_link2_index = random.randint(0, len(links) - 1)
      
          # Fork a snapshot with the link failures
          FAIL_SNAPSHOT_NAME = "fail_snapshot"
          bf.fork_snapshot(
              BASE_SNAPSHOT_NAME,
              FAIL_SNAPSHOT_NAME,
              deactivate_interfaces=[links.loc[failed_link1_index].Interface,
                                     links.loc[failed_link2_index].Interface],
              overwrite=True)
      
          # Run a differential reachability question
          answer = bf.q.differentialReachability(
              headers=HeaderConstraints(dstIps=pop_prefix)
          ).answer(
              snapshot=FAIL_SNAPSHOT_NAME,
              reference_snapshot=BASE_SNAPSHOT_NAME
          )
      
          # A non-empty returned answer means changed forwarding behavior
          # We print the bad failure scenario and exit
          if len(answer.frame()) > 0:
              show(links.iloc[[failed_link1_index, failed_link2_index]])
              break
      
      Interface IPs Remote_Interface Remote_IPs
      32 seattle[GigabitEthernet2/0] 2.12.21.1 philadelphia[GigabitEthernet1/0] 2.12.21.2
      31 seattle[GigabitEthernet1/0] 2.12.22.1 sanfrancisco[GigabitEthernet0/0] 2.12.22.2

      We see that there is a failure scenario under to which the network is not robust, that is, the failure will lead to a change in the forwarding behavior of at least some flows. This scenario is the failure of two links that connect Seattle to Philadelphia and San Francisco. This is unexpected because Seattle has another link that connects it to the rest of the network and should generally be available for traffic.

      Let us diagnose this situation to understand the problem. To begin, we first see which flows are impacted.

      [6]:
      
      show(answer.frame())
      
      Flow Snapshot_Traces Snapshot_TraceCount Reference_Traces Reference_TraceCount
      0 Start Location: hongkong
      Src IP: 3.0.1.2
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: hongkong
        ORIGINATED(default)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: hongkong
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      1 Start Location: hongkong interface=GigabitEthernet0/0
      Src IP: 3.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      2 Start Location: hongkong interface=GigabitEthernet1/0
      Src IP: 3.0.1.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: hongkong
        RECEIVED(GigabitEthernet1/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      3 Start Location: hongkong interface=GigabitEthernet2/0
      Src IP: 90.90.90.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: hongkong
        RECEIVED(GigabitEthernet2/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: hongkong
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      5. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      4 Start Location: seattle
      Src IP: 10.23.21.2
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: seattle
        ORIGINATED(default)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: seattle
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      5 Start Location: seattle interface=GigabitEthernet0/0
      Src IP: 10.23.21.1
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: seattle
        RECEIVED(GigabitEthernet0/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      2. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      3. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      6 Start Location: singapore
      Src IP: 10.13.22.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: singapore
        ORIGINATED(default)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: singapore
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      7 Start Location: singapore interface=GigabitEthernet0/0
      Src IP: 10.13.22.2
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: singapore
        RECEIVED(GigabitEthernet0/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      8 Start Location: singapore interface=GigabitEthernet1/0
      Src IP: 3.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: singapore
        RECEIVED(GigabitEthernet1/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      9 Start Location: singapore interface=GigabitEthernet2/0
      Src IP: 30.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: singapore
        RECEIVED(GigabitEthernet2/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      5. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      6. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: singapore
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.2.2, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: hongkong
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 3.0.1.1, Routes: [ibgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      4. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      6. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      7. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      10 Start Location: tokyo
      Src IP: 10.23.21.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: tokyo
        ORIGINATED(default)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: tokyo
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      11 Start Location: tokyo interface=GigabitEthernet0/0
      Src IP: 3.0.1.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      12 Start Location: tokyo interface=GigabitEthernet1/0
      Src IP: 10.23.21.1
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: tokyo
        RECEIVED(GigabitEthernet1/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4
      13 Start Location: tokyo interface=GigabitEthernet2/0
      Src IP: 30.0.2.3
      Dst IP: 2.128.0.0
      IP Protocol: ICMP (type=8, code=0)
      NO_ROUTE
      1. node: tokyo
        RECEIVED(GigabitEthernet2/0)
        NO_ROUTE(Discarded)
      1 EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.22.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 2.12.22.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet1/0)
      3. node: sanfrancisco
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.21.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.23.11.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      4. node: washingtondc
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.101.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)

      EXITS_NETWORK
      1. node: tokyo
        RECEIVED(GigabitEthernet2/0)
        FORWARDED(Forwarded out interface: GigabitEthernet1/0 with resolved next-hop IP: 10.23.21.2, Routes: [bgp (Network: 2.128.0.0/16, Next Hop: ip 10.23.21.2)])
        TRANSMITTED(GigabitEthernet1/0)
      2. node: seattle
        RECEIVED(GigabitEthernet0/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.12.21.2, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.101.4),ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      3. node: philadelphia
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet3/0 with resolved next-hop IP: 2.23.12.3, Routes: [ibgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet3/0)
      4. node: losangeles
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0 with resolved next-hop IP: 2.34.201.4, Routes: [bgp (Network: 2.128.0.0/24, Next Hop: ip 2.34.201.4)])
        TRANSMITTED(GigabitEthernet2/0)
      5. node: houston
        RECEIVED(GigabitEthernet1/0)
        FORWARDED(Forwarded out interface: GigabitEthernet2/0, Routes: [connected (Network: 2.128.0.0/24, Next Hop: interface GigabitEthernet2/0)])
        TRANSMITTED(GigabitEthernet2/0)
        EXITS_NETWORK(Output Interface: GigabitEthernet2/0, Resolved Next Hop IP: 2.128.0.0)
      4

      We see that when the links fail, if we ignore flows that end in Seattle (whose links have failed), a general pattern is that Asia loses connectivity to US. Given the network topology, this is quite surprising because after those failure we would have expected Asia to be able to reach US via Europe.

      To investigate further into the root cause, we ask Batfish to show how the BGP RIB in the two cases differ. We do so using the bgpRib question and comparing the two snapshots as in the differential reachability question. We focus on the impacted destination prefix 2.128.0.0/16.

      [7]:
      
      diff_routes = bf.q.bgpRib(network="2.128.0.0/16").answer(snapshot=FAIL_SNAPSHOT_NAME,
                                                               reference_snapshot=BASE_SNAPSHOT_NAME)
      diff_routes
      
      [7]:
      
      Node VRF Network Entry_Presence Snapshot_Status Reference_Status Snapshot_Next_Hop Reference_Next_Hop Snapshot_Next_Hop_IP Reference_Next_Hop_IP Snapshot_Protocol Reference_Protocol Snapshot_AS_Path Reference_AS_Path Snapshot_Metric Reference_Metric Snapshot_Local_Pref Reference_Local_Pref Snapshot_Communities Reference_Communities Snapshot_Origin_Protocol Reference_Origin_Protocol Snapshot_Origin_Type Reference_Origin_Type Snapshot_Originator_Id Reference_Originator_Id Snapshot_Received_From_IP Reference_Received_From_IP Snapshot_Received_Path_Id Reference_Received_Path_Id Snapshot_Cluster_List Reference_Cluster_List Snapshot_Tunnel_Encapsulation_Attribute Reference_Tunnel_Encapsulation_Attribute Snapshot_Weight Reference_Weight Snapshot_Tag Reference_Tag
      0 hongkong default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 10.23.21.2 None 10.23.21.2 None ibgp None 2 None 50 None 350 None ['2:3'] None ibgp None igp None 3.1.1.1 None 3.1.1.1 None 1 None [] None None None 0 None None
      1 losangeles default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 2.1.1.2 None 2.1.1.2 None ibgp None None 0 None 100 None [] None ibgp None igp None 2.1.1.2 None 2.1.2.1 None 2 None [33620481] None None None 0 None None
      2 losangeles default 2.128.0.0/16 Unchanged ['BEST'] ['BEST'] ip 2.1.1.1 ip 2.1.1.1 2.1.1.1 2.1.1.1 ibgp ibgp 0 0 100 100 [] [] ibgp ibgp igp igp 2.1.1.1 2.1.1.1 2.1.2.1 2.1.2.1 1 1 [33620481] [33620481] None None 0 0 None None
      3 philadelphia default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 2.1.1.2 None 2.1.1.2 None ibgp None None 0 None 100 None [] None ibgp None igp None 2.1.1.2 None 2.1.1.2 None 1 None [] None None None 0 None None
      4 philadelphia default 2.128.0.0/16 Unchanged ['BEST'] ['BEST'] ip 2.1.1.1 ip 2.1.1.1 2.1.1.1 2.1.1.1 ibgp ibgp 0 0 100 100 [] [] ibgp ibgp igp igp 2.1.1.1 2.1.1.1 2.1.1.1 2.1.1.1 1 1 [] [] None None 0 0 None None
      5 sanfrancisco default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 2.1.1.2 None 2.1.1.2 None ibgp None None 0 None 100 None [] None ibgp None igp None 2.1.1.2 None 2.1.1.2 None 1 None [] None None None 0 None None
      6 sanfrancisco default 2.128.0.0/16 Unchanged ['BEST'] ['BEST'] ip 2.1.1.1 ip 2.1.1.1 2.1.1.1 2.1.1.1 ibgp ibgp 0 0 100 100 [] [] ibgp ibgp igp igp 2.1.1.1 2.1.1.1 2.1.1.1 2.1.1.1 1 1 [] [] None None 0 0 None None
      7 seattle default 2.128.0.0/16 Only in Reference None ['BEST'] None discard None AUTO/NONE(-1l) None aggregate None None 0 None 100 None [] None aggregate None igp None 2.1.1.2 None 0.0.0.0 None None None [] None None None 32768 None None
      8 singapore default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 10.23.21.2 None 10.23.21.2 None ibgp None 2 None 50 None 350 None ['2:3'] None ibgp None igp None 3.1.1.1 None 3.10.1.1 None 1 None [50987265] None None None 0 None None
      9 tokyo default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 10.23.21.2 None 10.23.21.2 None bgp None 2 None 50 None 350 None ['2:3'] None bgp None igp None 2.1.1.2 None 10.23.21.2 None None None [] None None None 0 None None
      10 washingtondc default 2.128.0.0/16 Only in Reference None ['BEST'] None ip 2.1.1.2 None 2.1.1.2 None ibgp None None 0 None 100 None [] None ibgp None igp None 2.1.1.2 None 2.1.2.1 None 2 None [33620481] None None None 0 None None
      11 washingtondc default 2.128.0.0/16 Unchanged ['BEST'] ['BEST'] ip 2.1.1.1 ip 2.1.1.1 2.1.1.1 2.1.1.1 ibgp ibgp 0 0 100 100 [] [] ibgp ibgp igp igp 2.1.1.1 2.1.1.1 2.1.2.1 2.1.2.1 1 1 [33620481] [33620481] None None 0 0 None None

      We see that routers in Asia (Hongkong, Singapore, and Tokyo) and Seattle do not have any BGP routes to the prefix in the failure snapshot, which they did in the reference snapshot. The missing route in Seattle can be explained via missing routes in Asia since Seattle depended on Asia after losing its two other links.

      That Europe still has the routes after the failure alerts us to the possibility of improper filtering of incoming routes in Asia. So, we should check on that. There are many ways to analyze the incoming route filters; we’ll use the definedStructures question of Batfish to extract necessary definitions that we need to view.

      [8]:
      
      # View defined structures of type 'route-map' on 'hongkong'
      bf.q.definedStructures(types='route-map', nodes="hongkong").answer()
      
      [8]:
      
      Structure_Type Structure_Name Source_Lines
      0 route-map as1_to_as3 configs/hongkong.cfg:[119, 120]

      We see the route map as1_to_as3 is defined on line 119 and 120. Now we can quickly navigate to the lines in the config file, as showing below.

      [9]:
      
      # See the config lines where the route map as1_to_as3 is defined
      !cat networks/failure-analysis/configs/hongkong.cfg | head -121 | tail -4
      
      !
      route-map as1_to_as3 deny 100
       match ip address 102
      !
      

      We see that the route map is denying routes that match the access-list ‘102.’ Let’s look at the definition of this list, which is on lines 115-117 per the defined structures list above.

      [10]:
      
      # See the config lines where the access list '102' is defined
      !cat networks/failure-analysis/configs/hongkong.cfg | head -118 | tail -5
      
      !
      access-list 102 permit ip host 1.0.1.0 host 255.255.255.0
      access-list 102 permit ip host 1.0.2.0 host 255.255.255.0
      access-list 102 permit ip host 2.128.0.0 host 255.255.0.0
      !
      

      We see that this list includes the prefix of interest, which is 2.128.0.0/16 on the last line. Thus, the route map inadvertently blocks the prefix, thus disconnecting Asia from US when Seattle or its links fail.

      Without Batfish, it would have been hard to find this vulnerability, but the Chaos Monkey style testing enabled by Batfish makes it easy to find such vulnerabilities before they cause a service outage.

      Summary

      This notebook demonstrates how Batfish help analyze forwarding behavior in network failures. Specifically, 1. bf.fork_snapshot can clone a snapshot from another with deactivated interfaces and nodes; 2. differentialReachability can check all forwarding behavior changes for all flows between two snapshots; 3. We can build on top of the basic functions to create more involved analysis such as Chaos Monkey testing.


      Get involved with the Batfish community

      Join our community on Slack and GitHub.

      Cloud networking

      Analyzing public cloud and hybrid networks

      Public cloud and hybrid networks can be hard to debug and secure. Many of the standard tools (e.g., traceroute) do not work in the cloud setting, even though the types of paths that can emerge are highly complicated, depending on whether the endpoints are in the same region, different regions, across physical and virtual infrastructure, or whether public or private IPs of cloud instances are being used.

      At same time, the fast pace of evolution of these networks, where new subnets and instances can be spun up rapidly by different groups of people, creates a significant security risk. Network engineers need tools than can provide comprehensive guaratnees that services and applications are available and secure as intended at all possible times.

      In this notebook, we show how Batfish can predict and help debug network paths for cloud and hybrid networks and how it can guarantee that the network’s availability and security posture is exactly as desired.

      [1]:
      
      # Import packages
      %run startup.py
      bf = Session(host="localhost")
      
      
      def show_first_trace(trace_answer_frame):
          """
          Prints the first trace in the answer frame.
      
          In the presence of multipath routing, Batfish outputs all traces
          from the source to destination. This function picks the first one.
          """
          if len(trace_answer_frame) == 0:
              print("No flows found")
          else:
              show("Flow: {}".format(trace_answer_frame.iloc[0]['Flow']))
              show(trace_answer_frame.iloc[0]['Traces'][0])
      
      def is_reachable(start_location, end_location, headers=None):
          """
          Checks if the start_location can reach the end_location using specified packet headers.
      
          All possible headers are considered if headers is None.
          """
          ans = bf.q.reachability(pathConstraints=PathConstraints(startLocation=start_location,
                                                                 endLocation=end_location),
                                headers=headers).answer()
          return len(ans.frame()) > 0
      
      Initializing the Network and Snapshot

      SNAPSHOT_PATH below can be updated to point to a custom snapshot directory. See instructions for how to package data for analysis.

      [2]:
      
      # Initialize a network and snapshot
      NETWORK_NAME = "hybrid-cloud"
      SNAPSHOT_NAME = "snapshot"
      
      SNAPSHOT_PATH = "networks/hybrid-cloud"
      
      bf.set_network(NETWORK_NAME)
      bf.init_snapshot(SNAPSHOT_PATH, name=SNAPSHOT_NAME, overwrite=True)
      
      Your snapshot was successfully initialized but Batfish failed to fully recognized some lines in one or more input files. Some unrecognized configuration lines are not uncommon for new networks, and it is often fine to proceed with further analysis. You can help the Batfish developers improve support for your network by running:
      
          bf.upload_diagnostics(dry_run=False, contact_info='<optional email address>')
      
      to share private, anonymized information. For more information, see the documentation with:
      
          help(bf.upload_diagnostics)
      
      [2]:
      
      'snapshot'
      

      The network snapshot that we just initialized is illustrated below. It has a datacenter network with the standard leaf-spine design on the left. Though not strictly necessary, we have included a host srv-101 in this network to enable end-to-end analysis. The exit gateway of the datacenter connects to an Internet service provider (ASN 65200) that we call isp_dc.

      The AWS network is shown on the right. It is spread across two regions, us-east-2 and us-west-2. Each region has two VPCs, one of which is meant to host Internet-facing services and the other is meant to host only private services. Subnets in the public-facing VPCs use an Internet gateway to send and receive traffic outside of AWS. The two VPCs in a region peer via a transit gateway. Each VPC has two subnets, and we have some instances running as well.

      The physical network connects to the AWS network using IPSec tunnels, shown in pink, between exitgw and the two transit gateways. BGP sessions run atop these tunnels to make endpoints aware of prefixes on the other side.

      You can view configuration files that we used here. The AWS portion of the configuration is in the aws_configs subfolder. It has JSON files obtained via AWS APIs. An example script that packages AWS data into a Batfish snapshot is here.

      hybrid-cloud-network

      Analyzing network paths

      Batfish can help analyze cloud and hybrid networks by showing how exactly traffic flows (or not) in the network, which can help debug and fix configuration errors. Batfish can also help ensure that the network is configured exactly as desired, with respect to reachability and security policies. We illustrate these types of analysis below.

      First, lets define a couple of maps to help with the analysis.

      [3]:
      
      #Instances in AWS in each region and VPC type (public, private)
      hosts = {}
      hosts["east2_private"] = "i-04cd3db5124a05ee6"
      hosts["east2_public"] = "i-01602d9efaed4409a"
      hosts["west2_private"] = "i-0a5d64b8b58c6dd09"
      hosts["west2_public"] = "i-02cae6eaa9edeed70"
      
      #Public IPs of instances in AWS
      public_ips = {}
      public_ips["east2_public"] = "13.59.144.125" # of i-01602d9efaed4409a
      public_ips["west2_public"] = "54.191.42.182" # of i-02cae6eaa9edeed70
      
      Paths across VPCs within an AWS region

      To see how traffic flows between two instances in the same region but across different VPCs, say, from hosts["east2_private"] to hosts["east2_public"], we can run a traceroute query across them as follows.

      In the query below, we use the name of the instance as the destination for the traceroute. This makes Batfish pick the instance’s private (i.e., non-Elastic) IP (10.20.1.207). It does not pick the public IP because that those IPs do not reside on instances but are used by the Internet gateway to NAT instance’s traffic in and out of AWS (see documentation). If an instance has multiple private IPs, Batfish will pick one at random. To make Batfish use a specific IP, supply that IP as the argument to the dstIps parameter.

      [4]:
      
      # traceroute between instances in the same region, using SSH
      ans = bf.q.traceroute(startLocation=hosts["east2_private"],
                           headers=HeaderConstraints(dstIps=hosts["east2_public"],
                                                     applications="ssh")).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=i-04cd3db5124a05ee6 [10.30.1.166:49152->10.20.1.207:22 TCP (SYN)]'
      
      ACCEPTED
      1. node: i-04cd3db5124a05ee6
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: eni-05452497daf80ccb3 with resolved next-hop IP: 10.30.1.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: ip 10.30.1.1)])
        PERMITTED(~EGRESS_ACL~eni-05452497daf80ccb3 (EGRESS_FILTER))
        SETUP_SESSION(Incoming Interfaces: [eni-05452497daf80ccb3], Action: Accept, Match Criteria: [ipProtocol=TCP, srcIp=10.20.1.207, dstIp=10.30.1.166, srcPort=22, dstPort=49152])
        TRANSMITTED(eni-05452497daf80ccb3)
      2. node: subnet-0cb5f4c094bee5214
        RECEIVED(to-instances)
        FORWARDED(Forwarded out interface: vpc-0276455718806058a-vrf-tgw-attach-021f89744fac566dd with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.0.0/16, Next Hop: interface vpc-0276455718806058a-vrf-tgw-attach-021f89744fac566dd ip 169.254.0.1)])
        PERMITTED(acl-0b3d0f6b0978f09f8_egress (EGRESS_FILTER))
        TRANSMITTED(vpc-0276455718806058a-vrf-tgw-attach-021f89744fac566dd)
      3. node: vpc-0276455718806058a
        RECEIVED(subnet-0cb5f4c094bee5214-vrf-tgw-attach-021f89744fac566dd)
        FORWARDED(Forwarded out interface: tgw-06b348adabd13452d-tgw-rtb-00e37bc5142347b03 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface tgw-06b348adabd13452d-tgw-rtb-00e37bc5142347b03 ip 169.254.0.1)])
        TRANSMITTED(tgw-06b348adabd13452d-tgw-rtb-00e37bc5142347b03)
      4. node: tgw-06b348adabd13452d
        RECEIVED(vpc-0276455718806058a-tgw-rtb-00e37bc5142347b03)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.0.0/16, Next Hop: interface vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03 ip 169.254.0.1)])
        TRANSMITTED(vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03)
      5. node: vpc-0574d08f8d05917e4
        RECEIVED(tgw-06b348adabd13452d-tgw-rtb-00e37bc5142347b03)
        FORWARDED(Forwarded out interface: subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.1.0/24, Next Hop: interface subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5 ip 169.254.0.1)])
        TRANSMITTED(subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5)
      6. node: subnet-06a692ed4ef84368d
        RECEIVED(vpc-0574d08f8d05917e4-vrf-tgw-attach-0648110513acd6de5)
        PERMITTED(acl-09c0bb4e71ae5f9e4_ingress (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: to-instances, Routes: [connected (Network: 10.20.1.0/24, Next Hop: interface to-instances)])
        TRANSMITTED(to-instances)
      7. node: i-01602d9efaed4409a
        RECEIVED(eni-01997085076a9b98a)
        PERMITTED(~INGRESS_ACL~eni-01997085076a9b98a (INGRESS_FILTER))
        SETUP_SESSION(Originating VRF: default, Action: ForwardOutInterface(Next Hop: subnet-06a692ed4ef84368d, Next Hop Interface: to-instances, Outgoing Interface: eni-01997085076a9b98a), Match Criteria: [ipProtocol=TCP, srcIp=10.20.1.207, dstIp=10.30.1.166, srcPort=22, dstPort=49152])
        ACCEPTED(eni-01997085076a9b98a)

      The trace above shows how traffic goes from host["east2_private"] to host["east2_public"] – via the source subnet and VPC, then to the transit gateway, and finally to the destination VPC and subnet. Along the way, it also shows where the flow encounters security groups (at both instances) and network ACLs (at subnets). In this instance, all security groups and network ACLs permit this particular flow.

      This type of insight into traffic paths, which helps understand and debug network configuration, is difficult to obtain otherwise. Traceroutes on the live AWS network do not yield any information if the flow does not make it through, and do not show why or where a packet is dropped.

      Paths across AWS regions

      The traceroute query below shows paths across instances in two different regions.

      [5]:
      
      # traceroute between instances across region using the destination's private IP
      ans = bf.q.traceroute(startLocation=hosts["east2_public"],
                           headers=HeaderConstraints(dstIps=hosts["west2_public"],
                                                    applications="ssh")).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=i-01602d9efaed4409a [10.20.1.207:49152->10.40.2.80:22 TCP (SYN)]'
      
      DENIED_OUT
      1. node: i-01602d9efaed4409a
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: eni-01997085076a9b98a with resolved next-hop IP: 10.20.1.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: ip 10.20.1.1)])
        PERMITTED(~EGRESS_ACL~eni-01997085076a9b98a (EGRESS_FILTER))
        SETUP_SESSION(Incoming Interfaces: [eni-01997085076a9b98a], Action: Accept, Match Criteria: [ipProtocol=TCP, srcIp=10.40.2.80, dstIp=10.20.1.207, srcPort=22, dstPort=49152])
        TRANSMITTED(eni-01997085076a9b98a)
      2. node: subnet-06a692ed4ef84368d
        RECEIVED(to-instances)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7 ip 169.254.0.1)])
        PERMITTED(acl-09c0bb4e71ae5f9e4_egress (EGRESS_FILTER))
        TRANSMITTED(vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7)
      3. node: vpc-0574d08f8d05917e4
        RECEIVED(subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7)
        FORWARDED(Forwarded out interface: igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface igw-02fd68f94367a67c7 ip 169.254.0.1)])
        TRANSMITTED(igw-02fd68f94367a67c7)
      4. node: igw-02fd68f94367a67c7
        RECEIVED(vpc-0574d08f8d05917e4)
        PERMITTED(~DENY~UNASSOCIATED~PRIVATE~IPs~ (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: backbone with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: interface backbone ip 169.254.0.1)])
        TRANSFORMED(SOURCE_NAT srcIp: 10.20.1.207 -> 13.59.144.125)
        TRANSMITTED(backbone)
      5. node: isp_16509
        RECEIVED(To-igw-02fd68f94367a67c7-backbone)
        FORWARDED(Forwarded out interface: To-Internet with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: interface To-Internet ip 169.254.0.1)])
        DENIED(Block outgoing traffic using reserved addresses (EGRESS_FILTER))

      We see that such traffic does not reach the destination but instead is dropped by the AWS backbone (ASN 16509). This happens because, in our network, there is no (transit gateway or VPC) peering between VPCs in different regions. So, the source subnet is unaware of the address space of the destination subnet, which makes it use the default route that points to the Internet gateway (igw-02fd68f94367a67c7). The Internet gateway forwards the packet to aws-backbone, after NAT’ing its source IP. The packet is eventually dropped as it is using a private address as destination. Recall that using the instance name as destination amounts to using its private IP.

      The behavior is different if we use the public IP instead, as shown below.

      [6]:
      
      # traceroute betwee instances across region using the destination's public IP
      ans = bf.q.traceroute(startLocation=hosts["east2_public"],
                           headers=HeaderConstraints(dstIps=public_ips["west2_public"],
                                                    applications="ssh")).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=i-01602d9efaed4409a [10.20.1.207:49152->54.191.42.182:22 TCP (SYN)]'
      
      ACCEPTED
      1. node: i-01602d9efaed4409a
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: eni-01997085076a9b98a with resolved next-hop IP: 10.20.1.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: ip 10.20.1.1)])
        PERMITTED(~EGRESS_ACL~eni-01997085076a9b98a (EGRESS_FILTER))
        SETUP_SESSION(Incoming Interfaces: [eni-01997085076a9b98a], Action: Accept, Match Criteria: [ipProtocol=TCP, srcIp=54.191.42.182, dstIp=10.20.1.207, srcPort=22, dstPort=49152])
        TRANSMITTED(eni-01997085076a9b98a)
      2. node: subnet-06a692ed4ef84368d
        RECEIVED(to-instances)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7 ip 169.254.0.1)])
        PERMITTED(acl-09c0bb4e71ae5f9e4_egress (EGRESS_FILTER))
        TRANSMITTED(vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7)
      3. node: vpc-0574d08f8d05917e4
        RECEIVED(subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7)
        FORWARDED(Forwarded out interface: igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface igw-02fd68f94367a67c7 ip 169.254.0.1)])
        TRANSMITTED(igw-02fd68f94367a67c7)
      4. node: igw-02fd68f94367a67c7
        RECEIVED(vpc-0574d08f8d05917e4)
        PERMITTED(~DENY~UNASSOCIATED~PRIVATE~IPs~ (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: backbone with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: interface backbone ip 169.254.0.1)])
        TRANSFORMED(SOURCE_NAT srcIp: 10.20.1.207 -> 13.59.144.125)
        TRANSMITTED(backbone)
      5. node: isp_16509
        RECEIVED(To-igw-02fd68f94367a67c7-backbone)
        FORWARDED(Forwarded out interface: To-igw-0a8309f3192e7cea3-backbone with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 54.191.42.182/32, Next Hop: interface To-igw-0a8309f3192e7cea3-backbone ip 169.254.0.1)])
        TRANSMITTED(To-igw-0a8309f3192e7cea3-backbone)
      6. node: igw-0a8309f3192e7cea3
        RECEIVED(backbone)
        TRANSFORMED(DEST_NAT dstIp: 54.191.42.182 -> 10.40.2.80)
        FORWARDED(Forwarded out interface: vpc-00b65e98077106059 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.40.0.0/16, Next Hop: interface vpc-00b65e98077106059 ip 169.254.0.1)])
        TRANSMITTED(vpc-00b65e98077106059)
      7. node: vpc-00b65e98077106059
        RECEIVED(igw-0a8309f3192e7cea3)
        FORWARDED(Forwarded out interface: subnet-06005943afe32f714-vrf-igw-0a8309f3192e7cea3 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.40.2.0/24, Next Hop: interface subnet-06005943afe32f714-vrf-igw-0a8309f3192e7cea3 ip 169.254.0.1)])
        TRANSMITTED(subnet-06005943afe32f714-vrf-igw-0a8309f3192e7cea3)
      8. node: subnet-06005943afe32f714
        RECEIVED(vpc-00b65e98077106059-vrf-igw-0a8309f3192e7cea3)
        PERMITTED(acl-087574e8620270842_ingress (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: to-instances, Routes: [connected (Network: 10.40.2.0/24, Next Hop: interface to-instances)])
        TRANSMITTED(to-instances)
      9. node: i-02cae6eaa9edeed70
        RECEIVED(eni-087e18628dadd9b48)
        PERMITTED(~INGRESS_ACL~eni-087e18628dadd9b48 (INGRESS_FILTER))
        SETUP_SESSION(Originating VRF: default, Action: ForwardOutInterface(Next Hop: subnet-06005943afe32f714, Next Hop Interface: to-instances, Outgoing Interface: eni-087e18628dadd9b48), Match Criteria: [ipProtocol=TCP, srcIp=10.40.2.80, dstIp=13.59.144.125, srcPort=22, dstPort=49152])
        ACCEPTED(eni-087e18628dadd9b48)

      This traceroute starts out like the previous one, up until the AWS backbone (isp_16509) – from source subnet to the Internet gateway which forwards it to the backbone, after source NAT’ing the packet. The backbone carries it to the internet gateway in the destination region (igw-0a8309f3192e7cea3), and this gateway NATs the packet’s destination from the public IP to the instance’s private IP.

      Connectivity between DC and AWS

      A common mode to connect to AWS is using VPNs and BGP, that is, establish IPSec tunnels between exit gateways on the physical side and AWS gateways and run BGP on top of these tunnels to exchange prefixes. Incompatibility in either IPSec or BGP settings on the two sides means that connectivity between the DC and AWS will not work.

      Batfish can determine if the two sides are compatibly configured with respect to IPSec and BGP settings and if those sessions will come up.

      The query below lists the status of all IPSec sessions between the exitgw and AWS transit gateways (specified using the regular expression ^tgw- that matches those node names). This filtering lets us ignore any other IPSec sessions that may exist in our network and focus on DC-AWS connectivity.

      [7]:
      
      # show the status of all IPSec tunnels between exitgw and AWS transit gateways
      ans = bf.q.ipsecSessionStatus(nodes="exitgw", remoteNodes="/^tgw-/").answer()
      show(ans.frame())
      
      Node Node_Interface Node_IP Remote_Node Remote_Node_Interface Remote_Node_IP Tunnel_Interfaces Status
      0 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-06b348adabd13452d tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-1] 3.19.24.131 Tunnel1 -> vpn-vpn-01c45673532d3e33e-1 IPSEC_SESSION_ESTABLISHED
      1 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-06b348adabd13452d tgw-06b348adabd13452d[external-vpn-01c45673532d3e33e-2] 52.14.53.162 Tunnel2 -> vpn-vpn-01c45673532d3e33e-2 IPSEC_SESSION_ESTABLISHED
      2 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-0888a76c8a371246d tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-1] 34.209.88.227 Tunnel3 -> vpn-vpn-0dc7abdb974ff8a69-1 IPSEC_SESSION_ESTABLISHED
      3 exitgw exitgw[GigabitEthernet3] 147.75.69.27 tgw-0888a76c8a371246d tgw-0888a76c8a371246d[external-vpn-0dc7abdb974ff8a69-2] 44.227.244.7 Tunnel4 -> vpn-vpn-0dc7abdb974ff8a69-2 IPSEC_SESSION_ESTABLISHED

      In the output above, we see all expected tunnels. Each transit gateways has two established sessions to exitgw. The default AWS behavior is to have two IPSec tunnels between gateways and physical nodes.

      Now that we know IPSec tunnels are working, we can check BGP sessions. The query below lists the status of all BGP sessions where one end is an AWS transit gateway.

      [8]:
      
      # show the status of all BGP sessions between exitgw and AWS transit gateways
      ans = bf.q.bgpSessionStatus(nodes="exitgw", remoteNodes="/^tgw-/").answer()
      show(ans.frame())
      
      Node VRF Local_AS Local_Interface Local_IP Remote_AS Remote_Node Remote_Interface Remote_IP Address_Families Session_Type Established_Status
      0 exitgw default 65100 None 169.254.25.162 64512 tgw-06b348adabd13452d None 169.254.25.161 IPV4_UNICAST EBGP_SINGLEHOP ESTABLISHED
      1 exitgw default 65100 None 169.254.172.2 64512 tgw-06b348adabd13452d None 169.254.172.1 IPV4_UNICAST EBGP_SINGLEHOP ESTABLISHED
      2 exitgw default 65100 None 169.254.215.82 64512 tgw-0888a76c8a371246d None 169.254.215.81 IPV4_UNICAST EBGP_SINGLEHOP ESTABLISHED
      3 exitgw default 65100 None 169.254.252.78 64512 tgw-0888a76c8a371246d None 169.254.252.77 IPV4_UNICAST EBGP_SINGLEHOP ESTABLISHED

      The output above shows that all BGP sessions are established as expected.

      Paths from the DC to AWS

      Finally, lets look at paths from the datacenter to AWS. The query below does that using the private IP of the public instance in us-east-2 region.

      [9]:
      
      # traceroute from DC host to an instances using private IP
      ans = bf.q.traceroute(startLocation="srv-101",
                           headers=HeaderConstraints(dstIps=hosts["east2_public"],
                                                    applications="ssh")).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=srv-101 [203.0.113.12:49152->10.20.1.207:22 TCP (SYN)]'
      
      ACCEPTED
      1. node: srv-101
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 203.0.113.2, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 203.0.113.2)])
        TRANSMITTED(eth0)
      2. node: leaf1
        RECEIVED(Ethernet10)
        FORWARDED(Forwarded out interface: Ethernet1 with resolved next-hop IP: 10.10.11.1, Routes: [bgp (Network: 10.20.0.0/16, Next Hop: ip 10.10.11.1)])
        TRANSMITTED(Ethernet1)
      3. node: spine1
        RECEIVED(Ethernet1)
        FORWARDED(Forwarded out interface: Ethernet10 with resolved next-hop IP: 10.10.100.2, Routes: [bgp (Network: 10.20.0.0/16, Next Hop: ip 10.10.100.2)])
        TRANSMITTED(Ethernet10)
      4. node: exitgw
        RECEIVED(GigabitEthernet1)
        FORWARDED(Forwarded out interface: Tunnel1 with resolved next-hop IP: 169.254.25.161, Routes: [bgp (Network: 10.20.0.0/16, Next Hop: ip 169.254.25.161)])
        TRANSMITTED(Tunnel1)
      5. node: tgw-06b348adabd13452d
        RECEIVED(vpn-vpn-01c45673532d3e33e-1)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.0.0/16, Next Hop: interface vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03 ip 169.254.0.1)])
        TRANSMITTED(vpc-0574d08f8d05917e4-tgw-rtb-00e37bc5142347b03)
      6. node: vpc-0574d08f8d05917e4
        RECEIVED(tgw-06b348adabd13452d-tgw-rtb-00e37bc5142347b03)
        FORWARDED(Forwarded out interface: subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.1.0/24, Next Hop: interface subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5 ip 169.254.0.1)])
        TRANSMITTED(subnet-06a692ed4ef84368d-vrf-tgw-attach-0648110513acd6de5)
      7. node: subnet-06a692ed4ef84368d
        RECEIVED(vpc-0574d08f8d05917e4-vrf-tgw-attach-0648110513acd6de5)
        PERMITTED(acl-09c0bb4e71ae5f9e4_ingress (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: to-instances, Routes: [connected (Network: 10.20.1.0/24, Next Hop: interface to-instances)])
        TRANSMITTED(to-instances)
      8. node: i-01602d9efaed4409a
        RECEIVED(eni-01997085076a9b98a)
        PERMITTED(~INGRESS_ACL~eni-01997085076a9b98a (INGRESS_FILTER))
        SETUP_SESSION(Originating VRF: default, Action: ForwardOutInterface(Next Hop: subnet-06a692ed4ef84368d, Next Hop Interface: to-instances, Outgoing Interface: eni-01997085076a9b98a), Match Criteria: [ipProtocol=TCP, srcIp=10.20.1.207, dstIp=203.0.113.12, srcPort=22, dstPort=49152])
        ACCEPTED(eni-01997085076a9b98a)

      We see that this traffic travels on the IPSec links between the datacenter’s exitgw and the transit gateway in the destination region (tgw-06b348adabd13452d), and then makes it to the destination instance after making it successfully past the network ACL on the subnet node and the security group on the instance.

      A different path emerges if we use the public IP of the same instance, as shown below.

      [10]:
      
      # traceroute from DC host to an instances using public IP
      ans = bf.q.traceroute(startLocation="srv-101",
                           headers=HeaderConstraints(dstIps=public_ips["east2_public"],
                                                    applications="ssh")).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=srv-101 [203.0.113.12:49152->13.59.144.125:22 TCP (SYN)]'
      
      ACCEPTED
      1. node: srv-101
        ORIGINATED(default)
        FORWARDED(Forwarded out interface: eth0 with resolved next-hop IP: 203.0.113.2, Routes: [static (Network: 0.0.0.0/0, Next Hop: interface eth0 ip 203.0.113.2)])
        TRANSMITTED(eth0)
      2. node: leaf1
        RECEIVED(Ethernet10)
        FORWARDED(Forwarded out interface: Ethernet1 with resolved next-hop IP: 10.10.11.1, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: ip 10.10.11.1)])
        TRANSMITTED(Ethernet1)
      3. node: spine1
        RECEIVED(Ethernet1)
        FORWARDED(Forwarded out interface: Ethernet10 with resolved next-hop IP: 10.10.100.2, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: ip 10.10.100.2)])
        TRANSMITTED(Ethernet10)
      4. node: exitgw
        RECEIVED(GigabitEthernet1)
        FORWARDED(Forwarded out interface: GigabitEthernet3 with resolved next-hop IP: 147.75.69.26, Routes: [static (Network: 0.0.0.0/0, Next Hop: ip 147.75.69.26)])
        TRANSMITTED(GigabitEthernet3)
      5. node: isp_65200
        RECEIVED(To-exitgw-GigabitEthernet3)
        FORWARDED(Forwarded out interface: To-Internet with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 0.0.0.0/0, Next Hop: interface To-Internet ip 169.254.0.1)])
        PERMITTED(Block outgoing traffic using reserved addresses (EGRESS_FILTER))
        TRANSMITTED(To-Internet)
      6. node: internet
        RECEIVED(To-isp_65200)
        FORWARDED(Forwarded out interface: To-isp_16509 with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 13.59.144.125/32, Next Hop: interface To-isp_16509 ip 169.254.0.1)])
        TRANSMITTED(To-isp_16509)
      7. node: isp_16509
        RECEIVED(To-Internet)
        PERMITTED(Block incoming traffic using reserved addresses (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: To-igw-02fd68f94367a67c7-backbone with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 13.59.144.125/32, Next Hop: interface To-igw-02fd68f94367a67c7-backbone ip 169.254.0.1)])
        TRANSMITTED(To-igw-02fd68f94367a67c7-backbone)
      8. node: igw-02fd68f94367a67c7
        RECEIVED(backbone)
        TRANSFORMED(DEST_NAT dstIp: 13.59.144.125 -> 10.20.1.207)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.0.0/16, Next Hop: interface vpc-0574d08f8d05917e4 ip 169.254.0.1)])
        TRANSMITTED(vpc-0574d08f8d05917e4)
      9. node: vpc-0574d08f8d05917e4
        RECEIVED(igw-02fd68f94367a67c7)
        FORWARDED(Forwarded out interface: subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.1.0/24, Next Hop: interface subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7 ip 169.254.0.1)])
        TRANSMITTED(subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7)
      10. node: subnet-06a692ed4ef84368d
        RECEIVED(vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7)
        PERMITTED(acl-09c0bb4e71ae5f9e4_ingress (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: to-instances, Routes: [connected (Network: 10.20.1.0/24, Next Hop: interface to-instances)])
        TRANSMITTED(to-instances)
      11. node: i-01602d9efaed4409a
        RECEIVED(eni-01997085076a9b98a)
        PERMITTED(~INGRESS_ACL~eni-01997085076a9b98a (INGRESS_FILTER))
        SETUP_SESSION(Originating VRF: default, Action: ForwardOutInterface(Next Hop: subnet-06a692ed4ef84368d, Next Hop Interface: to-instances, Outgoing Interface: eni-01997085076a9b98a), Match Criteria: [ipProtocol=TCP, srcIp=10.20.1.207, dstIp=203.0.113.12, srcPort=22, dstPort=49152])
        ACCEPTED(eni-01997085076a9b98a)

      We now see that the traffic traverses the Internet via isp_65200 and the Internet gateway (igw-02fd68f94367a67c7), which NATs the destination address of the packet from the public to the private IP.

      Evaluating the network’s availability and security

      In addition to helping you understand and debug network paths, Batfish can also help ensure that the network is correctly configured with respect to its availability and security policies.

      As examples, the queries below evaluate which instances are or are not reachable from the Internet.

      [11]:
      
      # compute which instances are open to the Internet
      reachable_from_internet = [key for (key, value) in hosts.items() if is_reachable("internet", value)]
      print("\nInstances reachable from the Internet: {}".format(sorted(reachable_from_internet)))
      
      # compute which instances are NOT open to the Internet
      unreachable_from_internet = [key for (key, value) in hosts.items() if not is_reachable("internet", value)]
      print("\nInstances NOT reachable from the Internet: {}".format(sorted(unreachable_from_internet)))
      
      
      Instances reachable from the Internet: ['east2_public', 'west2_public']
      
      Instances NOT reachable from the Internet: ['east2_private', 'west2_private']
      

      We see that Batfish correctly computes that the two instances in the public subnets are accessible from the Internet, and the other two are not.

      We can compare the answers produced by Batfish to what is expected based on network policy. This comparison can ensure that all instances that are expected to host public-facing services are indeed reachable from the Internet, and all instances that are expecpted to host private services are indeed not accessible from the Internet.

      We can similarly compute which instances are reachable from hosts in the datacenter, using the query like the following.

      [12]:
      
      # compute which instances are reachable from data center
      reachable_from_dc = [key for (key,value) in hosts.items() if is_reachable("srv-101", value)]
      print("\nInstances reachable from the DC: {}".format(sorted(reachable_from_dc)))
      
      
      Instances reachable from the DC: ['east2_private', 'east2_public', 'west2_private', 'west2_public']
      

      We see that all four instances are accessible from the datacenter host.

      Batfish allows a finer-grained evaluation of security policy as well. In our network, our intent is that the public instances should only allow SSH traffic. Let us see if this invariant actually holds.

      [13]:
      
      tcp_non_ssh = HeaderConstraints(ipProtocols="tcp", dstPorts="!22")
      reachable_from_internet_non_ssh = [key for (key, value) in hosts.items()
                                         if is_reachable("internet", value, tcp_non_ssh)]
      print("\nInstances reachable from the Internet with non-SSH traffic: {}".format(
          sorted(reachable_from_internet_non_ssh)))
      
      
      Instances reachable from the Internet with non-SSH traffic: ['east2_public']
      

      We see that, against our policy, the public-facing instance allows non-SSH traffic. To see examples of such traffic, we can run the following query.

      [14]:
      
      ans = bf.q.reachability(pathConstraints=PathConstraints(startLocation="internet",
                                                             endLocation=hosts["east2_public"]),
                            headers=tcp_non_ssh).answer()
      show_first_trace(ans.frame())
      
      'Flow: start=internet interface=out [8.8.8.8:49152->13.59.144.125:3306 TCP (SYN)]'
      
      ACCEPTED
      1. node: internet
        RECEIVED(out)
        FORWARDED(Forwarded out interface: To-isp_16509 with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 13.59.144.125/32, Next Hop: interface To-isp_16509 ip 169.254.0.1)])
        TRANSMITTED(To-isp_16509)
      2. node: isp_16509
        RECEIVED(To-Internet)
        PERMITTED(Block incoming traffic using reserved addresses (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: To-igw-02fd68f94367a67c7-backbone with resolved next-hop IP: 169.254.0.1, Routes: [bgp (Network: 13.59.144.125/32, Next Hop: interface To-igw-02fd68f94367a67c7-backbone ip 169.254.0.1)])
        TRANSMITTED(To-igw-02fd68f94367a67c7-backbone)
      3. node: igw-02fd68f94367a67c7
        RECEIVED(backbone)
        TRANSFORMED(DEST_NAT dstIp: 13.59.144.125 -> 10.20.1.207)
        FORWARDED(Forwarded out interface: vpc-0574d08f8d05917e4 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.0.0/16, Next Hop: interface vpc-0574d08f8d05917e4 ip 169.254.0.1)])
        TRANSMITTED(vpc-0574d08f8d05917e4)
      4. node: vpc-0574d08f8d05917e4
        RECEIVED(igw-02fd68f94367a67c7)
        FORWARDED(Forwarded out interface: subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7 with resolved next-hop IP: 169.254.0.1, Routes: [static (Network: 10.20.1.0/24, Next Hop: interface subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7 ip 169.254.0.1)])
        TRANSMITTED(subnet-06a692ed4ef84368d-vrf-igw-02fd68f94367a67c7)
      5. node: subnet-06a692ed4ef84368d
        RECEIVED(vpc-0574d08f8d05917e4-vrf-igw-02fd68f94367a67c7)
        PERMITTED(acl-09c0bb4e71ae5f9e4_ingress (INGRESS_FILTER))
        FORWARDED(Forwarded out interface: to-instances, Routes: [connected (Network: 10.20.1.0/24, Next Hop: interface to-instances)])
        TRANSMITTED(to-instances)
      6. node: i-01602d9efaed4409a
        RECEIVED(eni-01997085076a9b98a)
        PERMITTED(~INGRESS_ACL~eni-01997085076a9b98a (INGRESS_FILTER))
        SETUP_SESSION(Originating VRF: default, Action: ForwardOutInterface(Next Hop: subnet-06a692ed4ef84368d, Next Hop Interface: to-instances, Outgoing Interface: eni-01997085076a9b98a), Match Criteria: [ipProtocol=TCP, srcIp=10.20.1.207, dstIp=8.8.8.8, srcPort=3306, dstPort=49152])
        ACCEPTED(eni-01997085076a9b98a)

      We thus see that our misconfigured public instance allows TCP traffic to port 3306 (MySQL).

      In this and earlier reachability queries, we are not specifying anything about the flow to Batfish. It automatically figures out that the flow from the Internet that can reach hosts["east2_public"] must have 13.59.144.125 as its destination address, which after NAT’ing becomes the private IP of the instance. Such exhaustive analysis over all possible header spaces is unique to Batfish, which makes it an ideal tool for comprehensive availability and security analysis.

      Batfish can also diagnose why certain traffic makes it past security groups and network ACLs. For example, we can run the testFilters question as below to reveal why the flow above made it past the security group on hosts["east2_public"].

      [15]:
      
      flow=ans.frame().iloc[0]['Flow']  # the rogue flow uncovered by Batfish above
      ans = bf.q.testFilters(nodes=hosts["east2_public"],
                            filters="~INGRESS_ACL~eni-01997085076a9b98a",
                            headers=HeaderConstraints(srcIps=flow.srcIp,
                                                      dstIps="10.20.1.207", # destination IP after the NAT at Step 3 above
                                                      srcPorts=flow.srcPort,
                                                      dstPorts=flow.dstPort,
                                                      ipProtocols=flow.ipProtocol)).answer()
      show(ans.frame())
      
      Node Filter_Name Flow Action Line_Content Trace
      0 i-01602d9efaed4409a ~INGRESS_ACL~eni-01997085076a9b98a Start Location: i-01602d9efaed4409a
      Src IP: 8.8.8.8
      Src Port: 49152
      Dst IP: 10.20.1.207
      Dst Port: 3306
      IP Protocol: TCP (SYN)
      PERMIT Security Group launch-wizard-1
      • Matched security group launch-wizard-1
        • Matched rule with description Connectivity test
          • Matched protocol TCP
          • Matched destination port 3306
          • Matched source address CIDR IP 0.0.0.0/0

      The “Trace” column shows that the flow was permitted because the security group “launch-wizard-1” has a matching rule called “Connectivity test.” (Perhaps someone added this rule to test connectivity but forgot to remove it.)

      Such introspection capability is indispensable for complex security groups and network ACLs. See this notebook for a more detailed illustration of these capabilities of Batfish.

      Summary

      Batfish allows you to analyze, debug, and secure your cloud and hybrid networks. It can shed light on different types of traffic paths between different types of endpoints (e.g., intra-region, cross-region, across hybrid links), and it can reveal the detailed availability and security posture of the network.

      Want to learn more? Come find us on Slack or GitHub

      Indices and tables