Format of vendor and supplemental data¶
Vendor formats¶
For vendors whose device configuration is not a single text file, additional preprocessing may be necessary before uploading data to Batfish.
AWS¶
Batfish understands AWS VPC configurations and analyzes it just like physical networks.
To use this functionality, place AWS configs in a top-level folder called aws_configs
.
The subfolders in this folder correspond to individual regions. If there is only one region,
then this level of hierarchy may be skipped.
The configuration files for a region should be the JSON output of the following API calls:
For EC2:
describe_addresses
,describe_availability_zones
,describe_customer_gateways
,describe_internet_gateways
,describe_network_acls
,describe_network_interfaces
,describe_instances
,describe_route_tables
,describe_security_groups
,describe_subnets
,describe_vpc_endpoints
,describe_vpcs
,describe_vpn_connections
,describe_vpn_gateways
For ES:
describe_elasticsearch_domains
For RDS:
describe_db_instances
This output can be collected using the AWS CLI or using the boto3 Python SDK.
An example script that packages AWS data into a Batfish snapshot is here.
An example snapshot, which includes both physical and AWS configs, is here. It is OK to have only AWS configs in a snapshot.
Cumulus Linux¶
Cumulus devices can be configured by editing individual files, such as /etc/network/interfaces
, /etc/cumulus/ports.conf
, and /etc/frr/frr.conf
or invoking the Network Command-Line Utility (NCLU)
Batfish supports processing of either NCLU configuration output, or the Cumulus configuration files themselves (concatenated into one file per device). We recommend using the configuration files, because Batfish can extract more data from them than from the NCLU output.
Note
If you are using the BGP Unnumbered feature on Cumulus devices, you will need to supply a Layer-1 topology file.
Cumulus configuration files (preferred)¶
Batfish processes the Cumulus configuration files concatenated into a single file per device. The format is as follows:
hostname (single line)
Contents of
/etc/network/interfaces
fileContents of
/etc/cumulus/ports.conf
fileContents of
/etc/frr/frr.conf
file
Batfish detects the boundary between each the concatenated files by looking for comments or declarations that typically occur at the start of each file. To ensure they are present for Batfish to find, we recommend adding them in case they are missing. Here’s an example bash snippet:
hostname=$(cat /etc/hostname)
(
# hostname
echo $hostname
# Signal start of /etc/network/interfaces
echo "# This file describes the network interfaces"
cat /etc/network/interfaces
# Signal start of /etc/cumulus/ports.conf
echo "# ports.conf --"
cat /etc/cumulus/ports.conf
# Signal start of /etc/frr/frr.conf
echo "frr version"
cat /etc/frr/frr.conf
) > $hostname.cfg
Cumulus: NCLU output (not preferred)¶
To retrieve the Cumulus configuration in the NCLU format, issue net show config commands
and save the output to a file.
F5 BIG-IP¶
F5 BIG-IP configuration spans multiple individual files, typically the following 3 configuration files:
Base
LTM
Routing [There are 2 formats for this routing configuration. Legacy
imish
and the newer structured format]
These files are typically in a folder named “config” as:
Base:
bigip_base.conf
LTM:
bigip.conf
Routing:
bigip_routing.conf
For Batfish to correctly process the F5 configuration, the 3 files need to be combined into a single file in a specific order.
As an example, let’s say you have these 3 files:
site1-f5-a-base.conf
site1-f5-a.conf
site1-f5-a-routing.conf
On any unix machine, simply run:
cat site1-f5-a-base.conf site1-f5-a.conf site1-f5-a-routing.conf > site1-f5-a-concat.cfg
Add site1-f5-a-concat.cfg
to the configs folder with the rest of the devices.
Note
The routing configuration MUST be the last thing copied into the file, otherwise Batfish will not be able to correctly parse the file.
Palo Alto Networks¶
Batfish supports Palo Alto Networks devices with or without Panorama.
From Panorama (preferred)¶
For devices managed through Panorama, with no configuration done directly on the device, you can pull configurations from Panorama without having to run any commands on the managed-devices themselves. If configuration is also done directly on the managed-device, you should follow the instructions for pulling configuration From individual devices
instead.
To pull these configurations through Panorama, use the pan-python
module as follows; note: replace <IP_ADDR>
and <LOGIN_USERNAME>
with the IP address and username for the Panorama device:
# Generate an API key and save in .panrc so subsequent commands don't require logging in again
# This only needs to be done once
panxapi.py -t panorama_tag -h <IP_ADDR> -l <LOGIN_USERNAME> -k >> .panrc
# Pull the XML Panorama config
panxapi.py -t panorama_tag -sxr > panorama_config.xml
# Convert the XML Panorama config into set-format
panconf.py --config pan-panorama_config.xml --set > panorama_config.set.txt
This will generate a single configuration (panorama_config.set.txt
) representing all firewalls managed by the specified Panorama device, and this config file is what Batfish needs to model the managed firewalls.
From individual devices¶
For each device, concatenate the following show commands into one file.
set cli config-output-format set
set cli pager off
show config pushed-shared-policy
show config pushed-shared-policy vsys <value> // run for each vsys
show config merged
The first two commands may not be available on all PAN-OS versions; just make sure that the output is NOT in XML format (first command) and that definitions are not truncated (second command).
Batfish data formats¶
Host JSON files¶
The host JSON files contain basic information about the hosts attached to the network, including their names, a pointer to their iptables configuration file, and their interfaces. An example file is:
{
"hostname" : "host1",
"iptablesFile" : "iptables/host1.iptables",
"hostInterfaces" : {
"eth0" : {
"name": "eth0",
"prefix" : "2.128.0.101/24"
}
}
}
iptables/host1.iptables
is the path relative to the snapshot where this host’s iptables configuration can be found. iptables configuration files should be in the format that is generated by iptables-save
.
Layer-1 topology file¶
Normally Batfish infers Layer-3 interface adjacencies based on IP address configuration on interfaces.
For instance, if there are two interfaces in the network with IP assignments 192.168.1.1/24
and 192.128.1.2/24
respectively,
Batfish will infer that these interfaces are adjacent.
However, you may override this behavior by supplying a Layer-1 topology file. In this case, Layer-3 adjacencies are computed by combining the supplied Layer-1 adjacencies with Layer-2 and Layer-3 configuration to get a more accurate model. This is especially useful if IP addresses are reused across the network on interfaces that are not actually adjacent in practice.
The expected Layer-1 topology file is a JSON file that has a list of edge records, where each edge record has node and interface names of the two ends.
See this file for an example.
Your file name should be layer1_topology.json
for it to be considered by Batfish.
ISP configuration¶
Batfish can model routers representing ISPs (and Internet) for a given network.
The modeling is based on a json configuration file (isp_config.json
),
which tells Batfish about the interfaces on border routers which peer with the ISPs.
An example file is:
{
"borderInterfaces": [
{
"borderInterface": {
"hostname": "as2border1",
"interface": "GigabitEthernet3/0"
}
}, {
"borderInterface": {
"hostname": "as2border2",
"interface": "GigabitEthernet3/0"
}
}
],
"filter": {
"onlyRemoteAsns": [],
"onlyRemoteIps": []
}
}
Here borderInterfaces
contains the list of interfaces on border routers which are meant to peer with the ISPs.
onlyRemoteAsns
(list of ASNs) and onlyRemoteIps
(list of IPs) provide a way to apply additional filter by restricting to ISPs having specific ASNs or IPs.
Warning
Batfish will not try to model any ISP routers in the absence of this configuration file.
An example network with ISP modeling configuration is here.