2204 views

Description

This lnowledge base article covers hoe layer 2 discovery information is processed.

What are layer 2 connections?

  • layer 2 connections are physical connections between network devices and between network devices and other components, such a servers.
  • The layer 2 algorithm uses several strategies, together with information from various tables to create Connects to:Connected by relationships in the CMDB.
  • These relationships can be created between the ports of the devices or between port of one device directly to another device (in case the target port was not found).
  • The results can be viewed in the dependency view map by applying the ‘Physical Network Connections’ dependency type on the relevant device.
  • When applying this dependency type, we always see a dependency between the devices and not between their ports (even if the ‘Connects to::Connected by’ relation in the CMDB is actually between the ports of the devices).


Required Plugins

  • Discovery plugin 


How It Works

  • Prior to New York, a script action called ‘Create Layer 2 connections’ listened to the ‘discovery.device.complete’ event and triggered the layer 2 connections algorithm for each such event.
  • Starting from New York, we created a new event for layer 2 connections creation and created a separate event queue for this event.
  • The name of the event queue is ‘physical_connections_creation’ and the new event is called ‘physical_connections.create’.​
  • A Script action named ‘Queue layer 2 connection calculations’ listens to the ‘discovery.device.complete’ event and creates a 'physical_connections.create’ event with the CI sys Id as a parameter in the ‘physical_connections_creation’ event queue.
  • A new job named ‘Layer 2 Connectivity Creation’ is responsible for the processing of the ‘physical_connections.create’ events.​
  • A Script action named ‘Create Layer 2 connections’ listens to these events and triggers the layer 2 connections algorithm for each such event.​
  • The purpose of this change is that the processing time of the layer 2 connections events will not influence the processing of other events and slow the event queue performance .​
  • This change was introduced via PRB1344266 (fixed in NY, MP6, LP10).​
  • In Orlando, we added an enhancement to reduce the amount of times the layer 2 connections algorithm is triggered.​
  • Instead of queuing the ‘physical_connections.create’ event for each ‘discovery.device.complete’ event, the ‘Queue layer 2 connection calculations’ script action finds the last time the layer 2 connections algorithm was triggered for the CI reported in the event. Only if the period of time passed since the last time the algorithm was triggered is more than one day (configurable by ‘sa.create_physical_connections.minutes_frequency’ property), a 'physical_connections.create’ event is created for this CI.​
  • This change was introduced via PRB1359923 (fixed in O, NP4, MP9).​
  • The ‘Create Layer 2 connections’ script action (that listens to the ‘physical_connections.create’ event) calls the java method NetworkPathManager.jsFunction_createPhysicalConnections(String deviceId).​
  • This method finds the layer 2 connections and creates the relations of type ‘Connects to::Connected by’ in the CMDB.​


Related Tables 

  • In order to create the layer 2 connections, we use the following tables:
    • discovery_switch_fwd_table
      • The forwarding table of the switches
    • discovery_device_neighbors
      • Used in order to find layer 2 connections between switches
      • Contains a field called neighbor_source which can be CDP or LLDP
    • discovery_switch_spanning_tree_table
      • Used in order to find layer 2 connections between switches.
    • discovery_switch_bridge_port_table
      • Used to map between port number in discovery_switch_fwd_table and interface index.
  • Port tables:
    • cmdb_ci_network_adapter
    • dscy_switchport
    • dscy_router_interface
  • All the tables are populated during the horizontal discovery of the device.


Layer 2 Connection Strategies  

  • PhysicalHostConnectionStrategy – Creates layer 2 connections between a discovered server, which is not a network device, and a network device.
  • VMLayer2ConnectionStrategy - Creates layer 2 connections between a VM and a network device.
  • NetworkDeviceLayer2ConnectionStrategy - Creates layer 2 connections between a network device and its neighbors.
  • SpanningTreeLayer2ConnectionStrategy - Creates a layer 2 connection between network device and the parent of the network device in the spanning tree.
  • JavaScriptLayer2ConnectionStrategy – Calls a java script function with empty implementation. We can add the function implementation on a customer instance in case we want to add another strategy. 


PhysicalHostConnectionStrategy 

  • Creates layer 2 connections between a discovered server, which is not a network device, and a network device.
  • Prerequisites:
    • Horizontal discovery of the server
    • Horizontal discovery of the network device.
  • Note: The discovery order matters. The switch needs to be discovered first in order the forwarding table, switch port table etc. will be populated. Then the server needs to be discovered. After the discovery of the server is complete, the layer 2 connections algorithm runs.
  • After the horizontal discovery was performed, we should validate the following:​
    • The network device has a forwarding table.​
    • The server has ports (network adapters or router interfaces).
  • Step 1: Find the ports (network adapters and router interfaces) of the server.​

  • Step 2: For each port, try to find if there is a layer 2 connection from the port:
    • Get the MAC address of the port.
    • Navigate to discovery_switch_fwd_table and search for records with this MAC address (and status field is not ‘self’).
    • For each such record, search if there are more records in the forwarding table with the same switch (configuration item) and port but with different MAC addresses.
    • Only in case there is only a single MAC on the port in the forwarding table of the switch, this switch is considered as a candidate (if we have more than one MAC it indicates that there is more than one source address so there is no layer 2 connection between the server and the switch).​
    •  In case the number of candidate switches does not exceed the value defined in sa.network.max_candidate_switches property (default is 2), create layer 2 connections to all the candidate switches.
  • Step 3: Try to find the port of the target switch and create the layer 2 connection
    • Navigate to discovery_switch_bridge_port_table and query this table by the switch (configuration item) and the port from the discovery_switch_fwd_table.
    • If a record found, get the interface index of the record.
    • Query dscy_switchport by the switch (configuration item) and interface index (interface number) in order to find the port.
    • If the port was not found, Query dscy_router_interface by the switch (configuration item) and interface index (number) in order to find the port.
    • If the port found in dscy_switchport table or dscy_router_interface table has install status absent, the layer 2 connection is created directly to the switch and not to the port.
    •  In case the port of the switch was found, create a relation of type ‘Connects to::Connected by’ between the port of the server to the port of the switch. Otherwise, create a relation of type ‘Connects to::Connected by’ between the port of the server to the switch itself.


VMLayer2ConnectionStrategy 

  • Creates layer 2 connections between a VM/ESX and a network device.​
  • Prerequisites:
    • Horizontal discovery of the network device.​
    • vCenter discovery (horizontal discovery of the ESX and all the VMs that are on this ESX) - should be done twice. ​
  • Note: The discovery order matters. The switch needs to be discovered first in order the forwarding table, switch port table etc. will be populated. Then the vCenter discovery needs to be performed. After the discovery of the VM or ESX is complete, the layer 2 connection algorithm runs.​
  • After the horizontal discovery was performed, we should validate the following:​
    • The network device has a forwarding table.​
    • The VM/ESX has ports (network adapters or router interfaces).​
    • Each VM has one of the following:​
      • 'virtualized by’ relation to the ESX
      • ‘instantiated by’ relation to VMWare Virtual Machine Instance that has ‘registered on’ relation to the ESX​
  • Step1:
    • Find the ESX of the VM and find all the VMs on this ESX. ​
    • Find the ports of the ESX and all the VMs (network adapters and router interfaces) and get the MAC addresses of those ports.​
    • Find all the VMware Virtual machine instances of the ESX and get the MAC addresses of their VMware network adapters (cmdb_ci_vmware_nic).​​
  • Step 2:
    • Find the ports (network adapters and router interfaces) of the VM.​
  • Step 3: For each port, try to find if there is a layer 2 connection from the port:
    • Get the MAC address of the port.​
    • Navigate to discovery_switch_fwd_table and search for records with this MAC address, status field is not ‘self’ and Absent=false. ​
    • For each such record, search if there are more records in the forwarding table with the same switch and port but with MAC addresses that don’t belong to the ESX or one of its VMs. Count the number of such exceptional MAC addresses.​
    • Only in case the number of exceptional MAC addresses is low (max 3 MAC addresses that are max 51% of all MAC addresses of the ESX and VMs), this switch is considered as candidate. The logic regarding the number of candidate switches allowed is the same as described in PhysicalHostConnectionStrategy.
  • Step 4: Try to find the port of the target switch and create the layer 2 connection
    • The same logic as in PhysicalHostConnectionStrategy.


NetworkDeviceLayer2ConnectionStrategy 

  • Creates layer 2 connections between a network device and its neighbors (other network devices).
  • Prerequisites:
    • Horizontal discovery of the network devices (switches or routers) – should be done twice.​
  • After the horizontal discovery was performed, we should validate the following:​
    • discovery_device_neighbors table was populated for at least one of the switches.​
  • Step 1: Find the ports of the switch.
  • Step 2: For each port, try to find if there is a layer 2 connection from the port:
    • Navigate to discovery_device_neighbors table and query the table by the switch (configuration item) and the port (origin interface).
    • Look at the neighbor address and neighbor interface of the record.
    • Ignore records that their neighbor interface is with install status = absent.
    • If we have exactly one neighbor with neighbor address or neighbor interface or both, we will create a layer 2 connection.
  • Step 3: create a layer 2 connection
    • In case the neighbor interface exists, create a relation of type ‘Connects to::Connected by’ between the origin interface to the neighbor interface. Otherwise, create a relation of type ‘Connects to::Connected by’ between the origin interface to the switch with the neighbor address.


JavaScriptLayer2ConnectionStrategy 

  • JavaScriptLayer2ConnectionStrategy is calling to createConnections (hostId) function in Layer2ConnectionStrategy script include.
  • The implementation of this function is empty and can be replaced by any other implementation on a customer instance in case we want to add another strategy or override an existing strategy.


System properties to enable/disable the feature

  • By default the layer 2 connections algorithm is active.
  • We can disable it by setting the ‘sa.create_physical_connections.active’ property to false. ​
  • On PRB1359923 (fixed in O, NP4, MP9), we added more system properties that enable executing the layer 2 algorithm only for certain types of devices:​
    • sa.create_physical_connections.vm.active​
    • sa.create_physical_connections.network_device.active​
    • sa.create_physical_connections.physical_host.active​

Known Problems 

  • Layer 2 connection is not presented in the dependency view (‘Connects to::Connected by’ relation exists in the cmdb, but does not appear in the dependency view when applying the ‘Physical Network Connections’ dependency type):
    • Run the following script in scripts-background:​
      • var npm = new SNC.NetworkPathManager();​
      • npm.getNetworkNeighbors(<host_sys_id>);​
    • If you get error which is similar to the following: ‚Syntax Error or Access Rule Violation detected by database (Unknown column 'cmdb$par10.a_ref_8' in 'where clause’)’, open a task to Persistence team (see similar cases/tasks: CSTASK008547, TASK475910).​


Running the layer 2 algorithm from Scripts-Background

  • Run the following script in scripts-background: ​
    • var npm = new SNC.NetworkPathManager();
    • npm.createPhysicalConnections(ciId);
  • In case of layer 2 connection between a server and a switch, the ciId should be the sys id of the server.​
  • In case of layer 2 connection between 2 switches, the ciId should be the sys id of one of the switches
  • To see the debug messages, set the ’sa.discolog.sys_log’ property to true before executing the algorithm. ​


Article Information

Last Updated:2020-11-02 08:00:39
Published:2020-11-02