In this blog, we will talk about one very common challenge that many telecom operators and enterprises face: the automation of documenting their infrastructure using network discovery tools. We will talk about what network discovery is, why it's an indispensable part of modern IT and telecom infrastructure, and how it assists the network administrator in maintaining a comprehensive and accurate view of the network. This guide will take you from basic understanding to the introduction of different protocols and tools used within network discoveries. Whether you are an experienced network engineer or just a beginner, this blog will equip you with the knowledge you need to fine-tune both performance and security in your network.
What is Network Discovery?
Let’s start with some basic definitions to explain what network discovery and the reconciliation process are.
Network Discovery is the process of automatically identifying and mapping all devices and their modules (such as cards, interfaces, etc.), as well as the connections between devices. This process can also include discovering logical resources like IP addresses, routing tables, MAC addresses, VLANs, logical sub-interfaces, and sometimes even provisioned services.
For network devices to be discoverable, they need to be accessible during the discovery process and configured in a way that allows a discovery engine to read the required information.
Reconciliation is the process of comparing the discovered network elements with the known or expected network documentation, usually documented in a network inventory system. If there are discrepancies (such as a missing device or an incorrect IP address), the reconciliation process can initiate certain actions, like notifying a user about the discrepancies or even updating the documentation.
In tandem, network discovery and reconciliation processes form the foundation for modern network management processes, as well as an invaluable resource for network planning engineers.
Purpose & Importance of Network Discovery
Network discovery is the foundation of modern service fulfillment and assurance processes, the two major verticals of OSS. The key goal of network discovery is the automated identification and mapping of all devices and connections within a network. The results of network discovery are used in the following areas:
- Network Inventory: to keep an accurate and updated inventory of all network devices with their physical and logical attributes.
- Automatic Troubleshooting: Network Topology and dependency knowledge facilitate quick root cause identification of network problems.
- Service Impact Analysis: Knowledge of the dependencies between devices and services allows gaining a better insight into the impact that might be caused by failures in the network, or even maintenance that is scheduled.
- Security: Provide identification of unauthorized or undocumented devices in the network that may pose risks to the security of the network (e.g. identification of wrong firmware version might impose a huge security back-hole into the network)
- Capacity Planning: Knowledge of the current and future capacity requirements in the network for peak performance, such as the number of free ports on an access device.
- Automated Remediation: With correct network information, automated tools can be programmed to identify conditions and invoke appropriate remedial action.
- Configuration Management: Identification of problematic configurations on network devices.
- Reporting: asset management / inventory reporting for finance
In this respect, by providing a precise image of the network infrastructure, network discovery represents the basis for automated fault resolution and network reconfiguration in response to any network issues that could be detected.
What are Network Discovery Protocols?
Network discovery protocols are the ways of communicating and fetching inventory data specialized in providing information about network infrastructure and essential in identifying devices on a network. In order to be efficient, a network discovery platform should be flexible enough in using various discovery protocols and should have intelligent algorithms that can stitch together the information it collects from them. Combining the data from multiple standard communication protocols like SNMP, ICMP, LLDP, CDP and others, may provide a more complete and correct view of the underlying network infrastructure.
Advanced algorithms analyze the data that is collected, determining relationships and dependencies. They also flag any unusual items to provide insights for network engineers to help manage the network and fix problems.
They contribute to a holistic approach, ensuring the platform is in position to adapt to a wide range of network environments and also deliver information in a reliable and actionable way.
Some of the major protocols that a modern network discovery platform use are:
- SNMP - Simple Network Management Protocol: This is a still widely used protocol used to gather and structure information from the managed devices on IP networks
- LLDP - Link Layer Discovery Protocol: This is a vendor-neutral protocol that is utilized by network devices for advertising their identity, capabilities, and other neighbors on the local network.
- CDP - Cisco Discovery Protocol: This is a proprietary protocol developed by Cisco; this is similar to LLDP but confined to Cisco devices. It enables Cisco devices to share information about themselves and find other Cisco devices that reside on the same network.
- MNDP - Mikrotik Neighbor Discovery Protocol: This is the proprietary protocol that is used by the MikroTik routers/dishes for the discovery of other MikroTik devices on the same network.
- ICMP - Internet Control Message Protocol: Although designed primarily for diagnostic and error-reporting functions, it is used in network discovery to obtain the list of devices currently active on a network with the ping utility.
- FDP - Foundry Discovery Protocol: This is another proprietary protocol by Foundry Networks, a subsidiary of Brocade Communications Systems, for the discovery and providing information of other devices inside the network.
- NDP - Neighbor Discovery Protocol: This protocol is employed with IPv6 for the purpose of neighbor device discovery, IP-to-MAC resolution, and configuration of network settings. It replaces the functionality of ARP used in IPv4.
- DHCP - Dynamic Host Configuration Protocol: An example of a network management protocol employed in automating the assigning of IP addresses and other network configuration parameters to devices on a network. DHCP helps connect devices to the network through automation by getting rid of the need for manual addressing of IP addresses.
Especially interesting for real-time updates is a telemetry approach which automatically collects and transmits data from remote sources providing a near real-time insight into the performance and behavior of systems. Telemetry provides immediate data on network topology, device status, and performance metrics. This makes the processes of network discovery much more accurate, hence giving a clear view into an updated network inventory almost immediately when a change happens is necessary for the best network performance. All of these protocols, along with telemetry, work together to provide a comprehensive view of the network, enabling effective management and troubleshooting.
How does Network Discovery Work?
Basically, network discovery works by systematically scanning the whole network to find all connected devices and their interrelationships. Generally speaking, this will involve the use of multiple protocols such as SNMP, LLDP, CDP, and others to gather information about each device's identity, capabilities, and operational status. A discovery engine sends queries or broadcasts to devices on the network that respond with details including IP addresses, MAC addresses, device types, and configuration settings. This information is then pieced together into one giant network map that gives a complete view of the topology of the network and the interrelations between devices.
Telemetry adds an additional level of precision by continuously monitoring the activity on the network and updating information in real time to make sure the network information is correct and accurate.
In some cases, discovery platform cannot communicate directly with the Network Devices themselves, and instead has to interact with a vendor-specific Element Management System (EMS). These EMS systems manage different segments of the network and provide an API interface to external systems for the purpose of accessing the inventory and status of the managed elements. In order to retrieve information about network devices behind EMS system, a discovery platform must be integrated with these EMS systems using variety of methods, such as REST, SOAP, CORBA, file parsing, DB access, etc.
This is an important integration for the reason that it
allows the discovery platform to create a holistic view of the network which
includes all networks segments regardless of protocols and vendors. The
described holistic approach enables network administrators to effectively
manage, troubleshoot, and optimize their network infrastructure.
What are the Benefits and Challenges of Network Discovery?
Network discovery forms the backbone of effective telecom operations. It builds up an accurate and complete picture of network infrastructure, including devices and interconnections between them and the device configurations. Without this basic knowledge, it becomes virtually impossible to manage and maintain the network effectively.
Key benefits of network discovery can be summarized in the following bullet-points:
- Efficient trouble-shooting: In case something goes wrong in the network, this could be spotted fast using a clearly outlined network map. These decreases service downtime, and customers are satisfied.
- Capacity Planning: Accurate network discovery data lets business owners plan for future growth or any expansion that may be required. This prevents any possible bottlenecks, and full performance is ensured.
- Security Management: Discovering firmware and software versions on network devices as well as topology of the network at different layers, significantly improves the capabilities of spotting the possible vulnerabilities in the network. Indirectly, it protects sensitive data from being accessed by unauthorized personnel.
- Cost Optimization: Network discovery can highlight underutilized or redundant resources that could be optimized to save money. This helps optimize resource utilization and aids in reducing operational expenses (e.g. identification of unused SFP modules in network devices).
- Compliance Adherence: Some industries have network compliance requirements. Network discovery facilitates a business's capability to demonstrate its adherence to such regulations.
- Support to financial asset management: helps provide updated inventory data to improve inventory management in ERP systems.
Why an OSS System Cannot Operate Without Discovery
In a real telco or enterprise environments, OSS systems carry out many different functions, like service activations, service monitoring, service reconfigurations, deactivations and many others, which relay on valid and precise up-to-date network information. With no discovery process, the OSS will operate based on incomplete or outdated information.
Some of the reasons why OSS systems cannot operate without network discovery can be summarized as follows:
- With distorted Inventory the OSS will have an inaccurate perception about the assets composing a network, thereby preventing accurate network management and maintenance.
- Service assurance processes cannot function normally without complete and accurate information about underlaying resources and services being mapped to those resources
- Incomplete or outdated information on network topology could lead to incorrect troubleshooting effort, time, and resources.
- Inaccurate inventory data would form the basis of decisions on capacity planning and resource allocation decisions, thus probably affecting network performance.
- Not having visibility into the network can leave vulnerabilities exposed to increase the risks of security breaches.
In other words, network discovery forms the foundation upon which a robust and efficient OSS can be built. With basic data that it provides to OSS, this then allows the system to manage the network at all levels of effectiveness and optimization, hence translation to better efficiency in business operations and customer satisfaction.
Challenges of Network Discovery
Network discovery faces a number of challenges due to the scale and complexity of the modern telecom and enterprise networks. Usually there are multiple vendors and protocols, making complete visibility a very difficult task to achieve. Sometimes the vendors do not follow the standards set on the discovery protocols and hence are not compatible with any other vendors.
Another big challenge in network discovery arises when parts of the network are managed by vendor-specific Element Management Systems, or EMS. This is very often a case with mobile networks (2G, 3G, 4G, 5G), SDN Networks, WiFi Networks, IoT networks, transport networks like DWDM and legacy non-IP based networks (e.g. SDH/SONET). In that case, the discovery platform does not have direct access to the managed elements. Instead, it has to integrate with the EMS through its API interface to learn about the inventory behind the EMS. But the complication arises because all these are proprietary EMS systems, having different APIs and data models. Seamless communication and correct fetching of data require deep understanding and adaptation to a specific API of EMS, which is naturally very time-consuming and resource-intensive. Any changes in updates to the API of the EMS disrupts the process of discovery, hence demanding continuous monitoring and maintenance of the integrations.
Once identified, these discovered elements should be mapped to their corresponding physical locations, which can include street name and number, specific rooms, buildings, or geographic coordinates. This mapping is challenging and error-prone task, but it is essential for effective network management, as it allows administrators to visualize the network’s layout on geographical maps and precisely dispatch field technicians when required.
Scalability is yet another challenge in that large networks grow and expand, which the discovery tool has to put up with in handling numerous volumes of data without any breakdowns. This is especially the case in dynamic environments like public clouds (Amazon AWS, Microsoft Azure, etc.) where device configurations or network topologies change regularly add to the challenge of keeping network maps accurate and current.
These can be overcome with several organizational strategies: device protocols and configurations can be standardized to avoid too much heterogeneity; enhanced security via encryption at various levels of communication, coupled with strong authentication methods, can prevent such possible vulnerabilities; scalability issues can be handled using cloud-based solutions supported with automated horizontal and vertical scaling on demand, so network discovery processes can continue unhindered with network growth. Besides, the employment of real-time monitoring and adaptive algorithms helps refresh the right network map in the dynamic environment. These approaches combined provide for effective and safe network discovery to improve network management and operational efficiency.
Future Trends in Network Discovery
Several transformative trends, driven by technological advancements and ever more complex network environments, will characterize the future of network discovery tools. The first among these is the integration of artificial intelligence and machine learning algorithms into network discovery tools. AI and ML can identify patterns and anomalies that might otherwise be missed by conventional methods, leading to quite a significant increase in the accuracy and efficiency of the process of network discovery.
Another trend to keep an eye on is the movement toward cloud-based network discovery solutions. Considering the fact that organizations have a tendency of migrating most of their operations to the cloud, this creates an urgent need for tools that can manage hybrid and multi-cloud network resources and perform network resource discoveries across all these environments simultaneously. This is where cloud-based solutions are offered with scalability and flexibility that make the dynamic nature of modern networks easier to handle. They also allow for better integration with other cloud services, applying a more integrated and holistic approach to network management.
Security remains paramount in network discovery, and future trends will embed a better orientation toward integrating robust security into the conduction of network discovery. It would involve the inclusion of encrypted communications, multi-factor authentication, and zero-trust architectures to keep the network discovery mechanism safe from every unwanted intervention. Given the increasing sophistication in cyber threats, in times to come, the security of the tools and processes used in network discovery will remain vital to letting the integrity and reliability of network operations remain intact.
Finally, the adoption of 5G and edge computing is expected to hit network discovery quite hard. The new layers of complexity brought by both technologies will raise the need for even more sophisticated discovery tools to manage the increased number of connected devices and the volume of data produced at the network edge.
By staying ahead of these trends, software vendors behind network discovery platforms, can ensure their customers that their network discovery processes remain effective and secure in an increasingly connected world.
UMBOSS & Network Discovery
The Automatic Discovery and Reconciliation module of UMBOSS, named ADM, literally takes network discovery for telecom providers and enterprise IT organizations to a whole new dimension. It automates the process of discovering a physical, logical, and virtual resources, thereby assuring a comprehensive view on network infrastructure and guaranteeing the precision of cataloging all active elements, topologies, and devices.
Key Benefits of UMBOSS ADM:
- Full Network Visibility: ADM discovers resources both in hybrid data centers and network environments.
- Continuous Monitoring: Automated schedule and event-driven discovery jobs keep network data up-to-date.
- Precise Inventory Management: Reconciles discovered data with inventory systems through flexible API integration.
- Integration: UMBOSS ADM can be easily integrated with any third party EMS/NMS system as well with any third party inventory systems making it perfect choice for complex multi-vendor environments, where inventory consolidation is an ongoing challenge.
While it may not need to be explicitly stated, it is inherently understood that UMBOSS ADM is natively integrated with other UMBOSS modules, such as the Event and Fault Management module, Performance Management module, Service Inventory, Address Book, and others.
How It Works?
ADM scans the networks using multiple protocols like SNMP, ICMP, CDP, LLDP, CLI, HTTP/S, and REST API. It deploys heuristic algorithms for multiple layers of topology verification, such as L2, L3, BGP, and OSPF. A scheduler automates the discovery jobs to keep continuous network monitoring. It performs data reconciliation with UMBOSS' internal inventory system (UMBOSS NDM) or any third-party inventory systems for resource data accuracy.
Business Outcomes:
In one such example, a major telecom operator used ADM for smoother network management, which resulted in operational costs reduced by 30% and also achieved an improvement in network reliability.
Have any questions? Want to learn more? Get in touch and let us know how we can help. Send us a message or book a demo today.
