Vulnerability scanning and penetration testing are two common forms of cyber security assessment that are conducted to help organisations evaluate, measure and mitigate information security risk.
Penetration testing actively exploits vulnerabilities and provides extensive remediation guidance, while vulnerability scanning (or vulnerability assessment) checks for known vulnerabilities and generates a risk exposure report.
The differences between the two are often not well understood, and the terms are often used interchangeably. The reality, however, is that vulnerability scanning and penetration testing are distinct assessments in their own right, and it is important to understand the differences before choosing which option is best for your business.
To clear up common misconceptions, here are the main ways that vulnerability scanning and penetration testing differ.
1. Aim
By identifying security risks, vulnerability scanning and penetration testing assessments help organisations to understand the areas in which they are weak and prioritise improvements.
Vulnerability scanning is an automated form of cyber security assessment that uses off-the-shelf software tools to assess the security of devices connected to a network. It is conducted with the aim of discovering as many security vulnerabilities as possible – principally those already disclosed publicly – in a short period of time.
Penetration testing is a more in-depth type of assessment that is conducted by professional white hat hackers. Unlike vulnerability scanning, it is focused on identifying more complex vulnerabilities that scanning tools aren’t able to easily identify. Another key difference between the two is that penetration testing is also focused on exploiting vulnerabilities. This helps to establish the extent to which hackers with malicious intent could gain unauthorised access to critical data and assets.
Some pentests, which incorporate scenario-based assessments, are focused on testing organisations’ ability to detect and respond to specific adversarial techniques, tactics and procedures (TTPs). Scenario-based tests are useful for helping to validate the effectiveness of security teams and the technologies they use.
2. Scope
Due to its automated nature, vulnerability scanning is typically much wider in scope than penetration testing. All devices that connect to a business’ networks (both wired and wireless), and which therefore have an IP address, should be scanned. Desktops, laptops, printers, routers, switches, hubs, servers and firewalls are therefore all in scope. Vulnerability scanning can be conducted to assess a particular IP or range of IPs, from inside and outside the network.
Being a more in-depth method of security assessment, the scope of penetration testing is typically much narrower than vulnerability scanning and is determined by the area(s) in need of testing. There are various types of penetration testing, ranging from internal and external network testing to application testing and wireless testing. As penetration testing involves the exploitation of vulnerabilities, a clearly defined scope is needed to ensure that testing won’t impact business operations and fall foul of the law.
3. Approach
Being an automated method of security assessment, vulnerability scanning is conducted using the same repeatable processes – making it useful for ensuring the consistency of assessments and measuring the impact of remedial activities.
The human-driven approach of penetration testing means that there is greater emphasis on a tester to ‘thinking outside the box’ in order to more closely replicate the approach of a real-life adversary. For this reason, no two pentests are the same. The fact that pen testing is also geared towards exploiting vulnerabilities means that it is better at helping organisations to understand the actual rather than theoretical risk that vulnerabilities pose. Vulnerabilities detected by scanning tools can often be classified as high risk, even if they are not widely exploited by attackers in the wild.
It is worth noting that the approach of pen testing will also depend upon the level of information shared with a tester prior to an engagement taking place. Whitebox penetration testing is based upon an assumed compromise, with full network and system information shared with the tester in advance, and is often used to limit the duration – and cost – of an engagement. In blackbox penetration testing, a tester receives no prior information about the network or application to be tested. This approach better replicates the tactics of a real-life attacker, but engagements are usually more expensive. Many organisations elect a greybox penetration test, where only basic information is shared with the tester.
4. Types of vulnerabilities identified
Vulnerability scanning identifies devices that are running an outdated operating system and applications. It also detects device configuration problems such as open ports and use of default or weak passwords. These flaws are linked to common vulnerabilities and exposures (CVEs) and are listed on publicly accessible databases such as those from Mitre and NIST.
Like vulnerability scanning, penetration testing uncovers known CVEs but is more focussed on the identification of exposures that are beneath the surface and won’t be detected by scanning tools. This includes cross-site scripting (XXS) and code injection vulnerabilities, as well as authentication, encryption and deep misconfiguration issues.
Unlike vulnerability scanning, penetration testing is also better at providing insights into what a hacker could do upon breaching a network, such as escalating user privileges to access critical systems and exfiltrate data. Pen testing can also incorporate simulated social engineering attacks, widening the scope to include people, not just networks and applications.
5. Duration
The time it takes to conduct vulnerability scanning is dependent upon a variety of factors, such as the size of the network being tested and the time of day a scan is initiated. It usually takes just a few hours to scan a small to medium sized business but an enterprise with ten of thousands of ports could take up to a day.
To help streamline the vulnerability scanning process, many organisations are electing to adopt an adaptive approach to vulnerability scanning. Adaptative vulnerability scanning tools detect changes to a network, such as the connection of a laptop or mobile device, and scan the new device immediately, rather than waiting for the next scheduled scan.
The time it takes to conduct pen testing varies depending on its scope and the capability of the tester. As a rough guide, the average time it will take an experienced ethical hacker to test a SaaS application is two to three days. A comprehensive internal and external network penetration test could take a week or more, including time for a tester to write up a formal report.
6. Frequency of testing
Owing to both the automated and less invasive nature of assessments, vulnerability scanning is conducted more regularly than penetration testing. Many large organisations perform vulnerability scanning on a weekly basis but as minimum it should be completed monthly.
Penetration testing, on the other hand, is usually conducted by organisations once or twice a year, but more frequently when there are infrastructure changes and key events such as product launches.
Compliance is also a key factor influencing the frequency that organisations should conduct vulnerability scanning and pen testing. To achieve Cyber Essentials Plus certification, for example, organisations are required to undertake annual internal and external scans. The Payment Card Industry Data Security Standard (PCI DSS) requires organisations that process cardholder data to perform quarterly vulnerability scans and annual PCI DSS penetration testing.
Four situations when you should consider a pen test for your business
7. Tools used
Vulnerability scanning is performed using automated vulnerability assessment (VA) tools. These tools work by scanning networks for known CVEs and running automated scripts to test for weaknesses. Hundreds of vulnerability scanning tools are openly available, ranging from free open-source tools to fully-fledged commercial applications. Before choosing a VA solution, organisations should consider a range of factors, such as the type of infrastructure to be tested, ease of integration, customisation options, quality of integrated threat intelligence, deployment options, and level of customer support offered.
To conduct penetration testing, ethical hackers utilise a wide range of tools. These range from specialist pen testing platforms (such as Cobalt Strike, Metasploit Pro and Kali Linux), to networking tools (such as Wireshark), and custom-developed tools and exploits written using Python, Java and PowerShell. Some penetration testers have also started leveraging Breach and Attack Simulation (BAS) tools to simulate common attack methods. In the right hands, the tools used by pen testers are very powerful but can take a lot of time and practice to master.
8. Resource and skillset requirements
Within many large organisations, vulnerability scanning is performed by security operations centre (SOC) teams. With vulnerability scanning largely automated, however, assessments don’t need to be conducted by dedicated cybersecurity specialists. Indeed, in SMEs, they are regularly performed by network administrators or other IT personnel.
Due to the level of technical skill required to perform penetration testing legally, ethically and to a high standard, professional pen testers require extensive IT experience and recognised industry qualifications. To become a CREST Registered Penetration Tester,for example, students need to have two to three years of regular and frequent practical experience. More experienced penetration testers will also have qualifications in areas such as infrastructure, web application and wireless testing plus, possess knowledge of specialist systems used across industries such as finance services and manufacturing.
Given the specialist training individuals require to perform pen testing, plus the time needed to continually develop skills and keep up to date with the latest adversarial techniques, most organisations commission independent third-party providers to perform pen testing rather than employ dedicated in-house staff. Another reason that third party testers are often preferred, is down to the fact that they can draw upon a range of security expertise to help support vulnerability remediation – an area that organisations benefit greatly from outside assistance.
9. Outcomes delivered
Almost all tools used for vulnerability scanning offer built-in reporting functionality to help IT and security teams understand the risk posed by any vulnerabilities discovered. Many VA solutions assign a critical severity score to vulnerabilities based on the Common Vulnerability Scoring System (CVSS). However, by failing to take into account whether vulnerabilities are being actively exploited in the wild, the value of common scoring can be limited, especially in scenarios where scanning identifies hundreds of vulnerabilities.
By providing context around ease of exploitation, penetration testing offers greater insight into the true risks posed by vulnerabilities. Importantly, pen testing assessments conducted by third parties will also include a more detailed level of remediation guidance to help address any vulnerabilities identified – although the quality of remediation support varies between companies.
While it is security teams that often perform or commission vulnerability scanning and penetration testing, they rarely have responsibility for the devices or applications being tested. For this reason, it is prudent to check whether assessment tools and any third-party security providers enlisted will provide sufficient information and support after the assessment to help facilitate swift and efficient remediation. Sharing the direct results of an auto-generated vulnerability scanning report with a device owner will almost certainly confuse rather than help this process.
Compliance should also be a key factor when considering the assessment outcomes required. Vulnerability scanning tools now include built-in reporting templates for compliance with PCI DSS and other information security requirements. Penetration testing reports can also be tailored to meet compliance needs.
10. Preparation
Vulnerability scanning and penetration testing assessments vary in the preparation needed to perform them. If undertaking vulnerability scanning in-house, organisations need to consider which VA tool to use and, if conducting internal network scanning, any hardware requirements. Most commercial VA vendors now support virtual appliances, therefore avoiding the need to install software on a dedicated physical server.
Thought also needs to be given to the best time to perform scanning. Vulnerability scanning can cause bandwidth issues on some networks and result in user and device account lock outs, meaning that it can be best to schedule them out of regular business hours. However, this does mean that some employees that connect devices to the network may not have them connected when scanning takes place.
The key to preparing for penetration testing is to ensure that there is a clearly defined scope for the tester to follow. This will help to ensure that testing is focused and that there is no risk of the tester accessing assets that are off limits.
Like vulnerability scanning, it may be desirable to conduct penetration testing out of business hours. If there are significant concerns about whether a pen test could disrupt operations, then testing can be confined to staging rather than production environments – although there is a risk that the two may be configured differently.
When conducting vulnerability scanning and penetration testing it may be necessary to inform all relevant stakeholders. If red teaming or scenario-based pen testing is being performed, however, it is not beneficial to inform the security team being assessed.
If there are plans to pen test public cloud environments, then it is important to check whether there any set rules of engagement that need to be followed. Microsoft’s rules around Office 365, for example, set restrictions on the assets that can be tested and requires organisations to report critical system security issues within 24 hours of discovery.
11. Cost
If performed in house, the cost of the latest commercial vulnerability assessment tools will depend upon a number of factors, such as the number of IPs, remediation features, and support for public cloud environments. Using an MDR provider or MSSP that offers a manged vulnerability assessment service can often prove a more cost-effective option, as they will be able to offer discounted pricing on VA tools and provide a wrap-around service that reduces other overheads, such as deployment and day-to-day management costs. A third-party provider will also provide remediation support.
Penetration testing by a professional company is typically charged on the basis of a day rate. Testing can be up to £1500/day, although it is usually dependent upon the experience of the tester and whether testing is performed on site or remotely. Exercise caution if a pen testing quotation is much cheaper than the competition – it may suggest that the tester is reliant on automated tools rather than human skill.
So, which assessment should you choose?
Given the constantly evolving threat landscape and speed of digital transformation, the need for organisations to continually assess their cyber security and address weaknesses is more important than ever. Vulnerability testing and penetration testing are two key cyber security assessments, each with their own advantages, and it is important to be clear about the differences between the two.
This considered, perhaps the question to ask is not which form of testing to choose, but how they can be best utilised alongside each other. A robust testing strategy will incorporate both approaches, allowing common threats to be eliminated as quickly as possible, without allowing more advanced vulnerabilities to slip through the net.
To learn more about vulnerability scanning and penetration testing (VAPT) plus how they compare, contact Redscan’s offensive security team today.
