The confusion is understandable because vulnerability scanning is often one of the techniques used during a penetration test. However, the two services have different objectives and provide different levels of assurance.
Personally, I have been on engagements where a company runs vulnerability scans but is surprised when additional security issues are raised or fails to understand the true risk of individual issues. Manual testing, including chaining issues together and exploiting them, helps showcase the true impact and risk associated with individual issues.
What is automated vulnerability scanning?
Automated vulnerability scanning uses specialised tools to identify known vulnerabilities, misconfigurations and missing security updates across systems and applications.
These tools commonly compare discovered assets against a database of known issues.
The benefits of vulnerability scanning include:
- Fast identification of known vulnerabilities.
- Lower cost compared to a full penetration test.
- Ability to perform assessments regularly.
- Broad coverage across large environments.
However, automated scanners have limitations. They typically identify vulnerabilities but do not fully understand the business context, security design or intended functionality of an application or system.
What is penetration testing?
Penetration testing is a manual security assessment performed by a security professional to identify, validate and safely demonstrate security weaknesses.
A penetration test often includes vulnerability scanning, but it goes significantly further by analysing how vulnerabilities can be exploited and what impact they could have on the organisation.
Key differences include:
- Findings are manually validated to reduce false positives.
- Security weaknesses are assessed within the context of the target environment.
- Business logic and authorisation flaws that automated tools struggle to detect can be identified.
- Multiple weaknesses can be chained together to demonstrate realistic attack paths.
- Exploitation may be performed where safe, authorised and within the agreed scope.
Illustrative example: web applications
A good example of the additional value provided by penetration testing is identifying access control vulnerabilities such as IDOR (insecure direct object reference).
These issues occur when an application fails to properly verify whether a user should be allowed to access a particular resource.
While scanners can identify many common vulnerabilities, access control flaws often require a tester to understand:
- The application’s functionality.
- Different user role permissions.
- Expected access controls.
- How data should be protected between users.
Scanners fail to understand the context of applications and what access controls are expected, and therefore fail to flag these issues.
Below is an example screenshot of an IDOR vulnerability from a PortSwigger training lab environment where a userid value can be identified by visiting a blog on the web application. This value can then be supplied to the id parameter at the /my-account endpoint, resulting in unauthorised access to another user’s account. In the real world, this could lead to an attacker gaining unauthorised access to another user’s account, revealing sensitive information or escalating permissions within the application.
Illustrative example: internal infrastructure
Infrastructure testing often demonstrates how seemingly moderate vulnerabilities can become high-risk when combined.
For example:
- Systems may send LLMNR (Link-Local Multicast Name Resolution) requests when DNS resolution fails.
- Attackers may be able to capture Net-NTLM (NT LAN Manager) authentication attempts through name-resolution poisoning.
- Other systems may permit NTLM relay attacks due to insecure SMB (Server Message Block) configurations (SMB signing not being enforced).
Individually, these findings may appear relatively low or medium risk. However, a penetration tester may demonstrate that combining multiple weaknesses allows an attacker to authenticate to other systems, access sensitive information or potentially gain elevated privileges.
The ability to validate and chain vulnerabilities together is one of the key differences between vulnerability scanning and penetration testing.
A scanner may identify the individual weaknesses, but it typically does not demonstrate how they could be combined during a real attack.
Below are some screenshots showing how LLMNR poisoning can be chained with an SMB relay attack in a training lab environment to demonstrate the potential impact.
In this instance, the connection was initiated by a domain administrator, poisoned and relayed to the domain controller, which resulted in NTLM hashes being dumped.
The administrator hash was then used to gain remote access to the system.
From my experience, chaining vulnerabilities reflects attack paths an attacker may exploit. Attackers do not look at individual issues and stop; they abuse anything that is vulnerable and will combine issues to get to their end goal.
The above is a great example of this: if the domain administrator password was strong, poisoning LLMNR to capture the authentication would only provide a hurdle as the hash could not be cracked easily. However, if the attacker identified that the domain controller had SMB signing disabled, this would show how the poisoning could be chained with relaying the authentication to the system, leading to a complete compromise of the domain environment.
In the real world, this would help demonstrate the true impact of individual issues to an organisation. For example, an attacker could use this level of access to deploy a ransomware attack against all domain-joined systems, resulting in financial loss and reputational damage.
When should you use each approach?
Both vulnerability scanning and penetration testing play important roles within security testing:
Vulnerability scanning
- Continuous security monitoring.
- Identifying missing patches.
- Regular compliance checks.
- Large-scale environments requiring frequent assessment.
Penetration testing
- Validating the effectiveness of security controls.
- Assessing real-world attack paths.
- Testing new applications and infrastructure.
- Understanding the true business impact of security weaknesses.
- Identifying additional vulnerabilities.
Vulnerability scanning is typically conducted for compliance purposes and to gain a broad overview of security issues, whereas penetration testing is vital, especially after significant changes to an environment (such as a new web application functionality or a change in network design), to gain a clear picture of what a real attacker could do and how they could impact your organisation.
A vulnerability scanner may flag a lot of vulnerabilities, which is great, but a penetration test will assess their impact and risk, helping organisations prioritise remediation of what matters most.
Conclusion
Automated vulnerability scanning and penetration testing should not be viewed as competing services.
Vulnerability scanning provides a cost-effective method of identifying known security issues on a regular basis. Penetration testing builds upon this by validating findings, uncovering weaknesses that automation may miss and demonstrating the real-world impact of security vulnerabilities.
The strongest security programmes typically use both approaches: regular vulnerability scanning to identify known issues quickly and periodic penetration testing to gain a deeper understanding of the organisation’s security posture.