Can’t I Just Do Attack Surface Discovery With Shodan?

If you want to understand how the internet works, read the requests for comments (RFC)s. If you want to know how the internet works, you have to start writing code.

Academically, there should be no difference between what you can learn about an organization using commercial software versus freemium and open-source tools because the protocols you use for discovery such as DNS, IP, HTTP, etc, are the foundation of the internet. Unfortunately, there is a difference, because RFC compliance isn’t a reality (anywhere I can tell), and people are constantly breaking things for reasons one has to assume range from:

• They have the dominant browser and don’t care if they break things for other people.
• The RFC didn’t factor in some edge case, so a network infrastructure provider deploys a workaround and doesn’t tell anyone.
• Or maybe someone simply configured their DNS server to respond affirmatively to any request with an IP address.

Adding to this complexity are security technologies that may start throttling requests to prevent you from enumerating their environment or just outright blocking your IP address. These complexities directly impact your ability to quickly and repeatedly collect accurate information about your environment. This is a problem because…

Time is of the essence when defending ephemeral environments

Asset management is a critical capability for any organization. Quite simply: You can’t protect what you don’t know about. In the battle against automated scanners that can probe the entire IPv4 address space within 45 minutes, any organization that leverages a continuous integration/continuous deployment (CI/CD) development model needs continuous monitoring to support that environment.

Gone are the days when any CISO should be allowed to turn a blind eye to what they don’t know about without being excoriated by their board. In fact, the question of what is being done from an attack surface management perspective should be one of the first questions board members are asking of their security leaders, because it’s the only way to ensure due diligence is being performed against becoming a target of opportunity (read ransomware).

ASM products provide frequent, accurate, and complete views of your environment

Performing effective reconnaissance of an organization as an offensive security researcher using an open toolkit is a multi-day process between research into the extent of an organization and every single one of its business units, offices, and IT deployed around the world and in the cloud, scanning, and then validating the data you get back. The outcome of this process is one you hope is a complete view of the target organization, but it doesn’t need to be because you’re just trying to find gaps. This is the mindset that open source reconnaissance tools are built with, which is quite a bit different than the needs of an ever-changing enterprise organization trying to maintain its security posture.

Commercial ASM products have to solve a different problem: They need to fulfill a continuous monitoring requirement, providing frequent snapshots of your environment that are both accurate and complete, so you can detect and respond to drift in your security posture. Here are a few advantages you should expect when investing in commercial ASM products:

• They have the necessary infrastructure to collect data at scale
  It’s incredible how many network security mechanisms assume an adversary will be performing their work from a single IP and over-respond to traffic that at worst may be considered impolite. Meanwhile, very little can be done to combat distributed botting other than finding patterns in the traffic itself or fingerprinting the botnet over time to filter out the traffic sources.By maintaining appropriate scanning infrastructure, an ASM provider can bypass these little annoyances without needing to throttle themselves or perform other evasive techniques. This ensures they are able to perform faster data collection with higher fidelity response. This is critical, because again, your infrastructure is in a constant state of change.

• Sophisticated AI and ML models for accurate attribution of assets:
  There is no way to get pristine data when querying third-party assets over the internet. The only way to ensure a complete view of a target environment is by over-collecting and filtering out false positives. This process can create substantial downstream manual work which directly impacts the timeliness and accuracy of your results.ASM providers solve this by validating findings across multiple data sets and utilizing AI and ML models to validate attribution to your organization. This type of cross-validation isn’t offered by open-source tools due to the specialized nature of these tools, even when they offer redundancy by validating against multiple sources using the same protocol.

• A professional user experience for managing findings:
  Simply knowing about an exposure is rarely enough to get it remediated in enterprise environments. Without the context of the severity and recommendations for how to remediate the issue, it can be difficult to know where to begin. Sure, there are open-source reconnaissance scripts that tie together different open-source tools and drop the output into a weak web user interface (UI), but this doesn’t fly with commercial products competing for your budget. With commercial products, you are also buying support and accountability, which alone can be worth the price tag.

The primary failing of an open-source tooling approach is one of design. Offensive scripts are not designed to provide an exhaustive inventory of an organization, but rather provide a path for identifying potential weak points that can be exploited.

This fundamental incompleteness means that a defensive security strategy relying on these tools inherits problems that may only be overcome with a heavy investment of time and resources. In the end, open source tools require you to trade speed for accuracy and data completeness, leaving you unable to respond to precisely the exposures that justified investing in an ASM program to begin with.