COMING from the business side, I have met and seen various vendors who promise heaven and earth to answer IT problems in your organization. There are the ‘yes’ guys who will always answer ‘yes’ when you ask if the solution can do this or that. There are the ‘deflectors’ who try to confuse or worse, mislead you when their solution cannot solve your IT issue. Then there are just the plain highfalutin ones who use terms such as AI or ML carelessly just to make a sale.

Now that I am on the side of the vendor, I have also met and seen fellow vendors- ambitious, innovative yet idealistic. For instance, there’s a vendor that sells the-only-anti-malware-that-you-will-need-for-your-organization. You don’t need perimeter security. Just install the solution to all your machines and you’re 100% protected from all attacks. Apparently, there are a lot of disclaimers and caveats in the Terms and Conditions, one is to assume that the attacks are known in their database and another is that the attacks should only be host-based.

I think as IT professionals, we have the responsibility to correct the ‘fake news’ in our own turf, similar to what scientists, doctors, lawyers, and other professions do to protect their respective reputations. As an IT security professional, I am both shocked and amazed at companies that claim that the entire VAPT can be automated and that their tool can do everything that a pen tester can do. I’ve seen a couple of different products on LinkedIn and some I’ve met and had a (heated) discussion.

I have listed 4 reasons why All-In-One/Automated Penetration Testing is a fallacy contrary to the claims of some companies that their solutions will replace actual pen testers.

By the way, one of the common misconceptions is that the Vulnerability Assessment (VA) activities and Penetration Testing (PT) activities are the same. They are not. To cut the story short, VA looks for existing vulnerabilities while PT exploits these vulnerabilities found. Some “self-proclaimed IT pundits” don’t even have a clear understanding of the definitions making the misinformation worse.

Anyway, so here are my reasons:

• Mens rea of the attacker
• • In the study of law, mens rea is defined as the intention or knowledge of wrongdoing that constitutes part of a crime. An attacker’s mens rea cannot be fully scoped by an automated tool. A tool can scope a certain known part of the assessment. But in the real world, exploits can be done by a gullible legitimate employee who accidentally clicks on a link that triggers the malware or a connivance/inside job to bypass stringent security measures. Scenarios mentioned can only be done by real people, not tools.
• An attacker’s out of the box perspective or the attack’s art (creativity)
• • The tool is limited by the signatures or known behaviors in its knowledge-based. Hackers/attackers are creative. For example, they will try to scan fast but not too fast so it can evade IDS tools. They will attempt to password guess but not reach the threshold and wait for a reset period before attempting to crack passwords again. The criminal mind is colorful and options are plentiful. Tools may have automating capabilities but limited to their applicability in actual testing.
• Timing and repetition attacks
• • There are attacks that require timing and repetition to actually exploit certain vulnerabilities. In a way, tools are a good complement for these attacks but it is the strategy of the attacker that dictates the success of the attack. For example, for applications that have so many pages of forms to fill before being allowed to submit, the tool alone cannot automate adding random data in all of these form fields. A human has to analyze and determine which parameters can the application accept and which can be used for automation.
• Logic attacks
• • Simply put, understanding logic, program flow, and its parameters are things that humans can handle easily compared to automated tools. Imagine if you are browsing an application and you encountered a transaction feature that requires you to input a 6-digit OTP from your registered phone through SMS. You know as a tester that you can automate a test that will input all possible combinations of 6-digits and use it to brute force the transaction. Tools, on the other hand, do not know that by default. Humans must still intervene. And the list goes on…

I think I am obliged to write this blog to emphasize that security testing involves both human testers and tools. They work hand in hand and the tools cannot work alone no matter how big the signature database is. The problem with these predatory solutions is that they promise too much, things that are too good to be true. Imagine if you use their tools and the tool didn’t find anything then you will feel secure. But a week later, you still get defaced through social engineering. So how would you respond?

Another very interesting and important advantage of using pen testers is the human tendency to exhaust all knowledge and techniques to find vulnerabilities. The hunger and desire of pen testers to find vulnerabilities is a big motivation to help the organizations find real security issues.