TOP 9 rules of productive outcome of Bug Bounties and Penetration Tests
Conducting penetration tests on your own is extremely difficult, so most companies decide to outsource this activity. There are some secrets that can help you take the maximum out of the pentersters’ work and maximize the benefits in a way, you would have never expected. Find out what rules to follow to make the most out of security tests from the new tutorial by Dawid Bałut, TestArmy’s Cyber Security Director.
Pentester and Bug Hunter with extensive experience who joined the security world more than half a decade ago. Since then he has worked as a Security Architect for corporations from Silicon Valley. Every day, he builds security systems, trains employees and automates all security processes.
Here is a list of activities I recommend you to undertake during and after the penetration test/bug bounty cycle. This list is based on the most common gaps I’ve noticed in the companies I’ve worked with. By implementing theese steps, you’ll be able to increase the penetration test’s ROI. If you’ve already spent the money, let’s make sure you’ve done it well and squeezed the maximum out of the investment.
- Provide the pentesters with all the information beforehand.
You want to make sure that pentesters don’t waste too much of their time on reconnaissance. Provide them with documentation on the product, videos presenting how your product works and a list of APIs and endpoints you want them to test. In general, penetration testers will know what to go after, however if you have hidden APIs, or want to speed up the process, it’s better if you provide them with sufficient information in advance.
- Don’t play games and don’t block pentesters as they do their job. You’re both in the same team and share the goal of making your organization safer.
The mindset shift is a huge thing. I’ve met a number of security teams that were literally fighting with external pentesters, because they were afraid of their position and ego. You hire pentesters to do the work for you and to provide maximum value to your business, it’s not exactly smart to waste your money on arguments with them and to block their testing environments. Depending on corporate culture, it may be a good idea to have a separate (such as for compliance/audit) team to coordinate external tests, in order to avoid a conflict of interest.
- Be humble and thoroughly follow pentesters recommendations in terms of issues remediation
Don’t fight it, when pentesters provide you information about the issue’s severity and its solving guidance. It is possible that with internal business knowledge you can perform better risk analysis and assess that the issue has a different level of severity for you. However, keep in mind that most often pentester know what they’re doing and it’s not their first assignment, so when they provide you with information about the issue, it’s likely to be legitimate. It’s generally a good idea to follow their remediation guidance, to make sure you’ve addressed the issue in-depth and then request re-tests.
- Find the cause of every single issue and learn what you can do better next time
Stay focused on the big picture and think about different places where the same issue might have happened, but wasn’t found by the pentesters. As opposed to simply fixing individual bugs, dig deeper into your codebase to find out if the same issue doesn’t happen in other applications. Then, review your software engineering practices to see what could be done better in order to stop these bugs from appearing in the first place.
- Have your engineering teams learn from the pentesters and study the pentest report
Don’t keep it all for yourself and have everyone learn from the report. Use it as an interesting exercise and learning opportunity. Pentests in most companies happen once or twice a year, so why not maximize the outcome? Thanks to this approach, software and QA engineers can learn how to apply new knowledge in their daily work, for instance how to add it to existing test cases.
- Make sure that the developers have complete understanding of all issues
You need to take over ownership of the reports and don’t just throw it at your employees. You want to make sure everyone has good grasp of security awareness, so they can address them well as well as use that knowledge in the future to build more robust apps. If something is complicated it won’t get easily remembered and put into action.
- Cover identified bugs with solid regression test cases
You don’t want the pentesters to keep finding identical bugs over and over again. Not only it’s a waste of money, but that’s an additional exposure, because if a regression pops up, attackers may make use of it before your next pentests discover it.
We perform penetration tests to improve the products and companies, so if you’re not going that extra mile, you’re leaving a lot on the table.
- Once in a while check the changes made by pentesters and visit their profiles used for testing to catch unexpected regressions
It makes a lot of sense to clone the activities that were performed by pentesters, and build standalone testing scenarios out of them. When you login to the pentesters’ account in the testing environment, you may notice much more bugs than were actually reported. Security testers focus on reporting security issues and most often don’t want to waste their time on reporting UI/UX issues, that may have been identified during security tests. So just checking their environment to see if e.g. unexpected encoding didn’t break the UI will be beneficial.
- Review logs to catch unidentified bugs
Some things aren’t exposed to the user, which means that security testers may touch some fragile systems and maybe had even break them, but not be aware of that. If you review the logs, you may find some unhandled exceptions, Internal Server Errors, and other indicators that allow you to improve the robustness of your code and systems.
When you make yourself comfortable with the logs generated by penetration testers, you’ll be able to create some security monitoring around your access/application logs and detect potentially malicious hacking attempts. If some keywords happen to appear in logs only during a pentesting engagement and never before, it means that potential attackers may perform tests in the same fashion and find them, too. If you create and alert system around that, you may be able to spot the attackers and respond to the threat swiftly.
In my career I’ve met only a handful of companies that take up such a wide range of activities during penetration tests. If you follow this guidance, you’ll optimize your spendings and will get more satisfaction from the general return of investment in security tests.