Here is share some humble application security advice from an old school practitioner. This advice is for practitioners and cybersecurity leaders (CISO, CSO, etc) alike. I have been a player in the application security (appsec) space for many years and I see the appsec space through a fairly wide lens of both offensive and defensive arenas. My lens factors in secure coding, layer 7 protective mechanisms, processes and things like pen testing. My background in these areas spans back to before the days of automated tools in pen testing, we did things manually and actually had to deeply understand stuff under the hood. Back then it was both an art and a science, I am not so sure these days. By 2005 I had professionally performed enough pen tests that I confidently wrote a book on the subject.
I can’t think of any modern day organization that does not have a business-centric Internet presence. This of course implies some web app, or web site. And these of course need protection. This protective journey starts way before the first request is responded to via some port open to the Internet (whether directly or via proxy). From my perspective, there are a few key areas relevant to securing software that are critical.
Before this discussion goes any further let me clearly state something. Appsec is a journey. One that you either wholeheartedly embrace or don’t bother at all. Too many appsec initiatives are driven by some externally mandated compliance, or it’s the scenario where some software engineering team is forced to do this, and frankly just doesn’t want to. Straight talk – out of all the software engineers you have met how many actually give a crap about security? 28 years in for me and that number is minuscule.
This means it is on us to be advisors and aim to positively influence. It’s on us to weave this into the day to day reality of other teams, but we have to be all in. You have to be deep about appsec and be committed. Otherwise feel free to just stop reading here.
Another point I will make here may upset some and that is ok. More straight talk for my security peers ….. you do no one any justice when you, or any “security” expert, comes to the table with software engineers to discuss their “insecure” coding, yet it is obvious that you have never actually coded anything yourself. A software engineer will see right through you and will silently (hopefully at least) be thinking, “what the %$#& do you know about what you are actually saying?”
Maturity is a big factor when pursuing the build out of an appsec program. One could argue that a certain level of maturity is inherent when an organization is even thinking about appsec formally. One major challenge will be your ability to positively influence the engineering culture of your organization. Proper influence is key here because a force fed program will get you limited results and I assure you there is not enough time to review every line of code written for some given cycle. Hence, your goal is to positively influence the relevant people and the process to want to be a part of this. A “shift left” process, for instance, should become a mutually desired business enabler.
Both sides, software engineering and cybersecurity, should be after the same goal of a secure, resilient, functional piece of customer facing software. So building a successful appsec program requires commitment from each side. This will require some tactful education on the side of cybersecurity leadership. And let’s face it, some organizations are just not wired, culturally, to actually have a good appsec program, if any at all.
Organizational Culture
Let’s start with the organizational culture. To keep this relevant let me be clear that not all organizations need a dedicated appsec team. The ones that do are generally building something and pushing that something out for customers to utilize. But for instance if your organizations entire tech stack consists of a bunch of integrated SaaS solutions there may not be much for an appsec team to do. Those orgs can probably get away with periodic consultants reviewing security configurations from the SaaS vendors and perusing the integrations for weak controls. But honestly those orgs are at the security mercy of the SaaS vendor.
Mature organizations want application security, and security in general, to silently just be there. This is ultimately the goal of security as a business enabler. The more silent it is the better. The challenge is, however, that security hurts and it costs (time, money, effort and resources). As much as people love to push the enablement issue security simply doesn’t actually enable anything in most typical businesses. Hence, why they see security as a necessary evil. Changing organizational culture is critical but you have to factor in the reality of what I just mentioned. Weaving security thinking into an organization’s culture, especially in the engineering space, is foundational. Without this an appsec program will fail.
In the spirit of not being that “department of NO”, or not being a blocking entity, it’s up to us to figure out how to be “enablers”. This “how” is not trivial and organizationally subjective. Shifting left is a very common way to ease into enablement. To me, this requires moving security elements (code scanning, vulnerability scans, DAST/SAST, pen tests, etc) to be impactful earlier in an engineering and/or automated build cycle. Accomplish that and you may just have enabled a better solution to be built and deployed.
Focal Areas
In order to build this appsec program you now realize you have to positively change the culture of an organization. This will take time and perseverance. It will also take focus and a sound strategy. Here are a few areas that should be front and center in your appsec journey:
SSDLC – The key focal area for positive impact will be the Software Development Life Cycle (SDLC). You need to help your organizations engineering entities transform this into a Secure SDLC (SSDLC). I suggest you take slow and small steps here as change is difficult to accept. This is especially so if you are an outsider (and you most likely are from their perspective) asking them to change.
Focusing on adding security value with changes/additions to an SDLC, the typical areas you will hear experts speak about are:
- secure code training (for software engineers)
- secure design reviews (typically at an architectural level)
- pen tests (internal and/or external)
- risk assessments
- regular advisories
- secure code review (when/where possible)
For those of us who have really done this we know it is never that cut and dry. Moreover, to accomplish all of that your appsec team and budget better be pretty hefty. Software engineers are not going to rejoice with open arms based on what you are asking them to do (more work, longer deadlines, harder testing, etc). You must choose wisely which of those areas you will initially push for and look to make allies who willingly engage. The iron fist approach hardly ever works.
Depending on the size of your team, budget, the size and numbers of the target engineering teams, and company support (organizational culture) you may find a great approach is to embed appsec folks into the engineering teams/squads. This will create institutional knowledge and tailor the appsec program based on that domain expertise they will gain. The downside is that it takes resources away from your normal operations, but my experience is that this is an acceptable cost.
Measure Maturity – Maturity matters and you should set a goal of formally tracking progress in respect to your appec program. Two popular frameworks in regards to establishing a baseline and measuring this over time are:
There is a nice side by side comparison of the two here. While both of these frameworks seem straightforward give them some thought. See which one fits best based on your intimacy with the culture of your organization. One note, it’s ok for your scores to drop every now and then. This is a space that is highly impacted by certain events. Take a Merger & Acquisition (M&A) event for example, you have little control of what you inherit. This could instantly drop your scores based on no fault of yours. So the scores are a great metric but one that requires you to go with flow a bit.
Build relationships – This area really encompasses two distinct areas, testing and operational work. Building a relationship with Quality Assurance (QA) testing teams may prove very beneficial. After all, functional testing can very well go hand in hand with some security testing. Having security functions injected into other areas, such as regression testing, may prove valuable as well.
While automation plays a big role here you may be disappointed to find that in 2022 there is still a lot of manual QA testing that is performed. As humbling as that may be do you actually think your automated appsec scanners can catch everything? Irrespective, building relationships such that discoveries are made prior to production releases will prove invaluable over time. So work with QA to for instance have pen test elements built into some of their processes. They can become total allies over time.
Part of the relationship ecosystem is all about ownership. We are in no way trying to “pass the buck” but other teams, such as software engineering, need to own part of the security responsibility. They will ultimately have more of an impact on day to day security decision making than anyone outside of their team(s). RACI charts can be effective in identifying ownership borders clearly so consider as part of your arsenal. The appsec team is there to advise and try to influence best decisions but generally wont have the authority to force much of anything.
Invest in the right tooling – Focus on solving problems and building solutions, not implementing products. This area is broad and really spans numerous key areas of an appsec program. The goal is to positively impact an entire ecosystem from the left and right. On the left there is the SSDLC and all the goals already mentioned, you will need to invest in some tooling. On the right there are architectural components, pen testing, and a host of other initiatives that range from the systems thinking to the active protective. Active protection, for instance, will require tooling as well. But remember, build solutions.
Automation is going to play a key role in the overall impact of your appsec program. Building security components into X as code (where X can be build or infrastructure, etc) initiatives can add a lot of value. Injecting blocking security mechanisms into your organizations CI/CD pipelines are a great way of designing guardrails directly into a some processes.
Other Areas
Set boundaries – The scope of your appsec program will be critical. Scope is very subjective per organization. In order to set your team up for success set the boundaries early. For example, is your program going to cover database security? What about protecting the connections from apps, APIs, etc to databases? What about file security and protection? In some organizations those elements belong to an appsec team and this needs to be defined clearly.
Gain intimacy – Make sure your appsec team gains some intimacy with all relevant software engineering processes (to include tech stacks, 3rd party libs, hosting environments, file transfer mechanism, build processes, etc). This intimacy will allow your program to be effective with some of its goals, especially the one related to building and implementing a SSDLC.
Intimacy also has a direct impact and/or outcome related to the relationships you want to build. Bi-directional communication will prove invaluable. Get into the weeds with software engineering teams and, assuming you earn their respect, you will quickly identify who will be an ally and a voice for your appsec program(s).
Build inventories – You can’t effectively protect what you are not aware of. In my experience software engineers are generally bad at documentation. Part of your appsec program should aim at inventorying the application landscape and what it is made of. Pay close attention to the hidden “gotchas”, such as the APIs that are built into an app’s infrastructure, and hosted on the same server and transport (i.e. HTTPS) mechanism. Don’t limit your viewpoint here and consider infrastructure components along with supply chain elements as a part of your inventory. A lot of interesting security problems are most likely lurking there. A good inventory should at least expose some of these areas of interest.
As part of the inventory consider also building a control inventory. Frameworks (i.e. MITRE ATT&CK, etc) can help keep this organized as well as help make sense of adversarial tactics, techniques, and procedures (TTP). Creating an inventory of this sort can expose areas in your attack surface that maay need attention.
Build a risk register – This risk register will be focused on your apps and solutions. This should provide clarity in terms of risk the organization faces on a regular basis.
Create opportunities – Take every opportunity to get your message across and show how some security initiatives can be seamless and painless. For example, if you have an engineering team that creates compiled code. Why not have a library built (i.e. shared or static lib) that performs numerous security related functions (i.e. input validation, header setting, encoding/decoding, etc)? Then have the engineers take a look at how easy certain things become when they call exposed functions in that library. That is a seamless way of getting your objective some traction.
Gather metrics that matter – You will need to relay the value add, and effectiveness, of your appsec program to the corporate executives and/or the C-suite. Focus on metrics that matter. Not all metrics will be relevant to an appsec program. Any type of cost savings is always welcome, while you can also measure and track program adoption. Strategically, use one of the frameworks (SAMM or BSIMM) discussed earlier to show progress of maturity.
Final Thoughts
Security leaders set strategy and create programs, but more importantly we advise. A solid appsec program is one of those areas where we advise an organization. It is on us to mold an appsec program and focus it to map to, and benefit, the business. The points I have made here should give you a good sense of how you can enable a business via an appsec program.
Designing a program takes effort and thought. Factor in the people, processes, technology, and culture of the organization. Factoring these elements in is a continuous process as things, and organizations, change. They have to adapt and overcome constantly. So does your program. As long as you are continuously improving, and keeping step with the business, you will get positive results.
Understand that the biggest impact will come from the partnerships you pursue. This will be crucial in terms of having positive influence on the culture of your organization. Support this with some, hopefully many, of the technical components discussed in this writing and you will not be disappointed with the results. Stay focused on the fact that your appsec team should exist to be advisory, as subject matter experts.
Cybersecurity, and appsec specifically, subjectively mean different things to organizations. For some organizations these elements are simply part of modern day business success. My humble advice can get wrapped up as this: position yourself as a facilitator and an advisor, one that can transparently (as much as is possible) leverage the advice provided here to actually enable safe application based business. I wish you the best with your appsec initiatives.