You are here

Organizational complexity and app sec: 4 ways to stay on track

public://pictures/John-Mello-Journalist.png
John P. Mello Jr., Freelance writer

There are many ways to predict how buggy a codebase will be. Having lots of changes from one version of a program to the next is one predictor. So are code complexity and overreliance on dependencies. The best predictor of all, however, may be organizational complexity.

Norwegian software programmer August Lilleaas recently blogged about a 2008 Microsoft study that concluded that organizational complexity was the No. 1 predictor of bug frequency in software. In a later study, organizational complexity placed lower as a predictor, but it was still relatively high—second in precision, and third in recall.

Precision compares the number of bugs predicted by a method to the actual number of bugs found. Recall identifies the number of modules with bugs predicted by a method. Precision is the more important metric, because a low precision rate means a method is creating false positives by tagging modules as buggy that were not.

The study found that trying to measure the complexity of the code is less useful than studying the organizational structure that produced the code when it comes to predicting bugs, said Jeff Williams, CTO and co-founder of Contrast Security, a maker of self-protecting software solutions.

"They're studying the whole organization, not just the software itself. That's really promising, but we have to look at a broader array of organizations, software, and bugs."
Jeff Williams

Marc Rix, product manager for Scaled Agile, added that complexity is one of those pieces of the organizational agility puzzle that doesn't get talked about as often as it should.

"In many parts of the industry, there's a big focus on tooling and changing architecture and not a lot of conversation about organizational complexity. What's interesting about this study is it presents organizational complexity as the leading factor in determining product quality."
Marc Rix

How can organizational complexity be reduced and software qualityand security—improved? Here are tips from top experts.

[ Discover how to reduce friction between dev and app sec in this Webinar. Plus: Learn how to build application security into your software with TechBeacon's guide ]

1. Keep your development teams small

Team size and software complexity are leading factors not only in creating bugs, but also in reducing engineering efficiency and effectiveness, noted Thomas Hatch, CTO and co-founder of SaltStack, a provider of intelligent IT automation software.

"The bug burden becomes heavier and heavier as projects scale and it becomes more and more difficult to onboard engineers for them."
Thomas Hatch

The number of communication handoffs increases exponentially with the number of people you have involved on a project, said Contrast Security's Williams.

"Too many cooks can spoil the soup, except software is a lot more complicated than soup."
—Jeff Williams

Scaled Agile's Rix added:

"As you add on engineers beyond the two-pizza team realm, communication management becomes unwieldy."

Other organizational tweaks to improve team performance include frequent code reviews and technical autonomy, said Jon Bottarini, senior security solutions engineer at HackerOne, which connects businesses with penetration testers and security researchers.

"No matter how many engineers you have working on a project, you need to have code reviews often. You also have to make sure those code reviews are blameless. They will go a long way to ensuring that the organization is pushing out code that is bug-free."
Jon Bottarini

Teams need autonomy to build quality applications, said Bo Lane, vice president of global engineering at Kudelski Security, a provider of tailored cybersecurity solutions.

Technical design decisions shouldn't be forced on them, Lane said.

"You can introduce complexity by mandating technology choices. In some cases, they may be required and necessary. In other cases, it's best to leave development teams to their own devices to select technology and design patterns that they're comfortable with."
Bo Lane

2. Reduce the percentage of your org working on the project

Software projects involve more than engineers and developers: Stakeholders are pulled into the process from other parts of the business, too. Architects may be contributing to the architecture of the project, for example, and product analysts may be writing requirements and prioritizing work. The more people from your organization working on a project, the more complex it will get, and with that complexity come more bugs.

The more engineers you have writing code, the more complex the codebase gets, and the more fragile and error-prone it becomes. By the same token, the more people in the organization involved in the solution, the more complex that solution gets, Scaled Agile's Rix said.

"When you've got a lot of people in the organization involved in the value stream, there's a tendency for things to get lost in translation and for what's implemented not to match the wants and needs of the people asking for it."
—Marc Rix

[ Take a deep-dive with our Application Security Trends and Tools Guide, which includes TechBeacon's 2019 App Sec Buyer's Guide. ]

3. Keep the developers and decision makers as close as possible

Conway's Law, formulated by computer programmer Melvin Conway in 1967, states, "Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations." That's especially true when it comes to communication between developers and decision makers.

When the executive responsible for setting the vision for an application or solution is as close as possible to the developer, that's when you can avoid a lot of misinterpretation and poor assumptions by the development team, said Kudelski 's Lane.

If you've got a product manager who tells a developer how he wants a feature to work and that developer implements that feature, they're more likely to get it correct than if that same feature goes from a product manager to an engineering manager to an engineering team to a junior developer, said Chris Eng, chief research officer for Veracode, a provider of cloud-based app intelligence and security verification services.

"That's why some of the modern development practices that are smaller-team-based are more effectively. You have a connection between the business side and the technical side so there's less room for something to get misinterpreted."
Chris Eng

4. Create good documentation to avoid developer churn problems

Microsoft's researchers discovered a connection between developer churn on a project and the frequency of bugs. The more ex-developers a project had, the greater number of bugs found in code.

"When a developer leaves, oftentimes there's no way to transfer knowledge effectively. A new developer needs to figure out everything again, and there's a good chance they'll not figure out the full context and make a mistake."
—Jeff Williams

Meanwhile, new developers on a project need time to acquire the knowledge they need to be effective. "As that's happening, there's more opportunity for bugs to be introduced into the software," said Kudelski's Lane.

What's more, when new developers are forced to work with old code, they tend to program around it or find a way to reroute its logic through new code. The more you do that, the more technical debt you build, and legacy code isn't tied into the latest codebase, said Veracode's Eng.

"It becomes barnacles. The more barnacles you have, the more untested or under-tested code you have, the more complexity you've  created that leads to bugs."
—Chris Eng

With properly documented code, some of the question marks can be removed from an ex-developer's code. "Distinctly documenting within the code what it's doing is going to be really helpful if the person writing the code leaves the company or the code is incorporated into a larger project," said HackerOne's Bottarini.

Code security remains the broader goal

While addressing organizational issues can improve code quality, it may not make the code more secure, Contrast Security's Williams said.

"I don't think these factors really affect whether a vulnerability is going to be created or not. They may prevent some kinds of bugs from being written, but not other kinds of bugs that don't have anything to do with how well you understand the code that you're writing."
—Jeff Williams

For example, code can be written to perform a function to meet certain use cases. A bug can keep the code from functioning properly, but once the bug is fixed, the code will appear to be working properly to the developer. What happens, though, when the function meets something outside the use case? "Unspecified behavior is something that often leads to security issues," Veracode's Eng said.

"Security bugs are going to be a factor in developers not knowing how to handle edge cases, and how to eliminate common security vulnerabilities. That's a separate issue from organizational complexity. That's an education and testing issue."
—Chris Eng

[ Get Report: Gartner Magic Quadrant for Application Security Testing ]