How devsecops is helping Australian companies modernise CI/CD for today’s risks

Application development moves quickly at Australia Post, the government-owned postal and package delivery giant that was already pushing hard into digital transformation and devops before the COVID-19 pandemic’s stay-at-home orders sent demand for its services through the roof.

As that demand increased, the company’s developers got progressively busier—and that pushed the company to look for a better way to manage and secure a continuous integration/continuous delivery (CI/CD) pipeline whose scope and intensity had outgrown the Atlassian development environment adopted by the Platform Engineering team more than five years ago.

With an increasing focus on cloud-based applications and delivery within Amazon Web Services, tools like Bamboo, BitBucket, and Crowd were struggling to keep up with an accelerating devops-driven development cycle in which Australia Post developers were pushing out 37 nonproduction and seven production deployments every business day.

What was initially a team of about 50 developers had ballooned out to 300 people across engineering, delivery, and business cohorts as well as central services like security, strategy, architecture, and enablement services—and the static CI/CD platform had failed to keep up.

“When it was put together years back, our tooling did an amazing job for what it was conceived as,” said engineering manager Nitin Sharma during a recent IQPC webinar, “but there was no active investment made to re-evaluate the needs of our developers.”

Recognition of the need to change led the team searching for a devsecops platform that would better support its new processes—and it ultimately embraced a GitLab CI/CD platform that has become the basis for a devops and devsecops revolution.

Since the change, Sharma said, “our security posture for sure has improved.” He cited improvements in secure-development processes like code signing, integration with the Checkmarx code-testing security tool, and a more tightly controlled “blast radius” that reduces the chance of compromised code.

Management support was key to driving the change, Sharma said, noting that successfully transitioning to devops and devsecops means “you have to consistently communicate with your stakeholders, and take them on the journey. People need to feel they are all in on this with you.”

Building a security culture via devops

Across Australia and the world, enterprises of all sizes are undergoing similar efforts as they shift security left by embracing the tenets of devsecops within their overall devops efforts, the next-generation development process that correlates application-development processes with the deployment of supporting cloud infrastructure.

As Sharma alluded, culture can be all the difference between successfully making the jump and failing. Yet the culture of development organisations in Australia and Asia-Pacific countries is less of an obstacle to devops adoption than in other parts of the world.

Just 37% of Asia-Pacific respondents to devops platform provider Puppet’s 2021 State of Devops Report said that culture was a barrier to the evolution of devops practices in their organisation—well below the 47% global average—while just 23% said that technology was more of an issue.

A “very specific set of challenges” were seen as cultural factors in impeding progress towards devops. These issues include cultures that discourage risk, have unclear responsibilities, deprioritise fast flow optimisation, and fail to include sufficient feedback loops—all of which accumulate over time and can cause stagnation that causes many organisations to plateau after only completing part of their devops transformation.

Such issues are common as companies transition to devops, wrote Amish Prajapati, ANZ southern region director at development platform provider Micro Focus, in a marketing post. He highlighted the risk of devops initiatives falling into the ‘J curve’—in which early quick wins give way to ponderous progress as morale suffers under the weight of continuous change.

Other key tactics include better understanding the technology value stream, aligning outputs with outcomes, using KPIs as a learning tool, and using smart approaches to re-engineer the organisation for devops.

Ultimately, said Darren Yeo, head of architecture and planning at Australian retailer Officeworks, lingering cultural issues – and the reality that security teams are usually desperately outnumbered by developers – mean teams need to work hard bring security practices out of the “ivory tower station that just sits there scanning stuff”.

Development and engineering teams at Officeworks “brought dev and ops together over the past five to eight years,” Yeo said, “and we are getting better at catching up with dev. But it’s still not instilled into the organic workflow.” With security teams generally comprising ten or fewer people—and development and engineering teams numbered in the hundreds—security practitioners were fighting an uphill battle to drive change themselves.

Yet continued inertia around devsecops is, Yeo said, becoming problematic in a world where “the digital experience is in hypergrowth and all apps and services are onto the Internet straight away,” he said. “So, you cannot even negotiate security. You have to bake it into the workflow—and the cultural shift in this computing direction needs to be ironed out all the time.”

The experience of devsecops advocates well reflects ongoing challenges around Australian implementations of devops itself. “Like any transformational journey, implementing devops will not be straightforward,” Micro Focus’s Prajapati wrote. “While there are many positive outcomes, there will also be multiple challenges.”