CumulusCI is often confused with continuous integration applications for good reason, it has “CI” in the name. This creates confusion about what role CumulusCI plays in a DevOps infrastructure. I’ll try to provide some historical context about how the name came about and hopefully clarify the role of CumulusCI in a DevOps process and infrastructure.
Our story begins in 2013 at the Salesforce Foundation (now known as Salesforce.org) when the decision was made to build out a product team for the first time. In probably the most significant moment of my professional life, I was lucky enough to land a spot as one of the first two members of that product team. My mandate was to figure out how to do CI/CD for ISV-style managed package development so we could release NPSP biweekly with push upgrades to the thousands of nonprofit orgs using it.
At the time, NPSP was the Nonprofit Starter Pack, an open source community built set of 5 managed packages. Our team’s goal was to figure out how to rearchitect NPSP into a product with a dedicated development team while not leaving behind any of the thousands of nonprofits using NPSP to run their organization. We settled on the interesting approach of creating a new extension package that extended the 5 original packages and migrated all the logic from those packages into the new package. That architectural decision is beyond the scope of this post but was the best way we could figure out how to achieve our goals. The new package we were building would eventually become NPSP 3.x, the Nonprofit Success Pack.
For the first year of our team, we called this effort Project Cumulus. Why that name? NPSP is a fundraising system and cumulus clouds are the clouds that “make it rain”. I won’t take credit/shame for that punny name, that was all our fearless leader Kevin Bromer.
The GitHub repository for the new package as named Cumulus (now renamed to NPSP). Since this was before Salesforce DX was even a thought in someone’s mind, I set out to implement a CI/CD process using the commonly recommended tooling, the Force.com Ant Migration Tool.
After a few months, we had a pretty functional CI/CD process running in Jenkins with full isolation of all feature development in feature branches and automated creation and test of beta versions on every merge to the main branch.
Some other ISVs expressed an interest in the process we had built. Since the Cumulus repository was open source, anyone could copy our Ant build scripts and a handful of Python scripts into their own repository to replicate our CI/CD process.
A few months after some ISVs had copied our scripts, I started to hear back about issues they were encountering. We had also hit those issues and implemented fixes in the Ant targets in the Cumulus repository. But since they were working on a copy of those scripts that they had modified for their own needs, there wasn’t an easy way for them to integrate the enhancements and fixes we developed in our Ant.
We decided to create a new repository called CumulusCI with a refactored version of the Ant and Python designed to allow projects to use that shared code base and configure their product specific automation by overriding a series of targets designed to provide hook points in the main Ant targets.
The name CumulusCI was quite accidental and definitely not selected based on a rigorous branding process with a vision toward what CumulusCI has grown into today.
CumulusCI is a framework for portable automation for Salesforce development projects in GitHub. The goal is to define the automation needs of a product in version control so that automation can be used in different user experiences throughout the product lifecycle.
The portable automation configured with CumulusCI replaces the often complex shell scripts used to define CI/CD jobs. Workflows created in CI systems such as GitHub Actions, Jenkins, or CircleCI generally have a single command calling out to CumulusCI’s CLI cci to run complex sequences of automation.
Put another way, CumulusCI abstracts most of the logic needed to run CI builds in a CI system into portable automation. That portability makes it easy to use whatever CI system you want and even to easily switch CI systems. It also allows product teams to leverage automation outside the context of CI such as for on-demand scratch org creation by team members through to the delivery of the product with tools such as MetaDeploy.
I often reflect on the series of happy accidents that led to the evolution of CumulusCI into the powerful, flexible, and mature Python based framework it is today. 9 years ago we had no idea we were creating what CumulusCI has become. I’ll chalk up the name CumulusCI as an unhappy accident and offer my apologies for the confusion the name causes. I also apologize for the decision I made years ago to refer to CumulusCI’s sequences of automation as “flows” and its branching model, an extension of GitHub Flow, “CumulusCI Flow”
Thanks for indulging me in a trip down memory lane. I find history interesting, but in the context of CumulusCI, the most important history is the years of ongoing development work that leads to amazing new features released every 2 weeks.