Velocity through hypergrowth with Roadie

Contentful is a modern content platform which lets you write and publish content to use in all of the locations your business is represented.

Over the past 4 years, the Contentful engineering organization has grown from around 40 engineers to approximately 250 engineers. This phenomenal growth has stretched all corners of the organization and required them to redefine aspects of their culture along the way.

One of the ways Contentful maintained engineering effectiveness through this growth was by introducing Backstage, provided by Roadie.

I met with Contentful staff engineer, Enrique Amodeo Rubio (Linkedin), to learn how Roadie Backstage has benefitted them.

Enrique has been a staff engineer at Contentful for 4 years. He was originally hired to help guide the team through the rapid transition from startup to scale-up, while maintaining the quality and productivity they were already known for.

Introducing a service catalog

In late 2019, Enrique and his Automation Tools team decided they needed a service catalog to track the multitude of microservices and teams springing up in the company. They set an initial goal of establishing a clear record of ownership for every software component and team.

Despite being extremely new at the time, the Spotify pedigree and availability of commercial support from Roadie made Backstage a compelling bet.

Enrique and his team knew that they would make faster progress if they focussed on the Contentful specific parts of the initiative, and brought in experts to manage Backstage. They reached out to Roadie and we began working together for the first time.

InnerSourcing: Maintain velocity through hypergrowth

InnerSourcing is an important philosophy at Contentful. It has been the key to maintaining productivity as Contentful has grown rapidly over the past 4 years.

If an API or library is available internally but it is missing one particular feature you need, you look to contribute the feature rather than opening a ticket on the backlog of that team and waiting for them to get to it. InnerSourcing allows teams to remain unblocked, reduces lead time for new features, and keeps everyone shipping at a high rate.

Effective InnerSourcing has been enabled by 3 powers delivered by Roadie Backstage.

1. A clear record and sense of ownership of code.
2. Relative homogeneity across codebases.
3. Availability of documentation and API Specs.

A record of ownership

Before contributing a change to another team’s service, it’s a good idea to have a conversation with the team about their priorities and how they would like you to contribute. To do this you must be able to easily identify and contact the owner of the service.

Roadie Backstage fulfills this function at Contentful. More than 90% of their fleet is tracked in the tool and anyone can find the owner of a service quickly. Each team has their Slack channel and email available.

A tangential benefit of a clear record of ownership is that it’s easier to get the right people together if a service has a problem in production. This can be the difference between a minor blip and extended downtime.

Codebase homogeneity

Another way to increase the efficiency of innerSourcing is to increase the homogeneity of the codebases in the company.

If every codebase looks completely unique, the learning curve is steep when trying to contribute a change to a service you are not used to working on. It may be written in a different language, with different technologies, coding standards and philosophies.

To combat this, Contentful used the custom plugins feature of Roadie to build a frontend for their own service scaffolder. Using this Backstage plugin, any Contentful engineer can quickly and easily create a new service from a predefined template. Since most new services are created from these templates, the consistency of the codebases and production environments is increased, and it is easier to contribute a change.

They also used a custom Tech Radar plugin to guide teams towards choosing broadly supported technologies. By placing this in Roadie Backstage where it is easily found, they ensure the tech radar contributes to the shared understanding.

TechDocs & API Specs

Of course, if you’re going to contribute a change to another team’s service, it would be helpful if they had some docs so you could get the service running locally and understand how it works.

Roadie Backstage helps here too. Engineers at Contentful are converting their technical documentation into Backstage TechDocs format so the documentation can be easily found by others who want to contribute a change.

If you’re looking to change an endpoint on a dependent service, you need to know what the API looks like so you can visualize your change. Again, Roadie Backstage provides the answer. By offering a centralized space for API Specs, any engineer in Contentful can look up a service API and understand what they can do with it.

Working with Roadie

When they decided to set out on this InnerSourcing journey, Enrique’s team knew that they didn’t have infinite resources at their disposal. They determined that they could move more quickly if they focussed on defining the standards and building the Contentful specific parts of the toolchain. They knew Backstage was the right answer to enable engineer effectiveness, but they didn’t want to spend their own time on setting it up and maintaining it. That’s why they decided to work with Roadie. Overall, Enrique says the experience of working with Roadie has been excellent.

The collaboration has been very good. The support is high quality with a quick turnaround. It’s obvious the team at Roadie has deep knowledge of the inner-workings of Backstage.

I also like that Roadie are deeply involved in the community. I didn’t want to end up on a private fork of Backstage, or locked into a proprietary service catalog.

The results so far

Roadie Backstage is not even a year old at Contentful, but it has already been a huge success.

The scaffolder, built into Backstage, is now the de facto way to create new services. Teams who use it gain the benefit of moving more quickly while adhering to strong engineering practices supported by their wider organization.

The service metadata contained in the catalog-info.yaml file has also been a success at Contentful. Adoption of the YAML file occurred spontaneously by teams throughout the engineering organization, with no pressure from upper management. The value of having a YAML descriptor for each service, and being able to look it up in Roadie Backstage, was self-evident. Today 90% of services and teams have their own YAML descriptor.

Adoption of docs and API specs is more nascent but the staff engineers group are bought into the idea and have moved all of their documentation into Backstage where it can serve as an example to other teams.

Overall, teams are moving more quickly at scale than they would have been able to without this initiative.