How to Scale APIs for Rapidly Growing Organizations

As companies and their products grow, so do the needs of their systems and codebases. One major problem that rapidly growing API providers need to address is scaling their APIs to meet the growing demand. On the bright side, the fact that a company’s API needs scaling is an extremely good sign -- the need for scaling implies something positive. Most companies need to think about scaling on two different dimensions: design and system. Optimizing the design of your API would enable hundreds of engineers and multiple teams to be able to work seamlessly and launch features. Second, optimizing the system would ensure that your API design can handle the growing demands from customers in terms of fulfilling their requests. Both of these dimensions need to be taken into account and baked into your API architecture from day one. 

Scaling your design with your organization

In an organization that is expanding in size, its API design will grow exponentially complex if not designed in the right way. As the number of engineers working on your API increases, so will the divergence in your codebase and internal styles of your API. Let’s take the example of an organization that has a singular API team maintaining three products, but the growing demand for each different product means that the company soon needs individual teams owning and growing the different products. 

Figure 1: As an API grows, it may become separate products requiring separate teams

In theory, this does not sound that hard -- one could easily create three different teams which could independently own and operate the different slices of the API. However, in practice, there are multiple major issues that arise due to such an organizational split:

• Development velocity
• Prone to errors
• Lack of consistency
• Low maintainability

With the right set of principles and tools, rapidly growing companies can ensure they avoid the pitfalls mentioned above. 

Don’t model your API based on your organizational design 

One common mistake to make is to model your API similar to how your organization looks like. This seems like the logical division of labor to make. You will often hear a leader say, “We already have a payments team, and they should own the payments API.” Modeling ownership and the design of your API similar to how your organization is modeled is a recipe for failure. This is because organizations are always in flux -- people come and go as structures in an organization are constantly changing. On the flip side, what your customers want is always constant. This is an extremely non-intuitive way to organize your APIs, but APIs should be designed for longevity, and almost no organizational structure can stand the test of time. Therefore, designing your API with the lens of a customer first approach is a lot more likely to be future proof. Leaders should deeply think about how and why customers use their products. This lens enables companies to ensure their APIs are designed for the long term.

Introduce a client-advocate role in your organization

As companies are scaling or they foresee a larger demand for their products and APIs, it is important to hire for or create space for a client-advocate in the organization. The job function here is to obsessively think about the needs of the clients and the users of the API. Having someone in the organization play this role will amplify the voices of the end users and bring forth a picture of your APIs that is desired by your customers. In organizations that are scaling, it becomes increasingly hard to fight for the design choices that ultimately benefit the customer. Competing product priorities, deadlines, and lack of understanding of customer needs can easily end up muddying the water when it comes to the sanctity of an API design. Companies need a voice in the room whose sole purpose is to make the life of users better. The best API design ideas almost always come from outside the organization and aren't incepted by the drivers or engineers of the API, but the customers of if. One of the best tools a scaling company can adopt is hiring client-advocates within the organization. This role should have power to impact the core design of your API and be consulted on engineering decisions that can have any changes on the interface of the design.

Versioning as an antipattern

This is one of the vastly misunderstood concepts in designing APIs. First, it’s important to understand that any serious API platform and design will have to have versioning baked into the infrastructure from day one. However, “versioning as an antipattern'', is more about a mindset. Great APIs are built for longevity — they never have to retire any resources. This mindset and ideology needs to be adopted from the early days of the API. Build your v1 as if your API would stay on v1 for the rest of its existance. This means that rapid iteration and fundamental changes to your API should absolutely be off limits. Everytime there is a proposal for making a backwards incompatible change that fundamentally changes a resource, one should critically ask if the design iteration should just be a new resource altogether. In addition, your API interfaces and contracts should always abstract away your implementation details. This ensures that any additions you do to your API will also not be exposed to your clients.

Emphasize your user journeys and workflows

More often than not, organizations incorrectly start focusing on the number of endpoints when they are rapidly growing and scaling. It is not uncommon to see organizations doing press releases and API updates of new endpoints being launched almost every other week. While this does indicate activity, it might not always indicate impact. What growing organizations should really think about is the journey that the user will take when accomplishing a certain task with their API. If the task is to make a payment, and your customers need to make seven consecutive API calls, that might not be the best experience on their end to build out a payment workflow. While some consecutive calls can never be avoided, it is important to think about your endpoints from the perspective of the workflow of the users. In companies where this isn’t done, they end up with a plethora of different endpoints which do not correlate with the needs of the end users, thereby, prioritizing activity over impact.

Horizontal and vertical scaling

Different components of the API have different needs and need to be scaled differently. This is one of the key components of scaling the needs of your APIs to avoid the major pitfalls such as lack of consistency and lack of maintainability. It is important to think about which parts of the API would need to serve as horizontal components and which ones would need vertical independence. Doing this exercise would ensure that you don’t end up with multiple teams owning and operating more than they should. Ideally, this means that the organization keeps a horizontal platform team that manages the core components of the API and being product agnostic, and then letting different vertical product teams iterate on different products offered by the API.

Figure 2: Consider which parts of the API serve as horizontal components and which need vertical independence

This would allow for consistent patterns and design choices within your API. A core foundations team that deals with all common architectural and system level concerns of an API will unlock product teams to only focus on what’s important -- the needs of the customer. This model would allow different product teams to execute independently and only focus on the business logic of delivering a great product, while offloading platform level problems to the core foundations team. What components the foundational team own depends on the use cases of different organizations, and therefore, it is of paramount importance that every organization identifies these horizontal components and scales them differently. 

Conclusion

The need for scaling APIs is a great problem any company can have -- however, it is extremely important that it is done in a principled way to avoid detrimental design decisions that will inevitably cause long term problems with velocity of iteration and a codebase that is hard to maintain. When thinking about scaling, a lot of companies and leaders only focus on the system, and underestimate the value of scaling their design. Companies that are rapidly scaling can unlock a massive amount of value and future dividends if scaling design and system go hand in hand.