Kelvin Welter

Tactics for Streamlining Development – From Design Docs to API Integration

At Bitboundaire, we constantly seek ways to deliver maximum value in the shortest time possible. In this article, we will share some practices we recently applied in implementing a new feature from scratch for one of our clients. While they may seem simple, these practices have provided significant benefits in organizational matters, workflow, collaboration, and documentation. In other words, we are working more efficiently and thus delivering value more rapidly. All of this with the practices outlined below.

 

Design Docs: The Beginning of the Journey

Design Docs, RFCs, ADRs, call them what you will. Despite their different purposes, all these formats have something in common: they encourage collaboration in addressing a problem. These docs help us expose our ideas and solutions to a particular problem and consequently bring us quick feedback from other colleagues. Therefore, whenever possible, we try to start a feature from a Design Doc.

In the feature we worked on from scratch, which served as the basis for this article, things were no different. The starting point for implementing the solution was not a code editor but a design doc in Notion, accompanied by various sketches and diagrams in Excalidraw. But why start this way? What are the benefits?

 

Why a Design Doc?

The Design Doc served us for various purposes in this feature (and others as well), such as:

 

  • Sharing and Validation: We shared the implementation idea with colleagues, seeking suggestions and validation. Are we heading in the right direction? Is there another way to do this? Is all of this necessary? Are we forgetting to consider any factors? All these questions can be answered during the review of the design doc by other colleagues.
  • Deep Thinking: It is very common to believe (irrationally) that we have a solution almost complete in our heads, and we can proceed with it. However, almost always, we realize later that the solution we decided to proceed with was shallow and overlooked many edge cases. Describing a solution in a design doc before proceeding with it forces us to think deeply about the proposed solution, enabling the development of a more robust solution to be implemented.
  • Initial Planning: The design doc not only addresses the technical part of a project but also serves to align expectations regarding deadlines and milestones to be achieved during implementation.
  • Dynamic Documentation: We take advantage of the space of a design doc to also centralize the decisions and changes made during the implementation and maintenance of a project. Thus, in one place, it is possible to understand the entire project, without discrepancies between the content of the design doc and the actual implementation of a project

 

This brief (non-exhaustive) list of advantages of writing a design doc before implementing a project brings a very interesting idea between the lines: documentation is not done at the end of a project, but throughout the project. Personally, I believe in this idea and have always had better experiences when this idea was applied. Therefore, I always prefer to start a project with a design doc. But what format should we follow when writing this document?

 

How We Did It

The format of a design doc is not fixed. Ideally, this format should meet the needs and reality of each company and team that undertakes to elaborate this document. Here at Bitboundaire, in my team and for the specific project that inspired this article, this was the format we used:

 

  • Context and Scope: Brief description of why the feature is being developed and its objectives.
  • Goals and Non-Goals: Clear definition of the feature’s scope. Listing what we are trying to solve and also what we do not want to solve in such a project.
  • Technical Design: Details about the architecture, focusing on initial design, new design, service layer, API layer, and implementation plan.
  • New Requirements: Sections added to document solutions and changes caused by new requirements.
  • Supporting Material: Useful links to delve deeper into the project, solution, and tools mentioned in the document.
  • Open Questions: Space for readers’ doubts and comments. However, since we used Notion to write the document, this section became obsolete. This is because Notion allows readers to comment on any text passage, making questions much more dynamic and contextualized.

 

Again, remember that this format is a decision entirely based on each team’s reality. After seeing the format we use here, you may feel the lack of some sections or feel that some sections are unnecessary. That’s okay. The important thing is that you adapt your design doc to your reality. However, regardless of your reality, don’t forget to add diagrams. They are powerful!

 

Diagrams: A Visual Perspective

One of the actions that greatly improved the quality of the design doc was the addition of diagrams whenever we saw fit. These diagrams greatly facilitated the understanding of the ideas we were trying to convey, providing a clearer view of flows and components. The diagram below is a (not real) example of those we included in the design doc.

 

It is important to note that, although diagrams have their role, they do not replace the need for complementary text for a complete understanding. It is very difficult to fully understand a diagram without the aid of text complementing the conveyed idea. Therefore, we use diagrams in the following format:

 

  1. Contextualization of the image
  2. Image itself
  3. Complementation about the shown image

 

Thus, we can combine texts and images to clarify all kinds of complex concepts and ideas within the project. This facilitates and greatly expands readers’ understanding. Thus, with a great understanding of the document, we receive much more feedback from readers. And this is extremely valuable to us.

 

Rapid Feedback: Refining Implementation

The Design Doc phase provided rapid and valuable feedback, both technical and non-technical. This included reviews from colleagues, who identified issues such as circular dependencies and interesting changes to be made in the API layer design. This feedback saved many hours of rework and strengthened the solution’s “final” design.

In addition to technical feedback, we involved the product team in the Design Doc review, making it accessible even to non-engineers. This resulted in valuable suggestions and questions that positively influenced the implementation.

So, after a few iterations made in the design doc, we had a much more robust solution than the one proposed at the beginning and a clear implementation plan from the API layer to the database. However, implementation details are not the focus here, but rather, the strong collaboration we had during implementation and will see next.

 

API Layer: Coordination between Backend and Frontend

Implementing the API layer can often seem simple, especially to the backend engineer. However, it can easily become a bottleneck in the project. And often, this bottleneck is not perceived by the backend, who believes they have finished their work when their code is deployed and the frontend engineer has been notified by them that they can start working on integration. This type of practice is very common but not ideal. Here, we decided to move forward with two points to prevent integration from becoming a bottleneck.

 

The Backend Coordinates

One of the biggest mistakes when designing a backend solution is to believe that small logic details can be left for frontend personnel to implement. This is one of the things that can most delay the frontend and degrade their codebase. This is because, no matter how small the logic we would like to implement on the frontend, we will be delaying the implementation of other components and screens. When we decide – usually unconsciously – that the frontend will be responsible for building the text to be displayed on screen X, we are deciding that the frontend will waste time that could be spent building screen Y or component Z to implement something that the backend could have already prepared for consumption.

Furthermore, it is important to emphasize that (almost always) the backend will be slightly ahead in implementation than the frontend. The backend tends to finish its part before the frontend. Therefore, building logic on the backend tends to balance this game a bit and makes the frontend not so “behind.” Conversely, when the frontend is responsible for small logics, we increase the distance between the progress of the frontend and backend in the project.

And I mention that the frontend would be “lagging,” but the truth is that the backend is too! Because if the frontend hasn’t finished, the project isn’t finished! Nobody wins, everyone loses.

Therefore, the API design in this project was entirely designed so that the responses were ready to be thrown on the frontend screen. Without the need for sorting, filtering, text construction, or any other type of logic/business rule. The concern of the frontend was exclusively focused on building screens, components, and integration with the backend, which we also tried to make as facilitated as possible, as we will see next.

 

Integration Documentation

The codebase implementing our project uses GraphQL in the API layer. An excellent technology, but one that can bring some confusion to frontend integration. It is a consensus among the frontend devs on my team that integration with GraphQL is more difficult and time-consuming than with a RESTful API. Therefore, it is always important to try to facilitate integration as much as possible, which can easily become a bottleneck for project progress.

 

To do this, we decided to try to facilitate integration through robust documentation, aiming to tackle 2 problems that occur during frontend integration:

 

  1. Not knowing exactly which fields to fetch in the GraphQL API: Many backend devs believe that their work ends with the deployment of their code. Leaving it to the frontend to build the GraphQL query that best suits the screen to be built. However, this work has a lot of friction for the frontend dev, who does not have the backend context and the correct tools to discover which query is “ideal” for a particular screen. Thus, the frontend loses a lot of time building something that could be done in a matter of minutes by the backend. Overfetching can also occur at this time.
  2. Not knowing exactly where each request field should be displayed on the screen: Frontend devs have reported confusion in understanding where each field returned by the backend should be placed. Of course, in simple queries, this is an easy and even intuitive matter to discover. However, complex screens may require complex queries, with several levels of nested fields. Thus, it brings great difficulty in understanding how to fit each field on the screen.

 

Both problems mentioned above are capable of causing significant delay in the integration between back and front, with the delay being proportional to the complexity of the screen. Therefore, we took two simple and quick actions, but very effective in combating delays caused by these problems:

 

  1. We assemble the query for the frontend, according to the screen preview passed to us: Assembling queries by the backend team, especially the dev who built the feature, is a very quick process. We can assemble the query in a matter of minutes, leaving already ready all the part of inputs and fields to be returned.
  2. We assemble an integration diagram: To make life easier for the frontend dev, who needs to know where each field should be placed, we created a diagram with two elements: the screen preview to be assembled and the query to be used. Between these elements, we draw arrows to show where each field should be placed.

 

And so, with these small actions, we achieved much faster integration between back and front. Moreover, this more active participation of the backend team in integration brings another great advantage: agility in resolving doubts. That is, a frontend dev tends to contact a backend much faster if they are having problems with what the backend has passed to them! This is because the backend should have passed everything without errors, if there is any error, it is certainly not the frontend. Whereas if the frontend is assembling the query and has errors, there will be much greater resistance to contact a backend to solve a problem.

 

Gradual Implementation

Finally, it is always important to remember how efficient the gradual delivery of a project is compared to “big bang” deliveries, where everything is delivered at once and the chances of something going wrong are high.

 

The gradual delivery of a project allows for the gradual receipt of feedback, avoiding a flood of feedback and issues reaching the team all at once. This was very important around here. This short cycle between implementation and feedback allowed us to work more quickly and reach the actual project launch with no last-minute adjustments needed.

 

Conclusion

And thus, with all the practices described in this article, we achieved a great implementation pace in the project: we went from zero to project validation in just 7 working days.

Personally, I believe that a major factor in developing this feature from scratch, without major friction and surprises, was our concern with simplicity.

 

  • We simplified the design doc
  • We simplified the API layer
  • We simplified the value delivery

 

Simplicity increases collaboration, collaboration increases speed.

I hope you enjoyed this article. Thank you for making it this far. That’s all for today!

Leave a Reply

Your email address will not be published. Required fields are marked *

Come with us!

Do you love technology as we do? Subscribe to our newsletter and receive articles about everything surrounding state of art software engineering.

Popular posts

Popular tags