..

Designing Api For Microservice In Golang

1. Backstory

More than a year ago, few other developers and I decided to try Golang to build microservices for our new project. Previously, we used Python frameworks such as Django, Flask, and Starlette/FastAPI. They all have their ups and downs, especially FastAPI. It comes with Swagger UI and a data validation layer using Pydantic.

Python is okayish most of the time, and it is still one of my go-to languages. However, I always wanted to try Go on a large project, and the rest of the developers also had the same idea, so we gave it a try.


2. Intro

I am not going to explain to you what the microservice is. If you want to know, go and read Martin Fowler article. It is a good read, and even if you know what it is, I still recommend you to read it.

In this article, I will tell you what functionalities my service needs to provide, what tools/frameworks I used to implement those. Lastly, how I set up testing with CI, and dockerize it so that my service integrates nicely with the project overall. So let me break into down into several sections

  1. Frameworks I used for the functionalities I need to provide
  2. Local development setup, configuration management, and dockerization
  3. Testing on local and CI (also visualizing the test coverage)
  4. How my service integrates with other services in the project overall


3. Frameworks

Web Service and Database

Since it is an API, I need a database (used Postgres) for storage, a web framework (Gin) with Pop as the ORM, and Soda CLI to handle database migration.

Previously, I used database driver (sqlx) instead of Pop. In most cases, I would recommend ORM over SQL drivers mainly for one reason. To not write SQL queries inside the application code. If you still want to use SQL driver, that is also not an issue because you can still use database migration tools like Soda CLI to handle the migrations separately. However, I would not suggest dropping the database migration tool a good idea because it makes your life so much easier. Trust me, doing database migration with .sql files is a nightmare.

It is very similar to Django with all those migration tools, etc, for those coming from Python. The only difference is, I am just taking out whatever frameworks I need for the requirement instead of using the whole ecosystem.

If you are into the complete package, you can look into the Buffalo ecosystem. However, I do not like frameworks like Django anymore because it is kinda overkilled. Even in Python, I prepare smaller frameworks like Starlette with no template engines out of the box but, you can install it only if you need it.

In addition to all those, my service also needs to provide Async Job/Queue for internal services communication. Therefore, I used Machinery with the Redis backend. Additionally, the service I am working on needs to schedule some tasks, and the Machinery is to trigger those cronjobs. I will explain further in detail later in the article how my team and I handled the cronjob among our microservices.

Configuration Management

In every service, there will be four application environments in general. Developing, Testing, Staging, and Production. Your application will need to store some configurations like database URL string, debug flag, HTTP server port, etc. Your database URL string will be different between staging and production among the four stages I mentioned above, and the configuration management tool is quite helpful to handle it.

Traditionally, people handle it by the environment variables, and it works obviously. But using the config management tool is far cleaner. For my service, I used Viper to handle configs.

In every service, there will be four application environments in general. Developing, Testing, Staging, and Production. Your application will need to store some configurations like database URL string, debug flag, HTTP server port, etc. Your database URL string will be different between staging and production among the four stages I mentioned above, and the configuration management tool is quite helpful to handle it.

Traditionally, people handle it by the environment variables, and it works obviously. But using the config management tool is far cleaner. For my service, I used Viper to handle configs with Cobra to provide CLI arguments for the service.

Why do I need that? I want to build my API service into one standalone binary file that can provide several functionalities such as run migration, run the webserver, etc.

Example:

service run migration
service run api


4. Local Development Setup


TODO: still not finish writing