How To Improve API Part 3
Asynchronous Process (written in golang)

source image : https://amoniac.eu/services/golang

Hello, we meet again on the third part, this part is a continuation of part 1 and part 2. I suggest for you guys to read the previous parts, before you read this part 😃.

In this section we will talk about other ways to improve our API, but interestingly, it is different as before, we will not use other dependencies like postgresql or redis, we will maximize our code flow.

Wow, hold on, I know you must be thinking there are lots of ways to maximize code flow to improve the API. Yes, I know, but we only covered one thing, namely asynchronous process. Asynchronous process is a process or function that executes a task “in the background” without the user having to wait for the task to finish, so the process can run in parallel.

Wait wait, if we can do that, why don't we use this way to call all functions?, oh it's not that easy bro, because there are some limitations to doing that, there is one most important thing if you guys want to implement this, there should be no dependency between one process with other processes (it is forbidden for each process to require one or some response field from another process).

I’ll give you a simple example of what should be done synchronously and what can be done asynchronously in real life.

There are 3 tasks :

  • washing clothes using washing machine (assume that the washing machine don’t support drying clothes)
  • drying clothes
  • sweeping floor

From 3 tasks above, you are picking 2 tasks, washing clothes and drying clothes. You can only dry clothes after you have finished washing clothes. You can’t wash clothes and dry clothes at the same time, because drying clothes requires clean clothes from the washing machine, so you have to complete both tasks one by one.

If you are picking 2 others task like washing clothes and sweeping the floor, you can do both in parallel, you just click the start button on the washing machine and leave it then you can sweep the floor. You can do both tasks at the same time because sweeping the floor doesn’t require clean clothes and vice versa (washing clothes does not require cleaning the floor first).

Both example have a different flow. When you guys check on the code for getting user detail, we can see the same flow as the second example. We have to return UserProfile , UserFamily , and UserTransportation for our response. Those 3 fields function don’t have any dependency, UserTransportation doesn’t require a response from UserFamily and UserProfile ,it happens to the others, so we can call these 3 functions asynchronously.

If you are still confused what we are going to do, maybe this diagram will make you more easily to understand what we want to do. We want to change the flow like this :

flow diagram GetUserDetail

Preparation

note : the code shown in this section is incomplete. I just highlight added code from the last part (part 2), so if you want to see the full code, i will give the github at the end of story 👌.

On this part only the main file is changed. We set goroutine for calling 3 functions to get detail data (UserProfile , UserFamily , and UserTransportation)

Main (adding asynchronous process)

There are 3 components added on this function :

  1. Wait Group for synchronize all the goroutine. There are 3 parts for this function :
    - wg.Add to determine the number of goroutines executed
    - wg.Done to notify that the goroutine on which the method was called has completed. We declare it by defer, as you know in golang, any function which declared by defer will only be executed right before returning that function
    - wg.Wait is used for the blocking process, the program execution process will not be forwarded to the next line, before the previously declared number of goroutines have all finished

    On my example, function wg.Add(1) is called 3 times (for getting dataUserProfile , UserFamily , and UserTransportation), then wg.Done() must also be called 3 times, so that the process can continue to the next line.
  2. Mutex for making the data can only be consumed (read / write) by one goroutine, so it helps us to avoid race condition. There are 2 parts for this function :
    - lock is used to indicate that all operations on the line after the code will only have one goroutine that can do it at a time. If there are multiple goroutines executing concurrently, they must be queued.
    - unlock is used to re-open operation access to previously locked variables

    On my example if there are more than one error (maybe error when called UserProfile and UserTransportation), the error will append to variable errs one by one.
  3. Goroutine ( go ) for calling the 3 functions (UserProfile , UserFamily , and UserTransportation) asynchronously.

There are no rules that make you have to create function like this. You can edit the function as good as you want, maybe if you want to declare wg.Add(3) on the first line, so you don’t have to declare wg.Add(1) one by one it doesn’t matter as long as the sum ofwg.Add is equal to wg.Done , or maybe you want to create a different error handling, no problem, your code is yours 👌.

Testing

Now, we want to test execution time by running the main file go run main.go on terminal. After apps already started, go to your browser and open this link :

http://127.0.0.1:8080/user/{user_id}

fill {user_id} by random number from 1–2288 (because we only create 2288 users) and it will return detail user data.

If you guys want to try time execution without asynchronous process you can checkout branch part 2, since i didn’t make toggle button configuration for enable / disable this (like redis cache in part 2). I purposely didn’t make toggle button because it can be duplicate the code (one way using asynchronous the other doesn’t), so i suggest you to checkout branch part 2 , you can check and compare execution time differences there.

Comparison Result

1. Time execution when not using goroutine (synchronous process)

print log when not using goroutine

2. Time execution when using goroutine (asynchronous process)

print log when using goroutine

Summary

For the comparison below we can conclude that, after we change the flow to asynchronous to get the data (profile, family, and transport), it can reduce the execution time. Before we used the asynchronous process, the whole process took an average of 1.4 ms, but after using asynchronous flow, the process was reduced to below 1 ms on average. It reduces execution time by ~30% for just a simple API. You can imagine if you had to call multiple complex endpoints and had to wait one by one (assume that there are no dependencies for each endpoint), maybe you could reduce them a bit more, so this flow would suit your use case.

However, it all depends on what use case you are dealing with. Please take a not, i have already mentioned it before that asynchronous process can only be implement where there are no dependency needed between each process, thus it can only be used on certain use case.

You can see and clone all of the code and data on my github, you can test it by yourself and edit it as what you want. I have create 4 branch on that github, the first one is master (the latest = already using asynchronous process), the second is for part 1 (indexing database), and the third is part 2 (caching redis), and the last is part 3 (asynchronous process), so you can check all the improvement for each part by your self. Happy reading all, Thank you! 😄