π—ͺπ—΅π˜† Medium π—±π—Άπ˜π—°π—΅π—²π—± 𝗑𝗼𝗱𝗲𝗝𝗦 𝗳𝗼𝗿 π—šπ—Ό.

π—ͺπ—΅π˜† Medium π—±π—Άπ˜π—°π—΅π—²π—± 𝗑𝗼𝗱𝗲𝗝𝗦 𝗳𝗼𝗿 π—šπ—Ό.

Β·

2 min read

Medium's feed was not that good a few years ago. The job of their recommendation engine was to show as many personalized stories to the user as they could.

In 2018, their codebase was entirely a NodeJS monolith, and their feed not rendering quickly enough. They were having thousands of stories and could only rank about 150 stories on their feed. Their end goal was able to source-recommend as many candidates as possible whenever a user goes to their platform.

So, they started writing a microservice from scratch and named it Rex.

Although NodeJS was a powerful tool, it wasn't the best fit for this kind of problem. The problem with Node is that it is single-threaded.

This single-threaded model works great when each request's computation is simple enough. But, fetching huge data to put a ranked feed together is a CPU-intensive task. Their heavy lifting tasks are:

  1. Sourcing the stories to rank

  2. Getting the data to rank stories

  3. Using different ranking services to order those stories effectively

Each of these tasks and subtasks in them has many separate IO calls. As IO operations take time, that time is utilized for other requests. When the main thread's control is given to other requests, it takes time making the previous request slower.

Enter Golang Medium chose Golang as the language for its microservice Rex. Golang works very well with CPU-intensive tasks. The vital role is played by GoRoutines. Splitting work into go-routines doesn't hog the CPU even for multiple requests.

This single reason is enough for them to write a new microservice in Go. Other reasons were that:

  1. Go makes writing efficient and clean code easier.
  2. Medium already had a few small microservices running in Golang and hence this decision is obvious in that case.

Although NodeJS is great, we can see why Go is being used by many for CPU-intensive use-cases.