I think it’s safe to assume you’ve heard of environment variables. It’s such an intuitive idea. You put a bunch of variables in the .env file that lets you change the behaviour of your code depending on your environment.

Hi, I am Sameeran, a Backend Engineer at FamPay. My job is to solve problems, and bring the most unique solutions to the table. And that's the approach we're going to take through this blog. So let's dive deeper!

Environment Variables haven’t always been around… they came about in 1979 with the release of Unix version 7. But since then, they have become an integral part of every system.

via GIPHY

The Problem

There is a major drawback when it comes to environment variables:

Once the system starts… the variables are set…

When you start a server, the server picks the variables up from the env file and sticks with them. If you wish to change the value of an environment variable while the server is running, your only option is to restart the server. It’s pretty clear how this is an issue in production…

via GIPHY

So how DO you change the behaviour of your system live in production?

… this is where Config Variables come into play.

The Solution

What makes this idea so special is how NOT complex it is. And also how far you can take it to make your system ready to handle issues on its own, something we will cover in a later blog. But for now, let’s understand what they are…

The Requirements

Suppose you’re running a library management system, a classic...

via GIPHY

This system is a bit different from the usual. The librarian wants to be able to stop all books from being borrowed whenever she feels like it. Let’s see what our options are.

The Plan

via GIPHY

Alright, we need a way to let the librarian control whether books are allowed to be borrowed or not. A good option would be to store the value as a boolean, in-memory right? Every time a person attempts to borrow a book, just access it, see if borrowing is allowed, and respond accordingly.

That would be perfect… but not very scalable. Imagine this librarian goes on to manage multiple libraries… I’m not sure if that’s a thing, but let’s imagine it anyway… You will eventually reach a point where you would need multiple instances of your system to handle the now increased load.

Multiple instances… each with their memory… disconnected from all the others… and resets when the instance restarts. How do you maintain the same value everywhere?

So an in-memory solution is probably not the best idea. A better way to achieve this is to store this ‘switch’ in the database… access it when someone tries to borrow a book, and respond accordingly. You won’t have to worry about inter-instance communication or memory reset, or the change not propagating to all instances… none of it.

Well alright… but then where in the DB do we store this?

You create a new table… and maybe call it ConfigVariables

That may seem like overkill, but we’re not really working on library management systems run by only one person in real life either. Bigger systems have many such requirements where behaviour needs to be changed at will and on the fly.

The Execution

This is what the table could look like:

Let’s break it down:

• The name field is the human readable name. For our reference.
• The key field is for a unique key that needs to be assigned to every config variable. In case of BORROWING_ALLOWED that key is CV1.
• The value is the current value of the config variable. This is what we will change.

However, we need some help when trying to access these values.

There are a few caveats that we need to consider. For one, it is not necessary that we store boolean values only. In .env files, that’s not a problem, but in a database table, we need to define a type for the field.

Another is what if we have set no value for a variable. We need to be have default values for each variable. To avoid these problems, we create helper functions to read and write to these config variables instead of accessing them directly. Here they are:

And there it is. That’s our solution. All you need to do now is get the librarian an admin dashboard of some kind where she can press buttons and stop people from borrowing books to her heart’s content 😊.

The Cons

If at this point you’re getting the feeling that you can throw your .env file out the window… you should close the window. Even though Config Variables are very powerful, there are problems with them.

The biggest one being Security.

In larger systems, multiple maintainers deal with the admin panel, and not all of them are meant to have access to secrets. Secrets like API keys, or credentials to certain services. Moreover, such secrets rarely need to change at all. Which is why they must go into the .env file.

The other issue is extra DB calls.

Sure, one DB call does not sound like much, but we’re talking about one extra DB call per request in systems that receive thousands of requests per minute or maybe even more. To make matters worse, you cannot even cache these results, because they may change any time.

So no, Config Variables are not the solution to world hunger… and they’re definetly not a replacement for environment variables. They are a tool to help you make some of your environment variables be less problematic to change.

The Conclusion

It’s obvious now that Config Variables are indispensable to any system at scale. Even here at FamPay, they are integral. We use them in cases where we expect down-times from our dependencies and other APIs to temporarily pause our services and so on.

But their potential does not end here. See, one major difference between environment variables and Config Variables is that the latter is accessible programmatically. Meaning you can change it through your code… you can have the system control its value… without you having to do anything.

But that’s something we will explore in a later blog. So stay tuned!

via GIPHY

Found this article interesting and want to share your insights? Drop in an email at sameeran.b@fampay.in!