Minimal amout of code needed

It is no secret that I don't like dependencies. I don't like dependencies and I hate external dependencies. But there is something I hate even more: small and very small dependencies.

What qualifies as a small dependency?

Small dependencies are libraries and packages that can be replaced by less than 200 lines of code at the moment when it gets introduced in a project. The small dependency that I most often see are testing frameworks in C++ projects so I will go ahead and use them as example of widespread "bad practice"^[1].

Why "simple" won the race when working on SStorage?

SStorage is made of a bunch of different parts. We try to test those parts and verify our test coverage regularly.

For that we use a minimalist Makefile and a minimalist test framework (both combined are 124 lines of code), and the main reason for that is: we don't need more, and if we did, adding more would be extremely easy.

Let's compare that with Catch2: Catch2 is rather featurefull, actually, it has so many features we will almost never use them all. With 16k lines of code, it is quite a large codebase. Here is a small comparison of code size of frameworks:

Now what do I mean by accomodating new features? I mean "Is it obvious how to add a feature to the framework when you need it and do not have knowledge of the inner workings or the framework?", and my experience has it that learning on the internals of something larger than 5'000 lines of code is complex, takes time, takes effort, and is generally less obvious than reading 140 lines of code and understanding them. This means that by having a very small framework, one can learn all the subtleties of the framework with a single glance (or a few more depending on your screen size).

This is why our distributed database and cloud storage doesn't use one of the front-running frameworks for its testing but a simple and primitive one: easy to check, easy to modify.

Yet another "small" dependency

Dependencies do not come alone. Do you remember when I listed Boost.Test as being around 10k lines of code? Well, it just happens that Boost.Test has dependencies:

set(_boost_test_dependencies
  Boost::algorithm # 7.7k
  Boost::assert # 134
  Boost::bind # 4.8k
  Boost::config # 8k
  Boost::core # 3.7k
  Boost::detail # 1.5k
  Boost::exception # 2.1k
  Boost::function # 2.2k
  Boost::io # 803
  Boost::iterator # 4.1k
  Boost::mpl # 77k
  Boost::numeric_conversion # 3.2k
  Boost::optional # 3.1k
  Boost::preprocessor # 143k
  Boost::smart_ptr # 9k
  Boost::static_assert # 105
  Boost::type_traits # 13.8k
  Boost::utility # 4.6k
)

It actually has about 290'000 lines of code worth of dependencies^[2]. That is quite a lot of code. Lots of code under a lot of scrutiny, and maybe lots of code you don't feel as you already use most of Boost in your project, but I would not call that insignificant.

In other languages, ExpressJS is 14k lines of code, but with its direct dependencies downloaded (by that I mean all dependencies except npm and its dependencies) it is a whooping 344k lines of code split in the library and its 286 dependencies. The blog you are currently reading has a codebase of 144k lines of code, but wait, it has dependencies. And with its dependencies, that is 2'800'000 lines of code in the codebase. For your information, counting this many lines of code took 2 whole minutes on my computer.

I find it unacceptable for my software to sit on this many dependencies for features that are mostly unused. Writing distributed software is complicated enough, worrying about computer security is tiresome enough, but baby-sitting several million lines of code worth of dependencies for security reasons is too much.

Short conclusion

If 3% of my lines of code contain a bug, my 140 lines of code contain around 5 bugs. If Boost.Test code have 0.1% of their lines of code containing a bug, it contains 290 bugs! That means that if their code is 30 times less buggy than mine, it still contains 50 times more bugs!

What is a "not small" dependency?

Acceptable dependencies are hard to replace. Replacing an entire PostgreSQL driver is a complex task. Rewriting SQLite3 is not realistic (and testing it as much as the real deal? impossible).

Those are unavoidable dependencies.

They are not realistic to avoid entirely.

Libraries made to interface with things are generally unavoidable, whenever it is the operating system, complex file formats or network protocols, libraries to interface with hardware or, finally, libraries to do complex mathematical operations.

Dependencies in SStorage

The only dependencies currently in SStorage are internal dependencies. Those dependencies handle both network protocol writing tools, tools that may not be provided by an operating system or that can be useful in the absence of one. They also provide code that we integrate from other sources like cryptographic primitives.

The reason for this internal dependency is code reuse within the project as the software is split into several parts.

Conclusion

Dependency management and codebase management is the work of a developer. As much as dependency seem to make one "gain" time, careless dependency management costs time down the line in terms of maintenance, security and in terms of complexity of the codebase that finds itself expanded beyond the need.

I invite you to check the information at https://nekoit.xyz/, join us on Discord or Telegram, or follow me on Mastodon [email protected] to receive notifications of my followup posts.

---

1. I name them bad practice not because unit testing is terrible (it is actually amazing) but for reasons I explain here, bear with me. ↩︎

2. I never count the standard library of a programming language as a dependency since it is generally shipped with said language. ↩︎