Drowning in a Sea of Complexity
When's the last time you looked a piece of code and went "ugggh, I don't want to try to figure out what this does, it's so complex?!" Yesterday? Today? The reason why you're reading a blog and not reading/writing code? ;)
We programmers reach for complexity like it’s goddamn instinctual.
Wait, let me back up. What is complexity?
According to Rich Hickey, it’s the interweaving/interleaving/braiding of things. For instance, if you have a function with “and” in the name, you’re complecting (adding complexity to) the function. It should be separate functions.
I just mentioned functions… not methods. Classes, for instance, complect data and operations on data (functions). According to the object-oriented tradition, this is a good thing. It provides encapsulation and a whole slew of other things. Sometimes this is necessary and sometimes not. When was the last time you wrote a class where the class was the only thing that operated on its internal state? You may have started out with good intentions for that class, but you know what they say about the road to hell.
But, where am I going with this? Complexity == bad, simplicity == good? Well, yeah, I doubt anyone is going to disagree with that, but let’s dive in a bit deeper.
How many things can you hold in your head at one time? A common accepted value is 7 +/- 2 [citation needed]. Great, but that’s for digits, integers, very simple concepts. How many variables can you keep in your head? How much state? How many classes operating on one another at the same time? I’m guessing a lot fewer than 7 unless you’re extraordinary and then you might hit 7. But the thing is, that’s not much different than hitting 2. It’s not an order of magnitude better. So what do we do?
“Well, then we reach for abstractions!” you, imaginary person on the other side of this conversation, say.
Great, abstractions. In theory they sound like the silver bullet. If I create a container that encapsulates a ton of other things, then I’ve reduced the number of things I need to keep track of. One problem though, abstractions tend to be leaky.
How many of them are leaky? Hah! Trick question! All of them! What do I mean by leaky though? It’s pretty simple; an abstraction leaks when you need to know something about the underlying implementation. That’s it. And that’s why abstractions can be so difficult to deal with. Needing to keep the underlying implementation details in your head on top of the local state and it’s no wonder we write buggy, broken code.
Two of my favorite abstractions to reach for in this discussion are SQL and ORM's. In theory, ORM's make your life soooo much easier! Instead of writing SQL you just use the framework's classes. Until you can't. Either due to performance or a complex business requirement, you have to drop down to SQL. Bam, the abstraction has leaked as is no longer relevant. And, since you know this is likely to happen in most use-cases for ORM's, why pick one up in the first place? Because it's easy and who doesn't want easy? In my opinion, a large number of "easy" abstractions get you nothing, all they do is kick the can a little further down the road. At some point, you’ve gotta pick up the can and figure out where it came from.
So, what do we do? My answer is look to the data that we’re schlepping around. When you boil down what developers do, we manipulate data. That’s it. Whether that data comes in the form of submitted form data, a stream of stock market prices, or a timer timing out. It’s all data. That’s it. Data.
So why are we building (leaky!) abstractions misdirections to hide the data from us? I spent two weeks, that I’m never getting back by the way, adding an extractor/mapper/persister to an existing microservice. This microservice was already consuming JSON, was already mapping other bits of the JSON, and was already persisting the mapped bits to a database.
TWO WEEKS. To persist an extra bit of the exact same JSON.
Maybe I’m a terrible dev, it’s possible. Or maybe we, as an industry, are drowning in complexity we added ourselves in the hopes that we’d be able to keep more things in our head at once. I’m sure we had great reasons at the time to reach for complexity in the name of "easy". Business constraints, feature requests, broken code, etc.
“No matter how far you have gone on the wrong road, turn back.” Turkish Proverb [citation needed]
Again, it all goes back to data. A new/old concept seems to be rising up again. Data Driven Development. Think about that for a minute. What it means is going back to our roots as developers. Moving and manipulating data. That’s all we do. And there’s no shame in doing that! Businesses pay us a lot of money to move data back and forth!
Instead of thinking about our programs as collections of objects, we should be thinking of them as data pipelines. Data flowing with steps transforming the data along the way.
This doesn’t only apply to code, by the way. It applies to architecture as well. Microservices are the extension of the object oriented principles to an architecture level. And here’s the real question, is all of the additional work and complexity really getting you anything? Or are you distributing your monolith and introducing network “stuff” as a potential hazard?
So, how do you start, imaginary other person in this conversation?
With simplicity. Keep things unentangled. If you're using an ORM and you're able to stop, do so. Learn SQL, learn what's going on in the database underneath you. Gain that knowledge and you'll know more than 75% of other developers [citation needed]. Re-examine the tools of your craft. If you're using a framework because it's the only one you know, maybe it's time to branch out. But whatever you do, write simply. Take the time to understand a problem before you reach for a keyboard. Only once you've understood the problem should you start writing. And when you do, do the simplest thing you can to solve the problem. Other developers (including future-you) will thank you.
For future listening...