The Other Value in Open Source
By James Moore
Many software developers (both new and old) suffer from a debilitating shortcoming. When presented with a new (to them) problem, they often say:
I can solve this
Then they set out to solve the challenge.
And Why is this bad?
Well, I’m not saying it’s necessarily bad, but it’s far from optimal. Let me explain by providing a bit of my history.
When I was 19 years old I joined the US Navy. College, at this point, wasn’t for me. I was a bit disillusioned by traditional education, and I wanted to do something different. I scored well on the ASVAB (an aptitude test) which allowed me enter the Naval Nuclear Training Program, the most mentally demanding military program. Honestly I didn’t know what I was getting into… This program was the single most challenging thing I’ve ever done and probably ever will do.
In two years of rigorous training I went from a fairly uneducated teenager to skilled operator of a nuclear power plant on a submarine. Think about that, at 21 years old I was operating one of the most complex machines ever created. Interestingly, I’d estimate the average age of the engineering department was probably around 25 years old, so the entire power plant was being operated and maintained by extremely young adults.
So how do you take a uneducated teenager, whose brain isn’t even fully formed, and teach them to operate a nuclear power plant?
A bit of military brainwashing (let’s ignore this for now), and lot’s of hard work… but hard work by itself won’t get you there, you must work hard and smartly.
I had to learn the background information for all systems I’d eventually be working on: Physics, Chemistry, Math, Metallurgy, Heat Transfer / Fluid Flow, Electrical Theory, Electronic Theory, Radiology… Then I needed to piece this background knowledge together to understand the various systems and how they worked together. The typical schedule during training included classroom training 7AM-> 5PM, then homework/studying which usually took 30-50 additional hours each week, plus the typical military stuff like Physical Training and Personal inspections and all sorts of other BS.
Obviously I wasn’t a scientist when I joined and I didn’t think like one... I struggled to think abstractly and see patterns (initially anyway). To get to where we needed to be in 2 years, I had to stand on the shoulders of those before me.
While learning, I didn’t solve problems per say, I simply learned from those who had spent countless days, months and years solving the hard problems before me. There was no time for me to solve a problem that had already been solved, I needed to understand the problem and study the solution quickly so I could move on to the next topic.
I gained a lot from this experience, but the biggest takeaway was: I learned how to learn fast by leveraging the knowledge created by those who came before me.
The “problem” often is the wrong problem
Software developers have a natural tendency or desire to tackle challenges we see as solvable. Doing this can be addictive as our brain often rewards itself with a small dose of dopamine. The problem is that we often solve problems that have already been solved.
Some of you may be thinking, when I solve a problem on my own I learn the subject extremely well. This is probably true, but the questions I ask you are: Is your solution optimal and was it worth it? In other words is your solution the best and was the amount of effort that went into solving the challenge worth the knowledge gained? I doubt it.
About The Brain
We like to think of ourselves as smart problem solvers who can think through a set of inputs and create a solution. This is sort of true, but it’s not the entire story and can be very misleading.
It turns out we aren’t actually that good at thinking. True thinking is slow and unreliable. In fact most of the time when we're “thinking”, we're really just engaged in memory retrieval. Remembering how we solved this particular problem in the past, or recognizing something abstract about the problem and solving it in terms of a similar problems we’ve solved or seen solved in the past.
Thinking is the hardest work there is, which is the probable reason why so few people engage in it. - Henry Ford
Memory our Tool of First Resort
It’s good to have a working understanding of how our brains work. The simplest and most helpful model I’ve seen breaks the process into 3 parts:
- Our environment provides inputs (problems that can be solved)
- Working memory (our consciousness)
- Long term memory
Working memory is the real bottleneck in our brains. I like to think of it as a small cache, we can only put some many things into our working memory before we have an overflow and productive thought grinds to a halt. Our brains don’t want to rely on working memory, it automatically draws from our long term memory when possible. Let me give you a few examples...
The Brain in Action
If you drive a car, think back to when you were first learning.. Several months ago I purchased a 34ft recreational vehicle. The wife and I decided to travel the country for a year, doing the digital nomad thing. Let me tell you, the first few days of driving this large vehicle, which btw was also pulling my VW, was extremely stressful. My mind was churning with thoughts like: Am I too far to the right or the left, do I have a large enough gap between me and the other vehicles, do I have enough room to make this turn, I sure hope the VW is still on the back… Initially my mind had no existing memory to rely on, however within a few days driving was much more pleasant. I didn’t need to use my working memory to drive, all those questions and concerns had been answered and stored in memory. I could mostly drive without actively thinking about it.
When working on a puzzle or game, you may not even recognize it, but you’re brain is categorizing moves as productive or unproductive and storing that knowledge in your long term memory. There was a popular game a few years ago named 2048, a simple game that became very popular and rather addictive. I played it for about a week after it first came out, eventually I figured out how to solve it and stopped playing. Recently I noticed the 2048 app on my phone and decided to play again. It took me a few games, to shake off the rust, but I solved it pretty quickly this time around. I didn’t have to re-solve the problem, my mind naturally pulled from its long term memory, almost unconsciously and I quickly solved it again.
One more example, when I was a child a friend of mine showed me a magic trick that was amazing, but didn’t tell me how he did it. This was before the internet, so I couldn’t just google it. I spent many hours trying to figure out how he did the magic trick, and I just couldn’t quite figure it out. After some serious persuasion, my friend finally showed me how to do the magic trick, it was so easy to understand once someone had shown me. To this day, I still remember this trick.
My point here is that there is an easy way and a hard way to learn things. The easy way is also the optimal way, why waste our time solving something that has already be done. We’re not any better off than we’d be if we just studied someone else's solution, and we’d save ourselves countless hours… Honestly, do you have time for that?
It is a profoundly erroneous truism, repeated by all copybooks and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them. - John Searle
So how do you learn smartly from those that came before you? There are a lot of obvious answers: Books, Blogs, videos… Those are all important parts of learning, but often they’re just presenting factual knowledge, not solutions to complex problems. It’s fairly easy to learn the component parts of a language / library / technology. The hard part is composing those pieces in an optimal way to solve actual problems.
There are trailblazers in our industry, those people who are solving new problems and creating knowledge. We know of these people, they are creating languages, frameworks, libraries and applications. They have spent countless hours thinking about problems, with a wealth of knowledge and experience at their disposal. The only way for a noobie to catch up to these people, and possibly even pass them, is to follow the paths they’ve made.
Where is the Path to Follow?
The path many of these trailblazers have left is Open Source Software, it’s the most valuable gift given to developers. To be clear, I’m not talking about the value of the software, I’m talking about the value of the source code to other developers. Studying other’s source code allows you gain knowledge in minutes or hours… Knowledge that took trailblazers days, months or even years to create. It’s literally developer gold, unfortunately it’s the gift that is often unrecognized and underutilized.
The Crazy Phenomenon of Open Source
For a bit of perspective, imagine that you had worked for weeks or months on a problem. Eventually you had a good working solution. Consider the immense value you’d place on your solution. Many people wouldn’t want to freely share what they had worked so hard to accomplish. In that regard, open source is bizarre and awesome. It’s a crazy phenomenon that you as a learning developer must take advantage of.
Genius is 1 percent inspiration, 99 percent perspiration. - Thomas Edison
To be a good developer, you must build up knowledge in your long term memory, as fast as possible... There is no other options! So read others source code, learn from them. Think about their code, keep an eye out for patterns. Memorize the gold nuggets you discover. Eventually you’ll be able to solve other problems by abstracting things you’ve learned in other areas, and applying that knowledge to the current problem.
P.S. Also thank those open source developers at every opportunity, they deserve our gratitude. They owe us nothing, we owe them much.