We should have some ways of coupling programs like garden hose - screw in another segment when it becomes necessary to massage data in another way. Doug McIlroy, Bell Labs – October 11, 1964
At Luminal, we're unabashed fans of Unix and the Unix Philosophy. Last year, we marked the 49th anniversary of the Unix pipe (|) and we'd be remiss not to mark this year's golden date.
You can revisit last year's #Pipeday post to learn more about why the | set the course for a standard software interface, as well as the accompanying modularity in computing. For the 50th anniversary, we thought it would be fun to turn it over to members of our growing team to find out what the | means to them.
Pipes mean simplicity and composability for me. Simple functions tend to be easy to reason about, write, test, and maintain. Composable functions can be strung together to create new functionality. The union of simplicity and composability means a system can be built from understandable and correct parts, which shortens development time and increases the likelihood of success.
Whenever someone talks about good tool design, the "Unix Philosophy" always comes up. The idea is that you have simple tools, where each tool does one thing and does that one thing well, and then you compose those tools to build complex tools. This design philosophy creates one of the most efficient and effective systems of computing for getting real work done. The brilliant innovation that made it all possible is the pipe. Having a simple mechanism to pipe one tool into another, into another, into another means that a small set of tools can be combined in limitless ways to quickly solve problems.
I vividly remember discovering the pipe while playing with an old Debian box where I learned how powerful computers really are. Piping programs together completely changed they way I started thinking about problems; [it] made me start looking for patterns in problems and decomposing them into steps that I could solve with a Unix pipeline. I know other people have had the same experience. Whether it's running
awk, or running an HTTP request through a CGI script, I think it's safe to say that pipes have shaped the way we use computers to solve problems, large and small.
To me, the pipe is an invitation to computing. With the pipe, any program written can immediately communicate with any existing program. The pipe really touches on the promise of computing. With the pipe, there is no difference between proprietary and free and open source, compiled and interpreted, or new and old.
Funny. Fifty years ago the monolith problem was already being solved - discrete, simple processes connected in different ways to solve many problems. Funny, because that's how we are thinking of microservices these days ...
About 15 years ago, I found a website that provided "free shell"; I had no idea what that was but I had heard you could do things in a "shell" that you couldn't do in Windows. When I logged in and was faced with a command line with no graphical elements, I was absolutely confounded and had no idea what to do. After reading about how to list files and other things, I found out about the Pipe. For the first time in my life I was exposed to the idea of composability, being able to combine parts and stack computation, like pieces of Lego, to build something greater. It was a fantastic feeling of power.
Pipes are proto-ReST. We don't need no stinkin' contracts to get things done! Imposing strongly typed and defined interfaces means building slow-moving systems, and for things like system calls, that's a good thing. For hacking around, it's not such a good thing. Programmers like to experiment, and experimentation leads to innovation. Pipes were the first technique I'm aware of that allowed this kind of loosely coupled integration approach. With ReST, this pattern has shown it can scale to the entire Internet. That's remarkable and shows that clear, simple ideas are best. Keep it simple, and people will play around. Playing around is grossly underrated.
The pipe symbol has been my close computing partner for so long, I cannot imagine working without it. The simple pipe symbol has enabled me to do tasks like convert my CD collection to MP3s and to parse system log files for errors at my job. It's easy enough to understand without a fancy computer science degree. The pipe symbol is a powerful tool that's very easy to teach to beginners in Unix or the MacOS terminal.
The pipe is a great symbol for a concept crucial to modern computing - composition. It's hard to imagine a world without composition now, but in such a world we would have only monoliths and sadness. With it, we have ... well, just about everything good that we have today.
There once was a long command,
Writing it was tough to withstand,
But when it was cut into pipes,
It went to and fro through the system - no gripes! Unix, you. are. the. promised. land!