Our Functional Future or: How I Learned to Stop Worrying and Love Haskell

I'd been aware of functional programming and Haskell in particular for some time, but my attempts to learn it had been less than successful. The problem wasn’t lack of resources; there are some good ones out there, e.g., Learn You A Haskell For Great Good and What I Wish I Knew When Learning Haskell. Maybe it was the novelty of functional programming to me or my lack of dedication, but even after substantial effort, I still couldn't understand the first line of code in our Ludwig DSL compiler. It was disappointing.

Programming languages aren’t new to me. Having recently completed a Ph.D. in computational fluid dynamics, my "higher-level" languages were C, CUDA, and Fortran. I picked up Python and Go—two primary languages at Fugue—when I joined the team in 2014. Shortly after, several Haskell programmers came on board to design our Ludwig DSL and write its compiler. (Fugue’s users declare infrastructure using this new DSL.)

After attending the ICFP and CUFP in Vancouver, Canada, and participating in a week of on-site training by Well-Typed, I’ve become reasonably confident that, despite some acute challenges, the future of our industry will be tied to functional programming.

The Path to Adoption

The community is aware of the difficulty of adoption. Amanda Laucher gave the extraordinary CUFP keynote focusing on the battle for getting functional programming accepted in the industry.

Functional programming has won all the technical and theoretical battles (at least, most functional programmers think so!), but it’s losing the war on adoption. Pain points, left to the developer to sort out, include issues with tooling, documentation, learning resources, and a concise and uniform standard library. Simon Marlow's talk on Facebook's high-profile move to Haskell echoed these themes. The project required as much time developing tools and educational resources as it did building the actual product.

The rapid acceptance of Stack and Stackage does show the community's ability to create great solutions that solve developers’ problems. Specifically, Stack tackled two major problems with using Cabal, a system for building and packaging Haskell libraries and programs: (1) library compatibility through long term support releases, and (2) more robust sandboxing and caching to dramatically reduce project build times. Stackage gives a self-consistent LTS release, but for only 4-6 months. More tools like these and their continued improvement likely will have a positive impact on expanding Haskell’s use and on adoption of functional programming in general, but we’ve got to move forward assertively with their development.

In my post-conference efforts to learn, I worked to a general knowledge of Haskell when I started to port some of my smaller projects to it. I made progress and would show my Haskell co-workers my code. Their support was incredible, their advice focused and constructive. However, I found that a lot of mistakes kept coming back to this: not using the correct libraries. In particular, there were several libraries that implemented a printf-like function including Text.Printf on Hackage. Coming from Go's standard library, I expected the top search result with a canonical name to be what I was looking for. You can imagine my shock when I found out, not only was the Text.Printf library not used often or regarded as the best (that would be Text.Format), but it was not type safe! By formatting a string in my program, I was giving up much of the safety I thought I was gaining by using Haskell! Finding the right libraries and functions continues to be an issue, whether it's array and vector for arrays or the small army of fold functions scattered about.

There Are Some Guarantees in Life

Despite obstacles and hesitations, the benefits of functional programming are immense. The key is this: language guarantees make code more robust, help to mitigate errors, and eliminate entire classes of bugs.

A guarantee provided by a functional programming language is an assurance like "data is immutable." If you pass data into a function, you don't have to worry about that function modifying that data. It's impossible. One big class of bugs that occurs in other languages (Python, Go, Java, etc.) is that you'll pass in a data structure to a function and that function may change the data structure in an unexpected way.

In addition, many bugs can be traced back to missing or incorrect error handling code. Haskell has language features to eliminate this type of bug. Rather than handling errors, exceptions, and logic in the same piece of code, Haskell separates error from logic by placing them in contexts (more technically referred to as monads). Contexts completely describe how to process invalid input, freeing the logic to only worry about valid input.

In most languages, if a nullable value is being passed around, it is up to each function to check whether that particular value is null and take the appropriate action. If a function developer forgets this check, an error may go uncaught and produce undesirable results. John Carmack made salient points when he said:

everything that is syntactically legal, that the compiler will accept, will eventually wind up in your codebase. and that's why i [sic] think that static typing is so valuable, because it cuts down what can make it past those hurdles.

Mistakes and lapses will make it into your code base. With Haskell, a nullable value can be wrapped in a Maybe context, which indicates that there may be a value or Nothing. The functional nature of Haskell allows developers to write functions that act on values without worrying whether or not the values are actually there. The definition of Maybe provides the check and appropriate action wherever the Maybe context is used.

That last statement is incredibly powerful. Now instead of depending on multiple developers to handle corner cases uniformly across large codebases, the error handling is defined once for the context. The definition carries across every use of the context by every function without developers needing to worry.

Predefined contexts including Maybe for nullable values and Either for multiple types of values or error support can get developers quite far. Each of these contexts contains the complete description of how to handle the corner cases when a function is applied. It's easy to provide custom contexts as well.

With the contexts provided in Haskell and other functional programming languages, developers can eliminate a type of bug we find occasionally at Fugue. As much as we try to avoid it, we occasionally find an access to a None in Python or an ignored non-nil error in Go. Using contexts in a functional language eliminates this class of errors. A developer cannot forget to add error handling to a new piece of logic, because it has already been handled by the context.

At Fugue, our product aims to tame the challenges of distributed computing in the cloud, starting with Amazon Web Services. Dealing with eventual consistency and failure is the rule in our product, not the exception. Fugue requires a significant amount of error checking and handling when it interacts with remote services. We rely on discipline to limit the amount of code that touches AWS and to provide the proper error handling.

But what happens if we find AWS interactions scattered throughout our code in places they should not be? In Python and Go, we need to trust that each function avoids performing these interactions. There is no way to know if each does other than inspecting the source code. This creates huge headaches for a developer using this function, especially if that person doesn't know that it has an external (non-functional) dependency.

In Haskell, any external interaction is placed inside an IO context. IO is used in Haskell to show that the associated value is the result of a state mutation. The mutation can be internal (e.g., mutating a data structure, which is rare in functional programming) or external (e.g., file or network access). As with the other contexts, a developer can still write functions that aren’t concerned with the fact that the value came from IO. However, the results stay in the context. It becomes very difficult to hide the fact that a function has external dependencies—this can be done for those brave enough to have unsafePerformIO in a pull request. So, the possibility of a bug from a hidden mutation drops significantly in functional languages because of a context around mutations.

Fugue utilizes a microservices architecture making it easy to add new languages into the core product. I continue to write in Go and Python, but also am taking advantage of the guarantees of Haskell for development, with its easy, clean, safe code. The benefits of functional languages are clear; it's just a matter of making them more attractive to developers. We’ve got challenges to overcome, but I look forward to our functional future.