Why I'm Stepping Back from Scalaz
Many of my readers know I have promoted and contributed to Scalaz for a little over two years now.
After many wonderful collaborations with some of the most talented Scala developers I know, I have decided to step back from Scalaz, and pull ZIO, my most significant work, into a separate organization—one that is not associated with Scalaz in any way.
I hope this post make it clear why I am doing this, and what it means for the future of functional programming in Scala.
Community Build Controversy
As most of the functional Scala community knows by now, all projects in the Scalaz organization, as well as some that depend on Scalaz, were recently removed from the Scala community build.
The community build is used as a regression suite for the Scalac compiler, and helps ensure that changes to the compiler do not break innovative or widely-used libraries in the Scala ecosystem.
When I was made aware of this event, I requested more information and asked for a reversal.
When it became clear to me there would be no reversal or official explanation, I made a final statement, which expressed my disappointment and desire for better community stewardship, even while assuming good-faith all around.
Ultimately, some good came out of the discussions:
- The build is now more faithfully rebranded as the Lightbend build;
- It is clear that which projects are included is entirely at the discretion of Lightbend employees, without input from the community.
- There is a chance that ZIO (which did not depend on Scalaz, but lived inside the Scalaz organization), will eventually make it back into the Lightbend build, helping protect against compiler regression.
While not the outcome I hoped for, these steps do represent progress toward more transparency, and I would hope only the Lightbend project be relocated to the Lightbend organization at some point in the future, to better convey the fact the build does not reflect what the Scala community uses or necessarily wishes to include in the build.
The Birth of ZIO
ZIO is a library for asynchronous and concurrent programming in Scala. While not dependent on Scalaz, the library has been hosted inside the Scalaz organization on Github, and has therefore been affected by the decision to remove all Scalaz projects from the Lightbend build.
ZIO was hosted inside Scalaz for historical reasons.
In early 2017, Vincent Marquez, a friend of mine and fellow functional Scala developer, asked me if I would be willing to contribute to the next version of Scalaz (Scalaz 8), which had a long tradition of pushing the boundaries of functional programming in Scala.
I said yes, and began looking for something interesting to work on.
For many years, I’ve been interested in how functional programming can improve asynchronous and concurrent programming. Having written several versions of Future, in several programming languages, and having authored the first version of Aff (the de facto asynchronous effect system for PureScript), working on the next-generation Scalaz effect system seemed like a good fit.
So I signed up to work on the Scalaz 8 IO monad.
I had heretical ideas about how such an effect type should be designed: I wanted built-in concurrency powered by fibers; I wanted resource-safety with iron-clad guarantees; I wanted fine-grained interruption (termed async exceptions in Haskell); and I wanted performance to radically improve on last-generation effect types.
While my ideas were extremely controversial at the time, I presented my work at Scala IO 2017 and Scale by the Bay 2017, and the reception was so overwhelmingly positive that it went on to permanently change the direction of the Cats IO data type and Cats Effect type class hierarchy.
Development of the Scalaz 8 effect system proceeded quickly, much faster than development of Scalaz 8 itself. Meanwhile, both Scalaz 7.x and Cats developers were interested in using the effect system even while still early in development. The demand was so high that other developers pulled the effect type out into a standalone project called ioeffect—which happened in about 24 hours and without any help from me.
When this happened, it became clear to me the best way to help the entire functional Scala community was to pull the effect system out of Scalaz 8, and make it a proper standalone project, without any dependencies on Scalaz or Cats, but with full support for both libraries in separate modules.
On June 11, 2018, ZIO was officially born.
ZIO Drifts from Scalaz
Over the coming months, ZIO got its own chat room on Gitter, its own set of contributors (most of whom did not use Scalaz, and some of whom used Cats!), and its own unique culture—which is overwhelmingly positive, supportive, and focused on mentoring and growing Scala developers.
ZIO was becoming its own thing, independent of Scalaz, even while it lived on within the same organization.
The Scalaz Connection
Having contributed to Scalaz, and worked (mostly on ZIO) within the Scalaz organization for more than two years, I can say unreservedly there are some amazing things about the Scalaz community.
Firstly, many of the people who work on Scalaz are very gifted, selfless individuals, who prize community service above all else. For example, Kenji Yoshida, Tomas Mikula, and others.
Secondly, there is a culture of technical excellence in Scalaz. Technical discussions are encouraged, even if they are critical. Contributors understand that they are not their code, and they welcome suggestions to improve their work.
Yet regardless of these cultural highlights, it is impossible to ignore the controversial history of the project. And at the center of this controversy is the project’s founder, Tony Morris.
Tony Morris is a long-time functional programmer, an accomplished teacher who has donated countless hours to developers, patiently teaching them everything from the elementary to the advanced (I remember him writing 50 lines of code to explain to me how List is a free monoid!).
Yet, while many (including myself) will attest to Tony’s generous assistance and mentorship over the years, there are others who have been offended or hurt by Tony’s communication style, or have found his relentless critiques of Scala to be emotionally draining.
I have witnessed Tony insult others on a number of occassions—not just critique their work, mind you, but engage in ad hominem. While, as far as I have seen, this was always in response to (more subtle and socially sanctioned) ad hominem, it does not mitigate the very real hurt caused by Tony’s actions.
The effects of this over time are twofold:
- Scalaz proper contributors are not easily offended and are much more direct (selection bias);
- A few people refuse to interact with the Scalaz project because they have been hurt or are afraid of being hurt.
Tony Morris is not the only person to scare developers away from Scala communities, of course. Some developers publicly refuse to contribute to Typelevel projects (and others, privately, so I am told), citing the actions of Lars Hupel or—before they “stepped down” from Typelevel—the actions of Travis Brown and Stew O’Connor, who still show up in the chat rooms and comment on project issues.
I’m not privy to everything, but as far as I can tell, Tony’s actions have been more publicly visible than those of Lars, Travis, or Stew, and have led to more public backlash.
Modern Day Scalaz
With the Scalaz mentorship program I led last year, the organization saw a huge influx of new contributors, who knew nothing about the history of the project or Tony’s communication style.
Most of the new contributors now work in the ZIO project, which is isolated from the main Scalaz project, but there are other Scalaz projects that work in their own silos (Parsers, Schema, etc.).
Although Tony Morris stopped contributing to Scalaz years ago (after he gave up on Scala), his ghost still haunts the halls of Scalaz to this day.
Tony is no longer present in the day-to-day, but he still shows up occassionally in the Scalaz chat room. Very rarely, he comments on issues in the Scalaz project. And, as he will likely do for many years to come, Tony continues to heavily criticize Scala on technical grounds.
For better or worse, the project founded by Tony Morris will probably always be associated with Tony Morris.
ZIO != Scalaz
I believe that every project and every community needs to be judged based on its own merits, and with the community build controversy, it has become clear to me this is not happening right now.
ZIO is not Scalaz, and I am not Tony Morris.
Many first-time contributors and users of ZIO have stated in public or in private that ZIO has one of the friendliest, most welcoming, and most supportive environments of any open source community.
The project has attracted more than a hundred diverse contributors from all around the world, some of whom chose ZIO to be their first contribution to the world of open source software.
I strongly believe that ad hominem will never help, but will make every situation worse. I believe that rather than trade barbs, leaders of a community should turn the other cheek (which may not be fair but it is noble), using empathy and nonviolent communication to heal and bind together.
It is not up to me to force my own personal communication philosophy on the open source projects of others. But it is up to me to follow my own path, and create the kind of communities I want to see in this world.
So in that spirit, and for both social as well as technical reasons, on April 29th I announced that ZIO would be leaving the Scalaz organization, and moving to a new ZIO organization on Github.
This is a brand new start for an exciting young project that will henceforth be judged on its own merits, for its own culture.
Life After Scalaz
With ZIO moving outside of Scalaz, I hope to continue to build upon the culture we have created, mentoring new generations of functional Scala developers, and building an environment where everyone feels supported and empowered to become the best version of themselves.
I hope also to retain the many positive aspects of the Scalaz culture, such as a commitment to technical excellence, and a willingness to hear technical critiques and act on them without ego (easier said than done!).
I hope to continue the Scalaz tradition of questioning dogma.
For example, unlike many of my functional Scala comrades, I believe the key to making functional programming in Scala feel natural is to embrace all of Scala’s features, including subtyping, intersection types, and variance (without compromising purity); and to reject many Haskellisms, such as monad transformers, which have poor ergonomics and performance in Scala.
I believe that functional Scala programmers need to roll up their sleeves and help make Scala even better at functional programming, rather than criticize from the side-lines (something I have been guilty of myself!).
I do not anticipate contributing significantly to the current versions of Scalaz or Cats. Rather, I hope, and will encourage, a unification between Cats and Scalaz—a single base of contributors all collectively working toward making functional Scala wildly successful in industry.
As you might suspect, I have a few (heretical) ideas about what a unified architecture could look like, and I think it quite likely these ideas will materialize, in an appropriate way, and at a suitable time.
Closing Thoughts
I’m very grateful for the chance to participate in the development of the Scalaz library—a great library that first made functional programming in Scala viable.
I’m grateful that many of the relationships I’ve made in the Scalaz community will continue to live on, even as I step back from the project to focus on the new ZIO organization, and surrounding ecosystem.
I wish both Scalaz and Cats contributors the best of luck helping programmers derive value from functional programming. And I want to express my sincere hope that we find a way to bring together the whole functional Scala community, with a culture that’s positive, inclusive, supportive, and professional.
With work and a little luck, I believe that we can heal past hurt, mend trust, build bridges not walls, and end up doing something amazing that will be remembered for a generation of programmers.
See you all online at @jdegoes or in the new ZIO Chat Room!
Addendum: I want to make it clear that Tony Morris and I have been and remain friends, and that despite different goals and philosophies, I highly respect his expertise and contributions to functional programming, and I welcome his positive contributions to any conferences, events, or projects I’m involved with, subject to the same criteria as anyone else. I also acknowledge in writing this post that it is an incomplete picture, and if you’re interested in the full story of why Tony communicates in the way he does, you should talk to Tony.