I love open-source: for me it is great way to develop any product, to acquire new skills, to have fun and to make something useful for the community. I am not an open-source rock-star (at least not yet :D) but I have created and contributed to tens of projects (take a look at my GitHub profile). Some of them got a bit of attention like WorldEngine, JavaParser or EffectiveJava. I am also an avid open-source user: almost daily I have to choose some open-source program or library to use or to contribute to. So I evaluate open-source projects regularly. I am also lucky enough to be in touch with many open-source developers, some of which I have interviewed for this blog.
So I want to share a few aspects I think are fundamental for the success of open-source projects.
The basic principles
One issue I have seen is many open-source projects is the lack of time from the committers: someone get excited with a new idea, build a project, the project get some traction and after a while life happens and the amount of patch to reviews pile up, questions remain unanswered for a long time and the new features take forever to be implemented. I do not claim I have a solution for that but I think that we can alleviate the problem by:
- automate as much as possible: write documentation once instead to answer ten times the same question, run tests automatically instead of correcting the same errors over and over, etc.
- get more contributors to share the burden of maintining the project. To do that you need to create a “contributor-friendly” project
To translate these general principles in practice I think there are five aspects to consider.
1. Automated tests
I think automated tests are very important in general in software development but they are especially important when it comes to open-source because they offer a way to automatically evaluate a contribution. In all of my projects I use some tool for continuos integration: each time I receive a pull request the tests are automatically run and a symbol show clearly if all the tests pass with the new patch or something need to be fixed. It is great because it helps new contributors to get confidence. It also help contributors which have very few time to reduce the time spent in code reviews: tests already guarantee a minimum quality standard. In this way the human reviewer can focus on other aspects such as the architectural coherence and the respect of stylistic guidelines.
2. Documentation
I know, no one likes to write documentation but it is a necessity for a variety of reasons.
First of all it is a way to communicate decisions long after the discussions which led to them. Today I discuss with other contributors about some design decision in my project, I write it down and a new contributor joining six months from now can read it. This helps to mantain the architectural integrity of the project.
Then there are users who want just to take benefit from the project. Without documentation to read they are either never adopting your project or keep asking the same questions over and over.
It is important to understand that there are different kinds of documentation for different audiences. The two main ones are users and contributors. For users documentation I suggest using readthedocs, while for contributors I normally use some form of wiki. There are also specific formats for certain domains. For example java libraries should use javadoc like we do for JavaParser.
3. Reactiveness
A few days ago I received an e-mail about a bug I reported over three years ago for an Eclipse project.
It was the first comment I received to this issue: the problem is probably not present anymore, I cannot help verify that because in the meantime I stopped using Eclipse. This is because I got the first answer over three years after I reported the bug.
Providing a proper solution to a problem can be difficult and require an effort that we are able to spend immediately. However I think that it is vital to react to user requests, questions and bug reports in a timely manner, perhaps just acknowledging that we received their message and that we are going to look into it. If we do not do so the user will get the feeling of being facing a dead project, with no one available to help or develop the product.
I realize this is much easier to say than to apply and I admit there are issues which have been open for far too long in the projects in which I am involved. However sometimes just providing a short and partial answer can help, and a few times after a short answer from me the user itself provided a solution and eventually became a contributor.
In any case if you still reply after 2 years and a half there will be projects which do worse than you…
4. Community
If you are doing things right your open-source project should solve exactly one problem. Normally your project could be a useful piece in a larger solution to a larger problem, perhaps part of a tool-chain. By finding out which are other related projects you could find way to collaborate and benefit from each other, providing more value.
For example, we built this great world generator named WorldEngine and we realized that the maps generated could be used inside tiled games. So we found out that Tiled is a great editor for tiled maps which has its own format named TMX. There are libraries to manipulate TMX files for a lot of programming languages. We understood that writing an exporter to TMX for our world generator would permit to a lot of game designers to use our project, so we started working on it.
We discussed it on the Tiled forum and we got useful feedback.
Finding related projects and start collaborating can increase the value of all the projects involved and lead to mix contributors. It basically means new energies, new ideas and new users.
5. Be open to new contributions
When you start working on an open-source project you are very excited and it is amazing to see your baby growing. At that point you are very attached to it and you would like to maintain control on it, be sure that it evolves according to your ideas and remain really your project. I think instead you should involve other people as early as possible and give up as much control as possible. There are at least two reasons for that. The first one is that other people have great ideas too. If you do not give spaces to contributors for proposing ideas and improve your project it will remain constrained by your time and your energy. The second aspect is that you want your project to survive you. Hopefully you will not hit by a bus very soon but you could end up having less time to dedicate to the project because of a new job or perhaps because you start focusing more on some other open-source project. That is perfectly fine if someone else can keep the project alive as you step down.
I know it is hard to not be in control but I think it is necessary to grow your project. For example WorldEngine was born from merging two projects: lands, which was my little project, and WorldSynth, created by Bret Curtis. We merged our projects and started mantaining WorldEngine together. Eventually we received amazing contributions by other people. This is an incomplete list:
- Evan Sampson contributed the amazing implementation of the Holdridge life zones model and improved a lot the ancient-looking-map, biome, precipitation and temperature generators. Thanks a million!
- Ryan contributed the Windows binary version and discussed Lands on Reddit bringing a lot of users. Thanks a million!
- stefan-feltmann made Lands depends on pillow instead that on PIL (which is deprecated). This could also help when moving to Python 3. Thanks a million!
- Russell Brinkmann helped saving the generation parameters in the generated world (so that we can use it to generate the same world again, for example), improved the command line options and added tracing information (useful for understanding the performance of the various generation steps)
- Joshua Coppola implemented the satellite view. Thanks a lot, it looks gorgeous!
- Stephan made WorldEngine make heavy use of numpy, helping to speed up the generation. He also made world-generation much more reproducible and helped improve compatibility with Python 3.
It was possible because we were open to new contributions. Sure, WorldEngine is not my own little project where I control every single decision. It is owned by a community, a small community perhaps, but definitely a community able to produce something much better of what I could by myself.
Conclusions
These are just some aspects that I think we should consider when running an open-source project. I tried to stay on practical aspects and provide examples but I am sure there are many other aspects I did not touch. I would suggest to take a look at this interview on the problems with open-source and this post on making your project friendly for newcomers. This is very important because you are going to need help along the way.
I think that open-source projects are mostly run and used by professionals. Participation in open-source projects is becoming more and more a way to evaluate developers. While participating wins over non-participating I think that it is not enough: we should apply the same professionalism we apply to our daily job also to open-source projects. With the additional constraint of having a limited amount of time to split between several projects.
It is not easy and personally I see there are many ways in which I could improve. I just hope this list could help you and me to make our projects a bit better. I would love to hear your stories about open-source.
| Reference: | 5 key aspects for a successful open-source project from our JCG partner Federico Tomassetti at the Federico Tomassetti blog. |
