08 Mar

Want features? Just code

Somebody just sent an email on a user’s mailing list for an open-source scientific package entitled Feature foo: why is package bar not up to the task?” (names hidden to avoid pointing directly to the responsible of my wrath). To quote him:

Is there ANY plan for having such a module in package bar?? I think (personally) that this is a MUST DO. This is typically the type of routines that I hear people use in e.g., idl etc. If this could be an optimised, fast (and easy to use) routine, all the better.

As some one who spends a fair amount of time working on open source software I hear such remarks quite often. I am finding it harder and harder not to react negatively to these emails. Now I cannot consider myself as a contributor to package bar, and thus I can claim that I am not taking your comment personally.

Why aren’t package not up to the task? Will, the answer is quite simple: because they are developed by volunteers that do it on their spare time, late at night too often, or companies that put some of their benefits in open source rather in locking down a market. 90% of the time the reason the feature isn’t as good as you would want it is because of lack of time.

I personally find that suggesting that somebody else should put more of the time and money they are already giving away in improving a feature that you need is almost insulting.

I am aware that people do not realize how small the group of people that develop and maintain their toys is. Borrowing the figure below from Fernando Perez’s talk at Euroscipy, the number of people that do 90% of the grunt work to get the core scientific Python ecosystem going is around two handfuls:

Commits per contributor in various scientific Python packages, from Fernando Perez

I’d like to think that this recruitment problem is a lack of skill set: users that have the ability to contribute are just too rare. This is not entirely true, there are scores of skilled people on the mailing lists. The poster himself mentioned his email that he was developing a package. I personally started contribution not knowing anything about software development. I struggled, I did the grunt work like maintaining wikis, answer questions on mailing list, and writing documentation. These easier tasks were useful to the community, I think, but must importantly, they taught me a lot because I was investing energy in them.

If people want things to improve, they will have more successes sending in pull requests than messages on mailing list that sound condescending to my ears.
I hope that I haven’t overreacted too badly :), that email turned me on. That said, I am not sure that people realize how much they owe to the open source developers breaking their backs on the packages they use.
All credit for images goes to Fernando Perez

10 Responses to “Want features? Just code”

  1. Kamil Kisiel Says:

    There’s also the other piece of the puzzle: projects need to make it clear what the process is to contribute and make it as easy as possible. Building, testing, and submitting code should be painless in order to attract contributors. The core developers also need to be responsive to contributors and not let things linger for too long, otherwise it’s easy for contributors (especially new ones) to get demotivated.

    Fortunately many of the specific projects you mention in this post have pretty clear contribution guidelines :)

  2. LionKimbro Says:

    A lot of the time, package maintainers just say “no” no matter how many people send in working code. And then like princesses they cry out, “why are there no suitors?”

  3. gael Says:

    @LionKimbro: if the code is sent actually following the guidelines? Not in my experience. It is true that many people simply dump their code in a project and do not seem to worry about integration maintenance or documentation. Such code doesn’t get in.

  4. Quick Links March 9 | Planet3.0 Says:

    [...] I Want You for Scientific Python. [...]

  5. Jorgen Says:

    It must be tough to get these types of comments again and again. I sincerely believe that people who ask these kind of questions really do not understand how free open-source software development works. I think it’s OK to write a blog entry like this to make people understand that there are people behind the process who dedicate precious time and effort. Otherwise for them open source software development is like a black box that occasionally emits a new version of software they like to use.

  6. Brendan Says:

    Yes, quite - but they’re not being entirely useless, they’re telling you what they want. It may only be an edge case, but if you’ve got the choice to do a number of things, you can (choose to) be influenced by what the users are asking for. More often than not my users pass in silence. Getting any feedback at all actually helps me keep working on something.

  7. Chris2048 Says:

    But consider that the effort to fix a bug and the effort to ask a bug be fixed are different, also the risk of fixing a bug and it being rejected (and the emotional hit) versus the risk of asking and it not being fixed are also relevant. I also think many authors make it unreasonably hard to contribute such as that it is *much* easier for them to fix a bug than an outsider, measured in the time it takes to fix; This has nothing to do with procedures, but a little to do with how readable the code is, and a *lot* to do with how modular the code is (i.e. how much knowledge is required to fix a particular component). The fact that a small proportion of the community makes a lot of the commits doesn’t mean that they are the only ones who care to contribute, it could also mean that authors make it hard for anyone else *but* themselves to contribute.

  8. Pierre Raybaut Says:


    From a certain point of view, I found your post very comforting. Sometimes, I’m quite desperate by some emails from users believing I’m 100% of my time working on my open-source projects and that I should instantly answer to all their very specific requests (they often use expression of the kind you mentioned: “the MUST have feature”, or “without this feature, the software is not usable”…). They seem to forget how much it can cost for us to work on our spare time for nothing but the pleasure to contribute, being part of a community and eventually some gratitude and glory (!).
    But from time to time, I receive great supporting emails with nothing but kind words, and that is a great motivation.

    Anyway, keep up the good work, Gaël: your strong involvement will always be greatly appreciated.


  9. gael Says:

    Granted, but it simply doesn’t scale.

    However, your points are also the reasons why I fight hard nowadays in projects I contribute to to favor readability and simplicity of the codebase.

  10. gael Says:

    @Pierre Raybaut: Same to you Pierre, thanks for your efforts: I can tell you that they pay.

Leave a Reply