The coding sprint will take place the 8 and 9 September at INRIA Saclay, near Paris, in the room K110 (building K).

For those who cannot make it, it will be possible to participate using the IRC chan (#scikit-learn on irc.freenode.net).

We will start at 9am (Paris time), and a sketch of the planning can be found here. In particular:

• More docs! we still need tutorials: features selection, model selection, cross-validation, etc..
• Make the pipeline object really work + illustration in different contexts.
• Clean up and doc for bayesian approaches.
• Implementation of PCA (fit + transform).
• FastICA (adapt the CanICA code)
• LDA : Covariance estimators (Ledoit-Wolf) and add transform.
• Preprocessing routines (center, standardize) with fit transform.
• Anything that you have a particular interest in.

Do not hesitate to send on the mailing list some advices on this (incomplete…) list, and see you next week!

scikit-learn is a Python module for efficient and easy machine learning using scipy and numpy.

So I generated an SVG image with all the countries. It was generating by tracing a bitmap, so it has a lot of imperfections, but being an SVG with each (major) country as a different object, it can be used to create a colored-code world map. I am uploading it here under a public-domain license. Enjoy!

My stance on code design

I call code design the process of designing the architecture of a piece of software: what are the objects it uses? how do they interact? how is the information passed around?…

My view of code design and software engineering has progressively evolved to favor extreme simplicity over sophistication. I believe that a good programmer should know design patterns, powerful language features, libraries dark corners, and not use them unless absolutely necessary.

Some rules of thumb

Here are some rules that I apply nowadays when writing code that I would like to last (I am aware that some of them go against well-advertised best practices).

• Keep it as simple a possible, really! Experimental results have shown that the tractability of a code base goes down as the square of the number of interactions, and thus much quicker than the number of lines in a project. Each time you add a line, think about it: can you make simpler? If not you’ll have to find resources to maintain your project as fixing bugs or adding features will grow harder.
• Design for the 80% usecases. In the same vein, a small decrease in the requirements can make your project much simpler [Woodfield1979]. Corner cases and minor usecases should not make the whole project complex and hard to maintain. If you can, give up on what is bringing in complexity. If you cannot, isolate it, and don’t let it sit at the core of your design.
• Don’t design for the future. Again the same core idea: don’t start planing for all the usecases, and all the difficulties that you haven’t encountered, you will most certainly design wrong, and chances are that you’ll add complexity that you do not use. Design simple, design cleanly and refactor as you go, based on concrete problems.
• Don’t be clever. Each time you do a clever trick, whoever has to read and maintain this code will have to understand it (that person may be you, in a few years). Chances are that they’ll get it wrong and start by loosing a lot of time.
• Repeating yourself may actually be OK. This is a case of practicality beats purity. Repeating code is really a bad thing in software design, because it leads to an increased number of lines to debug, and tends to hinder reusability. However, adding complexity in order to save a few lines of duplicated code will cost you more in the long run.
• Use objects sparingly. Object are great, but are they always need? An object with a single method eval can probably simply be implemented by a function. The limitation of objects is that they all have a different behavior. As a result, the users and maintainers of your codebase will first have to understand how all your classes interact before understanding your code. This also means that there is a lot of benefit in making many different classes that have the same interface.
• Avoid abstractions and levels of indirection. The more levels of code piled on top one of the other, the more layers your maintainer is going to have to inspect to find were the bug might be. An abstraction hides another object or algorithm. To debug code, chances are that all the black boxes will first have to be opened.

Coding for others to debug

“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” - Brian W. Kernighan

You may think that I am overemphasizing simplicity at the cost of functionality. Well, think about the future of your code. The net is full of unmaintained and abandoned code. If you want your project to grow and have a future, you will probably need people to help you. For a given purpose, the easiest the code is to read and debug, the more chances you will have to pick momentum.

• Titus Brown: Writing (Python) Code that Doesn’t Suck
• Peter Norvig: Teach yourself programming in 10 years
• Paul Stachour and David Collier-Brown: You Don’t Know Jack About Software Maintenance
• Greg Wilson: Software carpentry: a course in software engineering

This is why we would like to have people with different problems, applications, and backgrounds to pitch in.

It will be a two-days sprint. Everyone is welcome, so just fill in the doodle, so that we can choose the date?

And do not hesitate to suggest some topics that you would like to be addressed during the sprint, and to discuss them on the mailing-list!

Vincent Michel is organizing the sprint. If you have questions about the sprint, you are welcomed to contact me, but please do put him in Cc to.

There are many libraries around to specify what I call a ’signature’ for an object, in other words a list of attributes that define its parameter set. I have heavily used Enthought’s Traits library for this purpose, but the concept is fairly general and can be found eg in ORMs (Object Relational Mappers) or web frameworks.

Specification of this interface of parameters may be used to answer a variety of needs:

• Typing: in the case of an ORM, to generate UIs, or for better error management, it may be desirable to have some control on the types of certain attributes of an object. In this case, specifying the signature corresponds to laying out a data model for the object.
• Reactive programming: using properties to react to changes to attributes, one can fully specify the API of an object in terms of these attributes. This gives a message-passing like programming style that can be very well suited to parallel-computing in particular because it can easily be made thread-safe.

Signatures for statistical learning objects

Recently, I considered the signature pattern in a new context. In the scikit-learn, we are interested in statistical learning. This entails fitting models to data and often tuning parameters to select a model that fits best (a problem called model selection). Each of our models is an object that implements a couple of key methods to fit to the data and to apply to new data (fit and predict).

The approach that we are currently taking for model selection is (more or less) to generate a list of models with different parameters and fit and test them on the data.

A very nice feature would be to find out the parameters to vary simply by inspecting the objects, and such a desire recently got us discussing of defining signatures for our objects. I must confess that I am a bit weary as this means either depending on a signature library, or building one. We don’t want to grow our dependencies, and most signature-definition code that I know involve meta-programming tricks to avoid code duplication.

Solving the simple problem: avoiding type checking

Today, I had to bite the bullet, because we were in a situation in which we had to instantiate new models from the existing one during model selection. For technical reasons, using a copy.copy to create these new models was not a great idea, and it was better to have the minimal list of parameters required. Here come signatures again.

After a bit of messing around with the code, I realized that typing information was useless, and most probably harmful, to our immediate goals and that I just needed the names of the relevant attributes. I finally settled down to the following solution (which might still change):

• All parameters need to be specified as keyword arguments of the __init__. The __init__ may not have positional arguments or ‘*’ arguments. Attributes on the objects have the same names as the __init__ parameters.
• A simple base class, with couple of methods relying on a simple use of the inspect module to find the signature of the __init__.

class BaseEstimator(object):
    @classmethod
    def _get_param_names(cls):
        args, varargs, kw, default = inspect.getargspec(cls.__init__)
        assert varargs is None, (
            'scikit learn estimators should always specify their '
            'parameters in the signature of their init (no varargs).'
            )
        # Remove 'self'
        args.pop(0)
        return args

    def _get_params(self):
        out = dict()
        for key in self._get_param_names():
            out[key] = getattr(self, key)
        return out

    def _set_params(self, **params):
        valid_params = self._get_param_names()
        for key, value in params.iteritems():
            assert key in valid_params, (’Invalid parameter %s ‘
                ‘for estimator %s’ %
                (key, self.__class__.__name__))
            setattr(self, key, value)

The full code can be seen here and adds a bit more features, such as a clever __repr__.

What I like about this solution is that it (almost) does not use metaprograming, and avoids code duplication without forcing any specific pattern on the developer subclassing BaseEstimator.

The next step

This approach solves my immediate problem, but not the bigger one of finding what values can the different parameters take when varied for model selection. Of course this second problem is much more complicated, and maybe it is not worth solving it: the framework could very easily be bringing in more problems than it solves.

However, it seems that a fairly easy way of specifying possible values for parameters would be to decorate the __init__, giving the possible parameters to be tested during the model selection:

    @cv_params(l1=np.logspace(1e-4, 1, 10))
    def __init__(self, l1=.5, fit_intercept=True)
	# ...

All the decorator has to do is to store the information in an attribute attached to the __init__ (and probably to check that the parameters it was given are valid arguments, in order to raise errors early). Methods on the class can later inspect this information for model selection, or GUI building (data-model specification will probably require some typing language, rather than a simple list of possible parameters).

Once again, here we would be avoiding the difficulty of specifying type information in a non restrictive way, but avoiding a problem that we don’t have to solve is probably a good idea.

Euroscipy in numbers

The attendance this year was huge: there was a grand total of 160 who came to EuroScipy, with 140 that came only to the tutorials, and 130 only the conference. This up by almost a factor of 3 compared to last year’s
EuroScipy, more than last year’s SciPy conference in Passadena, and almost as much as this year’s SciPy conference in Austin that hosted 180 person. We had people coming from 16 country, and as far as New Zealand, the US, or Turkey. Research lab, education, and industry (small to large companies) were all well represented, with approximately a third of the delegates coming from the industry. Similarly, many different scientific field were discussed, ranging from landscape ecology to pure math.

There were 2 tutorial tracks with 10 tutorial slots in each track. We had 2 keynotes from Hans Petter Langtangen and Konrad Hinsen. With regards to the contributed talks, the conference this year was highly selective. We received 52 propositions. We unfortunately could accept only 30 of them, which corresponds to an acceptance rate of 58%. Finally, we had 18 lightning talks.

A warm and friendly atmosphere

As an organizer, I was really pleased to find out how much people were relaxed and friendly. This certainly facilitates discussions during the breaks. And the ambiance was undoubtedly warm: 140 people with laptops in a room without air conditioning in the Paris summer :).

Of course during the evenings, many people met to continue the passionate discussions in restaurants and bars.

Trends I noticed

What one remembers from a conference is obviously biased by personal interests. With that disclaimer, here are the recurrent and important topics that I noticed, both in the talks, but also in the coffee break discussions:

• Parallel computing, in particular making it easy to do parallel computing.
  Konrad’s keynote had many interesting directions to explore. (talks: Playdoh, DANA).
• Code generation. In the various conferences I have been to recently, I heard much talking about symbolic manipulation of numerical problems to generate optimal computing kernels (talks: Efficient computation tutorial, Theano, Algorithmic Differentiation.
• Data management, with problems such as provenance tracking for reproducibility (talks: Sumatra, Knowledge management tutorial).

Finally installation problems of scientific tools were the subject of many discussions, as each year. One thing that I did notice, is that people stopped simply blaming each others and acknowledged that nobody knew how to fix the problem. Somebody even pointed out that installing any major scientific code was not a piece of cake. Hans Petter and others said that they had solved the problem by relying on a virtual machine and Ubuntu.

Konrad has also blogged, giving his own view of the conference.

Thanks

The conference could happen only because of the help of many people. First we need to thank our sponsors: Enthought, Python Academy, Pytables, and especially our host Ecole Normale Supérieure, which not only provided us with the rooms, but also made sure that everything was going well with the sound system, the projection, or the access to the building. With regards to organization and planing, Nicolas and I received a lot of help from Emmanuelle Gouillart.

On Thursday May 20th, at midnight Samoa time, we will turn off the abstract submission on the conference site. Up to then, you can modify the already-submitted abstract, or submit new abstracts.

We are very much looking forward to your submissions to the conference.

Gaël Varoquaux
Nicolas Chauvat

EuroScipy 2010 is the annual European conference for scientists using Python. It will be held July 8-11 2010, in ENS, Paris, France.
Links: Conference website,  Call for papers,  Practical information

The registration for EuroScipy is finally open.

To register, go to the website, create an account, and you will see a ‘register to the conference’ button on the left. Follow it to a page which presents a ’shoping cart’. Simply submitting this information registers you to the conference, and on the left of the website, the button will now display ‘You are registered for the conference’.

The registration fee is 50 euros for the conference, and 50 euros for the tutorial. Right now there is no payment system: you will be contacted later (in a week) with instructions for paying.

We apologize for such a late set up. We do realize this has come as an inconvenience to people.

Do not wait to register: the number of people we can host is limited.

An exciting program

Tutorials: from beginners to experts

We have two tutorial tracks:

• Introductory tutorial: to get you to speed on scientific programming with Python.
• Advanced tutorial: experts sharing their knowledge on specific techniques and libraries.

We are very fortunate to have a top notch set of presenters.

Scientific track: doing new science in Python

Although the abstract submission is not yet over, I can say that we are going to have a rich set of talks, looking at the current submissions. In addition to the contributed talks, we have:

• Keynote speakers: Hans Petter Langtangen and Konrard Hinsen, two major player of scientific computing in Python.
• Lightning talks: one hour will be open for people to come up and present in a flash an interesting project.

Publishing papers

We are talking with the editors of a major scientific computing journal, and the odds are quite high that we will be able to publish a special issue on scientific computing in Python based on the proceedings of the conference. The papers will undergo peer-review independently from the conference, to ensure high quality of the final publication.

Call for papers

Abstract submission is still open, though not for long. We are soliciting contributions on scientific libraries and tools developed with Python and on scientific or engineering achievements using Python. These include applications, teaching, future development directions, and current research. See the call for papers.

I am very much looking forward to passionate discussions about Python in science in Paris

This might not be a bad news, because it might mean that the conference will be completely free. This would mean that we would be able to provide lunch which is a pity as there is nothing like eating with a bunch of passionate experts to learn new tricks, but it would not hamper the conference in any other way, as the rooms are already booked and various little expenses covered.

If we manage to sort out payments in the next weeks, the fee should be 50 euros for the 2 days of tutorial, and between 50 and 100 euros for the full conference, depending on exactly what catering we offer.

Anyhow, we should open the registration real-soon, with or without payment. We will need to have some formal registration, as the number of people that can fit in the rooms will be limited.

All in all, with or without registration fees, it should be possible to make it to Euroscipy keeping expenses low: we have indicated a few cheap accommodation on the practical details page, and it is easy to get good food for a good price in the area.

Also, even if you are not interested in scientific research done using Python, the tutorials are a unique opportunity: we are having top-notch experts presenting with two tracks, one to get beginners up to speed and efficient in a couple of days, and the other for exploring advanced subjects. I know the speakers, and I can tell you that I won’t be talking in the corridor, but sitting with my laptop and listening to them. People pay large chunks of money for such training, usually.