Gaël Varoquauxhttps://gael-varoquaux.info/2023-11-27T00:00:00+01:00People underestimate how impactful Scikit-learn continues to be2023-11-27T00:00:00+01:002023-11-27T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2023-11-27:/programming/people-underestimate-how-impactful-scikit-learn-continues-to-be.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">François Chollet rightfully said that people often underestimate the
impact of scikit-learn. I give here a few illustrations to back his
claim.</p>
</div>
<p>A few days ago, François Chollet (the creator of Keras, the library that
that democratized deep learning) <a class="reference external" href="https://twitter.com/fchollet/status/1727186047115882624">posted</a>:</p>
<a class="reference external image-reference" href="https://twitter.com/fchollet/status/1727186047115882624"><img alt="Tweet from François Chollet: "People underestimate how impactful scikit-learn continues to be"" class="align-center" src="../programming/attachments/chollet_scikit_learn_impact.png" /></a>
<p>Indeed, scikit-learn continues to be the most popular machine …</p><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">François Chollet rightfully said that people often underestimate the
impact of scikit-learn. I give here a few illustrations to back his
claim.</p>
</div>
<p>A few days ago, François Chollet (the creator of Keras, the library that
that democratized deep learning) <a class="reference external" href="https://twitter.com/fchollet/status/1727186047115882624">posted</a>:</p>
<a class="reference external image-reference" href="https://twitter.com/fchollet/status/1727186047115882624"><img alt="Tweet from François Chollet: "People underestimate how impactful scikit-learn continues to be"" class="align-center" src="../programming/attachments/chollet_scikit_learn_impact.png" /></a>
<p>Indeed, scikit-learn continues to be the most popular machine learning in
surveys:</p>
<div class="figure align-center">
<a class="reference external image-reference" href="../programming/attachments/kaggle_survey_library_2022.png"><img alt="" src="../programming/attachments/kaggle_survey_library_2022.png" /></a>
<p class="caption">Most popular machine-learning framework, according to <a class="reference external" href="https://www.kaggle.com/kaggle-survey-2022">a Kaggle survey</a></p>
</div>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Scikit-learn is probably the most used machine-learning library</p>
</div>
<p>This popularity is sometimes underestimated as scikit-learn is a small player
in terms of funding and size of the team, in particular
compared to giants such as tensorflow and pytorch. Size is limited
by nature of the project, based on a community without a strong commercial
entity backing the project.</p>
<p>We target different technology than tensorflow and pytorch: we have
by design let the big players focus on deep learning, which demands much
more resources. Rather, we have focused on classic machine learning,
believing that it serves other important needs. While such technologies
make less the news, they are used a lot, and scikit-learn is massively
used:</p>
<table border="1" class="noborder docutils align-center">
<caption><strong>Usage statistics</strong> (from github)</caption>
<colgroup>
<col width="33%" />
<col width="33%" />
<col width="33%" />
</colgroup>
<tbody valign="top">
<tr><td><a class="reference external image-reference" href="https://github.com/scikit-learn/scikit-learn"><img alt="sklearn_header" src="../programming/attachments/scikit-learn_header.png" /></a></td>
<td><a class="reference external image-reference" href="https://github.com/pytorch/pytorch/"><img alt="pytorch_header" src="../programming/attachments/pytorch_header.png" /></a></td>
<td><a class="reference external image-reference" href="https://github.com/tensorflow/tensorflow"><img alt="tensorflow_header" src="../programming/attachments/tensorflow_header.png" /></a></td>
</tr>
<tr><td><a class="reference external image-reference" href="https://github.com/scikit-learn/scikit-learn"><img alt="sklearn_used_by" src="../programming/attachments/scikit-learn_used_by.png" /></a></td>
<td><a class="reference external image-reference" href="https://github.com/pytorch/pytorch/"><img alt="pytorch_used_by" src="../programming/attachments/pytorch_used_by.png" /></a></td>
<td><a class="reference external image-reference" href="https://github.com/tensorflow/tensorflow"><img alt="tensorflow_used_by" src="../programming/attachments/tensorflow_used_by.png" /></a></td>
</tr>
</tbody>
</table>
<p>By not focusing on deep learning, does scikit-learn risk to become
outdated? Surveys show that simple models such as linear models or models
based on trees (including boosting) are actually the most used models:</p>
<div class="figure align-center">
<a class="reference external image-reference" href="../programming/attachments/popular_ml_algorithm_2022.png"><img alt="" src="../programming/attachments/popular_ml_algorithm_2022.png" /></a>
<p class="caption">Most popular machine learning algorithm, according to <a class="reference external" href="https://www.kaggle.com/code/dhirajkumar612/kaggle-survey-2022-data-analysis">a kaggle
survey</a>
(apologies for the small fonts on the figure, I did not generate it)</p>
</div>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Gradient Boosted Trees is a good go-to model</p>
</div>
<p>There is a lot of hype surrounding deep learning, but it is most
often not the right tool do tackle tabular data. Tabular data has
different properties than images or text: it comes with heterogeneous
columns which make sense by themselves, and tree-based models have the
right inductive bias <a class="reference external" href="https://proceedings.neurips.cc/paper_files/paper/2022/hash/0378c7692da36807bdec87ab043cdadc-Abstract-Datasets_and_Benchmarks.html">[Grinsztajn et al 2023]</a>.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://proceedings.neurips.cc/paper_files/paper/2022/hash/0378c7692da36807bdec87ab043cdadc-Abstract-Datasets_and_Benchmarks.html"><img alt="" src="../programming/attachments/benchmark_tree_models.png" /></a>
<p class="caption"><strong>Benchmark comparing models on tabular data while tuning
hyper-parameters</strong> (from <a class="reference external" href="https://proceedings.neurips.cc/paper_files/paper/2022/hash/0378c7692da36807bdec87ab043cdadc-Abstract-Datasets_and_Benchmarks.html">Grinsztajn et al 2023</a>) Each value corresponds to the test score of the
best model (on the validation set) after a specific time spent doing
random search. The
ribbon corresponds to the minimum and maximum scores on these 15
shuffles.
Models HistGradientBoostingTree, GradientBoostingTree, and
RandomForest come from scikit-learn. FTtransformer, Saint, ResNet and
MLP are all deep learning architecture, with FT transformer and Saint
models specifically developed for tabular data.</p>
</div>
<p>As we can see, scikit-learn’s <a class="reference external" href="https://scikit-learn.org/stable/modules/ensemble.html#histogram-based-gradient-boosting">HistGradientBoosting</a> really shines in terms of good prediction performance for small computational costs. We strive to facilitate datascience: make it lightweight, give good documentation and APIs.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Linear models and tree-based models are there to stay. They answer strong
needs for many application settings and they come with small
operational cost.</p>
<p>In my opinion, where scikit-learn could really grow to be even more
relevant is to integrate better in a broader ecosystem going from
databases to putting to production, being more “enterprise ready” :).</p>
Comité de l’intelligence artificielle: vision et stratégie nationale2023-09-20T00:00:00+02:002023-09-20T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2023-09-20:/science/comite-de-lintelligence-artificielle-vision-et-strategie-nationale.html<div class="topic">
<p class="topic-title">English summary</p>
<p>I have been appointed to the government-level panel of experts on AI,
to set the national vision and strategy in France.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>J’ai l’honneur d’être <a class="reference external" href="https://www.gouvernement.fr/communique/comite-de-lintelligence-artificielle">nommé au comité de l’intelligence artificielle du gouvernement Français</a>.</p>
<p>La mission qui nous est confiée d’éclairer l’action publique …</p><div class="topic">
<p class="topic-title">English summary</p>
<p>I have been appointed to the government-level panel of experts on AI,
to set the national vision and strategy in France.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>J’ai l’honneur d’être <a class="reference external" href="https://www.gouvernement.fr/communique/comite-de-lintelligence-artificielle">nommé au comité de l’intelligence artificielle du gouvernement Français</a>.</p>
<p>La mission qui nous est confiée d’éclairer l’action publique autour de
l’intelligence artificielle, une technologie qui peut impacter beaucoup
d’aspects de la société.</p>
<p>Le comité comprend des experts de profils très variés, allant du
jeune entrepreneur à l’économiste connu mondialement. La difficulté va être de considérer
l’ensemble des liens entre progrès technologique et société. Nous allons
chercher à dégager de la vision, rassembler beaucoup d’expertises
d’acteurs différents sur
différents
sujets, appuyer nos projections sur l’état actuel des connaissances
scientifiques.</p>
<p>Je ne partagerai pas les travaux du comité en avance de phase: il y aura
un travail nécessaire pour établir du consensus, travail qui prend du temps.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Cette mission dépasse mon cadre habituel, celui de la recherche académique
ou de la création de logiciels. Je fais cela parce que je crois que pour
que la technologie ait le meilleur impact sur la société, il doit y avoir
un va-et-vient entre la création technologique et les changements
sociétaux. Si nous, scientifiques, décidons de nous concentrer uniquement
sur notre travail académique et technique, nous perdons le contrôle de la
façon dont la société adopte notre technologie; nous laissons ce contrôle
aux personnes qui décident d’utiliser leur énergie pour agir, influencer,
profiter directement de ces technologies. En tant que chercheur en sciences informatiques, travaillant à
la fois sur l’IA fondamentale et sur les applications dans le domaine de
la santé, je dispose d’une expertise qu’il est important d’apporter à la
table. En tant que fonctionnaire, je pense que je peux et que je dois
éclairer le débat : je suis moins exposé au risque de conflits
d’intérêts, je suis payé par l’argent public pour être utile au public.</p>
<p>Ce travail n’est néanmoins pas une prise de position politique: je suis
scientifique et non élu. Le pouvoir du comité n’est pas de faire les
décisions politiques, mais d’informer du possible. C’est un travail de
synthèse et de médiation.</p>
2022, a new scientific adventure: machine learning for health and social sciences2023-01-31T00:00:00+01:002023-01-31T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2023-01-31:/science/2022-a-new-scientific-adventure-machine-learning-for-health-and-social-sciences.html<p>A retrospective on last year (2022): I embarked on a new scientific
adventure, assembling <a class="reference external" href="https://team.inria.fr/soda/">a team</a> focused on
developing machine learning for health and social science. The team has
existed for almost a year, and the vision is nice shaping up. Let me
share with you illustrations of where we …</p><p>A retrospective on last year (2022): I embarked on a new scientific
adventure, assembling <a class="reference external" href="https://team.inria.fr/soda/">a team</a> focused on
developing machine learning for health and social science. The team has
existed for almost a year, and the vision is nice shaping up. Let me
share with you illustrations of where we are at. This is extracted from
our yearly report which will be public later, but I have sometimes edited
it a bit to add personal context.</p>
<div class="contents topic" id="highlights">
<p class="topic-title">Highlights</p>
<ul class="simple">
<li><a class="reference internal" href="#a-new-team-soda" id="toc-entry-1">A new team: Soda</a></li>
<li><a class="reference internal" href="#the-scientific-vision" id="toc-entry-2">The scientific vision</a><ul>
<li><a class="reference internal" href="#application-context-richer-data-in-health-and-social-sciences" id="toc-entry-3">Application context: richer data in health and social sciences</a></li>
<li><a class="reference internal" href="#applications-raise-specific-data-science-challenges" id="toc-entry-4">Applications raise specific data-science challenges</a></li>
<li><a class="reference internal" href="#our-research-axes" id="toc-entry-5">Our research axes</a></li>
</ul>
</li>
<li><a class="reference internal" href="#some-notable-results-of-2022" id="toc-entry-6">Some notable results of 2022</a><ul>
<li><a class="reference internal" href="#learning-on-relational-data-aggregating-across-many-tables" id="toc-entry-7">Learning on relational data: aggregating across many tables</a></li>
<li><a class="reference internal" href="#validating-probabilistic-classifiers-beyond-calibration" id="toc-entry-8">Validating probabilistic classifiers: beyond calibration</a></li>
<li><a class="reference internal" href="#reweighting-randomized-trials-for-generalization-finite-sample-error-and-variable-selection" id="toc-entry-9">Reweighting randomized trials for generalization: finite sample error and variable selection</a></li>
<li><a class="reference internal" href="#challenges-to-clinical-impact-of-ai-in-medical-imaging" id="toc-entry-10">Challenges to clinical impact of AI in medical imaging</a></li>
<li><a class="reference internal" href="#privacy-preserving-synthetic-educational-data-generation" id="toc-entry-11">Privacy-preserving synthetic educational data generation</a></li>
</ul>
</li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="a-new-team-soda">
<h2><a class="toc-backref" href="#toc-entry-1">A new team: Soda</a></h2>
<div class="figure align-right">
<img alt="" src="attachments/2022/team_2022.jpg" style="width: 400px;" />
<p class="caption">The team in early 2022 (it has grown a lot since)</p>
</div>
<p>At <a class="reference external" href="https://www.inria.fr/en">Inria</a>, we have teams assembling multiple
tenured researchers around a scientific project. Last year, we assembled
a new team called <a class="reference external" href="https://team.inria.fr/soda/">Soda</a>, which stands for
“social data”, but above all is a fun name.</p>
<p>In a year, the team grew like crazy (to be honest, this had been baking
for a little while). We are now around 25 people.
There are 4 PIs (Marine le Morvan, Judith Abécassis, Jill-Jênn Vie, and
myself); and the engineers working on scikit-learn at Inria are also part
of the team.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="the-scientific-vision">
<h2><a class="toc-backref" href="#toc-entry-2">The scientific vision</a></h2>
<p class="align-right"><em>Machine learning to leverage richer, more complex, data for
social-sciences and health</em></p>
<div class="section" id="application-context-richer-data-in-health-and-social-sciences">
<h3><a class="toc-backref" href="#toc-entry-3">Application context: richer data in health and social sciences</a></h3>
<p>Opportunistic data accumulations, often observational, bare great
promises for social and health sciences. But the data are too big and
complex for standard statistical methodologies in these sciences.</p>
<p><strong>Health databases</strong> Increasingly rich health data is accumulated
during routine clinical practice as well as for research. Its large
coverage brings new promises for public health and personalized medicine,
but it does not fit easily in standard biostatistical practice because it
is not acquired and formatted for a specific medical question.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Social, educational, and behavioral sciences</strong> Better data sheds new
light on human behavior and psychology, for instance with on-line
learning platforms. Machine learning can be used both as a model for
human intelligence and as a tool to leverage these data, for instance
improving education.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="applications-raise-specific-data-science-challenges">
<h3><a class="toc-backref" href="#toc-entry-4">Applications raise specific data-science challenges</a></h3>
<p><strong>Data management: preparing dirty data for analytics</strong> Assembling,
curating, and transforming data for data analysis is very labor
intensive. These data-preparation steps are often considered the number
one bottleneck to data-science. They mostly rely on data-management
techniques. A typical problem is to establishing correspondences between
entries that denote the same entities but appear in different forms
(entity linking, including deduplication and record linkage). Another
time-consuming process is to join and aggregate data across multiple
tables with repetitions at different levels (as with panel data in
econometrics and epidemiology) to form a unique set of “features” to
describe each individual.</p>
<div class="sidebar">
The <a class="reference external" href="https://project.inria.fr/dirtydata/">Dirty Data project</a> paved the way.</div>
<p>Progress in machine learning increasingly helps automating data
preparation and processing data with less curation.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Data science with statistical machine learning</strong> Machine learning can
be a tool to answer complex domain questions by providing non-parametric
estimators. Yet, it still requires much work, eg to go beyond point
estimators, to derive non-parametric procedures that account for a
variety of bias (censoring, sampling biases, non-causal associations), or
to provide theoretical and practical tools to assess validity of
estimates and conclusion in weakly-parametric settings.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="our-research-axes">
<h3><a class="toc-backref" href="#toc-entry-5">Our research axes</a></h3>
<div class="section" id="representation-learning-for-relational-data">
<h4>Representation learning for relational data</h4>
<p>I dream of deep-learning methodology for relational databases, from
tabular datasets to full relational databases. The stakes are <em>i)</em> to
build machine-learning models that apply readily to the raw data so as to
minimize manual cleaning, data formatting and integration, and <em>ii)</em> to
extract reusable representations that reduce sample complexity on new
databases by transforming the data in well-distributed vectors.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="mathematical-aspects-of-statistical-learning-for-data-science">
<h4>Mathematical aspects of statistical learning for data science</h4>
<p>I want to use machine learning models as non-parametric estimators, as I
worry about the impact of mismodeling on conclusion. However, for a given
statistical task, the statistical procedures and validity criterion need
to be reinvented. Soda contributes statistical tools and results for a
variety of problems important to data science in health and social
science (epidemiology, econometrics, education). These fields lead to
various statistical topics:</p>
<ul class="simple">
<li>Missing values</li>
<li>Causal inference</li>
<li>Model validation</li>
<li>Uncertainty quantification</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="machine-learning-for-health-and-social-sciences">
<h4>Machine learning for health and social sciences</h4>
<p>Soda targets applications in health and social sciences, as these can
markedly benefit from advanced processing of richer datasets, can have a
large societal impact, but fall out of mainstream machine-learning
research, which focus on processing natural images, language, and voice.
Rather, data surveying humans needs another focus: it is most of the time
tabular, sparse, with a time dimension, and missing values. In term of
application fields, we focus on the social sciences that rely on
quantitative predictions or analysis across individuals, such as policy
evaluation. Indeed, the same formal problems, addressed in the two
research axes above, arise across various social sciences:
<strong>epidemiology, education research, and economics</strong>.
The challenge is to develop efficient and trustworthy machine learning
methodology for these high-stakes applications.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="high-quality-data-science-software">
<h4>High-quality data-science software</h4>
<p>Societal and economical impact of machine learning requires easy-to-use
practical tools that can be leveraged in non-specialized organizations
such as hospitals or policy-making institutions.</p>
<p>Soda incorporates the core team working at Inria on <strong>scikit-learn</strong>, one
of the most popular machine-learning tool world-wide. One of the missions
of soda is to improve scikit-learn and its documentation, transfering the
understanding of machine learning and data science accumulated by the
various research efforts.</p>
<p>Soda works on other important software tools to foster growth and health
of the Python data ecosystem in which scikit-learn is embedded.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
</div>
<div class="section" id="some-notable-results-of-2022">
<h2><a class="toc-backref" href="#toc-entry-6">Some notable results of 2022</a></h2>
<p>I am listing here a small number of the achievements of the team, because
I find them inspiring.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="learning-on-relational-data-aggregating-across-many-tables">
<h3><a class="toc-backref" href="#toc-entry-7">Learning on relational data: aggregating across many tables</a></h3>
<p>For many machine-learning tasks, augmenting the data table at hand with
features built from external sources is key to improving performance. For
instance, estimating housing prices benefits from background information
on the location, such as the population density or the average income.</p>
<div class="figure align-right">
<img alt="" src="attachments/2022/aggregating.png" style="width: 300px;" />
<p class="caption">Often, data must be assembled across multiple tables into a single
table for analysis. Challenges arise due to one-to-many relations,
irregularity of the information, and the number of tables that may be
involved.</p>
</div>
<p>Most often, a major bottleneck is to <strong>assemble this information across
many tables</strong>, requiring time and expertise from the data scientist. We
propose <strong>vectorial representations of entities (e.g. cities) that capture
the corresponding information</strong> and thus can replace human-crafted
features. In <a class="reference external" href="https://link.springer.com/article/10.1007/s10994-022-06277-7">Cvetkov-Iliev 2023</a>, we
represent the relational data on the entities as a graph and adapt
graph-embedding methods to create feature vectors for each entity. We
show that two technical ingredients are crucial: modeling well the
different relationships between entities, and capturing numerical
attributes. We adapt knowledge graph embedding methods that were
primarily designed for graph completion. Yet, they model only discrete
entities, while creating good feature vectors from relational data also
requires capturing numerical attributes. For this, we introduce KEN:
Knowledge Embedding with Numbers. We thoroughly evaluate approaches to
enrich features with background information on 7 prediction tasks. We
show that a good embedding model coupled with KEN can perform better than
manually handcrafted features, while requiring much less human effort. It
is also competitive with combinatorial feature engineering methods, but
much more scalable. Our approach can be applied to huge databases, for
instance on general knowledge graphs as in YAGO, creating <strong>general-purpose
feature vectors reusable in various downstream tasks</strong>.</p>
<div class="figure align-center">
<img alt="" src="attachments/2022/entity_types_with_names.png" style="width: 100%;" />
<p class="caption"><strong>Entity embeddings of YAGO (wikipedia)</strong> (2D-representation using
UMAP). The vectors are downloadable from
<a class="reference external" href="https://soda-inria.github.io/ken_embeddings">https://soda-inria.github.io/ken_embeddings</a>} to readily augment
data-science projects.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="validating-probabilistic-classifiers-beyond-calibration">
<h3><a class="toc-backref" href="#toc-entry-8">Validating probabilistic classifiers: beyond calibration</a></h3>
<div class="figure align-right">
<img alt="" src="attachments/2022/grouping_loss.png" style="width: 360px;" />
<p class="caption">Validating probabilistic predictions of classifiers must go account
not only for the average error given an predicted score, but also for
the dispersion of errors.</p>
</div>
<p>Ensuring that a classifier gives reliable confidence scores is essential
for informed decision-making, in particular in high-stakes areas such as
health. For instance, before using a clinical prognostic model, we want
to establish that for a given individual is attributes probabilities of
different clinical outcomes that can be indeed trusted. To this end,
recent work has focused on miscalibration, <em>i.e.</em>, the over or under
confidence of model scores.</p>
<p>Yet calibration is not enough: even a perfectly calibrated classifier
with the best possible accuracy can have confidence scores that are far
from the true posterior probabilities, if it is over-confident for some
samples and under-confident for others. This is captured by the grouping
loss, created by samples with <strong>the same confidence scores but different
true posterior probabilities</strong>. Proper scoring rule theory shows that given
the calibration loss, the missing piece to characterize individual errors
is the grouping loss. While there are many estimators of the calibration
loss, none exists for the grouping loss in standard settings. In
<a class="reference external" href="https://arxiv.org/abs/2210.16315">Perez-Level 2023</a>, we propose an
estimator to approximate the grouping loss. We show that modern neural
network architectures in vision and NLP exhibit grouping loss, notably in
distribution shifts settings, which highlights the importance of
pre-production validation.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="reweighting-randomized-trials-for-generalization-finite-sample-error-and-variable-selection">
<h3><a class="toc-backref" href="#toc-entry-9">Reweighting randomized trials for generalization: finite sample error and variable selection</a></h3>
<div class="figure align-right">
<img alt="" src="attachments/2022/reweighting_trial.png" style="width: 360px;" />
<p class="caption">There may be a sampling bias between a randomized trial and the
target population.</p>
</div>
<p>Randomized Controlled Trials (RCTs) are ideal experiments to establish
causal statement. However, they may suffer from limited scope, in
particular, because they may have been run on non-representative samples:
some RCTs over- or under- sample individuals with certain characteristics
compared to the target population, for which one wants conclusions on
treatment effectiveness. Re-weighting trial individuals to match the
target population can improve the treatment effect estimation.</p>
<p>In <a class="reference external" href="https://hal.science/hal-03822662">Colnet 2022</a>, we establish the
exact expressions of the bias and variance of such reweighting procedures
- also called Inverse Propensity of Sampling Weighting (IPSW) - in
presence of categorical covariates for any sample size. Such results
allow us to compare the theoretical performance of different versions of
IPSW estimates. Besides, our results show how the performance (bias,
variance, and quadratic risk) of IPSW estimates depends on the two sample
sizes (RCT and target population). A by-product of our work is the proof
of consistency of IPSW estimates. Results also reveal that IPSW
performances are improved when the trial probability to be treated is
estimated (rather than using its oracle counterpart). In addition, we
study <strong>choice of variables</strong>: how including covariates that are not
necessary for identifiability of the causal effect may impact the
asymptotic variance. Including covariates that are shifted between the
two samples but not treatment effect modifiers increases the variance
while non-shifted but treatment effect modifiers do not.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="challenges-to-clinical-impact-of-ai-in-medical-imaging">
<h3><a class="toc-backref" href="#toc-entry-10">Challenges to clinical impact of AI in medical imaging</a></h3>
<p>I have worked for many years on research in computer analysis of medical
images. In particular, I am convinced that machine learning bears many
promises to improve patients’ health. However, I cannot be blind to the
fact that a number of systematic challenges are slowing down the progress
of the field.</p>
<p>In <a class="reference external" href="https://www.nature.com/articles/s41746-022-00592-y">Varoquaux & Cheplygina</a>, we tried to take
a step back on these challenges, from limitations of the data, such as
biases, to research incentives, such as optimizing for publication. We
reviewed roadblocks to developing and assessing methods. Building our
analysis on evidence from the literature and data challenges, we showed
that at every step, potential biases can creep.</p>
<ul class="simple">
<li>First, larger datasets do not bring increased prediction accuracy and
may suffer from biases.</li>
<li>Second, evaluations often miss the target, with evaluation error larger
than algorithmic improvements, improper evaluation procedures and
leakage, metrics that do not reflect the application, incorrectly chosen
baselines, and improper statistics.</li>
<li>Finally, we show how publishing too often leads to distorted incentives.</li>
</ul>
<p>On a positive note, we also discuss on-going efforts to counteract these
problems and provide recommendations on how to further address these
problems in the future.</p>
<p>This was a fun exercise. I realize that I still need to sit on it and
introspect how it has shaped my research agenda, because I think it has
pushed me to choose specific emphases (such as model evaluation, or
focusing on rich data sources).</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="privacy-preserving-synthetic-educational-data-generation">
<h3><a class="toc-backref" href="#toc-entry-11">Privacy-preserving synthetic educational data generation</a></h3>
<p>Soda also works on other applications than health, for instance
education. In this direction, I would like to highlight work in which I
did not participate, by Jill-Jenn Vie, another PI of the team.</p>
<p>Institutions collect massive learning traces but they may not disclose it
for privacy issues. Synthetic data generation opens new opportunities for
research in education. <a class="reference external" href="https://hal.inria.fr/hal-03715416">Vie 2022</a>
presented a generative model for educational data that can preserve the
privacy of participants, and an evaluation framework for comparing
synthetic data generators. We show how naive pseudonymization can lead to
re-identification threats and suggest techniques to guarantee privacy. We
evaluate our method on existing massive educational open datasets.</p>
<p>The tension between privacy of individuals and the need for datasets for
open science is a real and important one.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<hr class="docutils" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>This was just a quick glance of what we do at soda, and we are just
warming up. I am super excited about this research. I hope that it will
matter.</p>
<p>I truely believe that more and better machine learning can help health
and social science to draw new insight from new datasets.</p>
</div>
</div>
My Mayavi story: discovering open source communities2022-07-10T00:00:00+02:002022-07-10T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2022-07-10:/programming/my-mayavi-story-discovering-open-source-communities.html<p class="align-right"><em>The Mayavi Python software, and my personal history: A thread on
Python and scipy ecosystems, building open source codebase, and
meeting really cool and friendly people</em></p>
<img alt="" class="align-right" src="attachments/mayavi/mayavi_ets.png" />
<p>I am writing today as a goodbye to the project: I used to be one of the
core contributors and maintainers but have been …</p><p class="align-right"><em>The Mayavi Python software, and my personal history: A thread on
Python and scipy ecosystems, building open source codebase, and
meeting really cool and friendly people</em></p>
<img alt="" class="align-right" src="attachments/mayavi/mayavi_ets.png" />
<p>I am writing today as a goodbye to the project: I used to be one of the
core contributors and maintainers but have been inactive for a while for
lack of time. Out of common agreement, we recently removed my commit
rights to limit security risks.</p>
<p>Mayavi brought my so much!</p>
<div class="section" id="the-start-of-my-adventure-with-mayavi">
<h2>The start of my adventure with Mayavi</h2>
<img alt="" class="align-right" src="attachments/mayavi/example_magnetic_field_lines.jpg" />
<p>I got involved around 2007: I needed 3D visualization of magnetic fields as I was designing coils for my PhD <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>This led to an example in the Mayavi docs <a class="reference external" href="http://docs.enthought.com/mayavi/mayavi/auto/example_magnetic_field_lines.html">http://docs.enthought.com/mayavi/mayavi/auto/example_magnetic_field_lines.html</a></td></tr>
</tbody>
</table>
<p>I started as an early user of Mayavi2, a rewrite of Mayavi, and
eventually joined Prabhu Ramachandran and Enthought as a contributor.</p>
</div>
<div class="section" id="what-is-mayavi">
<h2>What is Mayavi?</h2>
<p>Mayavi is a scientific 3D visualization library in Python.</p>
<p>It enables interactive visualization to understand complex information in
3D, such as multi-physics fields, combined with <a class="reference external" href="https://docs.enthought.com/mayavi/mayavi/mlab.html">simple scripting</a> to integrate in a
broader scientific computing flow.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Mayavi was designed and founded around 2000 by Prabhu Ramachandran, a
researcher in computational fluid dynamics at IIT Bombay and long-time
open-source and Python figure.</p>
<p>The key idea was to make VTK, a powerful C++ visualization library,
easily useful with a Python interface.</p>
<p>Mayavi bridged the gap between the C++ data structures, and efficient Python data structures, exposing without copies to numpy arrays.</p>
<p>It uses tools from Enthought (namely the entought tool suite) for an
interactive GUI built on a Python object model: fully scriptable (the
vision in explained in <a class="reference external" href="https://hal.archives-ouvertes.fr/hal-00502548">an article Prabhu and I wrote</a> )</p>
<div class="figure align-right">
<img alt="" src="attachments/mayavi/mayavi_application.png" />
<p class="caption">Mayavi is a full-blown interactive application</p>
</div>
<div class="figure align-right">
<img alt="" src="attachments/mayavi/mayavi_mlab.jpg" />
<p class="caption">Mayavi is also a Python library, for full scripting</p>
</div>
</div>
<div class="section" id="working-on-mayavi-taught-me-code-and-communities">
<h2>Working on Mayavi taught me code and communities</h2>
<div class="figure align-right">
<img alt="" src="attachments/mayavi/mayavi_ipython.png" />
<p class="caption">Mayavi used within an interactive IPython – an image from the
<a class="reference external" href="https://ieeexplore.ieee.org/abstract/document/5725237">Mayavi paper</a></p>
</div>
<p>I joined to help with the “mlab” interface, for even simpler Python
scripting built upon functions. My goal was to make Mayavi natural to
matlab and matplotlib users, a product vision which was probably
important to push popularity even further.</p>
<p>I was an isolated PhD student in a physics lab, emboldened by a
discussion with Fernando Perez, I started contributing and discussing
with Prabhu Ramanchandran. I remember my first skype discussion with
Prabhu, I was very intimidated.</p>
<p>Understanding this large codebase was hard! And yet, slowly but surely, I
started making more and more meaningful contribution: on mlab, than on
the broader codbase, fixing bugs, a lot of work on documentation and
examples…</p>
<div class="figure align-right">
<img alt="" src="attachments/mayavi/scipy_conf.jpg" />
<p class="caption">Prabhu and myself are in this scipy conference group picture! From <a class="reference external" href="https://slideshare.net/enthought/scientific-computing-with-python-webinar-august-28-2009">https://slideshare.net/enthought/scientific-computing-with-python-webinar-august-28-2009</a></p>
</div>
<p>Then Enthought funded my overseas travel to the scipy conference: a big
deal for me, as I was a peniless PhD student.</p>
<p>My Mayavi story is that of meeting amazing people in the Python, scipy,
and pydata world; people who believe in building a tool stack to
democratize scientific computing; people from all over the world,
friendly, welcoming, passionate.</p>
<p>It founded my belief in communities.</p>
<p>This adventure led me to learn software engineering (<a class="reference external" href="https://software-carpentry.org/">Software carpentry</a> really helped getting started) to
work at Enthought (a software startup central to scientific computing in
Python), to change career from physics to computing, join Inria (French
national research in maths and computing), and I do other open source
projects…</p>
<p>Mayavi was crucial to my personal adventure. Thank you Prabhu! Thank you
Enthought! Thank you the Scipy community!!</p>
</div>
2021 highlight: Decoding brain activity to new cognitive paradigms2022-02-24T00:00:00+01:002022-02-24T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2022-02-24:/science/2021-highlight-decoding-brain-activity-to-new-cognitive-paradigms.html<p class="align-right"><em>Broad decoding models that can specialize to discriminate
closely-related mental process with limited data</em></p>
<div class="topic">
<p class="topic-title">TL;DR</p>
<p>Decoding models can help isolating which mental processes are implied
by the activation of given brain structures. But to support a broad
conclusion, they must be trained on many studies, a difficult problem
given …</p></div><p class="align-right"><em>Broad decoding models that can specialize to discriminate
closely-related mental process with limited data</em></p>
<div class="topic">
<p class="topic-title">TL;DR</p>
<p>Decoding models can help isolating which mental processes are implied
by the activation of given brain structures. But to support a broad
conclusion, they must be trained on many studies, a difficult problem
given the unclear relations between tasks of different studies. We
contributed a method that infers these links from the data. Their
validity is established by generalization to new tasks. Some
cognitive neuroscientists prefer qualitative consolidation of
knowledge, but such approach is hard to put to the test.</p>
</div>
<div class="section" id="context-infering-cognition-from-brain-imaging">
<h2>Context: Infering cognition from brain-imaging</h2>
<p>Often, when interpreting functional brain images, one would like to
conclude on the indvidual’s on-going mental processes. But this
conclusion is not directly warranted by brain-imaging studies, as they do
not control the brain activity, but rather engage the participant via a
cognitive paradigm made of psychological manipulations <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>. <em>Brain
decoding</em> can help grounding such <em>reverse inferences</em> <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>, by using
machine learning to predict aspects of the task.</p>
<p>But a brain decoding model can seldom support broad reverse-inference
claims, as typical decoding models are trained in a given study that
samples only a few aspects of cognition. Thus the decoding model only
concludes on the interpretation of the brain activity in the studies’
narrow scope.</p>
<p>Another challenge is that of statistical power. Most functional brain
imaging studies comprise only a few dozen subjects, compromising
statistical power <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>, even more so when using machine learning <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a>.
While there exists large acquisition efforts, these must focus on broad
psychological manipulations that do not probe fine aspects of mental
processes.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Poldrack 2006, <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1364661305003360">Can cognitive processes be inferred from
neuroimaging data?</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Poldrack 2011, <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S0896627311009895">Inferring Mental States from Neuroimaging Data:
From Reverse Inference to Large-Scale Decoding</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Poldrack 2017, <a class="reference external" href="https://www.nature.com/articles/nrn.2016.167">Scanning the horizon: towards transparent and
reproducible neuroimaging research</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>Varoquaux 2018, <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811917305311">Cross-validation failure: Small sample sizes lead
to large error bars</a></td></tr>
</tbody>
</table>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="contribution-informing-specialized-decoding-questions-from-broad-data-accumulation">
<h2>Contribution: Informing specialized decoding questions from broad data accumulation</h2>
<p>In <a class="reference external" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008795">Mensch 2021</a>,
we designed a machine-learning method that can <strong>jointly analyze many
unrelated functional imaging studies to build representations associating
brain activity to mental processes</strong>. These representations can then be
used to <strong>improve brain decoding in new unrelated studies</strong>, thus bringing
statistical-power improvements even to experiments probing fine aspects
of mental processes not studied in large cohorts.</p>
<p>One roadblock to accumulating information across
cognitive neuroimaging studies is that all probe different, yet related,
mental processes. Framing them all in the same analysis faces the lack of
universally-adopted language to describe cognitive paradigms. Our prior
work <a class="footnote-reference" href="#footnote-5" id="footnote-reference-5">[5]</a> on this endeavior –the quest for universal decoding across
studies–, relied on describing each experimental paradigm in an ontology
of cognitive processes and psychological manipulations. However, such
approach is not scalable. Here, rather, we infered the latent structure
of the tasks from the data, without explicitely modeling the links
between studies. In my eye, this was a very important ingredient of our
work, and it is non trivial that it enables improving the decoding of
unrelated studies.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-5">[5]</a></td><td>Varoquaux 2018, <a class="reference external" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006565">Atlases of cognition with large-scale human brain
mapping</a></td></tr>
</tbody>
</table>
<p>Capturing <em>representations</em> was key to transfering across study:
representations of brain activity captured distributed brain structures
predictive of behavior; representations of tasks across studies captured
decompositions of behavior well explained by brain activity. Of course,
the representations that we extracted were not as sharp as the stylized
functional modules that have been manually compiled from decades of
cognitive-neuroscience research.</p>
<p>From a computer-science standpoint, we used a deep-learning architecture.
This is the first time that we witnessed a
deep-learning architecture outperforming well-tuned shallow baselines on
functional neuroimaging data <a class="footnote-reference" href="#footnote-6" id="footnote-reference-6">[6]</a>. This success is likely due to the
massive amount of data that we assembled: as our method can
readily work across studies, we were able to apply it to 40000
subject-level contrast maps.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-6" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-6">[6]</a></td><td>There have been many reports of deep architectures on functional
brain imaging. However, in our experience, good shallow benchmarks
are hard to beat.</td></tr>
</tbody>
</table>
<div class="figure align-center">
<img alt="" src="attachments/2021_highlights/mston.png" />
<p class="caption">Our deep-learning architecture</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="a-research-agenda-that-does-not-win-all-hearts">
<h2>A research agenda that does not win all hearts</h2>
<p>Our underlying research agenda is to <strong>piece together
cognitive-neuroimaging evidence on a wide variety of tasks and mental
processes</strong>. In cognitive neuroscience, such consolidation of knowledge
is done via review articles, that assemble findings from many
publications into a consistent picture on how tasks decompose on
elementary mental processes implemented by brain functional modules. The
literature review and the ensuing neuro-cognitive model are however verbal
by nature: assembling qualitative findings. I, for one, would like to
have quantitative tools to foster big-picture view. Of course, the
challenge with quantitative approaches as ours is to capture all
qualitative aspects of the question.</p>
<p>Over the years that I have been pushing these ideas, I find that they are
met with resistance from some elite cognitive neuroscientists who see
them as unexciting at best. The same people are enthusiastic about new
data-analysis methods to dissect in fine details brain responses with a
detailed model of a given task, despite limited statistical power and
external validity. My feeling is that <strong>the question of how
various tasks are related is perceived as belonging to the walled garden
of cognitive neuroscientists, not to be put to the test by statistical
methods</strong> <a class="footnote-reference" href="#footnote-7" id="footnote-reference-7">[7]</a>.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-7" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-7">[7]</a></td><td><a class="reference external" href="https://journals.plos.org/ploscompbiol/article/peerReview?id=10.1371/journal.pcbi.1008795">The second round of review of our manuscript</a>
certainly felt as if the method was judged by cognitive-neuroscience
lenses, and not the validity of the data analysis that it entailed.</td></tr>
</tbody>
</table>
<p>Yet, as clearly exposed by Tal Yarkoni in his <a class="reference external" href="https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/generalizability-crisis/AD386115BA539A759ACB3093760F4824">Generalization crisis</a>,
drawing conclusions on mental organization from a few repetitions of a
tasks is at risk of picking up idiosyncrasies of the task or the stimuli.
A starting point of our work (<a class="reference external" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008795">Mensch 2021</a>)
was the fall of statistical power in cognitive neuroscience, documented
by <a class="reference external" href="https://www.nature.com/articles/nrn.2016.167">Poldrack 2017</a>, but
one reviewer censored this argument <a class="footnote-reference" href="#footnote-8" id="footnote-reference-8">[8]</a>. This exchange felt to me as <strong>a
field refusing to discuss publicly its challenges</strong>, which leaves no room for
methods’ researchers such as myself to address them.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-8" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-8">[8]</a></td><td><a class="reference external" href="https://journals.plos.org/ploscompbiol/article/peerReview?id=10.1371/journal.pcbi.1008795">Comments in the first review</a></td></tr>
</tbody>
</table>
</div>
Hiring an engineer and post-doc to simplify data science on dirty data2021-10-29T00:00:00+02:002021-10-29T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2021-10-29:/programming/hiring-an-engineer-and-post-doc-to-simplify-data-science-on-dirty-data.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Join us to work on reinventing data-science practices and tools to
produce robust analysis with less data curation.</p>
</div>
<p>It is well known that data cleaning and preparation are a heavy burden to
the data scientist.</p>
<img alt="" class="align-center" src="attachments/big_data_borat_cleaning_data.png" style="width: 400px;" />
<div class="section" id="dirty-data-research">
<h2>Dirty data research</h2>
<p>In the <a class="reference external" href="https://project.inria.fr/dirtydata/">dirty data project</a>, we
have been conducting machine-learning research …</p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Join us to work on reinventing data-science practices and tools to
produce robust analysis with less data curation.</p>
</div>
<p>It is well known that data cleaning and preparation are a heavy burden to
the data scientist.</p>
<img alt="" class="align-center" src="attachments/big_data_borat_cleaning_data.png" style="width: 400px;" />
<div class="section" id="dirty-data-research">
<h2>Dirty data research</h2>
<p>In the <a class="reference external" href="https://project.inria.fr/dirtydata/">dirty data project</a>, we
have been conducting machine-learning research to see how better
statistical models could readily ingest non-curated data, and reduce the
need of data preparation for data science. We now have a growing
understanding of the problems, theoretical and practical, which lie
across statistical and database topics.</p>
<p>Machine learning leads to different tradeoffs than traditional
inferential statistics (because it can rely on more powerful model). For
instance, we now have a good understanding of the case of missing values:
in <a class="reference external" href="https://arxiv.org/abs/2106.00311">Le Morvan et al</a>, we showed that
with traditional methods, ignorable missingness <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> and “good”
imputation are important, but it turns out for prediction, flexible
predictors are what matters and they can work on any missingness
mechanism.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>“Missing at Random”, where missingness is independent of the
hidden values</td></tr>
</tbody>
</table>
<p>Similarly, we have made good progress on tolerating normalization errors
and typos. We find that rather to attempt to deduplicate the entries or
fix the typos, it is best to represent similarities and ambiguities to
a flexible learning algorithm. The simplest and most reliable methods are
implemented in the <a class="reference external" href="http://dirty-cat.github.io/">dirty-cat</a> library, to
facilitate the life of data-scientists</p>
</div>
<div class="section" id="reinventing-data-science">
<h2>Reinventing data science</h2>
<p>With this understanding (and even more exciting on-going research), we
want to revisit data science. Machine-learning can provide flexible
models for many usages of data science. Our goal is to use it to help
assembling and analyzing datasets while minimizing human efforts. For
this, we need tools that can answer typical data-science questions using
machine learning and starting from the raw data, often spread in multiple
files or multiple tables of a databases. Building these tools requires
data-science research, a new vision of data-science APIs, and careful
software crafting.</p>
</div>
<div class="section" id="join-us-in-this-adventure">
<h2>Join us in this adventure</h2>
<p>We have an <a class="reference external" href="https://project.inria.fr/dirtydata/team/">awesome team</a>,
with a great mix of people of different seniority, different expertise
(statistics, machine learning, databases, software engineering), sharing
offices with the <a class="reference external" href="https://scikit-learn.fondation-inria.fr/home/">scikit-learn at Inria</a>. But we have too many
exciting ideas, so we are growing this team.</p>
<div class="section" id="a-data-science-engineer-new-software-with-new-ideas">
<h3>A data-science engineer: new software with new ideas</h3>
<p>We are looking for someone with a background in data science or numerical
Python programming to join us, to help with designing a new data-science
library, evolving from <a class="reference external" href="http://dirty-cat.github.io/">dirty-cat</a>, and
to help with data-science experimentation for the research.</p>
<p>We like people who care about data, designing good tools, and have vision
about data science. We are happy to consider different level of
experience. Apply on <a class="reference external" href="https://jobs.inria.fr/public/classic/fr/offres/2021-04182">the job offer</a>.</p>
</div>
<div class="section" id="a-post-doc-researcher-science-joining-data-engineering-to-deep-learning">
<h3>A post-doc researcher: science joining data engineering to deep learning</h3>
<p>We will soon be announcing a post-doc position to join the team for
research in this scope. We are interested in questions around learning on
relational or tabular data, or learning data integration. We have plenty
of ideas to explore around embeddings in databases, learning to
aggregate, learning on sets, graph neural networks for databases, or
distributional matching for entity and schema alignment.
We expect to be borrowing tools (conceptual and practical) from deep
learning, but to blending them with techniques from data integration,
knowledge graphs, and databases.</p>
<p>The job posting will be out soon, but I am running out of the office
right now for vacations (work-life balance also matters to us).</p>
<div class="topic">
<p class="topic-title">Diversity is important</p>
<p><a class="reference external" href="https://project.inria.fr/dirtydata/team/">Our team</a> is not as
diverse as I would like it to be (though probably doing better than
typical computer-science team). We love diverse candidates. Do not
hesitate.</p>
</div>
</div>
</div>
Hiring someone to develop scikit-learn community and industry partners2021-09-14T00:00:00+02:002021-09-14T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2021-09-14:/programming/hiring-someone-to-develop-scikit-learn-community-and-industry-partners.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>With the growth of scikit-learn and the wider PyData ecosystem, we
want to recruit in the Inria scikit-learn team for <a class="reference external" href="https://recrutement.inria.fr/public/classic/en/offres/2021-04058">a new role</a>.
Departing from our usual focus on excellence in algorithms,
statistics, or code, we want to add to the team someone with some
technical understanding, but an …</p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p>With the growth of scikit-learn and the wider PyData ecosystem, we
want to recruit in the Inria scikit-learn team for <a class="reference external" href="https://recrutement.inria.fr/public/classic/en/offres/2021-04058">a new role</a>.
Departing from our usual focus on excellence in algorithms,
statistics, or code, we want to add to the team someone with some
technical understanding, but an eye for people dynamics. Are you
passionate about developing open-source communities for data science?
This job is a unique opportunity.</p>
<p class="last">The mandate will be on the one hand to develop the wider community
behind scikit-learn, on the other hand to foster the foundation’s
partnerships, as this is our funding.</p>
</div>
<div class="section" id="context-scikit-learn-inria-foundation">
<h2>Context: Scikit-learn @ Inria foundation</h2>
<div class="section" id="the-growth-of-scikit-learn">
<h3>The growth of Scikit-learn</h3>
<img alt="" class="align-right" src="../programming/attachments/scikit-learn-logo.png" style="width: 200px;" />
<p>Scikit-learn is used massively, from schools to major companies. It
underpins business-intelligence analysis or automates processes. Its
reliability is crucial for the enterprise. Its well-documented methods
help data-scientists run to valid analyses.</p>
<p>Scikit-learn has hugely grown and is still growing in terms of userbase
and expectation of quality. These days, the development team is large,
with many grass-root volunteering and some contributors spending a
sizeable fraction of their work time.</p>
<div class="figure align-center">
<img alt="" src="../programming/attachments/sklearn_website_access.png" style="width: 450px;" />
<p class="caption">Number of monthly website access</p>
</div>
</div>
<div class="section" id="scikit-learn-inria-foundation">
<h3>Scikit-learn @ Inria foundation</h3>
<p><strong>Birth of a foundation</strong>
To ensure reliable funding to a small core of scikit-learn developers, we
set up a foundation <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> a few years ago. The goal was to make sure that
we did not lose our experienced developers.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>See <a class="reference external" href="http://gael-varoquaux.info/programming/a-foundation-for-scikit-learn-at-inria.html">the motivating announcement</a> and the <a class="reference external" href="https://scikit-learn.fondation-inria.fr">website</a>.</td></tr>
</tbody>
</table>
<p><strong>Achieving sustainability</strong>
The resulting structure is set up to provide a career path to a few of
our core people. As a consequence, it is a French legal entity, acting as
an employer, funded via sponsorship agreement with a few
of major economic users of scikit-learn (check out <a class="reference external" href="https://scikit-learn.fondation-inria.fr">the list of our
sponsors</a>). The priorities of
the team are set <a class="reference external" href="https://scikit-learn.fondation-inria.fr/how-are-the-priorities-of-the-consortium-defined/">jointly between the sponsors and the open-source
community</a>. The setup is not without flaws, in particular it forces us to employ people <a class="reference external" href="https://www.inria.fr/en/centre-inria-saclay-ile-de-france">on Campus</a>, but it enables giving proper benefits to these contributors.</p>
<p><strong>The team</strong> The <a class="reference external" href="https://scikit-learn.fondation-inria.fr/people/">scikit-learn team at Inria foundation</a> currently comprises 4
very experienced developers. In addition, we have other sources of
funding –research projects, <a class="reference external" href="https://www.fun-mooc.fr/en/courses/machine-learning-python-scikit-learn/">the scikit-learn MOOC</a> –
that we use to create a larger team (currently 3 full-time positions).
Finally, various researchers on campus are heavily invested in
scikit-learn or related projects such as joblib. As a result, the amount
of technical skills is staggering.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Long story short, we want to add new DNA to this awesome team: someone
into peopleware as much as software.</strong></p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="mandate">
<h2>Mandate</h2>
<p>The goal of <a class="reference external" href="https://recrutement.inria.fr/public/classic/en/offres/2021-04058">the new position</a> is
to talk both to our wider open-source world and our corporate partners.
Both are crucial to fostering growth for scikit-learn.</p>
<p>The <a class="reference external" href="https://recrutement.inria.fr/public/classic/en/offres/2021-04058">official job posting</a>
doesn’t convey as well as I would like what is behind this position. I’m
probably to blame :).</p>
<div class="section" id="growing-our-open-source-community">
<h3>Growing our open-source community</h3>
<img alt="" class="align-right" src="../programming/attachments/herdingcats.jpg" style="width: 300px;" />
<p>As both the scikit-learn and the PyData community have grown,
communication becomes a bottleneck. There are so many little things to
make an open-source community productive: facilitating on-boarding,
dividing efficiently the workload, documenting well the decision making,
organizing fun sprints, making sure that issue triaging is efficient…</p>
<p>We are looking for someone passionate about open-source
communities and who wants to be herding such cats.</p>
</div>
<div class="section" id="increasing-our-corporate-visibility">
<h3>Increasing our corporate visibility</h3>
<p>Scikit-learn is one of the most used data-science tools. However, talking
to senior decision makers, their perception sometimes differs. Indeed, we
are competing for visibility with many powerful actors.</p>
<p>We must communicate beyond the open-source world to develop
a strong brand for scikit-learn. Good communication will help us find new
sponsors, a key ingredient of growth and sustainability for scikit-learn.</p>
<p>We need to communicate on our progresses and our actions, as people are
often surprised by the breadth of our contributions <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>.</p>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>for instance, the foundation team has contributed <a class="reference external" href="https://youtu.be/UVL4LFy8ch0?t=1437">improvements in
CPython itself</a> , maintains
<a class="reference external" href="https://github.com/cloudpipe/cloudpickle">cloudpickle</a> a central
component of the data ecosystem).</td></tr>
</tbody>
</table>
<p>As a foundation, we need to be transparent and accountable, which is
harder than it seems.</p>
</div>
</div>
<div class="section" id="a-good-fit">
<h2>A good fit</h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/randychiu/4602851011/"><img alt="One Man Band, CCby2.0 from randychiu" class="align-right" src="../programming/attachments/one_man_band.jpg" style="width: 250px;" /></a>
<p>We are looking for someone into open source, but also who likes writing
blog posts, social networks, organizing events, presenting scikit-learn,
and improving processes.</p>
<p>We believe that such a job is best done by someone who has some technical
interest in scikit-learn: good advocacy needs with good understanding.</p>
<p>Maybe this sounds daunting? Few people have all the skills, let alone the
experience. We are actually more <strong>looking for a passionate and promising
candidate</strong>, whatever the length of the resume. We believe that
<strong>talented people can learn</strong>, when they like what they do.</p>
<p>This is a job about open-source, for open source! It’s not a perfect job:
we have many administrative constraints in running the foundation, we are
paying ourselves less than a non-open-source job.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Apply now</strong></p>
<p>We are looking forward to your application. You can submit them on
<a class="reference external" href="https://recrutement.inria.fr/public/classic/en/offres/2021-04058">the official job offer</a></p>
</div>
</div>
2020: my scientific year in review2021-01-05T00:00:00+01:002021-01-05T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2021-01-05:/science/2020-my-scientific-year-in-review.html<p>The year 2020 has undoubtedly been interesting: the covid19 pandemic
stroke while I was on a work sabbatical in Montréal, at the <a class="reference external" href="https://www.mcgill.ca/neuro/">MNI</a> and the <a class="reference external" href="https://mila.quebec/">MILA</a>,
and it pushed further my interest in machine learning for health-care.
<strong>My highlights this year revolve around basic and applied data-science
for health</strong>.</p>
<div class="contents topic" id="highlights">
<p class="topic-title">Highlights …</p></div><p>The year 2020 has undoubtedly been interesting: the covid19 pandemic
stroke while I was on a work sabbatical in Montréal, at the <a class="reference external" href="https://www.mcgill.ca/neuro/">MNI</a> and the <a class="reference external" href="https://mila.quebec/">MILA</a>,
and it pushed further my interest in machine learning for health-care.
<strong>My highlights this year revolve around basic and applied data-science
for health</strong>.</p>
<div class="contents topic" id="highlights">
<p class="topic-title">Highlights</p>
<ul class="simple">
<li><a class="reference internal" href="#mining-electronic-health-records-for-covid-19" id="toc-entry-1">Mining electronic health records for covid-19</a></li>
<li><a class="reference internal" href="#machine-learning-for-dirty-data" id="toc-entry-2">Machine learning for dirty data</a><ul>
<li><a class="reference internal" href="#supervised-learning-with-missing-values-beyond-imputation" id="toc-entry-3">Supervised learning with Missing values: beyond imputation</a></li>
<li><a class="reference internal" href="#machine-learning-without-normalizing-entries" id="toc-entry-4">Machine-learning without normalizing entries</a></li>
</ul>
</li>
<li><a class="reference internal" href="#making-sense-of-brain-functional-signals" id="toc-entry-5">Making sense of brain functional signals</a><ul>
<li><a class="reference internal" href="#neuroquery-brain-mapping-any-neuroscience-query" id="toc-entry-6">NeuroQuery: brain mapping any neuroscience query</a></li>
<li><a class="reference internal" href="#a-high-resolution-brain-functional-atlas" id="toc-entry-7">A high-resolution brain functional atlas</a></li>
</ul>
</li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="mining-electronic-health-records-for-covid-19">
<h2><a class="toc-backref" href="#toc-entry-1">Mining electronic health records for covid-19</a></h2>
<p class="align-right"><em>Hospital databases are rich and messy</em></p>
<p><strong>Hospital databases</strong>
In March, we <a class="reference external" href="https://www.inria.fr/en/scikiteds-visualization-tool-monitoring-flow-sick-patients">teamed up with the hospital around Paris</a> that were suffering from a severe overload due to a new pathology,
covid-19. The challenge was to extract information from the huge
databases of the hospital management system: What were the characteristic
of the patients? How were the resources of the hospital evolving? In the
treatments that were empirically attempted, which were most efficient?</p>
<p>The hospital databases are hugely promising, because <strong>they offer at
almost no cost information on all the patients that go through the
hospital</strong>. As we were dealing with a conglomerate of 39 hospitals, this
information covers millions of patients each year: an excellent
epidemiological coverage.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Challenging data science</strong>
Our work was classic data science: we did a lot of data management,
crafting SQL queries and munging pandas dataframes to create data tables
for statistics and visualizations. We interacted strongly with the
hospital management and the doctors to understand the information of
interest. As we moved forward it became clear that behind each “simple”
question, there were challenges of statistical validity. We did not want
to produce a figure that was misleading. Typical challenges were:</p>
<ul class="simple">
<li>Information needed complicated transformations (such as following a
patient hoping across hospitals to capture the patient status)</li>
<li>Information was represented differently in the differently hospitals</li>
<li>Incorrect inputs prevented aggregation (such as erroneous entry data
after the exit date, or missing values)</li>
<li>The database had biases compared to the ground truth (simple oxygen
therapy acts more often unreported than complicated invasive
ventilation)</li>
<li>Censoring effects prevented the use of naive statistics (after 20 days
of epidemic outburst most hospital stays are short simply because
patients have entered the hospitals recently)</li>
<li>A lot of information was present as unnormalized text, sometimes in
long hand-written notes, full of acronyms and errors due to character
recognition.</li>
<li>The data were of course often a consequence of treatment policy (the
choices of the medical staff in terms of patient handling and
measures), and hence not directly interpretable in causal or
interventional terms.</li>
</ul>
<p>These challenges were very interesting to me, as they related directly to
my research agenda of <a class="reference external" href="https://project.inria.fr/dirtydata/">facilitating the processing of “dirty data”</a> (more on that below).</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Most of the work that we did was not oriented toward publication, but
rather to address urgent needs of the hospitals. Some scholarly
contributions did come out:</p>
<ul class="simple">
<li>Part of the extracted data are consolidated worldwide for medical
studies (<a class="reference external" href="https://www.nature.com/articles/s41746-020-00308-0">Brat et al, Nature Digital Medicine 2020</a>).</li>
<li>We used causal-inference methods to estimate the treatment effects of
HCQ with and without Azithromycin (<a class="reference external" href="https://www.medrxiv.org/content/10.1101/2020.06.16.20132597v1">Sbidian et al, MedRxiv 2020,</a>.</li>
<li>The data are used in follow up medical studies (eg associating
mortality and obesity <a class="reference external" href="https://onlinelibrary.wiley.com/doi/full/10.1002/oby.23014">Czernichow et al, Obesity 2020,</a> )</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Biomedical entity recognition</strong> A major AI difficulty in this work is
recognizing biomedical entities, such as conditions or treatments, in the
various texts. Coincidentally, we had been working on simplifying the
state of the art pipelines for biomedical entity linking. While this
research work was not used on the hospital data, because it was too
bleeding edge, it led to an AAAI paper (<a class="reference external" href="https://arxiv.org/abs/2012.08844">Chen et al, AAAI 2021</a>) on a state-of-the model for
biomedical entity linking that is much more lightweight than current
approaches.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="machine-learning-for-dirty-data">
<h2><a class="toc-backref" href="#toc-entry-2">Machine learning for dirty data</a></h2>
<p class="align-right"><em>Machine learning methods that can robustly ingest non-curated data.</em></p>
<p>The <a class="reference external" href="https://project.inria.fr/dirtydata/">Dirty Data project</a>, that we
undertook a few years ago, is really bearing its fruits.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="supervised-learning-with-missing-values-beyond-imputation">
<h3><a class="toc-backref" href="#toc-entry-3">Supervised learning with Missing values: beyond imputation</a></h3>
<p>The classic view on processing data with missing values is to try and
<em>impute</em> the missing values: replace them by probable values (or better,
compute the distribution of the unobserved values given the observed
ones). However, such approach needs a model of the missing-values
mechanism; this is simple only when the values are missing at random.
When have been studying the alternative view based on directly computing
a predictive function to be applied data with missing values.</p>
<div class="figure align-center">
<img alt="" src="attachments/2020_highlights/mnar_versus_mcar.png" style="width: 500px;" />
<p class="caption"><strong>Missing-values mechanisms</strong>: black dots are fully-observed data
points, while grey ones are partially observed. The left panel
displays a missing-at-random situation, where missingness is
independent of the underlying values. On the contrary, in a
missing-not-at-random situation (right panel), whether values are
observed or not depends on the underlying values (potentially
unobserved).</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><a class="reference external" href="http://proceedings.mlr.press/v108/morvan20a.html">Le Morvan et al, AIStats 2020</a> studied the
seemingly-simple case of a linear generative mechanism and showed that,
with missing values, the optimal predictor was a complex, piecewise
linear, function of the observed data concatenated with the
missing-values mask. This function can be implemented with a neural
network with ReLu activation functions, fed with data where missing
values are replaced by zeros and corresponding indicator features are
added.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>To go one step further, we noticed that the optimal predictor uses the
correlation between features (<em>eg</em> on fully-observed data) to compensate
for missing values.</p>
<div class="figure align-center">
<img alt="" src="attachments/2020_highlights/compensation_effects.jpeg" style="width: 700px;" />
<p class="caption"><strong>Compensation effects</strong>: The optimal predictor uses the correlation
between features to compensate when a value is missing.</p>
</div>
<p><a class="reference external" href="https://neurips.cc/virtual/2020/public/poster_42ae1544956fbe6e09242e6cd752444c.html">Le Morvan et al, NeurIPS 2020</a>
devise a neural-network architecture that efficiently captures these
links across the features. Mathematically, it stems from seeking good
functional forms to approximate the expression of the optimal predictor,
that can be derived for various missing-values mechanisms. A non-trivial
result is that a simple functional form can approximate the optimal
predictor under very different mechanisms.</p>
<div class="figure align-right">
<img alt="" src="attachments/2020_highlights/neumiss_nb_parameters.jpeg" />
<p class="caption"><strong>Better parameter efficiency</strong></p>
</div>
<p>The resulting architecture needs much less parameters (depth or width)
than a fully-connected multi-layer perceptron to predict well in the
presence of missing values. This, in turns, leads to better performance
on limited data size.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="machine-learning-without-normalizing-entries">
<h3><a class="toc-backref" href="#toc-entry-4">Machine-learning without normalizing entries</a></h3>
<p>A challenge of data management is that the same information may be
represented in different ways, typically with different strings denoting
the same, or related entities. For instance, in the following table, the
<em>employee position title</em> column contains such non-normalized
information:</p>
<blockquote>
<table border="1" class="docutils">
<colgroup>
<col width="13%" />
<col width="47%" />
<col width="40%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Sex</th>
<th class="head">Employee Position Title</th>
<th class="head">Years of experience</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>Male</td>
<td>Master Police Officer</td>
<td>23</td>
</tr>
<tr><td>Female</td>
<td>Social Worker IV</td>
<td>17</td>
</tr>
<tr><td>Male</td>
<td>Police Officer III</td>
<td>12</td>
</tr>
<tr><td>Female</td>
<td>Police Aide</td>
<td>9</td>
</tr>
<tr><td>Male</td>
<td>Electrician I</td>
<td>4</td>
</tr>
<tr><td>Male</td>
<td>Bus Operator</td>
<td>15</td>
</tr>
<tr><td>Male</td>
<td>Bus Operator</td>
<td>22</td>
</tr>
<tr><td>Female</td>
<td>Social Worker III</td>
<td>13</td>
</tr>
<tr><td>Female</td>
<td>Library Assistant I</td>
<td>3</td>
</tr>
<tr><td>Male</td>
<td>Library Assistant I</td>
<td>5</td>
</tr>
</tbody>
</table>
</blockquote>
<p>Typos, or other morphological variants (such as varying abbreviations)
often make things worse. We found many instances of such challenges in
electronic health records.</p>
<p>In a data-science analysis, such data has categorical meanings, but a
typical categorical data representation (as a one-hot encoder) breaks:
there are too many categories, and in machine learning, the test set
might come with new categories.</p>
<p>The standard practice is to curate the data: represent the information in
a normalized way, without morphological variants, and separating the
various bits of information (for instance the type of job from the rank).
It typically requires a lot of human labor.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="figure align-right">
<img alt="" src="attachments/2020_highlights/gamma_poisson_encoding.png" style="width: 600px;" />
<p class="caption">The original categories and their continuous representation on latent
categorical features inferred from the data.</p>
</div>
<p><a class="reference external" href="https://ieeexplore.ieee.org/abstract/document/9086128">Cerda & Varoquaux, TKDE 2020</a> give two
efficient approaches to encode such data for statistical analysis
capturing string similarities. The most interpretable of these approaches
represents the data by continuous encoding on latent categories inferred
automatically from recurrent substrings.</p>
<p>This research is implemented in the <a class="reference external" href="https://dirty-cat.github.io/stable/">dirty-cat</a> Python library, which is making
rapid progress.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="making-sense-of-brain-functional-signals">
<h2><a class="toc-backref" href="#toc-entry-5">Making sense of brain functional signals</a></h2>
<p class="align-right"><em>Turning brain-imaging signal into insights</em></p>
<p>Brain imaging, and in particular functional brain imaging, is amazing,
because it gives a window on brain function, whether it is to understand
cognition, behavior, or pathologies. One challenge that I have been
interested in, across the years, is how to give systematic sense to these
signals, in a broader perspective than a given study.</p>
<div class="section" id="neuroquery-brain-mapping-any-neuroscience-query">
<h3><a class="toc-backref" href="#toc-entry-6">NeuroQuery: brain mapping any neuroscience query</a></h3>
<p>Systematically linking mental processes and disorders to brain structures
is a very difficult task because of the huge diversity of behavior.</p>
<p>In <a class="reference external" href="https://elifesciences.org/articles/53385">Dockes et al, elife 2020</a> we used text mining on a
large number of brain-imaging publications to predict where in the brain
a given subject of study (in neuroscience, behavior, and related
pathologies) would report findings.</p>
<p>With this model, we built a web application, <a class="reference external" href="https://neuroquery.org">NeuroQuery</a> in which the user can type a neuroscience
query, and get a brain map of where a study on the topic is like to
report findings.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="a-high-resolution-brain-functional-atlas">
<h3><a class="toc-backref" href="#toc-entry-7">A high-resolution brain functional atlas</a></h3>
<p class="align-right"><em>Regions to summarize the fMRI signal</em></p>
<p>Atlases of brain regions are convenient to summarize the information of
brain images, turning them into information easy to analyse. We have long
studied the specific case of functional brain atlases, extracting and
validating them from brain imaging data. <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811920306121">Dadi NeuroImage 2020</a>
contributes a high-resolution brain functional atlas, DiFuMo. This atlas
can be browsed or downloaded <a class="reference external" href="https://parietal-inria.github.io/DiFuMo/">online</a>.</p>
<div class="figure align-right">
<img alt="" src="attachments/2020_highlights/difumo.jpg" style="width: 400px;" />
<p class="caption">The functional regions, at dimension 512.</p>
</div>
<p>The atlas comes with various resolutions, and all the structures that it
segments have been given meaningful names. In the paper, we showed that
using this atlas to extract functional signals led to better analysis for
a large number of problems compare to the atlases commonly used. We thus
recommend this atlas for instance to extract Image-Derived Phenotypes in
population analysis, where the huge size of the data requires to work on
summarize information.</p>
<div class="figure align-center">
<img alt="" src="attachments/2020_highlights/putamen_difumo.png" />
<p class="caption">The region capturing the right hemisphere putamen.</p>
</div>
</div>
</div>
Technical discussions are hard; a few tips2020-05-28T00:00:00+02:002020-05-28T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2020-05-28:/programming/technical-discussions-are-hard-a-few-tips.html<!-- Emma, Eliz, Rashema, Ralf Gommers to read this -->
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This post discuss the difficulties of communicating while developing
open-source projects and tries to gives some simple advice.</p>
</div>
<p>A large software project is above all a social exercise in which technical
experts try to reach good decisions together, for instance on github
pull requests. But communication is difficult, in …</p><!-- Emma, Eliz, Rashema, Ralf Gommers to read this -->
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This post discuss the difficulties of communicating while developing
open-source projects and tries to gives some simple advice.</p>
</div>
<p>A large software project is above all a social exercise in which technical
experts try to reach good decisions together, for instance on github
pull requests. But communication is difficult, in particular between
diverging points of view. It is easy to
underestimate how much well-intended persons can misunderstand
each-other and get hurt, in open source as elsewhere. Knowing why
there are communication challenges can help, as well as applying a few
simple rules.</p>
<img alt="" class="align-right" src="../programming/attachments/communication.png" style="width: 300px;" />
<div class="contents topic" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#maintainer-s-anxiety" id="toc-entry-1">Maintainer’s anxiety</a></li>
<li><a class="reference internal" href="#contributor-s-fatigue" id="toc-entry-2">Contributor’s fatigue</a></li>
<li><a class="reference internal" href="#communication-is-hard" id="toc-entry-3">Communication is hard</a></li>
<li><a class="reference internal" href="#little-things-that-help" id="toc-entry-4">Little things that help</a></li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>The first challenge is to understand the other’s point of view: the
different parties see the problem differently.</p>
<!-- TODO: put a few things in bold -->
<div class="section" id="maintainer-s-anxiety">
<h2><a class="toc-backref" href="#toc-entry-1">Maintainer’s anxiety</a></h2>
<div class="section" id="open-source-can-be-anxiety-generating-for-the-maintainers">
<h3>Open source can be anxiety-generating for the maintainers</h3>
<p>Maintainers ensure the quality and the long-term life of an open-source
project. As such, <strong>they feel responsible for any shortcoming in
the product</strong>. In addition, they often do this work because they care,
even though it may not bring any financial support.
But they can quickly become a converging point of anxiety-generating
feedback:</p>
<ul class="simple">
<li>Code has bugs; the more code, the more bugs. Watching a issue tracker
fill up with a long list of bugs is frightening to people who
feel in charge.</li>
<li>Given that maintainers are visible and qualified, they become the
target of constant requests for attention: from pleas to prioritize a
specific issue to solicitations for advice.</li>
<li>A small fraction of these interactions come as plain
aggressions. I have been insulted many times by unsatisfied
users. Each time, it hurts me a lot. My policy is to
disengage from the conversation, but I am left shaking and staring at
my computer in the evening.</li>
</ul>
<div class="side-hanging small sidebar">
<p class="first sidebar-title"><strong>Related writings</strong></p>
<p>Ralf Gommers discusses <a class="reference external" href="https://rgommers.github.io/2019/06/the-cost-of-an-open-source-contribution/">the cost of an open source
contribution</a>, from the point of view of the maintainer.</p>
<p>Ilya Grigorik suggests: <a class="reference external" href="https://www.igvita.com/2011/12/19/dont-push-your-pull-requests/">Don’t push your pull request</a>.</p>
<p class="last">Brett Cannon: <a class="reference external" href="https://snarky.ca/setting-expectations-for-open-source-participation/">Setting expectations for open source participation</a></p>
</div>
<p>The more popular a project, the more weight it puts on its maintainers’
shoulders. A consequence is that <strong>maintainers are tired</strong>, and can
sometimes approach discussions in a defensive way. Also, we may be plain
scared of integrating a code that we do not fully comprehend.</p>
<p>Open-source developers may even, unconsciously, adopt a simple, but
unfortunate, protection mechanism: being rude. The logic is flawless: if
I am nasty to people, or I set unreasonnable expectations, people will let me alone.
Alas, this strategy leads to toxic environments. It not only makes people
unhappy but also harms the community dynamics that ground the excellence
of open source.</p>
</div>
<div class="section" id="the-danger-abusive-gatekeeping">
<h3>The danger abusive gatekeeping</h3>
<!-- add a image of puppy? And a gate? -->
<p>A maintainer quickly learns that every piece of code, no matter how cute
it might be, will give him or her work in the long run, <a class="reference external" href="https://snarky.ca/setting-expectations-for-open-source-participation/#submittingacontribution">just like a puppy</a>. This
is unavoidable given that the complexity of code grows faster than its number of
features <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>, and, even for a company as rich as Google,
project maintenance becomes intractable on huge projects <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>.</p>
<div class="side-hanging docutils container">
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td><a class="reference external" href="https://ieeexplore.ieee.org/document/1702600">An Experiment on Unit Increase in Problem Complexity, Woodfield 1979</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>To quote tensorflow developers
<a class="reference external" href="https://github.com/tensorflow/tensorflow/pull/33460">“Every [code addition] takes around 16 CPU/GPU
hours of [quality control]. As such, we cannot just run every
[code addition] through the [quality control] infrastructure.”</a></td></tr>
</tbody>
</table>
</div>
<p><strong>A maintainer’s job is to say no often</strong>, to protect the project. But,
as any gatekeeping, it can unfortunately become an excercise in unchecked
power. Making objective choices for these difficult decisions is hard,
and we all tend naturally to trust more people that we know.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Most often we are not aware of our shortcomings, let alone are we doing
them on purpose.</p>
</div>
</div>
<div class="section" id="contributor-s-fatigue">
<h2><a class="toc-backref" href="#toc-entry-2">Contributor’s fatigue</a></h2>
<p>A new contributor starting a conversation with a group of seasoned
project maintainers may easily <strong>feel an imposter</strong>. The
new contributor knows less about the project. In addition, he or she is engaging
with a group of people that know each-other well, and is not yet part of
that <em>inner</em> group.</p>
<p>This person does not know the code base, or the conventions, and must <strong>make
extra efforts</strong>, compared to the seasoned developers, to propose a
contribution suitable for the project. Often, he or she does
not understand fully the reasons for the project guidelines, or for the
feedback given. Request for changes can easily be seen as trifles.</p>
<p>Integrating the contribution can often be a lengthy process –in
particular in scikit-learn. Indeed, it will involve not only shaping up
the contribution, but also learning the skills and discovering the
process. These <strong>long cycles can undermine motivation</strong>: humans need
successes to feel enthusiasm. Also, the contributor may legitimately
worry: Will all these efforts be fruitful? Will the contribution make its
way to the project?</p>
<p>Note that for these reasons, it is recommended to start contributing with
very simple features, and to seek feedback on the scope of the
contribution before writing the code.</p>
<p>Finally, contributors are seldom paid to work on the project, and there
is no single line of command that makes decisions and controls incentives
for all the people on the project. No one is responsible when things go
astray, which means that the weight falls on the shoulder of the
individuals.</p>
<!-- fun pictures, to relax atmosphere, but only later, first write and
review -->
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>The danger behind the lengthy cycle of reviews and improvements needed to
contribute is <strong>death by a thousands cuts</strong>. The contributor looses
motivation, and no longer finds the energy to finish the work.</p>
<div class="grey docutils container">
<p><strong>How about users?</strong></p>
<p>This article is focused on developers. Yet, users are also an
important part of the discussion around open source.</p>
<p>Often communication failures with users are due to frustration.
Frustration of being unable to use the software, of hitting a bug, of
seeing an important issue still not addressed. This frustration stems
from incorrect expectations, which can often be traced to
misunderstanding of the processes and the dynamics. Managing
expectations is important to improve the dialogue, via the
documentation, via notes on the issue tracker.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="communication-is-hard">
<h2><a class="toc-backref" href="#toc-entry-3">Communication is hard</a></h2>
<p>Communication is hard: messages are sometimes received differently than
we would like. <strong>Overworked people discussing very technically
challenging issues</strong> only makes the matter worse. I have seen people not
come across well, while I know they are absolutely lovely and caring.</p>
<p>We are human beings; we are limited; we misunderstand things, and we have
feelings.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Emotions</strong> –
My most vivid memory of a communication failure was when I was a sailing
instructor. Trainees that were under my responsibility had put themselves
at risk, causing me a lot of worry. During the debrief, I was angry. My
failure to convey the messages without emotional loading undermined my
leadership on the group, putting everybody at risk for the rest of the
week.</p>
<p>Inability to understand the others’ point of view, or to communicate
ours, can bring in emotions. Emotions most often impedes technical
communication.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Limited attention</strong> –
We, in particular maintainers, are bombarded with email, notifications,
text and code to read.
As a consequence, it is easy to read things too fast, to stop in the
middle, to forget.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Language barriers</strong> –
Most discussions happen in English; but most of us are not native English
speakers. We may hide well our difficulties, but nuances are often lost.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Clique effects</strong> –
Most interactions in open source are done in writing, with low
communication bandwidth. It can be much harder to convince a maintainer
on the other side of the world than a colleague in the same room. Schools
of thoughts naturally emerge when people work a lot together. These
create bubbles, where we have the impression that everything we say is
obvious and uncontroversial, and yet we fail to convince people outside
of our bubble.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="little-things-that-help">
<h2><a class="toc-backref" href="#toc-entry-4">Little things that help</a></h2>
<p>Communication can improved by continuously working on it <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>.
It may be obvious to some, but it personally took me many years to learn.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Training materials for managers often discuss communication, and
give tricks. I am sure that there are better references than my
list below. But that’s the best I can do.</td></tr>
</tbody>
</table>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="hear-the-other-exchange">
<h3>Hear the other: exchange</h3>
<div class="side-hanging small sidebar">
<p class="first sidebar-title"><strong>Related presentation</strong></p>
<p class="last"><a class="reference external" href="https://docs.google.com/presentation/d/1mEMjGQXErZC-mBeCt0quLz7b5ODQnehmfwwnCeggzcU/edit#slide=id.g5135b4b0eb_1_14">How can we have healthier technical discussions?</a> by Nathaniel J. Smith</p>
</div>
<p><strong>Foster multiway discussions</strong> – The goal of a technical discussion is to
come up to the best solution. Better solutions emerge via confronting
different points of view: a single brilliant individual
probably cannot find or recognize the best solution alone.</p>
<ul class="simple">
<li>Integrate input from as many perspectives as possible.</li>
<li>Make sure everyone feels heard.</li>
</ul>
<p><strong>Don’t seek victory</strong> – Most important to keep in mind is that giving
up on an argument and accepting the other point of view is a perfectly
valid option. I naturally biased to think that my view on topics dear to
me is the right one. However, I’ve learned that adopting the view of the
other could bring a lot to the social dynamics of a project: we are often
debating over details and the bigger benefit comes from moving forward.</p>
<p>In addition, if several very bright people have different conclusions
than me about something that they’ve thought a lot, who am I to disagree?</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="convey-ideas-well-pedagogy">
<h3>Convey ideas well: pedagogy</h3>
<p><strong>Explain</strong> – Give the premises of your thoughts. Unroll your thought
processes. People are not sitting in your head, and need to hear not only
your conclusion, but how you got there.</p>
<p><strong>Repeat things</strong> – Account for the fact that people can forget, and
never hesitate to gently restate important points. Reformulating
differently can also help explaining.</p>
<p><strong>Keep it short</strong> – A typical reading speed is around 200 words a
minute. People have limited time and attention span. The greatest help
you can provide to your reader is to condense your ideas: let us avoid
long threads that require several dozens of minutes to read and digest.
There is a tension between this point and the above. My suggestion:
remove every word that is not useful, move details to footnotes or
postscriptums.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="cater-for-emotions-tone">
<h3>Cater for emotions: tone</h3>
<div class="side-hanging small sidebar">
<p class="first sidebar-title"><strong>Related good advice</strong></p>
<p class="last"><a class="reference external" href="https://www.mozilla.org/en-US/about/governance/policies/participation/#expected-behavior">Mozilla participation guide, expected behavior section</a></p>
</div>
<p><strong>Stay technical</strong> – Always try to get to the technical aspect of the
matter, and never the human. Give specific code and wording suggestions.
When explaining a decision, give technical arguments, even if they feel
obvious to you.</p>
<p><strong>Be positive</strong> – Being positive in general helps people feeling happy and
motivated. It is well known that positive feedback leads to quicker
progress than negative, as revealed <em>eg</em> by studies of class rooms. I am
particularly guilty of this: I always forget to say something nice,
although I may be super impressed by a contribution. Likewise, avoid
negative words when giving feedback (stay technical).</p>
<p><strong>Avoid “you”</strong> – The mere use of the pronoun “you” puts the person we are
talking to in the center of message. But the message should not be about
the person, it should be about the work. It’s very easy to react
emotionally when it’s about us. The passive voice can be useful to avoid
putting people as the topic. If the topic is indeed people, sometimes “we”
is an adequate substitute for “you”.</p>
<p><strong>Assume good faith</strong> – There are so many misunderstandings that can
happen. People forget things, people make mistakes, people fail to convey
their messages. Most often, all these failures are in good faith, and
misunderstandings are legitimate. In the rare cases there might possibly
be some bad faith, accounting for it will only make communication worse,
not better. Along the same line, we should ignore when we feel assaulted
or insulted, and avoid replying in kind.</p>
<p><strong>Choose words wisely</strong> – The choice of words matter, because they convey
implicit messages. In particular, avoid terms that carry judgement
values: “good” or “bad”. For example “This is done wrong” (note that this
sentence already avoids “you”), could be replaced by “There might be more
numerically stable / efficient way of doing it” (note also the use of
precise technical wording rather than the generic term “better”).</p>
<p><strong>Use moderating words</strong> – Try to leave room for the other in the
discussion. Statements too assertive close the door to different points
of view: “this must be changed” (note the lack of “you”) should be
avoided while “this should be changed” is better. For this reason, this
article is riddled with words such as “tend”, “often”, “feel”, “may”,
“might”.</p>
<p><strong>Don’t blame someone else</strong> – If you feel that there is some pattern that
you would like to change, do not point fingers, do not blame others.
Rather, point yourself at the center of the story, find an example of
this pattern with you, and the message should be that “it is a pattern
that <em>we</em> should avoid. <em>“We”</em> is such a powerful term. It unites; it
builds a team.</p>
<p><strong>Give your understanding</strong> – If you feel that there is a misunderstanding,
explain how you are feeling. But do it using “I”, and not “you”, and
acknowledge the subjectivity: “I feel ignored” rather than “you are
ignoring me”. Even better: only talk about the feeling: “I am loosing
motivation, because this is not moving forward”, or “I think that am
failing to convey why this numerical problem is such an important issue”
(note the use of “I think”, which avoids casting the situation as
necessarily true).</p>
<div class="side-hanging small sidebar">
<p class="first sidebar-title"><strong>Implicit messages</strong></p>
<p class="last"><a class="reference external" href="https://en.wikipedia.org/wiki/Four-sides_model">The four sides</a>
view of communication highlights the multiple messages present even in
simple statements.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>I hope this can be useful. I personally try to apply these rules, because
I want to work better with others.</p>
<div class="topic">
<p class="topic-title">Thanks</p>
<p>to many who gave me feedback: Adrin Jalali, Andreas Mueller,
Elizabeth DuPre, Emmanuelle Gouillart, Guillaume
Lemaitre, Joel Nothman, Joris Van den Bossche, Nicolas Hug.</p>
</div>
<hr class="docutils" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>PS: note how many times I’ve used “you” above. I can clearly get better
at communication!</p>
</div>
</div>
Jean Dechoux, June 13rd 1923 – Feb 9th 20202020-02-16T00:00:00+01:002020-02-16T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2020-02-16:/personnal/jean-dechoux-june-13rd-1923-feb-9th-2020.html<p>Jean Dechoux was born between the first and the second world wars, in a
small French town, close to Germany. His family was that of poor
farmers, who would work in coal mines to make up for the small size of
their crops.</p>
<p>He grew to become a pulmonologist, heading …</p><p>Jean Dechoux was born between the first and the second world wars, in a
small French town, close to Germany. His family was that of poor
farmers, who would work in coal mines to make up for the small size of
their crops.</p>
<p>He grew to become a pulmonologist, heading a hospital department that
tended to the illnesses of his people. He became an intellectual,
traveling the world, an avid reader, and the author of multiple
publications on diseases of coal miners.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>The story of how Jean grew his education is worth telling. His native
language was not even French, but “Lorrain” dialect. His sisters worked
young. But he was able to go to school because the village priest had
perceived Jean’s intelligence and wanted him to go to the seminary.
However the second world war came. Jean eventually got drafted in the
German (Nazy) army. Being from Lorraine, he was considered a German, yet
not one to be fully trusted: his fate was to be sent to Stalingrad, as
cannon fodder. Mistreated during training, he catches tuberculosis and
escapes narrowly the front. During his recovery in the German army
hospitals, a chief doctor shelters him, declares him unfit for service,
and pushes him to study for the <em>abitur</em>, the German high-school degree.
Now Jean wants to become a doctor, and serves as a nurse in the German
hospitals.</p>
<p>When the allies’ army advances, Jean is taken prisoner of war, then
incorporated in the French army, and eventually released with war
compensations. He uses them for college studies, during which he meets
his wife-to-be, Nicole Lissacq. Nicole is more wealthy than him, and
receives a stipend, as a student of the famed “École Normale Supérieure”.
The rest is history: Jean is brilliantly successful during his medical
studies, and comes back to his native region, Lorraine, to work as a
doctor for the coal miners.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<img alt="" class="align-right" src="../personnal/attachments/jean_dechoux.jpg" style="width: 200px;" />
<p>Jean, as I knew him, was a profoundly open and kind person. He survived
tragedy in his family by becoming even more so. Despite his age, he was
modern: the first time that I saw wifi was at his place.</p>
<p>Jean was my grand father. I very much look up to him.</p>
<hr class="docutils" />
<p>.</p>
Survey of machine-learning experimental methods at NeurIPS2019 and ICLR20202020-01-22T00:00:00+01:002020-01-22T00:00:00+01:00Xavier Bouthillier & Gaël Varoquauxtag:gael-varoquaux.info,2020-01-22:/science/survey-of-machine-learning-experimental-methods-at-neurips2019-and-iclr2020.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">A simple survey asking authors of two leading machine-learning
conferences a few quantitative questions on their experimental
procedures.</p>
</div>
<p>How do machine-learning researchers run their empirical validation? In
the context of a push for improved reproducibility and benchmarking, this
question is important to develop new tools for model comparison. We …</p><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">A simple survey asking authors of two leading machine-learning
conferences a few quantitative questions on their experimental
procedures.</p>
</div>
<p>How do machine-learning researchers run their empirical validation? In
the context of a push for improved reproducibility and benchmarking, this
question is important to develop new tools for model comparison. We ran a
simple survey asking to authors of two leading conferences, NeurIPS 2019
and ICLR 2020, a few quantitative questions on their experimental
procedures.</p>
<p>A <a class="reference external" href="https://hal.archives-ouvertes.fr/hal-02447823">technical report on HAL</a> summarizes our
finding. It gives a simple picture of how hyper-parameters are set, how
many baselines and datasets are included, or how seeds are used.
Below, we give a very short summary, but please read (and <a class="reference external" href="https://hal.archives-ouvertes.fr/hal-02447823v1/bibtex">cite</a>)
<a class="reference external" href="https://hal.archives-ouvertes.fr/hal-02447823">the full report</a> if you are interested.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Highlights</strong>
The response rates were 35.6% for NeurIPS and 48.6%
for ICLR.
A vast majority of empirical works optimize model hyper-parameters,
thought almost half of these use manual tuning and most of the automatic
hyper-parameter optimization is done with grid search. The typical number
of hyper-parameter set is in interval 3-5, and less than 50 model fits
are used to explore the search space. In addition, most works also
optimized their baselines (typically, around 4 baselines).
Finally, studies typically reported 4 results per model per task to provide a measure of variance, and around 50% of them
used a different random seed for each experiment.</p>
<p><strong>Sample results</strong></p>
<div class="side-caption figure align-center">
<img alt="" src="../science/attachments/survey_of_ml_experimental_methods/hyper_parameter_optimization.png" style="width: 400px;" />
<p class="caption">How many papers with experiments optimized hyperparameters.</p>
</div>
<div class="side-caption figure align-center">
<img alt="" src="../science/attachments/survey_of_ml_experimental_methods/tuning_methods.png" style="width: 400px;" />
<p class="caption">What hyperparameter optimization method were used.</p>
</div>
<div class="side-caption figure align-center">
<img alt="" src="../science/attachments/survey_of_ml_experimental_methods/number_datasets.png" style="width: 400px;" />
<p class="caption">Number of different datasets used for benchmarking.</p>
</div>
<div class="side-caption figure align-center">
<img alt="" src="../science/attachments/survey_of_ml_experimental_methods/number_seeds_or_trials.png" style="width: 400px;" />
<p class="caption">Number of results reported for each model (ex: for different seeds)</p>
</div>
<p>These are just samples. Read <a class="reference external" href="https://hal.archives-ouvertes.fr/hal-02447823">the full report</a> for
more results.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>For reproducibility and AutoML, there is active research in benchmarking
and hyperparameter procedures in machine learning. We hope that the
survey results presented here can help inform this research. As this
document is merely a research report, we purposely limited
interpretation of the results and drawing recommendations. However, trends that stand out to our
eyes are, <cite>1)</cite> the simplicity of hyper-parameter tuning strategies
(mostly manual search and grid search), <cite>2)</cite> the small number of
model fits explored during this tuning (often 50 or less), which biases the
results and <cite>3)</cite> the small number of performances reported, which limits
statistical power. These
practices are most likely due to the high computational cost of fitting
modern machine-learning models.</p>
<div class="sidebar">
<p class="first sidebar-title">Code</p>
<p class="last">The code used for plotting and analysis is <a class="reference external" href="https://github.com/bouthilx/ml-survey-2020">on github</a>.</p>
</div>
<p><strong>Acknowledgments</strong> We are deeply grateful to the participants of
the survey who took time to answer the questions.</p>
2019: my scientific year in review2020-01-05T00:00:00+01:002020-01-05T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2020-01-05:/science/2019-my-scientific-year-in-review.html<p>My current research spans wide: from brain sciences to core data
science. My overall interest is to build <strong>methodology drawing insights from
data</strong> for questions that have often been addressed qualitatively. If I can
highlight a few publications from 2019 <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>, the common thread would be
computational statistics, from dirty …</p><p>My current research spans wide: from brain sciences to core data
science. My overall interest is to build <strong>methodology drawing insights from
data</strong> for questions that have often been addressed qualitatively. If I can
highlight a few publications from 2019 <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>, the common thread would be
computational statistics, from dirty data to brain images. Let me try to
give the gist of these progresses, in simple terms.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>It’s already 2020, I’m always late.</td></tr>
</tbody>
</table>
<div class="contents topic" id="highlights">
<p class="topic-title">Highlights</p>
<ul class="simple">
<li><a class="reference internal" href="#comparing-distributions" id="toc-entry-1">Comparing distributions</a></li>
<li><a class="reference internal" href="#predictive-pipelines-on-brain-functional-connectomes" id="toc-entry-2">Predictive pipelines on brain functional connectomes</a></li>
<li><a class="reference internal" href="#population-shrinkage-of-covariance" id="toc-entry-3">Population shrinkage of covariance</a></li>
<li><a class="reference internal" href="#deep-learning-on-non-translation-invariant-images" id="toc-entry-4">Deep learning on non-translation-invariant images</a></li>
<li><a class="reference internal" href="#open-science" id="toc-entry-5">Open science</a></li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="comparing-distributions">
<h2><a class="toc-backref" href="#toc-entry-1">Comparing distributions</a></h2>
<p class="align-right"><em>Fundamental computational-statistics work</em></p>
<p>What if you are given two set of observations and need to conclude on
whether they are drawn from the same distribution? We are interested in
this question for the <a class="reference external" href="https://project.inria.fr/dirtydata/">DirtyData</a>
research project, to facilitate analysis of data without manual curation.
Comparing distributions is indeed important to detect drifts in the data,
to match information across datasets, or to compensate for dataset
biases.</p>
<p>Formally, we are given two cloud of points (circle and crosses in the
figure below) and we want to develop a statistical test of whether the
distributions differ. There is an abundant literature on this topic, that I
cover in <a class="reference external" href="http://gael-varoquaux.info/science/comparing-distributions-kernels-estimate-good-representations-l1-distances-give-good-tests.html">a more detailed post on this subject</a>.
Specifically, when the observations have a natural similarity, for
instance when they live in a vector space, kernel methods are interesting
because they enable to estimate a representant of the underlying
distribution that interpolates between observations, as with <a class="reference external" href="https://en.wikipedia.org/wiki/Kernel_density_estimation">a kernel
density estimator</a>.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="http://papers.nips.cc/paper/9398-comparing-distributions-ell_1-geometry-improves-kernel-two-sample-testing"><img alt="" src="../science/attachments/comparing_distributions_l1/optimizing_position.png" style="width: 500px;" /></a>
<p class="caption">Two cloud of points, the corresponding distribution representants μ_P
and μ_Q (blue and orange), the difference between these
(black), and locations to measure this difference (red triangles).</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>With Meyer Scetbon, in
<a class="reference external" href="http://papers.nips.cc/paper/9398-comparing-distributions-ell_1-geometry-improves-kernel-two-sample-testing">Scetbon & Varoquaux, NeurIPS</a>,
we investigate how to measure best the difference between these
representants. We show that the best choice is to take the absolute value
of the difference (the l1 norm), while the default choice had so far been
the Euclidean (l2) norm. In a nutshell, the reason is that the difference
most like is dense when the distributions differ: zero almost nowhere.</p>
<p>We were able to show that the <a class="reference external" href="https://slideslive.com/38921490/interpretable-comparison-of-distributions-and-models">sophisticated framework</a>
for efficient and powerful tests in the
Euclidean case carries over to the l1 case. In particular, our paper
gives efficient testing procedures using a small number of locations to
avoid costly computation (the red triangles in the figure above), that
can either be sampled at random or optimized.</p>
<p>My hunch is that the result is quite general: the l1 geometry is better
than the l2 one on representants of distributions. There might be more
fundamental mathematical properties behind this. The drawback is that the
l1 norm is non smooth which can be challenging in optimization settings.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="predictive-pipelines-on-brain-functional-connectomes">
<h2><a class="toc-backref" href="#toc-entry-2">Predictive pipelines on brain functional connectomes</a></h2>
<p class="align-right"><em>Brain-imaging methods</em></p>
<p>Brain functional connectivity is increasingly used to extract biomarkers
of behavior and mental health. The long-term stakes are to ground
assessment of psychological traits on quantitative brain
data, rather than qualitative behavioral observations. But, to build
biomarkers, there are many details that go in estimating functional
connectivity from fMRI, something that I have studied for more than 10
years. With Kamalakar Dadi, in <a class="reference external" href="https://www.sciencedirect.com/science/article/abs/pii/S1053811919301594">Dadi et al</a>,
we ran thorough empirical benchmarks to find which methodological choices
for the various steps of the pipeline give best prediction across
multiple cohorts. Specifically, we studied 1) defining regions of
interest for signal extraction, 2) building a functional-connectivity
matrix across these regions, 3) prediction across subjects with
supervised learning on these features.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://www.sciencedirect.com/science/article/abs/pii/S1053811919301594"><img alt="" src="../science/attachments/2019_highlights/dadi_2019_highlights.png" style="width: 600px;" /></a>
<p class="caption">Summarizing our benchmark results.</p>
</div>
<div class="sidebar">
<p class="first sidebar-title">Recommendations</p>
<ul class="last simple">
<li>functional regions (eg from dictionary learning)</li>
<li>tangent-space for covariances</li>
<li>l2-logistic regression</li>
</ul>
</div>
<p>Results show the importance of defining regions from functional data,
ideally with a linear-decomposition method that produces soft
parcellations such as ICA or dictionary learning. To represent
connectivity between regions, the best choice is tangent-space
parametrization, a method to build a vector-space from covariance
matrices (more below). Finally, for supervised learning, a simple
l2-penalized logistic regression is the best option. With the huge popularity
of deep learning, it may surprise that linear models are the best
performer, but this is well explained by the amount of data at hand: a
cohort is typically less than 1000 individuals, which is way below the
data sizes needed to see the benefits of non-linear models.</p>
<p>A recent preprint, <a class="reference external" href="https://www.biorxiv.org/content/10.1101/741595v2.abstract">Pervaiz et al</a> from
Oxford, overall
confirms our findings, even though they investigated slightly
different methodological choices. In particular, they find tangent space
clearly useful.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>In my eyes, such benchmarking studies are important not only to improve
prediction, but also to reduce analytic variability that opens the door
to inflation of reported effects. Indeed, given 1000 individuals, the
measure of prediction accuracy of a pipeline is quite imprecise
(<a class="reference external" href="https://www.sciencedirect.com/science/article/abs/pii/S1053811917305311">Varoquaux 2018</a>).
As a consequence, trying out a bunch a analytic choices and
publishing the one that works best can lead to grossly optimistic
prediction accuracies. <strong>If we want trust in biomarkers, we need to
reduce the variability in the methods used to build them</strong>.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="population-shrinkage-of-covariance">
<h2><a class="toc-backref" href="#toc-entry-3">Population shrinkage of covariance</a></h2>
<p class="align-right">Statistics for brain signals</p>
<p>Estimating covariances is central for functional brain connectivity and
in many other applications. With Mehdi Rahim, in <a class="reference external" href="https://www.sciencedirect.com/science/article/abs/pii/S1361841518301014">Rahim et al</a>
we considered the case of a population of random processes with
related covariances, as for instance when estimating functional
connectivity from a group of individuals. For this, we combined two
mathematical ideas: that of using natural operations on covariance
matrices, and that of priors for mean-square estimation:</p>
<ul class="simple">
<li><strong>Tangent space</strong> Covariance matrices are positive-definite matrices,
for which standard arithmetics are not well suited <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>: subtracting
two covariance matrices can lead to a matrix that cannot be
the covariance of a signal. However, a group of covariance matrices can
be transformed into points in a vector space for which standard
distances and arithmetics respect the structure of
covariances (for instance Euclidean distance between these points
approximate KL divergence between covariances). This is what we call
the <em>tangent space</em>.</li>
</ul>
<table class="side-hanging docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Technically, covariance matrices live on a Riemannian manifold:
a curve surface inside <em>R^{n x n}</em> that has some metric
properties.</td></tr>
</tbody>
</table>
<ul class="simple">
<li><strong>James-Stein shrinkage</strong> To estimate the mean of <em>n</em> observations, it
is actually best not to compute the average of these, but rather to
push a bit this average toward a prior guess. The better the
guess, the more this “push” helps. The more the number of observations,
the more gentle this push should be. This strategy is known as
<a class="reference external" href="https://en.wikipedia.org/wiki/James%E2%80%93Stein_estimator">James-Stein shrinkage</a> and it
is in my opinion one of the most beautiful results in statistics.
It can be seen as a Bayesian posterior, but it comes with guarantees
that do not require the model to be true and that control estimation
error, rather than a posterior probability.</li>
</ul>
<p>James-Stein shrinkage is easily written for quadratic errors on vectors,
but cannot be easily applied to covariances, as they do not live in a vector
space and we would like to control a KL divergence rather than
a quadratic error. Our work combined both ideas to give an excellent
estimator of a family of related covariances that is also very
computationally efficient. We call it PoSCE: Population Shrinkage
Covariance Estimation.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://www.sciencedirect.com/science/article/abs/pii/S1361841518301014"><img alt="" src="../science/attachments/2019_highlights/posce.png" style="width: 600px;" /></a>
<p class="caption">Schema of the estimation strategy: projecting the covariances matrices
into a tangent space, shrinkage to a group mean, but taking in account
the anisotropy of the dispersion of the group, and projecting back to
covariances.</p>
</div>
<p>It is easy to see how accounting for group information in the estimation
of individual covariances can help stabilizing them. However, will it be
beneficial if we are interested in the differences between these
covariances, for instance to ground biomarkers, as studied above? Our
results show that it does indeed help building better biomarkers, for
instance to predict brain age. The larger the group of covariances used,
the larger the benefits.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://www.sciencedirect.com/science/article/abs/pii/S1361841518301014"><img alt="" src="../science/attachments/2019_highlights/posce_age_learning_curve.png" style="width: 500px;" /></a>
<p class="caption">Error in predicting brain aging decreases when more individuals are used
to build the biomarker.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="deep-learning-on-non-translation-invariant-images">
<h2><a class="toc-backref" href="#toc-entry-4">Deep learning on non-translation-invariant images</a></h2>
<p class="align-right">Computer vision</p>
<p>Brain images, in particular images of brain activity, are very different
from the natural images on which most computer-vision research focuses.
A central difference is that detecting activity in different parts of the
brain completely changes the meaning of this detection, while detecting a
cat in the left or the right of a picture on Facebook makes no
difference. This is important because many progresses of computer vision,
such as convolutional neural networks, are built on the fact that natural
images are statistically translational invariant. On the opposite, brain
images are realigned to a template, before being analyzed.</p>
<p>Convolutional architectures have been crucial to the successes of deep
learning on natural images because they impose a lot of structure on the
weights of neural networks and thus help fight estimation noise. For
predicting from brain images, the regularizations strategies that have
been successful foster spatially continuous structures. Unfortunately,
they have lead to costly non-smooth optimizations that cannot easily be
used with the optimization framework of deep learning, stochastic
gradient descent.</p>
<p>With Sergul Aydore, in <a class="reference external" href="http://proceedings.mlr.press/v97/aydore19a.html">Aydore et al, ICML</a>, we have introduced a
spatial regularization that is compatible with the deep learning toolbox.
During the stochastic optimization, we impose random spatial structure
via feature groups estimated from the data. These stabilize the input
layers of deep architecture. They also lead to iterating on smaller
representations, which greatly speeds up the algorithm.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="http://proceedings.mlr.press/v97/aydore19a.html"><img alt="" src="../science/attachments/2019_highlights/stochastic_grouping_mlp.png" style="width: 600px;" /></a>
<p class="caption">At each step of a stochastic gradient descent, we randomly pick a
feature-grouping matrix (itself estimated from the data), and use it
to reduce the data in the computations of the gradients, then invert
this reduction to update the weights.</p>
</div>
<p><a class="reference external" href="http://proceedings.mlr.press/v97/aydore19a.html">The paper</a> comes with
extensive empirical validation, including comparison to convolutional
neural networks. We benchmark the strategy on brain images, but also
on realigned faces, to show that the approach is beneficial for any
non-translational-invariant images. In particular, the approach greatly
speeds up convergence.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="http://proceedings.mlr.press/v97/aydore19a.html"><img alt="" src="../science/attachments/2019_highlights/stochastic_grouping_results.png" style="width: 600px;" /></a>
<p class="caption">Prediction accuracy as a function of training time – left: on
realigned faces – right: on brain images</p>
</div>
<p><a class="reference external" href="http://proceedings.mlr.press/v97/aydore19a.html">This paper</a> clearly
shows that <strong>one should not use convolutional neural networks on fMRI
data</strong>: these images are not translational invariant.</p>
<div class="sidebar">
<p class="first sidebar-title"><strong>Preprints</strong></p>
<p class="last">All papers are available as preprints, eg on <a class="reference external" href="http://gael-varoquaux.info/publications.html">my site</a>.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="open-science">
<h2><a class="toc-backref" href="#toc-entry-5">Open science</a></h2>
<p><strong>Open and reproducible science:</strong> Looking at all these publications, I
realize that every single one of them comes with code on a github
repository and is done on open data, which means that they can all be
easily reproduced. I’m very proud of the teams behind these papers.
Achieving this level of reproducibility requires hard work and
discipline. It is also a testimonial to a community investment in
software tools and infrastructure for open science that has been going on
for decades and gives the foundations on which these works build.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>A prize for scikit-learn:</strong> On this topic, a highlight of 2019 was also
that the work behind scikit-learn was acknowledged in <a class="reference external" href="../programming/getting-a-big-scientific-prize-for-open-source-software.html">an important
scientific prize</a>.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Why open science:</strong> Why do I care so much for open science? Because in
a world of uncertainty, the claims of science must be trusted and hence
built on transparent practice (think about science and global warming).
Because it helps putting our methods in the hands of a wider public,
society at large. And because it levels the ground, making it easier for
newcomers –young scientists, or developing countries– to contribute,
which in itself makes science more efficient.</p>
</div>
Comparing distributions: Kernels estimate good representations, l1 distances give good tests2019-12-08T00:00:00+01:002019-12-08T00:00:00+01:00Meyer Scetbon & Gaël Varoquauxtag:gael-varoquaux.info,2019-12-08:/science/comparing-distributions-kernels-estimate-good-representations-l1-distances-give-good-tests.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>Given two set of observations, are they drawn from the same
distribution? Our paper <a class="reference external" href="https://papers.nips.cc/paper/9398-comparing-distributions-ell_1-geometry-improves-kernel-two-sample-testing.html">Comparing distributions: l1 geometry
improves kernel two-sample testing</a>
at the <strong>NeurIPS 2019 conference</strong> revisits this classic statistical
problem known as “two-sample testing”.</p>
<p class="last">This post explains the context and the paper with a bit of hand …</p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p>Given two set of observations, are they drawn from the same
distribution? Our paper <a class="reference external" href="https://papers.nips.cc/paper/9398-comparing-distributions-ell_1-geometry-improves-kernel-two-sample-testing.html">Comparing distributions: l1 geometry
improves kernel two-sample testing</a>
at the <strong>NeurIPS 2019 conference</strong> revisits this classic statistical
problem known as “two-sample testing”.</p>
<p class="last">This post explains the context and the paper with a bit of hand
waiving.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="contents topic" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#the-context-two-sample-testing" id="toc-entry-1">The context: two-sample testing</a></li>
<li><a class="reference internal" href="#from-kernel-mean-embeddings-to-distances-on-distributions" id="toc-entry-2">From kernel mean embeddings to distances on distributions</a></li>
<li><a class="reference internal" href="#controlling-the-weak-convergence-of-probability-measures" id="toc-entry-3">Controlling the weak convergence of probability measures</a></li>
<li><a class="reference internal" href="#two-sample-testing-procedures" id="toc-entry-4">Two-sample testing procedures</a></li>
<li><a class="reference internal" href="#the-l1-metric-provides-best-testing-power" id="toc-entry-5">The L1 metric provides best testing power</a></li>
</ul>
</div>
<div class="section" id="the-context-two-sample-testing">
<h2><a class="toc-backref" href="#toc-entry-1">The context: two-sample testing</a></h2>
<p>Given two samples from two unknown populations, the goal of two-sample tests is
to determine whether the underlying populations differ with a statistical
significance. For instance, we may care to know whether the
McDonald’s and KFC use different logic to chose locations of restaurants
across the US. This is a difficult question: we have access to data points,
but not the underlying generative mechanism, that is probably governed by
marketing strategies.</p>
<img alt="" class="align-center" src="attachments/comparing_distributions_l1/map_KFC_McDo_simple.png" style="width: 70%;" />
</div>
<div class="section" id="from-kernel-mean-embeddings-to-distances-on-distributions">
<h2><a class="toc-backref" href="#toc-entry-2">From kernel mean embeddings to distances on distributions</a></h2>
<p>In the example of spatial distributions restaurants,
there is <strong>a lot of information in how close observed data
points lie in the original measurement space (here geographic coordinates)</strong>.
Kernel methods arise naturally to capture this information. They can be
applied to distributions, building representatives of distributions:
<a class="reference external" href="https://en.wikipedia.org/wiki/Kernel_embedding_of_distributions">Kernel embeddings of distributions</a>. The
mean embedding of a distribution P with a kernel k is written:</p>
<div class="formula">
<i>μ</i><sub><i>P</i></sub>(<i>t</i>) : = <span class="limits"><span class="limit"><span class="bigoperator integral">∫</span></span></span><sub>ℝ<sup><i>d</i></sup></sub><i>k</i>(<i>x</i>, <i>t</i>)<i>dP</i>(<i>x</i>)
</div>
<p>Intuitively, it is related to <a class="reference external" href="https://en.wikipedia.org/wiki/Kernel_density_estimation">Kernel Density Estimates (KDEs)</a> which
estimate a density in continuous space by smoothing the observed data
points with a kernel.</p>
<div class="figure align-right">
<img alt="" src="attachments/comparing_distributions_l1/kde.jpg" />
<p class="caption">Kernel mean embeddings for two distributions of points</p>
</div>
<p>For two-sample testing, kernel embeddings can be used to compute distances
between distributions, building metrics over the space of probability
measures. Metrics between probability measures can be defined via the
notion of Integral Probability Metric (IPM): as a difference of
expectations:</p>
<div class="formula">
<span class="text">IPM</span>[<i>F</i>, <i>P</i>, <i>Q</i>] : = <span class="limits"><sup class="limit"> </sup><span class="limit">sup</span><sub class="limit"><i>f</i> ∈ <i>F</i></sub></span>(𝔼<sub><i>x</i> ∼ <i>P</i></sub><span class="stretchy">[</span><i>f</i>(<i>x</i>)<span class="stretchy">]</span> − 𝔼<sub><i>y</i> ∼ <i>Q</i></sub><span class="stretchy">[</span><i>f</i>(<i>y</i>)<span class="stretchy">]</span>)
</div>
<p>where F is a class of functions. This definition is appealing because it
<strong>characterizes the difference between P and Q by the function for which
the expectancy differs most</strong>. The specific choice of class of function
defines the metric. If we now consider a kernel, it implicitly defines a
space of functions (intuitively related to all the possible KDEs
generated by varying data points): a Reproducible Kernel Hilbert Space
(RKHS). Defining a metric (an IPM) with a function class F as the unit
ball in such an RKHS, is known as the Maximum Mean Discrepancy (MMD). It
can be shown that, rather than computing the maximum, the MMD has a more
convenient expression, the RKHS distance between the mean embeddings:</p>
<div class="formula">
<span class="text">MMD</span>[<i>P</i>, <i>Q</i>] = ‖<i>μ</i><sub><i>P</i></sub> − <i>μ</i><sub><i>Q</i></sub>‖<sub><i>H</i><sub><i>k</i></sub></sub>
</div>
<p>For good choices of kernels, the MMD has appealing mathematical
properties to compare distributions. With kernels said to be
characteristic, eg Gaussian kernels, the MMD is a metric: MMD[P, Q] = 0
if and only if P = Q. Using the MMD for two-sample testing –given only
observations from the distributions, and not P and Q– requires using an
empirical estimation of the MMD. This can be done by computing the RKHS
norm in the expression above, which leads to summing kernel evaluations
on all data points in P and Q.</p>
<p>Our work builds upon this framework, but deviates a bit from the
classical definition of MMD as it addresses the question of which norm is
best to use on the difference of mean embeddings, µQ - µP (as well as
other representatives, namely the smooth characteristic function, SCF).
We consider a wider family of metrics based on the Lp distances between
mean emdeddings (p=2 recovers the classic framework):</p>
<div class="formula">
<i>d</i><sub><i>L</i><sup><i>p</i></sup>, <i>μ</i></sub>(<i>P</i>, <i>Q</i>) : = <span class="stretchy">(</span><span class="limits"><sup class="limit"> </sup><span class="limit"><span class="bigoperator integral">∫</span></span><sub class="limit"><i>t</i> ∈ ℝ<sup><i>d</i></sup></sub></span>|<i>μ</i><sub><i>P</i></sub>(<i>t</i>) − <i>μ</i><sub><i>Q</i></sub>(<i>t</i>)|<sup><i>p</i></sup><i>d</i>Γ(<i>t</i>)<span class="stretchy">)</span><sup>1 ⁄ <i>p</i></sup>
</div>
<p>where Γ is a Borel probability measure absolutely continuous.</p>
</div>
<div class="section" id="controlling-the-weak-convergence-of-probability-measures">
<h2><a class="toc-backref" href="#toc-entry-3">Controlling the weak convergence of probability measures</a></h2>
<p>We show that these metrics have good properties. Specifically, for p ≥ 1,
as soon as the kernel is bounded continuous and characteristic, these
metrics metrize the weak convergence. What this means is that these
metrics tend to zero if and only if P and Q weakly converge.</p>
<p>The <a class="reference external" href="https://en.wikipedia.org/wiki/Convergence_of_measures#Weak_convergence_of_measures">weak convergence of probability measures</a>
is a notion of convergence that is based <strong>not just on having events with
probabilities that are the same for the two distributions, but also that some events are
“close”</strong>. Indeed, classic convergence in probability just tells us that
the same observation should have the same probability in the two distributions. Weak convergence takes in account the topology of the
observations. For instance, to go back to the problem of spatial
distributions of restaurants, it does not only look at whether the
probabilities of having a Mc Donald’s or a KFC restaurant converge on
11th Wall Street, but also at restaurants are likely on 9th Wall Street.</p>
<p>A simple example to see why these matters is to consider two Dirac
distributions: spikes in a single point. If we bring these spikes closer
and closer, merely looking at the probability of events in the same exact
position will not detect any convergence until the spikes exactly
overlap.</p>
<p>Using kernel embeddings of distributions enables to capture the aspects
of convergence in the spatial domain because the kernels used give a
spatial smoothness to the representatives:</p>
<img alt="" class="align-center" src="attachments/comparing_distributions_l1/converging_diracs.png" style="width: 70%;" />
<p>Having a metric on probability distributions that captures the topology
of the observations is important for many applications, for instance when
fitting GANs to generate images: the goal is not to only capture that
images are exactly the same, but also that they maybe be “close”.</p>
</div>
<div class="section" id="two-sample-testing-procedures">
<h2><a class="toc-backref" href="#toc-entry-4">Two-sample testing procedures</a></h2>
<p>Now that we have built metrics, we can derive two-sample test statistics.
A straightforward way of doing it would involve large sums on all the
observations, which would be costly. Hence, we resort to a good
approximation by sampling a set of {Tj} locations from the distribution
Γ:</p>
<div class="formula">
<i>d̂</i><span class="scripts"><sup class="script"><i>p</i></sup><sub class="script"><i>ℓ</i><sub><i>p</i></sub>, <i>μ</i>, <i>J</i></sub></span>[<i>X</i>, <i>Y</i>] : = <i>n</i><sup><i>p</i> ⁄ 2</sup><span class="limits"><sup class="limit"> </sup><span class="limit"><span class="bigoperator">∑</span></span><sub class="limit"><i>j</i> = 1..<i>J</i></sub></span>|<i>μ</i><sub><i>X</i></sub>(<i>T</i><sub><i>j</i></sub>) − <i>μ</i><sub><i>Y</i></sub>(<i>T</i><sub><i>j</i></sub>)|<sup><i>p</i></sup>
</div>
<p>We show that this approximation maintains (almost surely) the appealing
metric properties, generalizing the results that were established by
<a class="reference external" href="http://papers.nips.cc/paper/5685-fast-two-sample-testing-with-analytic-representations-of-probability-measures">Chwialkowski et al 2015</a>
for the special case of the L2 metric.</p>
<div class="figure align-center">
<img alt="" src="attachments/comparing_distributions_l1/optimizing_position.png" style="width: 70%;" />
<p class="caption">Sampling at different positions</p>
</div>
<p>We further develop the testing procedures by showing that other tricks
known to improve testing with the L2 metric can be adapted to other
metrics, such as the L1 metric. Fast and performant tests can be obtained
by optimizing the test locations –using an upper-bound on the test power–
or by testing in the Fourrier domain, using the Smooth Characteristic
Function of the kernel. Even in the case of the L1 metric, the null
distribution of the test statistic can be derived, leading to tests that
can control errors without permutations.</p>
</div>
<div class="section" id="the-l1-metric-provides-best-testing-power">
<h2><a class="toc-backref" href="#toc-entry-5">The L1 metric provides best testing power</a></h2>
<p>Going back to our question of which norm on the difference of
distribution representative is best suited to detect, we show that when
using analytics kernels, such as the Gaussian kernel, the L1 metric
improves upon the L2 metric, which corresponds to the classic definition
of the MMD.</p>
<p>Indeed, analytic kernels are non-zero almost everywhere. As a result,
when P is different from Q, the difference between their mean embeddings
will be dense, as well as the differences between the representatives
that we use to build our tests (for instance the values at the locations
that we use to build the tests above). l1 norms capture better dense
differences than l2 norms –this is the reason why, used as penalties,
they induce sparsity.</p>
<img alt="" class="align-right" src="attachments/comparing_distributions_l1/l1_vs_l2.png" style="width: 150px;" />
<p>A simple intuition is that dense vectors tend to lie in the diagonals of
the measurement basis, as none of their coordinates are zero. On these
diagonals, the l1 norm is much larger than the l1 norm of vectors with
some zero, or nearly-zero coordinates.</p>
<div class="topic">
<p class="topic-title"><strong>Summary</strong></p>
<p>For a very simple summary, the story is that: to perform tests of
whether two distributions differs, it is useful to compute a “mean
Kernel embedding” –similar to a Kernel density estimate, but without
normalization– of each distribution, and consider the l1 norm of the
difference of these embeddings. They can be computed on a small number
of locations, either drawn at random or optimized. This approach is
reminiscent of looking at the total variation between the measures,
however the fact that it uses Kernels makes it robust to small spatial
noise in the observations, unlike the total variation for which events
must perfectly coincide in both set of observations (the total
variation does not metrize the weak convergence).</p>
</div>
<div class="topic">
<p class="topic-title"><strong>References</strong></p>
<p>The framework exposed here is one that was developed over a long line
of research, which our work builds upon. <a class="reference external" href="https://papers.nips.cc/paper/9398-comparing-distributions-ell_1-geometry-improves-kernel-two-sample-testing.html">Our paper</a>
gives a complete list of references, however, some useful review
papers are</p>
<ul class="simple">
<li>C.-J. Simon-Gabriel and B. Schölkopf. Kernel distribution
embeddings: Universal kernels, <em>characteristic kernels and kernel
metrics on distributions</em>, <a class="reference external" href="https://arxiv.org/abs/1604.0525">arXiv:1604.05251</a>, 2016.</li>
<li>A. Gretton, K.M. Borgwardt, M.J. Rasch, B. Schölkopf, A. Smola; <em>A
Kernel Two-Sample Test</em>, <a class="reference external" href="http://www.jmlr.org/papers/v13/gretton12a.html">JMLR, 2012</a>.</li>
<li><a class="reference external" href="https://slideslive.com/38921490/interpretable-comparison-of-distributions-and-models">The NeurIPS 2019 tutorial</a>,
by Gretton, Sutherland, and Jitkrittum, is extremely didactic and gives
a lot of big picture</li>
</ul>
</div>
<p>·</p>
</div>
Getting a big scientific prize for open-source software2019-12-01T06:00:00+01:002019-12-01T06:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2019-12-01:/programming/getting-a-big-scientific-prize-for-open-source-software.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">An important acknowledgement for a different view of doing science:
open, collaborative, and more than a proof of concept.</p>
</div>
<img alt="" class="align-right" src="attachments/sklearn_prize_academie/prize.jpg" style="width: 350px;" />
<p>A few days ago, Loïc Estève, Alexandre Gramfort, Olivier Grisel, Bertrand
Thirion, and myself received the <a class="reference external" href="https://www.academie-sciences.fr/fr/Laureats/prix-inria-academie-des-sciences-2019-vincent-hayward-equipe-scikit-learn-et-maria-naya-plasencia.html">“Académie des Sciences Inria prize for transfer”</a>,
for our contributions to the scikit-learn project …</p><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">An important acknowledgement for a different view of doing science:
open, collaborative, and more than a proof of concept.</p>
</div>
<img alt="" class="align-right" src="attachments/sklearn_prize_academie/prize.jpg" style="width: 350px;" />
<p>A few days ago, Loïc Estève, Alexandre Gramfort, Olivier Grisel, Bertrand
Thirion, and myself received the <a class="reference external" href="https://www.academie-sciences.fr/fr/Laureats/prix-inria-academie-des-sciences-2019-vincent-hayward-equipe-scikit-learn-et-maria-naya-plasencia.html">“Académie des Sciences Inria prize for transfer”</a>,
for our contributions to the scikit-learn project. To put things simply,
it’s quite a big deal to me, because I feel that it illustrates a change
of culture in academia.</p>
<div class="align-right docutils container">
Recognizing an open view of scientific contributions</div>
<p>It is a great honor, because the selection was made by the members of the
Académie des Sciences, very accomplished scientists with impressive
contributions to science. The “Académie” is the hallmark of fundamental
academic science in France. To me, this prize is also symbolic because it
recognizes an open view of academic research and transfer, a view that
sometimes felt as not playing according to the incentives. We started
scikit-learn as a crazy endeavor, a bit of a <em>hippy</em> science thing.
People didn’t really take us seriously. We were working on software, and
not publications. We were doing open source, while industrial transfer is
made by creating startups or filing patents. We were doing Python, while
academic machine learning was then done in Matlab, and industrial
transfer in C++. We were not pursuing the latest publications, while
these are thought to be research’s best assets. We were interested in
reaching out to non experts, while partners considered as
interesting have qualified staff.</p>
<div class="align-right docutils container">
Quality and openness, at the cost of quantity and control</div>
<p>No. We did it different. We reached out to an open community. We did
BSD-licensed code. We worked to achieve quality at the cost of quantity. We
cared about installation issues, on-boarding biologists or medical
doctors, playing well with the wider scientific Python ecosystem.
We gave decision power to people outside of Inria, sometimes whom we had
never met in real life. We made sure that Inria was never the sole actor,
the sole stake-holder. We never pushed our own scientific publications in
the project. We limited complexity, trading off performance for ease of
use, ease of installation, ease of understanding.</p>
<div class="figure align-right">
<object data="attachments/sklearn_prize_academie/sklearn_website_stats_white.svg" style="width: 25%;" type="image/svg+xml"></object>
</div>
<p>As a consequence, we slowly but surely assembled a large community. In
such a community, the
sum is greater than the parts. The breadth of interlocutors and cultures
slows movement down, but creates better results, because these results are
understandable to many and usable on a diversity of problems. The
consequence of this quality is that
we were progressively used in more and more places: industrial
data-science labs, startups, research in applied or fundamental
statistical learning, teaching. Ironically, the institutional world did
not notice. It got hard, next to impossible, to get funding <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>. A few years
ago, I was told by a central governmental agency that we, open-source
zealots, were destroying an incredible amount of value by giving away
for free the production of research <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>. The French report on AI, lead by a
Fields medal, cited tensorflow and theano –a discontinued software–, but
ignored scikit-learn; maybe because we were doing “boring science”?</p>
<p>But, scikit-learn’s amazing community continued plowing forward. We grew
so much that we were heard from the top. The prize from the Académie shows
that we managed to capture the attention of senior scientists with
open-source software, because this software is really having a worldwide
impact in many disciplines.</p>
<div class="figure align-center">
<img alt="" src="attachments/sklearn_prize_academie/academie_presentation.jpeg" style="width: 70%;" />
<p class="caption">Presenting scikit-learn at the Academie Des Sciences</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="align-right docutils container">
An accomplishment of the community</div>
<p>There were only five of us on stage, as the prize is for Inria permanent
staff. But this is of course not a fair account of how the project has
grown and what made it successful.</p>
<p>In 2011, at <a class="reference external" href="scikit-learn-nips-2011-sprint-international-thanks-to-our-sponsors.html">the first international sprint</a>,
I felt something was happening: Incredible people whom I had never met
before were sitting next to me, working very hard on solving problems
with me. This experience of being united to solve difficult problems is
something amazing. And I deeply thank every single person who has worked
on this project, the 1500 contributors, many of those that I have never
met, in particular <a class="reference external" href="https://scikit-learn.org/stable/about.html#authors">the core team</a> who is committed
to making sure that every detail of scikit-learn is solid and serves the
users. The team that has assembled over the years is of incredible
quality.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="align-right docutils container">
The promises of data science need open source</div>
<p>The world does not understand how much the promises of data science,
for today and tomorrow, need open source projects, easy to install and to use
by everybody. These projects are like <a class="reference external" href="https://www.fordfoundation.org/work/learning/research-reports/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure/">roads and bridges</a>:
they are needed for growth thought no one wants to pay for maintaining
them. I hope that I can use the podium that the prize will give us to
stress the importance of the battle that we are fighting.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Getting funding from the government implied too much politics and
risks. For these reasons, I turned to private donors, in a
<a class="reference external" href="https://scikit-learn.fondation-inria.fr/">foundation</a>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Inria <em>always</em> supported us, and often paid developers in my team
out of its own pockets.</td></tr>
</tbody>
</table>
<hr class="docutils" />
<p>PS: As an another illustration of the culture change toward openness in
science, it was announced during the ceremony that the <a class="reference external" href="https://en.wikipedia.org/wiki/Comptes_rendus_de_l%27Acad%C3%A9mie_des_Sciences">“Compte Rendu de
l’Académie des Sciences”</a> is becoming open access, without publication
charges!</p>
2018: my scientific year in review2019-01-03T00:00:00+01:002019-01-03T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2019-01-03:/science/2018-my-scientific-year-in-review.html<p>From a scientific perspective, 2018 <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> was once again extremely exciting
thank to awesome collaborators (at <a class="reference external" href="https://team.inria.fr/parietal/">Inria</a>, with <a class="reference external" href="https://project.inria.fr/dirtydata/">DirtyData</a>, and our <a class="reference external" href="https://scikit-learn.fondation-inria.fr/">local scikit-learn team</a>).
Rather than going over everything that we did in 2018, I would like to
give a few highlights: We published major work using <strong>machine learning to …</strong></p><p>From a scientific perspective, 2018 <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> was once again extremely exciting
thank to awesome collaborators (at <a class="reference external" href="https://team.inria.fr/parietal/">Inria</a>, with <a class="reference external" href="https://project.inria.fr/dirtydata/">DirtyData</a>, and our <a class="reference external" href="https://scikit-learn.fondation-inria.fr/">local scikit-learn team</a>).
Rather than going over everything that we did in 2018, I would like to
give a few highlights: We published major work using <strong>machine learning to
map cognition in the brain</strong>, We started a new research project on <strong>analysis
of non-curated data</strong> (addressing all of data science, beyond brain
imaging); And we worked a lot on <strong>growing scikit-learn</strong>.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>It’s already 2019, I am indeed late in posting this summary.</td></tr>
</tbody>
</table>
<div class="contents topic" id="highlights">
<p class="topic-title">Highlights</p>
<ul class="simple">
<li><a class="reference internal" href="#cognitive-brain-mapping" id="toc-entry-1">Cognitive brain mapping</a></li>
<li><a class="reference internal" href="#data-science-without-data-cleaning" id="toc-entry-2">Data science without data cleaning</a></li>
<li><a class="reference internal" href="#scikit-learn-growth-and-consolidation" id="toc-entry-3">Scikit-learn: growth and consolidation</a></li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="cognitive-brain-mapping">
<h2><a class="toc-backref" href="#toc-entry-1">Cognitive brain mapping</a></h2>
<p>We have been exploring <strong>how predictive models can help mapping cognition
in the human brain</strong>. In 2018, these long-running efforts led to important
publications.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="atlases-of-cognition-with-large-scale-human-brain-mapping">
<h3>Atlases of cognition with large-scale human brain mapping</h3>
<p>More than 6 years ago, with my student Yannick Schwartz, we started
working on <strong>compiling an altases of cognition across many cognitive
neuroimaging studies</strong>. This turned out to be quite challenging for several
reasons:</p>
<ul class="simple">
<li><strong>Formalizing the links between mental processes</strong> studied across the
literature is challenging. Strictly speaking, every paper studies a
different mental process. However, to build an atlas of cognition, we
are interested in finding commonalities across the literature.</li>
<li>While cognitive studies tend to target a specific mental function,
the psychological manipulations that they use also recruit many other
processes. For instance, a memory study might use a <em>visual n-back</em>
task, and hence recruit the visual cortex. The problem is more than an
experimental inconvience: <strong>varying details of an experiment may
trigger different cognitive processes</strong>. For instance, there are common
and separate pathways for visual word recognition and auditory word
recognition.</li>
<li>Simply <strong>detecting regions that are recruited in a given mental operation
leads to selecting the whole cortex</strong> with enough statistical power. Indeed
tasks are never fully balanced; reading might for instance require more
attention than listening.</li>
</ul>
<p>These challenges are related on the one hand to the problem of <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1364661305003360">reverse
inference</a>
<a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>, and on the other hand to that of mental-process decomposition, or
cognitive subtraction, both central to cognitive neuroimaging. They also
call for formal knowledge representation, <em>eg</em> by building ontologies,
which is a task harder than it might seem at first glance.</p>
<table class="side-hanging docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>In essence, the reverse inference problem arises because in a
cognitive brain imaging the observed brain activity is a consequence
of the behavior, and not a cause. While a conclusion that activity in
a brain structure causes a certain behavior is desirable, it is not
directly supported by a cognition neuroimaging experiment.</td></tr>
</tbody>
</table>
<p>In our work <a class="reference external" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006565">[Varoquaux et al, PLOS 2018]</a>,
we tackled these challenges to build atlases of cognition as follows:</p>
<ul class="simple">
<li>We assigned to each brain-activity image labels describing the
<em>multiple</em> mental processes related to the experimental manipulation</li>
<li>We used decoding –<em>ie</em> prediction of the cognitive labels from the brain
activity– to ground a principled <em>reverse inference</em> interpretation:
regions selected indeed do imply the corresponding behavior.</li>
<li>Regions in the atlas were built of brain structures that both implied
the corresponding cognition, and were triggered by it (conditional and
marginal link), to ground a strong selectivity:</li>
</ul>
<div class="figure align-center">
<a class="reference external image-reference" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006565"><img alt="" src="attachments/2018_highlights/mapping_types.png" style="width: 700px;" /></a>
</div>
<p>We applied these techniques to the data from 30 different studies,
resulting in a detailed break down of the cortex in functionally-specialized
modules:</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006565"><img alt="" src="attachments/2018_highlights/cognitive_regions.png" style="width: 700px;" /></a>
</div>
<p>Importantly, the validity of this decomposition in regions is established
by the ability of these regions to predict the cognitive aspects of new
experimental paradigms.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="predictive-models-avoid-excessive-reductionism-in-cognitive-neuroimaging">
<h3>Predictive models avoid excessive reductionism in cognitive neuroimaging</h3>
<div class="figure align-right">
<img alt="" src="attachments/2018_highlights/decoding.png" style="width: 400px;" />
</div>
<p>While machine learning is generally seen as an engineering tool to build
predictive models or automate tasks, I see in it a central method of
modern science. Indeed, it can distill <strong>evidence that generalizes</strong> from
vast –high dimensional– and ill-structured experimental data. Beyond
prediction, it can guide understanding.</p>
<p>With Russ Poldrack, we wrote an opinion paper <a class="reference external" href="https://hal.archives-ouvertes.fr/hal-01856412/">[Varoquaux & Poldrack,
Curr Opinion Neurobio 2019]</a> that details why
predictive models are important tools to building wider theories of brain
function. It reviews many exciting progresses in uncovering with
predictive models how brain mechanisms support the mind. It makes the
point that <strong>ability generalize is a fundamentally desirable priority of
scientific inference</strong>. Models that are grounded on explicit
generalization give a solid path to build broad theories of the mind.
Particularly interesting is generalization to significantly different
settings, <em>ie</em> going further than typical cross-validation experiments of
machine learning, where identical data are artificially split.</p>
<p>Something that is dear to my heart is that we are aiming for
<strong>quantitative generalization</strong>, while psychology often contents itself
with qualitative generalization.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="individual-brain-charting-a-high-resolution-fmri-dataset-for-cognitive-mapping">
<h3>Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping</h3>
<p>We are convinced about the importance of analyzing brain response across
multiple paradigms, to build models of brain function that generalize
across these paradigms. However, addressing such a research program by
aggregating multiple studies is hindered by data heterogeneity, due to
inter-individual differences or to differing scanners.</p>
<p>Hence, my team, <a class="reference external" href="https://team.inria.fr/parietal/">Parietal</a>, has
undertook a major data acquisition, the <a class="reference external" href="https://project.inria.fr/IBC">Individual Brain Charting
project</a>: <strong>scanning a few individuals
under a huge amount of cognitive tasks</strong>. The data acquisition will last
for many years, as the individuals come back to the lab for new
acquisitions. The images are of excellent quality, thanks to the unique
expertise of our scanning site, Neurospin, a brain-imaging research
facility.</p>
<p>The data are completely <strong>openly accessible</strong>: the raw data, preprocessed
data, statistical outputs, alongside with the processing script. We are
releasing new data as the project moves forward. This year, we published
the data paper <a class="reference external" href="https://www.nature.com/articles/sdata2018105">[Pinho et al, Scientific Data 2018]</a>.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title">Data accumulation in brain imaging</p>
<p>We are living exciting times, as <strong>there are more and more large volumes
of shared brain imaging data</strong>. <a class="reference external" href="https://openfmri.org/">OpenfMRI</a>
aggregates data in a consistent way across brain-imaging
studies. Large projects such as the Human Connectome Project, our
Individual Brain Charting project, or the UK BioBank, are designed
from the beginning to be shared. We are entering an era of
brain-image analysis on many terabytes of data, with dozens of
thousands of subjects, compounding hundreds of different clinical or
cognitive conditions.</p>
<p>Massive data accumulation opens exciting new scientific prospects,
and raises new engineering challenges. Some of these challenges are
to scale up neuroimaging data-processing practices, eg inter-subject
alignments at the scale of many thousands subjects. Some of these
challenges are new to neuroimaging: <strong>when compounding hundreds of
sources of data into an analysis, the human cost of data
integration becomes a major roadblock</strong>. As I have become convinced
that analysing more, and more diverse, data is an important way
forward, I have started working on data intergration per se.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="data-science-without-data-cleaning">
<h2><a class="toc-backref" href="#toc-entry-2">Data science without data cleaning</a></h2>
<div class="section" id="a-new-personal-research-agenda-dirtydata">
<h3>A new personal research agenda: DirtyData</h3>
<p>Challenges to integrating data in a statistical analysis are ubiquitous,
including in brain imaging. Data cleaning <a class="reference external" href="https://www.kaggle.com/surveys/2017">is recognized</a> as the number one time sink for
data scientists. When advising scikit-learn users, including very large
companies, I often find that the major roadblock is going from the raw
data sources to the data matrix that is input to scikit-learn.</p>
<p>A year ago, I started a new research focus, around the <a class="reference external" href="https://project.inria.fr/dirtydata">DirtyData project</a>. We now have a team with multiple
exciting collaborations, and funding. Our goal is to <strong>facilitate
statistical analysis of non-curated data</strong>. We hope to foster better
understanding of how powerful machine-learning models can cope with
imperfect, non homogeneous data. As we go, we will publish this
understanding, but also distribute code with new methods, and hopefully
influence common data-science practices and software. This is an exciting
adventure (and yes, <strong>we are hiring</strong>; see our <a class="reference external" href="https://project.inria.fr/dirtydata/job-offers">job offers</a> or contact me).</p>
<p>The topics are vast, at the intersection between database research and
statistics. In particular, it calls for integrating machine learning
with:</p>
<ul class="simple">
<li>Knowledge representation</li>
<li>Information retrieval</li>
<li>Information extraction</li>
<li>Statistics with missing data</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="similarity-encoding-analysis-with-non-normalized-string-categories">
<h3>Similarity encoding: analysis with non-normalized string categories</h3>
<p>While the DirtyData project is young, we already made progress for
analysis of <strong>dirty categories, ie categorical data represented with
strings that lack curation</strong>. These can have typos or other simple
morphological variants (<em>eg</em> “patient” vs “patients”), or they can have
more structured and fundamental differences, <em>eg</em> arising from the merge
of multiple data sources. This latter problem is well-known of database
research, where it is seen as a <em>record linkage</em> or <em>alignment</em> problem.</p>
<p>For statistical analysis, in particular machine learning, the problem
with these non-curated string categories is that they must be encoded to
numerical representations, and classic categorical encodings are not well
suited for them. For instance, one-hot encoding leads to very high
cardinality.</p>
<p>In <a class="reference external" href="https://hal.inria.fr/hal-01806175">Cerda et al (2018)</a>, we
contribute a simple encoding approach, <em>similarity encoding</em>, based on
interpolating one-hot encoding with string similarities between the
categories.</p>
<div class="figure align-center">
<a class="reference external image-reference" href="https://dirty-cat.github.io/stable/auto_examples/01_investigating_dirty_categories.html"><img alt="" src="attachments/2018_highlights/investigating_dirty_categories.png" style="width: 600px;" /></a>
</div>
<div class="figure align-right">
<a class="reference external image-reference" href="https://dirty-cat.github.io/stable/auto_examples/02_fit_predict_plot_employee_salaries.html"><img alt="" src="attachments/2018_highlights/predict_employee_salaries.png" style="width: 230px;" /></a>
</div>
<p>We ran an extensive empirical study, and show that <strong>similarity encoding
leads to better prediction accuracy without curation of the data</strong>,
outperforming all the other approaches that we tried. The paper is purely
empirical, but stay tuned: a theoretical analysis of why this is the case
is coming soon.</p>
<p>For the benefit of data scientists and researchers, we are released a
small Python package, <a class="reference external" href="https://dirty-cat.github.io/stable/">dirty-cat</a>,
for learning with dirty categories.</p>
<p>This is just the beginning of the DirtyData project, more exciting work
is under way.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="scikit-learn-growth-and-consolidation">
<h2><a class="toc-backref" href="#toc-entry-3">Scikit-learn: growth and consolidation</a></h2>
<img alt="" class="align-right" src="attachments/2018_highlights/scikit-learn-logo-notext.png" style="width: 150px;" />
<p>In 2018, a lot of my energy went to consolidating scikit-learn as a
project. Describing the work in detail is for another post. However, my
main efforts where around growing the team and working on sustainability.</p>
<ul class="simple">
<li>We established a <a class="reference external" href="https://scikit-learn.fondation-inria.fr/">scikit-learn foundation at Inria</a>, in which companies
partner with us to fund scikit-learn development. This took a lot of
effort to establish good partnerships and create the legal vessels.
Indeed, we want to make sure that the common effort is invested to make
scikit-learn better. For instance, working with Intel, who are somewhat
running an arms race for computing speed, we improved our test suite,
and are slowly but surely learning how to improve our speed.</li>
<li>A consequence of the foundation is that we are hiring to grow the team
(check out <a class="reference external" href="https://scikit-learn.fondation-inria.fr/people/">our open positions</a>). In 2018, my own
team grew, with more excellent people working on scikit-learn, but also
<a class="reference external" href="http://joblib.readthedocs.io/">joblib</a>, and even contributing to
core Python and numpy to improve <a class="reference external" href="https://github.com/python/cpython/pull/3895">parallel computing</a> and <a class="reference external" href="https://github.com/numpy/numpy/pull/12133">pickling</a>.</li>
<li>As the scikit-learn community is growing, it seemed important to
formalize a bit more how decisions are made. To me, an important aspect
was laying out clearly that the project is still governed by the
community, and not partners or people paid by the foundation. We have a
draft of a <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/12878">governance document</a>, that is
pretty much ready for merge. We also worked on a <a class="reference external" href="https://scikit-learn.org/dev/roadmap.html">roadmap</a>. It is a non binding
document, but it still was an interesting exercise.</li>
<li>Scikit-learn 0.20 was released, <a class="reference external" href="https://scikit-learn.org/dev/whats_new.html">with many enhancements</a>. And the 0.20 release
was followed by two minor releases, to make sure that our users got
robust code with backward compatibility.</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>We are busy finishing a few very interesting studies; next year will be
exciting! I hope that we will have much to say about population analysis
with brain imaging, which is a amazingly interesting subject.</p>
</div>
A foundation for scikit-learn at Inria2018-09-17T00:00:00+02:002018-09-17T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2018-09-17:/programming/a-foundation-for-scikit-learn-at-inria.html<p>We have just announced that a foundation will be supporting scikit-learn
at Inria <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>: <a class="reference external" href="https://scikit-learn.fondation-inria.fr">scikit-learn.fondation-inria.fr</a></p>
<div class="align-right docutils container">
Growth and sustainability</div>
<p>This is an exciting turn for us, because it enables us to receive private
funding. As a result, we will be able to have secure employment for some
existing core …</p><p>We have just announced that a foundation will be supporting scikit-learn
at Inria <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>: <a class="reference external" href="https://scikit-learn.fondation-inria.fr">scikit-learn.fondation-inria.fr</a></p>
<div class="align-right docutils container">
Growth and sustainability</div>
<p>This is an exciting turn for us, because it enables us to receive private
funding. As a result, we will be able to have secure employment for some
existing core contributors, and to hire more people on the team. The goal
is to help sustaining quality (more frequent releases?) and to tackle
some ambitious features.</p>
<div class="section" id="a-foundation-what-and-why">
<h2>A foundation? What and why?</h2>
<p>Open source lives and thrives by its base, the community of developers.
And scikit-learn is a fantastic example of these dynamics. Because of its
grass-root origins, it has focused on features that matter for the small
and the many, such as ease of use and statistical models that work well
in data-poor situations. Over the years, decisions have been based on
their technical merit, rather than the importance of displaying a list of
features that are trendy. A consequence of the breadth of contributors
with different backgrounds is the library tends to be well-suited for
many applications, including some models that are less mainstream.</p>
<div class="align-right docutils container">
People with dedicated time to support the community</div>
<p>That said, over time this is an increasing need for a core team of
maintainers. As the library gets bigger, is it more and more difficult to
have a full view of what is happening. Integration of new features,
quality assurances, and releases are best done by developers who can
dedicate a large amount of time to the library. Also, ambitious changes
to the library, such as improving the parallel computing engine, need
long efforts. For many years, we have always had people with dedicated
time to support the community. In France, we were going through hoops to
find public money to found them. As someone who has done this effort, I
can tell you that is a complicated one <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>.</p>
<p>The ability to receive money from sponsors will enable us to scale up our
operations. I was initially worried that we would have difficulties
finding partners that accepted to give us money without asking for
control on the project. However, I was proven wrong, and we have found a
small set of <a class="reference external" href="https://scikit-learn.fondation-inria.fr/en/home/#sponsors">great partners</a>.</p>
</div>
<div class="section" id="what-will-people-work-on-how-will-decisions-be-made">
<h2>What will people work on? How will decisions be made?</h2>
<p>It can be a difficult exercise to balance how money is used in a
community-driven project. The project should not loose its drive where
the community of developers is important. Interests of the sponsors
should not prime over interests of the user base.</p>
<p>We will make sure that the money that the foundation receives is invested
for the interest of the community. We have a technical committee that
supervises the activity of the foundation. Its decisions will be informed
by the community <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>. For this, we have an advisory board composed of
core contributors of scikit-learn. Beside the advisory board, the
technical committee also comprises a delegate from each sponsor. I am
excited about the input that our partners will provide us on
the priorities for them, as they represent various industries.
Voting power will be spread so that sponsors and community have the same
voting power.</p>
</div>
<div class="section" id="why-not-an-existing-foundation-such-as-numfocus-or-the-psf">
<h2>Why not an existing foundation such as NumFOCUS, or the PSF?</h2>
<p>There are several reasons why we choose this particular legal vessel. Our
endeavor is slightly from the prominent foundations in our ecosystem,
<a class="reference external" href="https://numfocus.org">NumFocus</a> and the <a class="reference external" href="https://www.python.org/psf">PSF (Python Software
Foundation)</a>.</p>
<p>The first important aspect is that we want to employ full-time
developers. Different countries have very different legal frameworks, and
it is really hard to transfer money overseas in a non profit. Physical
assets like employing people or owning real estate is even harder. We
needed something in France. And there might be a need for something else
in another country at some point.</p>
<p>Another reason to be embedded in the Inria foundation is that it is
giving us a really good deal. We basically get legal advice, accounting,
office space, and IT support, for an 8% overhead. This is an excellent
deal and is part of the sponsoring efforts that Inria will keep doing.</p>
<p>Last, we feel that a foundation targeting specifically scikit-learn can
raise money from different people than other foundations. I think that
there is value having multiple foundations seeking money for open-source
software. Indeed, a foundation builds a case and an image, to convince
donors. Different donors require a different case and a different image.
For instance the president of NumFOCUS <a class="reference external" href="https://twitter.com/aterrel/status/1039488246454083585">argues for a name less focused on
numerics</a>. Yet,
too wide of a scope can dilute the image.</p>
<p>We have in mind to make it easy for other foundations to support
scikit-learn. We have majors contributors at leading institutions, such
as Andreas Mueller at Columbia or Joel Nothman at Sydney university. It
is important that these institutions can easily gather donations too, in
the legal framework suited to their country. Hence the name reflects that
the foundation is embedded at Inria, leaving room for other initiatives.</p>
</div>
<div class="section" id="what-s-the-scope">
<h2>What’s the scope?</h2>
<p>The scope of our work is everything scikit-learn related. It is not the
whole pydata or scipy ecosystem: it is focused on scikit-learn. But we
will not hesitate contribute fixes and enhancements to neighboring
projects, like in the past, even all the way up to core Python <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a>.</p>
<hr class="docutils" />
<p>I’m am very excited. A strong team of full-time contributors will allow
us to do ambitious things with scikit-learn.</p>
<div class="topic">
<p class="topic-title"><strong>Join us</strong></p>
<p>We will be recruiting! See <a class="reference external" href="https://scikit-learn.fondation-inria.fr/people">our positions</a>. Come work with us
in Paris.</p>
</div>
<p>I want to end by thanking the amazing men and women who have been
contributing to scikit-learn, and are with us in this fantastic
adventure! The energy that is in this project is incredible. We are
are launching this effort thank to you, and to empower you even more.</p>
<img alt="" class="align-center" src="attachments/code_sklearn_crop.jpg" style="width: 90%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>I am quite proud that over the years, my group has employed
<a class="reference external" href="https://github.com/ogrisel">Olivier Grisel</a>, <a class="reference external" href="https://github.com/jorisvandenbossche">Joris van den Bossche</a> (working on pandas in
addition to scikit-learn), <a class="reference external" href="https://github.com/glemaitre">Guillaume Lemaître</a> (working on imbalanced-learn in
addition to scikit-learn), <a class="reference external" href="https://github.com/jeremiedbb">Jérémie du Boisberranger</a>,
<a class="reference external" href="https://github.com/tomMoral">Tom Moreau</a>,
<a class="reference external" href="https://github.com/lesteve">Loic Estève</a>,
<a class="reference external" href="https://github.com/fabianp">Fabian Pedregosa</a>, to name only a
few. All these people, and the many others students that we have
payed part time to work on software, have had an structuring
impact on our ecosystem, going beyond the bounds of scikit-learn
and touching many aspects of computing in Python. However, because
of the constraints of research funding in France, public money
forced my to hire them with short-term contracts.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Technically, it is a tax-deductible scikit-learn consortium inside
the Inria foundation, which is an non-profit entity related to Inria.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Details on the goverance of the foundation can be found at
<a class="reference external" href="https://scikit-learn.fondation-inria.fr/en/mission-and-governance">https://scikit-learn.fondation-inria.fr/en/mission-and-governance</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>For instance Olivier and Tom have been making parallelism more
robust in Python 3.7 (amongst various issues
<a class="reference external" href="https://bugs.python.org/issue33056">https://bugs.python.org/issue33056</a> and
<a class="reference external" href="https://bugs.python.org/issue31699">https://bugs.python.org/issue31699</a>). Olivier helped defining the
<a class="reference external" href="https://www.python.org/dev/peps/pep-0574/">new pickling protocol</a>, crucial to
efficient persistence.
This is hard work. Yet it is
important, because it benefits all libraries.</td></tr>
</tbody>
</table>
</div>
Sprint on scikit-learn, in Paris and Austin2018-08-01T00:00:00+02:002018-08-01T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2018-08-01:/programming/sprint-on-scikit-learn-in-paris-and-austin.html<p>Two weeks ago, we held a scikit-learn sprint in Austin and Paris. Here is
a brief report, on progresses and challenges.</p>
<div class="topic">
<p class="topic-title">Several sprints</p>
<p>We actually held two sprint in Austin: one open sprint, at the scipy
conference sprints, which was open to new contributors, and one core
sprint, for more …</p></div><p>Two weeks ago, we held a scikit-learn sprint in Austin and Paris. Here is
a brief report, on progresses and challenges.</p>
<div class="topic">
<p class="topic-title">Several sprints</p>
<p>We actually held two sprint in Austin: one open sprint, at the scipy
conference sprints, which was open to new contributors, and one core
sprint, for more advanced contributors. Thank you to all who joined
the scipy conference sprint. As I wasn’t there, I cannot report on
it.</p>
</div>
<div class="section" id="many-achievements">
<h2>Many achievements</h2>
<p>Too many things were done to be listed here. Here is brief overview:</p>
<ul class="simple">
<li><strong>Optics got merged</strong>: <a class="reference external" href="http://scikit-learn.org/dev/modules/clustering.html#optics">The optics clustering algorithm</a> is a
density-base clustering, as DBScan, but with hyperparameters more
flexible and easier to set. Our implementation is also more scaleable
for very large number of samples. The <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/1984">Pull request</a> was opened
in 2013, and got many many improvements over the years.</li>
<li><strong>Yeo-Johnson</strong>: <a class="reference external" href="http://scikit-learn.org/dev/modules/preprocessing.html#mapping-to-a-gaussian-distribution">The Yeo-Johnson transform</a>
is a simple parametric transformation of the data that can be used to
make it more Gaussian. It is similar to the Box-Cox transform but can
deal with negative data
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11520">PR</a>).</li>
<li><strong>Novelty versus outlier detection</strong>: Novelty detection attempts to
find on new data observations that differ from train data. Outlier
detection considers that even in the train data there are aberrant
observation. New modes in scikit-learn enable both usage scenario with
the same algorithms (see <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/issues/8693">this issue</a> and <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/10700">this
PR</a>).</li>
<li><strong>Missing-value indicator</strong>: a new transform that adds indicator columns
marking missing data
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/8075">PR</a>).</li>
<li><strong>Pypy support</strong>: pypy support was merged.
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11010">PR</a>).</li>
<li><strong>Random Forest with 100 estimators</strong> The default of <cite>n_estimator</cite> in
RandomForest was changed from 10, which was fast but statistically
poor, to 100 (<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11542">PR</a>).</li>
<li><strong>Changing to 5-fold</strong>: we changed to default of cross-validation from
3-fold to 5-fold
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11557">PR</a>).</li>
<li><strong>Toward release 0.20</strong>: most of the effort of the sprint was actually
spent on addressing issues for the 0.20 release: a long list of quality
improvements
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/milestone/24">milestone</a>).</li>
</ul>
</div>
<div class="section" id="scikit-learn-is-hard-work">
<h2>Scikit-learn is hard work</h2>
<div class="figure align-right">
<img alt="" src="attachments/dev_scikit-learn.png" style="width: 300px;" />
<p class="caption">Even for the almighty @amueller</p>
</div>
<p>Two days of intense group work on scikit-learn reminded me how much it is
hard work. I thought that it was maybe a good idea to try to illustrate
why.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<ul class="simple">
<li><strong>Mathematical errors</strong>: maintaining the library requires mathematical
understanding of the models. For instance Ivan Panico <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11585">fixed the sparse
PCA</a>, for
which the transform was mathematically incorrect.</li>
<li><strong>Numerical instabilities</strong>: sometimes, however, when models give a
result different from the expected one, this is due to numerical
instability. For instance, Sergul Aydöre <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11587">changed the tolerance for
certain variants of ridge</a></li>
<li><strong>Keeping examples and documentation up to date</strong>:
Each change requires changing all documentation and examples. We have a
lot these. For instance, Alex Boucault <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11557">had to update many examples and
documentation pages when changing the default cross-validation</a>.</li>
<li><strong>Clean deprecation path</strong>: We make sure that our changes do not break
users code, and therefore we provide a smooth update path, with
progressive deprecations. For instance, <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11557">the change of default
cross-validation</a> introduce
an intermediate step where the default is kept the same and warns that
it will change in two releases.</li>
<li><strong>Consistent behavior across the library</strong>:
One of the acclaimed values of scikit-learn is that it has a very
consistent behavior across different models. We enforce this by “common
tests”, that test some properties of the estimators altogether. For
instance, Sergul implemented <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11558">common tests for sample weights</a>.</li>
<li><strong>Extensive testing</strong>: We test many many things in scikit-learn:
that the code snippets in the documentation are correct, that <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/11421">the
docstring conventions</a> are
respected, that there are no deprecation errors raised, including from
our dependencies. As a results, continuous integration is a core part
of our development. During the sprint, we flooded our cloud-based
continuous integration, and as a result iteration really slowed down.
<a class="reference external" href="https://travis-ci.org/">TravisCI</a> were kind enough to fix this by
allocating us freely more computing power.</li>
<li><strong>Supporting many versions</strong>: Least by not least, one constraint that
makes development hard with scikit-learn is that we support many
different versions of Python, of our dependencies, of linear-algebra
libraries, and of operating system. This makes development harder, and
continuous integration slower. But we feel that this is very valuable
for a core library: narrowing the supported versions means that users
are more likely to end up in unsatisfiable dependencies situations,
where different parts of a project want different version numbers of a
dependency.</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p><strong>Dropping support for Python 2</strong></p>
<p>Supporting many version slows development. It also prevents
implementing new features: supporting Python 2 makes it harder to
provide better parallelism or traceback management.</p>
<p class="last">Python 3 has been out for 10 years. It is solid and comes with many
improvements over Python 2. Alongside with <a class="reference external" href="http://python3statement.org">many other projects</a>, we will be requiring Python 3 for
the future releases of scikit-learn (0.21 and later). scikit-learn
0.20 will be the last release to support Python 2. It will enable
us to develop faster a better toolkit.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="credits-and-acknowledgments">
<h2>Credits and acknowledgments</h2>
<div class="section" id="contributors-to-the-sprint">
<h3>Contributors to the sprint</h3>
<div class="sidebar">
<p class="first sidebar-title">Women contributors</p>
<p class="last">We deeply regret having only one woman in this long list of
contributors. We care about diversity and welcome contributors from
under-represented groups <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[*]</a>.</p>
</div>
<p><strong>In Paris</strong></p>
<ul class="columns simple">
<li>Albert Thomas, Huawey</li>
<li>Alexandre Boucaud, Inria</li>
<li>Alexandre Gramfort, Inria</li>
<li>Eric Lebigot, CFM</li>
<li>Gaël Varoquaux, Inria</li>
<li>Ivan Panico, Deloitte</li>
<li>Jean-Baptiste Schiratti, Telecom ParisTech</li>
<li>Jérémie du Boisberranger, Inria</li>
<li>Léo Dreyfus-Schmidt, Dataiku</li>
<li>Nicolas Goix</li>
<li>Samuel Ronsin, Dataiku</li>
<li>Sebastien Treguer, Independent</li>
<li>Sergül Aydöre, Stevens Institute of Technology</li>
</ul>
<p><strong>In Austin</strong></p>
<ul class="columns simple">
<li>Andreas Mueller, Columbia</li>
<li>Guillaume Lemaître, Inria</li>
<li>Jan van Rijn, Columbia</li>
<li>Joan Massich, Inria</li>
<li>Joris Van den Bossche, Inria</li>
<li>Loïc Estève, Inria</li>
<li>Nicolas Hug, Columbia</li>
<li>Olivier Grisel, Inria</li>
<li>Roman Yurchak, independent</li>
<li>William de Vazelhes, Inria</li>
</ul>
<p><strong>Remote</strong></p>
<ul class="columns simple">
<li>Hanmin Qin, Peking University</li>
<li>Joel Nothman, University of Sydney</li>
</ul>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="sponsors">
<h3>Sponsors</h3>
<ul class="simple">
<li><a class="reference external" href="https://franceisai.com/">France Is AI</a> payed the travel of the French
contributors to Austin</li>
<li>The NSF and the Sloan foundation payed the travel of the people from
Columbia.</li>
<li><a class="reference external" href="https://scipy2018.scipy.org">SciPy 2018</a> organizers (and sponsors) hosted the first part of the sprint in Austin,</li>
<li><a class="reference external" href="https://www.enthought.com/">Enthought</a> hosted the second part of the sprint in Austin,</li>
<li><a class="reference external" href="https://www.dataiku.com/">Dataiku</a> hosted us in Paris</li>
<li><a class="reference external" href="https://travis-ci.org/">TravisCI</a> raised our number of workers for
online testing</li>
<li><a class="reference external" href="https://www.meetup.com/Paris-Machine-learning-applications-group/">ParisML meetup</a> helped us with the organization</li>
</ul>
<p>Thank you all for the support</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Also thanks to Andy Mueller and Olivier Grisel for feedback on this blog post.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[*]</a></td><td>We aspire to treat everybody excatly the same way. However,
acknowledging the fact that there is currently a lack of diversity, we
are happy to do some outreach and give extra help onboarding
newcomers.</td></tr>
</tbody>
</table>
</div>
</div>
Our research in 2017: personal scientific highlights2017-12-31T00:00:00+01:002017-12-31T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2017-12-31:/science/our-research-in-2017-personal-scientific-highlights.html<p>In my opinion the scientific highlights of 2017 for <a class="reference external" href="https://team.inria.fr/parietal/">my team</a> were on multivariate predictive
analysis for brain imaging: a brain decoder more efficient and faster
than alternatives, improvement clinical predictions by predicting jointly
multiple traits of subjects, decoding based on the raw time-series of
brain activity, and a personnal …</p><p>In my opinion the scientific highlights of 2017 for <a class="reference external" href="https://team.inria.fr/parietal/">my team</a> were on multivariate predictive
analysis for brain imaging: a brain decoder more efficient and faster
than alternatives, improvement clinical predictions by predicting jointly
multiple traits of subjects, decoding based on the raw time-series of
brain activity, and a personnal concern with the small sample sizes we
use in predictive brain imaging…</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="a-fast-and-stable-brain-decoder-using-ensembling-frem">
<h2>A fast and stable brain decoder using ensembling: FReM</h2>
<p>We have been working for 10 years on methods for brain decoding:
predicting behavior from imaging. In particular, we developed state of
the art decoders based on <a class="reference external" href="http://ieeexplore.ieee.org/abstract/document/5711672/">total variation</a>.
In <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811917308182">Hoyos-Idrobo et al</a>
(<a class="reference external" href="https://hal.inria.fr/INRIA/hal-01615015v1">preprint</a>)
we used a different technique based on ensembling: combining many fast
decoders. The resulting decoder, dubbed <em>FReM</em>, predicts better, faster,
and with more stable maps than existing methods. Indeed, we have learned
that good prediction accuracy was not the only important feature of a
decoder.</p>
<div class="figure align-center">
<img alt="" src="attachments/2017_highlights/frem_benchmarks.png" style="width: 600px;" />
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="brain-imaging-to-characterize-individuals-joint-prediction-of-multiple-traits">
<h2>Brain imaging to characterize individuals: joint prediction of multiple traits</h2>
<p>In <em>population imaging</em>, individual traits are linked to their brain
images. Predictive models ground the development of imaging biomarkers.
In <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811917305438">Rahim et al</a>
(<a class="reference external" href="https://hal.inria.fr/hal-01547524/">preprint</a>), we showed that
accounting for multiple traits of the subjects when <em>learning</em> the
biomarker, gave a better prediction of the individual traits. For
instance, knowing the MMSE (mini mental state examination) of subjects
in a reference population helps derive better markers of Alzheimer’s
disease, even for subjects of unknown MMSE. This is an important step to
including a more complete picture of individuals in imaging studies.</p>
<div class="figure align-center">
<img alt="" src="attachments/2017_highlights/multi_output_decoder.jpg" style="width: 600px;" />
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="time-domain-decoding-for-fmri">
<h2>Time-domain decoding for fMRI</h2>
<p>In studies of cognition with functional MRI, the standard practice to
decoding brain activity is to estimate a first-level model that teases
appart the different experimental trials. It results in maps of regions
of the brains that correlate with each trial. Decoding is then run on
these maps, with supervised learning. The limitation of this approach is
that the experiment has to be designed with a good time separation
between each trial.</p>
<div class="figure align-right">
<img alt="" src="attachments/2017_highlights/time_domain_decoding.png" style="width: 300px;" />
</div>
<p>In <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811917306651">Loula et al</a>
(<a class="reference external" href="https://hal.inria.fr/hal-01576641/">preprint</a>) we designed a
<em>time-domain decoding</em> scheme, that starts from the raw brain activity
time-series and predicts model time-courses of cognition. From these, it
can classify the type of each trial. Importantly, it works better than
traditional approaches when the trials are not well separated. It thus
opens the door to decoding in experiments that were so far too fast.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="cross-validation-failure-the-dangers-of-small-samples">
<h2>Cross-validation failure: the dangers of small samples</h2>
<div class="figure align-right">
<img alt="" src="attachments/2017_highlights/sample_size_distribution.png" style="width: 300px;" />
</div>
<p>I wrote <a class="reference external" href="https://www.sciencedirect.com/science/article/pii/S1053811917305311">an opinion paper</a>
(<a class="reference external" href="https://hal.inria.fr/hal-01545002/">preprint</a>) on a problem of our
field that has been worrying me lot: <strong>often, we do not have enough
samples to assess properly the predictive power in neuroimaging</strong>.
Indeed, the typical predictive analysis in neuroimaging uses 100 samples.</p>
<div style="clear: both"></div><div class="figure align-right">
<img alt="" src="attachments/2017_highlights/binomial_cdf.png" style="width: 300px;" />
</div>
<p>The error distribution on the measure of prediction accuracy of a decoding
is at best given by a binomial. With around 100 samples, it yields
confidence bounds around ±7%. Analysis of neuroimaging studies reveals
larger error bars.</p>
<p>Such error bars, large compared to the effect of interest, undermine
publications using or developing predictive models in neuroimaging.
Indeed, they couple with the publication incentives in two ways. First,
studies that by chance observe an effect are published, while the others
end up unaccounted for in a <em>``file drawer``</em>. Second, minor
modifications to the data processing strategy give large but meaningless
differences on the observed prediction accuracy. These <em>researchers
degress of freedom</em> can hardly be checked in a review process or a
statistical test. The methods research, trying to improve decoders, is
hindered by such error bars and should consider multiple datasets to
gauge progress. Clinical neuroimaging, for biomarkers, must increase
sample sizes and face heterogeneity.</p>
<p>I believe that this is a major challenge for our field, and invite you to
read the paper if you are not convinced.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="convergence-proofs-for-last-year-s-blazing-fast-dictionary-learning">
<h2>Convergence proofs for last year’s blazing fast dictionary learning</h2>
<div class="figure align-center">
<img alt="" src="attachments/2017_highlights/online_dict_learning.png" style="width: 600px;" />
</div>
<p><a class="reference external" href="http://ieeexplore.ieee.org/abstract/document/8038072/">Mensch et al</a>
(<a class="reference external" href="https://hal.inria.fr/hal-01431618/">preprint</a>) is a long paper that
studies in detail our very fast dictionary learning algorithm, with
extensive experiments and convergence proofs. On huge matrices, such as
brain imaging data in population studies, hyperspectral imaging, or
recommender systems, is gives <strong>10 fold speedups for matrix factorization</strong>.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>We are busy finishing a few very interesting studies. Stay posted, next
year will be exciting!</p>
</div>
Beyond computational reproducibility, let us aim for reusability2017-09-19T12:10:00+02:002017-09-19T12:10:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2017-09-19:/programming/beyond-computational-reproducibility-let-us-aim-for-reusability.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Scientific progress calls for reproducing results. Due to limited
resources, this is difficult even in computational sciences. Yet,
reproducibility is only a means to an end. It is not enough by itself
to enable new scientific results. Rather, new discoveries must build
on reuse and modification of the state …</p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Scientific progress calls for reproducing results. Due to limited
resources, this is difficult even in computational sciences. Yet,
reproducibility is only a means to an end. It is not enough by itself
to enable new scientific results. Rather, new discoveries must build
on reuse and modification of the state of the art. As time goes, this
state of the art must be consolidated in software libraries, just as
scientific knowledge as been consolidated on bookshelves of
brick-and-mortar libraries.</p>
</div>
<div class="small docutils container">
I am reposting <a class="reference external" href="https://openlab-flowers.inria.fr/uploads/default/original/1X/65addc14bb2a6a7feaf7690865fa3708d5b0990f.pdf">an essay</a>
that I wrote on reproducible science and software libraries. The full
discussion is in <a class="reference external" href="https://openlab-flowers.inria.fr/t/ieee-cis-newsletter-on-cognitive-and-developmental-systems/129/1">IEEE CIS TC Cognitive and Developmental Systems</a>,
but I’ve been told that it is hard to find.</div>
<hr class="docutils" />
<p>Science is based on the ability to falsify claims. Thus, reproduction or
replication of published results is central to the progress of science.
Researchers failing to reproduce a result will raise questions:
Are these investigators not skilled enough? Did they misunderstand the
original scientific endeavor? Or is the scientific claim unfounded? For
this reason, the quality of the methods description in a research paper
is crucial. Beyond papers, computers —central to science in our digital
era— bring the hope of automating reproduction. Indeed, computers excel
at doing the same thing several times.</p>
<p>However, there are many challenges to computational reproducibility. To
begin with, computers enable reproducibility only if all steps of a
scientific study are automated. In this sense, interactive environments
—productivity-boosters for many— are detrimental unless they enable easy
recording and replay of the actions performed. Similarly, as a
computational-science study progresses, it is crucial to keep track of
changes to the corresponding data and scripts. With a
software-engineering perspective, version control is the solution. It
should be in the curriculum of today’s scientists. But it does not
suffice. Automating a computational study is difficult. This is because
it comes with a large maintenance burden: operations change rapidly,
straining limited resources —processing power and storage. Saving
intermediate results helps. As does devising light experiments that are
easier to automate. These are crucial to the progress of science, as
laboratory classes or thought experiments in physics. A software
engineer would relate them to unit tests, elementary operations checked
repeatedly to ensure the quality of a program.</p>
<div class="align-right docutils container">
Archiving computers in thermally-regulated nuclear-proof vaults?</div>
<p>Once a study is automated and published, ensuring reproducibility should
be easy; just a matter of archiving the computer used, preferably in a
thermally-regulated nuclear-proof vault. Maybe, dear reader, the
scientist in you frowns at this solution. Indeed, studies should also be
reproduced by new investigators. Hardware and software variations then
get in the way. Portability, <em>ie</em> achieving identical results across
platforms, is well-known by the software industry as being a difficult
problem. It faces great hurdles due to incompatibilities in compilers,
libraries, or operating systems. Beyond these issues, portability also
faces numerical and statistical stability issues in scientific computing.
Hiding instability problems with heavy restrictions on the environment is
like rearranging deck chairs on the Titanic. While enough freezing will
recover reproducibility, unstable operations cast doubt upon scientific
conclusions they might lead to. Computational reproducibility is more
than a software engineering challenge; it must build upon solid numerical
and statistical methods.</p>
<p>Reproducibility is not enough. It is only a means to an end, scientific
progress. Setting in stone a numerical pipeline that produces a figure is
of little use to scientific thinking if it is a black box. Researchers
need to understand the corresponding set of operations to relate them to
modeling assumptions. New scientific discoveries will arise from varying
those assumptions, or applying the methodology to new questions or new
data. Future studies build upon past studies, standing on the shoulders
of giants, as Isaac Newton famously wrote. In this process, published
results need to be modified and adapted, not only reproduced. Enabling
reuse is an important goal.</p>
<div class="align-right docutils container">
Libraries as reusable computational experiments</div>
<p>To a software architect, a reusable computational experiment may sound
like a library. Software libraries are not only a good analogy, but also
an essential tool. The demanding process of designing a good library
involves isolating elementary steps, ensuring their quality, and
documenting them. It is akin to the editorial work needed to assemble a
textbook from the research literature.</p>
<p>Science should value libraries made of code, and not only bookshelves.
But they are expensive to develop, and even more so to maintain. Where to
set the cursor? It is clear that in physics not every experimental setup
can be stored for later reuse. Costs are less tangible with computational
science; but they should not be underestimated. In addition, the race to
publish creates legions of studies. As an example, Google scholar lists
28000 publications concerning compressive sensing in 2015. Arguably many
are incremental and research could do with less publications. Yet the
very nature of research is to explore new ideas, not all of which are to
stay.</p>
<div class="align-right docutils container">
Identifying and consolidating major results for reuse</div>
<p>Computational research will best create scientific progress by
identifying and consolidating the major results. It is a difficult but
important task. These studies should be made reusable. Limited resources
imply that the remainder will suffer from “code rot”, with results
becoming harder and harder to reproduce as their software environment
becomes obsolete. Libraries, curated and maintained, are the building
blocks that can enable progress.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="grey docutils container">
If you want to cite this essay in an academic publication, please
cite the version in
<a class="reference external" href="https://openlab-flowers.inria.fr/t/ieee-cis-newsletter-on-cognitive-and-developmental-systems/129/1">IEEE CIS TC Cognitive and Developmental Systems</a>
(volume 32, number 2, 2016).</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Related posts</strong>:</p>
<ul class="simple">
<li><a class="reference external" href="software-for-reproducible-science-lets-not-have-a-misunderstanding.html">Software for reproducible science: let’s not have a misunderstanding</a></li>
<li><a class="reference external" href="mloss-2015-wising-up-to-building-open-source-machine-learning.html">MLOSS 2015: wising up to building open-source machine learning</a></li>
<li><a class="reference external" href="../science/publishing-scientific-software-matters.html">Publishing scientific software matters</a></li>
</ul>
</div>
Scikit-learn Paris sprint 20172017-06-23T00:00:00+02:002017-06-23T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2017-06-23:/programming/scikit-learn-paris-sprint-2017.html<object class="align-right" data="attachments/scikit-learn-logo.svg" style="width: 400px;" type="image/svg+xml"></object>
<p>Two week ago, we held in Paris a large international sprint on
<a class="reference external" href="http://scikit-learn.org">scikit-learn</a>. It was incredibly productive
and fun, as always. We are still busy merging in the work, but I think
that know is a good time to try to summarize the sprint.</p>
<div class="section" id="a-massive-workforce">
<h2>A massive workforce</h2>
<img alt="" class="align-center" src="attachments/sklearn_sprint_2017/P1060011.jpg" style="width: 100%;" />
<p>We had a …</p></div><object class="align-right" data="attachments/scikit-learn-logo.svg" style="width: 400px;" type="image/svg+xml"></object>
<p>Two week ago, we held in Paris a large international sprint on
<a class="reference external" href="http://scikit-learn.org">scikit-learn</a>. It was incredibly productive
and fun, as always. We are still busy merging in the work, but I think
that know is a good time to try to summarize the sprint.</p>
<div class="section" id="a-massive-workforce">
<h2>A massive workforce</h2>
<img alt="" class="align-center" src="attachments/sklearn_sprint_2017/P1060011.jpg" style="width: 100%;" />
<p>We had a mix of core contributors and newcomers, which is a great
combination, as it enables us to be productive, but also to foster the
new generation of core developers. Were present:</p>
<ul class="columns simple">
<li>Albert Thomas</li>
<li>Alexandre Abadie</li>
<li>Alexandre Gramfort</li>
<li>Andreas Mueller</li>
<li>Arthur Imbert</li>
<li>Aurélien Bellet</li>
<li>Bertrand Thirion</li>
<li>Denis Engemann</li>
<li>Elvis Dohmatob</li>
<li>Gael Varoquaux</li>
<li>Jan Margeta</li>
<li>Joan Massich</li>
<li>Joris Van den Bossche</li>
<li>Laurent Direr</li>
<li>Lemaitre Guillaume</li>
<li>Loic Esteve</li>
<li>Mohamed Maskani Filali</li>
<li>Nathalie Vauquier</li>
<li>Nicolas Cordier</li>
<li>Nicolas Goix</li>
<li>Olivier Grisel</li>
<li>Patricio Cerda</li>
<li>Paul Lagrée</li>
<li>Raghav RV</li>
<li>Roman Yurchak</li>
<li>Sebastien Treger</li>
<li>Sergei Lebedev</li>
<li>Thierry Guillemot</li>
<li>Thomas Moreau</li>
<li>Tom Dupré la Tour</li>
<li>Vlad Niculae</li>
</ul>
<ul class="simple">
<li>Manoj Kumar (could not come to Paris because of visa issues)</li>
</ul>
<p>And many more people participating remote, and I am pretty certain that I
forgot people.</p>
</div>
<div class="section" id="support-and-hosting">
<h2>Support and hosting</h2>
<p><strong>Hosting</strong>:
As the sprint extended through a French bank holiday and the week end,
we were hosted in a variety of venues:</p>
<ul class="simple">
<li><a class="reference external" href="http://lapaillasse.org">La paillasse</a>, a Paris bio-hacker space</li>
<li><a class="reference external" href="http://www.criteo.com">Criteo</a>, a French company doing word-wide
add-banner placement. The venue there was absolutely gorgeous, with a
beautiful terrace on the roofs of Paris. And they even had a social
event with free drinks one evening.</li>
</ul>
<p>Guillaume Lemaître did most of the organization, and at Criteo Ibrahim
Abubakari was our host. We were treated like kings during the whole stay;
each host welcoming us as well they could.</p>
<p><strong>Financial support by France is IA</strong>: Beyond our hosts, we need to thank
<a class="reference external" href="https://franceisai.com/">France is IA</a> who fund the sprint, covering
some of the lunches, accomodations, and travel expenses to bring in our
contributors from abroad (3000 euros travel & accomodation, and 1000
euros for food and a venue during the week end).</p>
</div>
<div class="section" id="some-achievements-during-the-sprint">
<h2>Some achievements during the sprint</h2>
<p>I would be hard to list everything that we did during the sprint (have a
look at the <a class="reference external" href="http://scikit-learn.org/dev/whats_new.html#version-0-14">development changelog</a> if you’re curious). Here are some</p>
<ul>
<li><p class="first">Quantile transformer, to transform the data distribution into uniform,
or Gaussian distributions
(<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/8363">PR</a>,
<a class="reference external" href="http://scikit-learn.org/dev/auto_examples/preprocessing/plot_all_scaling.html">example</a>):</p>
<p><strong>Before</strong></p>
<img alt="" src="attachments/sklearn_sprint_2017/original_distributions.png" style="width: 500px;" />
<p><strong>After</strong></p>
<img alt="" src="attachments/sklearn_sprint_2017/quantile_transform.png" style="width: 500px;" />
</li>
<li><p class="first">Memory saving by avoiding to cast to float64 if X is given as float32:
we are slowly making sure that, as much as possible, all models avoid
using internal representations of a dtype float64 when the data is
given as float32. This reduces significantly memory usage and can give
speed ups up to a factor of two.</p>
</li>
<li><p class="first">API test on instances rather than class. This is to facilitate testing
packages in <a class="reference external" href="https://github.com/scikit-learn-contrib/scikit-learn-contrib">scikit-learn-contrib</a>.</p>
</li>
<li><p class="first">Many small API fixes to ensure better consistency of models, as well as
cleaning the codebase, making sure that examples display well under
matplotlib 2.x.</p>
</li>
<li><p class="first">Many bug fixes, include fixing corner cases in our average precision,
which was dear to me (<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/9017">PR</a>).</p>
</li>
</ul>
<p><strong>Work soon to be merged</strong></p>
<ul class="simple">
<li>ColumnTransformer (<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/9012">PR</a>): from
pandas dataframe to feature matrix, by applying different transformers
to different columns.</li>
<li>Fixing t-SNE (<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/9032">PR</a>): our
t-SNE implementation was extremely memory-inefficient, and on top of
this had minor bugs. We are fixing it.</li>
</ul>
<p>There is a lot more of pending work that the sprint help moved forward.
You can also glance at the <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pulse/monthly">monthly activity report on github</a>.</p>
<p><strong>Joblib progress</strong></p>
<p><a class="reference external" href="https://pythonhosted.org/joblib/">Joblib</a>, the parallel-computing
engine used by scikit-learn, is getting extended to work in distributed
settings, for instance using dask distributed as a <a class="reference external" href="http://distributed.readthedocs.io/en/latest/joblib.html">backend</a>.
At the sprint, we made progress running a grid-search on Criteo’s Hadoop
cluster.</p>
<img alt="" class="align-center" src="attachments/sklearn_sprint_2017/P1060014.jpg" style="width: 100%;" />
</div>
Our research in 2016: personal scientific highlights2016-12-31T00:00:00+01:002016-12-31T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2016-12-31:/science/our-research-in-2016-personal-scientific-highlights.html<p>Year 2016 has been productive for science in <a class="reference external" href="https://team.inria.fr/parietal/">my team</a>. Here are some personal highlights:
bridging artificial intelligence tools to human cognition,
markers of neuropsychiatric conditions from brain activity at rest,
algorithmic speedups for matrix factorization on huge datasets…</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="artificial-intelligence-convolutional-networks-map-well-the-human-visual-system">
<h2>Artificial-intelligence convolutional networks map well the human visual system</h2>
<p><a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1053811916305481">Eickenberg et …</a></p></div><p>Year 2016 has been productive for science in <a class="reference external" href="https://team.inria.fr/parietal/">my team</a>. Here are some personal highlights:
bridging artificial intelligence tools to human cognition,
markers of neuropsychiatric conditions from brain activity at rest,
algorithmic speedups for matrix factorization on huge datasets…</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="artificial-intelligence-convolutional-networks-map-well-the-human-visual-system">
<h2>Artificial-intelligence convolutional networks map well the human visual system</h2>
<p><a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1053811916305481">Eickenberg et al</a>
(<a class="reference external" href="https://hal.inria.fr/hal-01389809/document">preprint</a>), showed that
convolutional networks –machine-learning tools developed in artificial
intelligence for image analysis– map well the human visual system. This
is interesting because it shows that cognitive vision and artificial
computer vision have evolved to similar architectures. It is not that
surprising, as they are both driven by the statistics of natural images.
From the point of view of inference in neuroscience, what I found really
interesting is that we demonstrated that our computational model of brain
activity generalizes across experimental paradigms. This is something new
to my knowledge.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="using-brain-activity-at-rest-to-predicting-autism-status-across-clinical-sites">
<h2>Using brain activity at rest to predicting Autism status across clinical sites</h2>
<p><a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1053811916305924">Abraham et al</a>
(<a class="reference external" href="https://arxiv.org/pdf/1611.06066">preprint</a>) used resting-state brain
activity to predict whether individuals were typical controls or
diagnosed with Autistic symptoms. The important aspect of this study
is that it was performed on a large data collection across many sites
that had not concerted each other during the acquisition. Given that
prediction was successful across sites, the study shows the viability of
extracting predictive biomarkers across inhomogeneous multi-site data. I
think that it is an important result for the future of psychiatric
neuroimaging research. The paper also highlights the aspects of the
predictive pipeline that were important for this success.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="dictionary-learning-for-massive-matrix-factorization">
<h2>Dictionary Learning for Massive Matrix Factorization</h2>
<p>On a pure machine-learning side, <a class="reference external" href="http://jmlr.org/proceedings/papers/v48/mensch16.html">Mensch et al</a> introduced a new
algorithm for matrix factorization that gives 10 times speedups compared
to the state of the art on absolutely huge datasets (Terabyte scales).
The key aspect is to combine online learning with random subampling that
exploits redundancies in the data. For neuroimaging, this algorithmic
advances is needed to tackle larger and larger resting-state data. We
will use it to scale predictive models to epidemiologic cohorts. The
original paper was purely heuristic but <a class="reference external" href="https://arxiv.org/pdf/1611.10041">later work</a> comes with proofs and we will soon
be submitting a very rich journal paper about this class of algorithms.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="a-guide-to-cross-validation-in-neuroimaging">
<h2>A guide to cross-validation in neuroimaging</h2>
<p>We published <a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S105381191630595X">a review on cross-validation for neuroimaging</a>
(<a class="reference external" href="https://arxiv.org/pdf/1606.05201">preprint</a>). While this may sound
less leading edge than other of our work, cross-validation is central to
everything we do. Doing it right is important. We learned some
interesting tradeoffs while doing the experiments for the review. One of
them is that for predictive models that are quite stable, such as SVMs,
it may be profitable to use default hyper-parameters than to tune them by
cross-validation. This is because with the small sample sizes typical of
neuroimaging cross-validation is fairly noisy.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Though not in my team, <a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1053811916306103">Liem et al</a>
(<a class="reference external" href="http://www.biorxiv.org/content/biorxiv/early/2016/11/07/085506.full.pdf">preprint</a>)
collaborated with us for a beautiful study showing multimodal prediction
of brain age from rest brain activity and brain anatomy. Interestingly,
they showed that discrepancy between predicted age and chronological age
captures cognitive impairment.</p>
<p>We have many interesting things in the pipeline, but it will be for next
year. On an unrelated note, I’ve been doing more <a class="reference external" href="http://www.flickriver.com/photos/gaelvaroquaux/popular-interesting/">art photography</a>
on my free time in 2016.</p>
</div>
Data science instrumenting social media for advertising is responsible for todays politics2016-11-11T00:00:00+01:002016-11-11T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2016-11-11:/programming/data-science-instrumenting-social-media-for-advertising-is-responsible-for-todays-politics.html<p><em>To my friends developing data science for the social media, marketing, and
advertising industries,</em></p>
<p>It is time to accept that we have our share of responsibility in the outcome of
the US elections and the vote on Brexit. We are not creating the
society that we would like. Facebook,
Twitter …</p><p><em>To my friends developing data science for the social media, marketing, and
advertising industries,</em></p>
<p>It is time to accept that we have our share of responsibility in the outcome of
the US elections and the vote on Brexit. We are not creating the
society that we would like. Facebook,
Twitter, targeted advertising, customer profiling, are harmful to truth
and have helped Brexiting and electing Trump. Journalism
has been replaced by social media and commercial content tailored to
influence the reader: your own personal distorted reality.</p>
<p>There are many deep reasons why Trump won the election. Here, as a
data scientist, I want to talk about the factors created by data science.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Rumor replaces truth</strong>: the way we, data-miners, aggregate and
recommend content is based on its popularity, on readership statistics.
In no way is it based in the truthfulness of the content. As a
result, Facebook, Twitter, Medium, and the like amplify rumors and
sensational news, with no reality check <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
<p>This is nothing new: clickbait and tabloids build upon it. However, social networking and
active recommendation makes things significantly worst. Indeed, birds of
a feather flock together, reinforcing their own biases. <strong>We receive
filtered information</strong>: have you noticed that every single argument you
heard was overwhelmingly against (or in favor of) Brexit? To make matters
even worse, our brain loves it: to resolve cognitive dissonance we avoid
information that contradicts our biases <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>.</p>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">We all believe more information when it confirms our biases</p>
</div>
<p>Gossiping, rumors, and propaganda have always made sane decisions
difficult. The <strong>filter bubble</strong>, algorithmically-tuned rose-colored
glasses of Facebook, escalate this problem into a major dysfunction of
our society. They amplify messy and false information better than
anything before. Soviet-style propaganda builds on a carefully-crafted
lies; post-truth politics build on a flood of information that does not
even pretend to be credible in the long run.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>Active distortion of reality</strong>: amplifying biases to the point that
they drown truth is bad. Social networks actually do worse: they give
tools for active manipulation of our perception of the world. Indeed, the
revenue of today’s Internet information engines comes from advertising.
For this purpose they are designed to learn as much as possible about the
reader. Then they sell this information bundled with a slot where the
buyer can insert the optimal message to influence the reader.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/benterrett/6929895752/"><img alt="" class="align-right" src="https://farm8.staticflickr.com/7212/6929895752_2e359557b8_z_d.jpg" style="width: 25%;" /></a>
<p>The Trump campaign used targeted Facebook ads presenting to
unenthusiastic democrats information about Clinton tuned to discourage
them from voting. For instance, <a class="reference external" href="http://www.theverge.com/2016/10/27/13434246/donald-trump-targeted-dark-facebook-ads-black-voters">portraying her as racist to black voters</a>.</p>
<p>Information manipulation works. The Trump campaign has been a smearing
campaign aimed at suppressing votes of his opponent. Release of
negative information on Clinton <a class="reference external" href="https://medium.com/@jonathonmorgan/we-are-more-than-our-partisanship-4ea179592c1f">did affect her supporter allegiance</a>.</p>
<p><strong>Tech created the perfect mind-control tool, with an eyes on
sales revenue. Someone used it for politics.</strong></p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>The tech industry is mostly socially-liberal and highly educated,
wishing the best for society. But it must accept its share of the blame.
My friends improving machine-learning for costumer profiling and ad
placement, <strong>you help shaping a world of lies and deception</strong>. I will
not blame you for accepting this money: if it were not for you, others
would do it. But we should all be thinking about how do we improve this
system. How do we use data science to build a world based on objectivity,
transparency, and truth, rather than Internet-based marketing?</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>References analysing the erosion of truth</strong></p>
<ul class="simple">
<li><a class="reference external" href="http://www.economist.com/news/briefing/21706498-dishonesty-politics-nothing-new-manner-which-some-politicians-now-lie-and">Must-read article in the economist on lies in politics</a></li>
<li><a class="reference external" href="https://en.wikipedia.org/wiki/Post-truth_politics">Wikipedia page on Post-truth politics</a></li>
<li><a class="reference external" href="http://nymag.com/selectall/2016/11/donald-trump-won-because-of-facebook.html">Donald Trump won because of Facebook</a></li>
<li><a class="reference external" href="http://inverseprobability.com/2016/06/23/the-real-story-behind-todays-referendum">The real story behind todays referendum</a> : Neil Lawrence’s analysis of the filter-bublle effect in Brexit</li>
<li><a class="reference external" href="http://users.polisci.wisc.edu/behavior/Papers/Toff&Kim2013.pdf">A 2013 academic study showing that twitter increases partisan
polarization</a></li>
</ul>
</div>
<div class="topic">
<p class="topic-title"><strong>Disgression: other social issues of data science</strong></p>
<ul class="simple">
<li>The tech industry is <strong>increasing inequalities</strong>, making the rich richer and
leaving the poor behind. Data-science, with its ability to automate
actions and wield large sources of information, is a major contributor
to these sources of inequalities.</li>
<li>Internet-based marketing is building <strong>a huge spying machine</strong> that
infers as much as possible about the user. The Trump campaign was able
to target a specific population, black voters leaning towards
democrats. What if this data was used for direct executive action? This
could come quicker than we think, given how intelligence agencies tap
into social media.</li>
</ul>
<p>I preferred to focus this post on how data-science can help distort truth.
Indeed, it is a problem too often ignored by data scientists who like to
think that they are empowering users.</p>
</div>
<!-- The wikileaks dumps of Clinton's mail resemble the
`Kompromat <https://en.wikipedia.org/wiki/Kompromat>`_ techniques used
by post-soviet regimes, using private information on opponents to
control them. -->
<p class="align-right">In memory of <a class="reference external" href="https://en.wikipedia.org/wiki/Aaron_Swartz">Aaron Schwartz</a>
who fought centralized power on Internet.</p>
<hr class="docutils" />
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Facebook was until recently using human curators, <a class="reference external" href="http://arstechnica.com/business/2016/08/facebook-fires-human-editors-algorithm-immediately-posts-fake-news/">but fired them,
leading to a loss of control on veracity</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>It is a well-known and well-studied cognitive bias that
<a class="reference external" href="https://en.wikipedia.org/wiki/Cognitive_dissonance">individuals strive to reduce cognitive dissonace and actively avoid
situations and information likely to increase it</a></td></tr>
</tbody>
</table>
<a class="reference external image-reference" href="https://www.flickr.com/photos/cdevers/4602805654"><img alt="" class="align-center" src="https://farm2.staticflickr.com/1376/4602805654_db8b6569fb_z_d.jpg" style="width: 80%;" /></a>
Unison 2.48 binaries for ARM2016-07-23T00:00:00+02:002016-07-23T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2016-07-23:/misc/unison-248-binaries-for-arm.html<p class="first last">I have built static binaries of Unision 2.48 for ARM</p>
<p>I have built static binaries of Unison 2.48 for ARM
Run on my NAS, the arm architecture is necessary to synchronize with the
recent Ubuntu.</p>
<ul class="simple">
<li><a class="reference external" href="../programming/attachments/unison-2.48.4-armel.zip">unison-2.48.4-armel.zip</a></li>
</ul>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p><strong>I will not support these binaries</strong></p>
<p class="last">I will not answer any questions or request on these binaries. I have
built them for my personal use and put them online in case it might be
useful for others.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="remark-on-backward-compatibility">
<h2>Remark on backward compatibility</h2>
<p>Why don’t the Unison devs ensure compatibility between minor version of
Unison?</p>
<p><strong>Breaking compatibility is bad practice, in particular between minor
versions</strong>. It breaks the trust that users have in updating the software.
Programmers complain that users always run old versions of
OSs/libraries/programs, but this is explained by the fear of stuff
breaking during upgrades.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="notes-to-build-these-binaries">
<h2>Notes to build these binaries</h2>
<p>I built this following instructions historically on
<a class="reference external" href="http://www.crutzi.info/unison/binary/armel">http://www.crutzi.info/unison/binary/armel</a></p>
<p>I retrieved these instructions from the wayback machine, and adapted them
to work on a more modern Debian system</p>
<p>To compile it, I used qemu and a Debian ARM image</p>
<div class="section" id="build-a-debian-system-under-qemu">
<h3>1. Build a debian system under Qemu</h3>
<p>Install the system (lasts a couple of hours, with some user input):</p>
<pre class="literal-block">
sudo apt install qemu-system-arm qemu-efi libguestfs-tools
wget -O installer-vmlinuz http://http.us.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/vmlinuz
wget -O installer-initrd.gz http://http.us.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/initrd.gz
# Create a drive
qemu-img create -f qcow2 hda.qcow2 5G
qemu-system-arm -M virt -m 1024 \
-kernel installer-vmlinuz \
-initrd installer-initrd.gz \
-drive if=none,file=hda.qcow2,format=qcow2,id=hd \
-device virtio-blk-device,drive=hd \
-netdev user,id=mynet \
-device virtio-net-device,netdev=mynet \
-nographic -no-reboot
</pre>
<p>Under Ubuntu:</p>
<pre class="literal-block">
sudo chmod 644 /boot/vmlinuz*
</pre>
<p>List the content on the /boot dir of the VM’s disk:</p>
<pre class="literal-block">
virt-ls -a hda.qcow2 /boot/
</pre>
<p>Copy the initrd and vmlinux:</p>
<pre class="literal-block">
virt-copy-out -a hda.qcow2 /boot/vmlinuz-3.16.0-6-armmp-lpae /boot/initrd.img-3.16.0-6-armmp-lpae .
</pre>
<p>Do symlinks:</p>
<pre class="literal-block">
ln -s initrd.img-3.16.0-6-armmp-lpae initrd.img
ln -s vmlinuz-3.16.0-6-armmp-lpae vmlinuz
</pre>
<p>The installed system is then booted with:</p>
<pre class="literal-block">
qemu-system-arm -M virt -m 1024 \
-kernel vmlinuz \
-initrd initrd.img \
-drive if=none,file=hda.qcow2,format=qcow2,id=hd \
-device virtio-blk-device,drive=hd \
-netdev user,id=mynet \
-device virtio-net-device,netdev=mynet \
-nographic -no-reboot -append "root=/dev/vda2"
</pre>
</div>
<div class="section" id="build-unison-under-the-debian-system">
<h3>2. Build unison under the debian system</h3>
<p>Download the unison source package from</p>
<p>Then compile the files within the qemu ARM environment:</p>
<pre class="literal-block">
apt-get update
apt-get upgrade
apt-get build-dep unison
wget https://github.com/bcpierce00/unison/archive/v2.48.15v4.tar.gz
tar -xvzf v2.48.15v4.tar.gz
cd unison-2.48.15v4
make UISTYLE=text NATIVE=true STATIC=true
</pre>
<p>You might need to remove the ‘-unsafe-string’ option as detailed in <a class="reference external" href="https://github.com/bcpierce00/unison/issues/211">https://github.com/bcpierce00/unison/issues/211</a></p>
<p>The binary will be in <cite>src/unison</cite></p>
</div>
</div>
Better Python compressed persistence in joblib2016-05-20T00:00:00+02:002016-05-20T00:00:00+02:00Alexandre Abadie & Gaël Varoquauxtag:gael-varoquaux.info,2016-05-20:/programming/new_low-overhead_persistence_in_joblib_for_big_data.html<p class="first last">New persistence in joblib enables low-overhead storage of big data contained in arbitrary objects</p>
<div class="section" id="problem-setting-persistence-for-big-data">
<h2>Problem setting: persistence for big data</h2>
<p><a class="reference external" href="https://pythonhosted.org/joblib/">Joblib</a> is a powerful Python package
for management of computation: parallel computing, caching, and
primitives for out-of-core computing. It is handy when working on so
called <strong>big data</strong>, that can consume more than the available RAM (several GB
nowadays). In such situations, objects in the working space must be
persisted to disk, for out-of-core computing, distribution of jobs, or
caching.</p>
<p>An efficient strategy to write code dealing with big data is to rely on
<strong>numpy arrays to hold large chunks of structured data</strong>.
The code then handles objects or arbitrary containers (list, dict) with
numpy arrays. For data management, joblib provides transparent disk
persistence that is very efficient with such objects. The internal
mechanism relies on specializing <a class="reference external" href="https://docs.python.org/3/library/pickle.html">pickle</a> to handle better numpy
arrays.</p>
<p><a class="reference external" href="https://github.com/joblib/joblib/pull/260">Recent improvements</a>
reduce vastly the memory overhead of data persistence.</p>
<div class="section" id="limitations-of-the-old-implementation">
<h3>Limitations of the old implementation</h3>
<p>❶ Dumping/loading persisted data <strong>with compression</strong> was a memory hog,
because of internal copies of data, limiting the maximum size
of usable data with compressed persistence:</p>
<img alt="" class="large" src="https://gael-varoquaux.info/programming/attachments/old_pickle_mem_profile.png" />
<p>We see the increased memory usage during the calls to <tt class="docutils literal">dump</tt> and
<tt class="docutils literal">load</tt> functions, profiled using the <a class="reference external" href="https://pypi.python.org/pypi/memory_profiler">memory_profiler package</a> with this <a class="reference external" href="https://gist.github.com/aabadie/7cba3385406d1cec7d3dd4407ba3f164">gist</a></p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>❷ Another drawback was that large numpy arrays (>10MB) contained in an
arbitrary Python object were dumped in separate <tt class="docutils literal">.npy</tt> file, increasing
the load on the file system <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="o">>>></span> <span class="kn">import</span> <span class="nn">joblib</span> <span class="c1"># joblib version: 0.9.4</span>
<span class="o">>>></span> <span class="n">obj</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">((</span><span class="mi">5000</span><span class="p">,</span> <span class="mi">5000</span><span class="p">)),</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">((</span><span class="mi">5000</span><span class="p">,</span> <span class="mi">5000</span><span class="p">))]</span>
<span class="c1"># 3 files are generated:</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl'</span><span class="p">,</span> <span class="n">compress</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl'</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl_01.npy.z'</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl_02.npy.z'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'/tmp/test.pkl'</span><span class="p">)</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<!-- XXX: announce content of post earlier
Let's now discover the new features and improvements that comes with
version 0.10.0. After that, we'll compare speed and memory consumption with
other libraries and discuss the results. Then we'll give some details about the
new internal implementation. -->
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="what-s-new-compression-low-memory">
<h2>What’s new: compression, low memory…</h2>
<p>❶ <strong>Memory usage is now stable</strong>:</p>
<img alt="" src="https://gael-varoquaux.info/programming/attachments/new_pickle_mem_profile.png" />
<p>❷ <strong>All numpy arrays are persisted in a single file</strong>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="o">>>></span> <span class="kn">import</span> <span class="nn">joblib</span> <span class="c1"># joblib version: 0.10.0 (dev)</span>
<span class="o">>>></span> <span class="n">obj</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">((</span><span class="mi">5000</span><span class="p">,</span> <span class="mi">5000</span><span class="p">)),</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">((</span><span class="mi">5000</span><span class="p">,</span> <span class="mi">5000</span><span class="p">))]</span>
<span class="c1"># only 1 file is generated:</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl'</span><span class="p">,</span> <span class="n">compress</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'/tmp/test.pkl'</span><span class="p">)</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<p>❸ <strong>Persistence in a file handle</strong> (ongoing work in a <a class="reference external" href="https://github.com/joblib/joblib/pull/351">pull request</a>)</p>
<p>❹ <strong>More compression formats are available</strong></p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title">Backward compatibility</p>
<p>Existing joblib users can be reassured: the new version is <strong>still
compatible with pickles generated by older versions</strong> (>= 0.8.4). You
are encouraged to update (rebuild?) your cache if you want to take
advantage of this new version.</p>
</div>
</div>
<div class="section" id="benchmarks-speed-and-memory-consumption">
<h2>Benchmarks: speed and memory consumption</h2>
<p>Joblib strives to have <strong>minimum dependencies</strong> (only numpy) and to
<strong>be agnostic to the input data</strong>. Hence the goals are to deal with any
kind of data while trying to <strong>be as efficient as possible with numpy arrays</strong>.</p>
<p>To illustrate the benefits and cost of the new persistence implementation, let’s
now compare a real life use case
(<a class="reference external" href="http://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_lfw_people.html">LFW dataset from scikit-learn</a>)
with different libraries:</p>
<ul class="simple">
<li>Joblib, with 2 different versions,
0.9.4 and master (dev),</li>
<li>Pickle</li>
<li>Numpy</li>
</ul>
<img alt="" class="large" src="https://gael-varoquaux.info/programming/attachments/persistence_lfw_bench.png" />
<p>The four first lines use non compressed persistence strategies, the last
four use persistence with zlib/gzip <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a> strategies. Code to reproduce the
benchmarks is available on this <a class="reference external" href="https://gist.github.com/aabadie/2ba94d28d68f19f87eb8916a2238a97c">gist</a>.</p>
<p>⚫ <strong>Speed</strong>: the results between joblib 0.9.4 and 0.10.0 (dev) are
similar whereas <strong>numpy and pickle are clearly slower than joblib</strong> in both
compressed and non compressed cases.</p>
<p>⚫ <strong>Memory consumption</strong>: Without compression, old and
new joblib versions are the same; with compression, the new joblib version is
much better than the old one.
<strong>Joblib clearly outperforms pickle and numpy in terms of
memory consumption</strong>. This can be explained by the fact that numpy relies on
pickle if the object is not a pure numpy array (a list or a dict with arrays for
example), so in this case it inherits the memory drawbacks from pickle. When
persisting pure numpy arrays (not tested here), numpy uses its internal save/load
functions which are efficient in terms of speed and memory consumption.</p>
<p>⚫ <strong>Disk used</strong>: results are as expected: non compressed files have
the same size as the in-memory data; compressed files are smaller.</p>
<div class="topic">
<p class="topic-title">Caveat Emptor: performance is data-dependent</p>
<p>Different data compress more or less easily. Speed and disk used will
vary depending on the data. Key considerations are:</p>
<ul class="simple">
<li><strong>Fraction of data in arrays</strong>: joblib is efficient if much of the
data is contained in numpy arrays. The worst case scenario is
something like a large dictionary of random numbers as keys and
values.</li>
<li><strong>Entropy of the data</strong>: an array fully of zeros will compress well
and fast. A fully random array will compress slowly, and use a lot
of disk. Real data is often somewhere in the middle.</li>
</ul>
</div>
</div>
<div class="section" id="extra-improvements-in-compressed-persistence">
<h2>Extra improvements in compressed persistence</h2>
<div class="section" id="new-compression-formats">
<h3>New compression formats</h3>
<p>Joblib can use new compression formats based on Python standard library modules:
<strong>zlib, gzip, bz2, lzma and xz</strong> (the last 2 are available for Python
greater than 3.3). <strong>The compressor is
selected automatically when the file name has an explicit extension</strong>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.z'</span><span class="p">)</span> <span class="c1"># zlib</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.z'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.gz'</span><span class="p">)</span> <span class="c1"># gzip</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.gz'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.bz2'</span><span class="p">)</span> <span class="c1"># bz2</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.bz2'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.lzma'</span><span class="p">)</span> <span class="c1"># lzma</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.lzma'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.xz'</span><span class="p">)</span> <span class="c1"># xz</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.xz'</span><span class="p">]</span>
</pre></div>
<p>One can tune the compression level, setting the compressor explicitly:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.pkl.compressed'</span><span class="p">,</span> <span class="n">compress</span><span class="o">=</span><span class="p">(</span><span class="s1">'zlib'</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.compressed'</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s1">'/tmp/test.compressed'</span><span class="p">,</span> <span class="n">compress</span><span class="o">=</span><span class="p">(</span><span class="s1">'lzma'</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.compressed'</span><span class="p">]</span>
</pre></div>
<p>On loading, joblib uses the magic number of the file to determine the
right decompression method. This makes loading compressed pickle transparent:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'/tmp/test.compressed'</span><span class="p">)</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<p>Importantly, the generated compressed files use a <strong>standard
compression file format</strong>: for instance, regular command line tools (zip/unzip,
gzip/gunzip, bzip2, lzma, xz) can be used to compress/uncompress a pickled file
generated with joblib. Joblib will be able to load cache compressed with those
tools.</p>
<div class="topic">
<p class="topic-title">Toward more and faster compression</p>
<p>Specific compression strategies have been developped for fast
compression, sometimes even faster than disk reads such as <a class="reference external" href="http://google.github.io/snappy/">snappy</a> , <a class="reference external" href="http://www.blosc.org/">blosc</a>, LZO or LZ4. With a file-like interface, they should be
readily usable with joblib.</p>
<p>In the benchmarks above, loading and dumping with compression is
slower than without (though only by a factor of 3 for loading). These
were done on a computer with an SSD, hence with very fast I/O. In a
situation with slower I/O, as <strong>on a network drive, compression could
save time</strong>. With faster compressors, compression will save time on most
hardware.</p>
</div>
</div>
<div class="section" id="compressed-persistence-into-a-file-handle">
<h3>Compressed persistence into a file handle</h3>
<p>Now that everything is stored in a
single file using standard compression formats, joblib can
persist in an <a class="reference external" href="https://github.com/joblib/joblib/pull/351">open file handle</a>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'/tmp/test.pkl'</span><span class="p">,</span> <span class="s1">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl'</span><span class="p">]</span>
<span class="o">>>></span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'/tmp/test.pkl'</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="o">>>></span> <span class="nb">print</span><span class="p">(</span><span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<p>This also works with compression file object available in the standard library,
like <tt class="docutils literal">gzip.GzipFile</tt>, <tt class="docutils literal">bz2.Bz2File</tt> or <tt class="docutils literal">lzma.LzmaFile</tt>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="kn">import</span> <span class="nn">gzip</span>
<span class="o">>>></span> <span class="k">with</span> <span class="n">gzip</span><span class="o">.</span><span class="n">GzipFile</span><span class="p">(</span><span class="s1">'/tmp/test.pkl.gz'</span><span class="p">,</span> <span class="s1">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="o">>>></span> <span class="n">joblib</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span>
<span class="p">[</span><span class="s1">'/tmp/test.pkl.gz'</span><span class="p">]</span>
<span class="o">>>></span> <span class="k">with</span> <span class="n">gzip</span><span class="o">.</span><span class="n">GzipFile</span><span class="p">(</span><span class="s1">'/tmp/test.pkl.gz'</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="o">>>></span> <span class="nb">print</span><span class="p">(</span><span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<p>Be sure that you use a decompressor matching the internal compression when
loading with the above method. If
unsure, simply use <tt class="docutils literal">open</tt>, joblib will <strong>select the right decompressor</strong>:</p>
<div class="highlight"><pre><span></span><span class="o">>>></span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'/tmp/test.pkl.gz'</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="o">>>></span> <span class="nb">print</span><span class="p">(</span><span class="n">joblib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">))</span>
<span class="p">[</span><span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">]],</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.47006195</span><span class="p">,</span> <span class="mf">0.5436392</span> <span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="mf">0.1218267</span> <span class="p">,</span> <span class="mf">0.48592789</span><span class="p">]])]</span>
</pre></div>
<div class="topic">
<p class="topic-title">Towards dumping to elaborate stores</p>
<p>Working with file handles opens the door to <strong>storing cache data in database blob or cloud
storage such as Amazon S3, Amazon Glacier and Google Cloud Storage</strong>
(for instance via the Python package <a class="reference external" href="https://github.com/boto/boto">boto</a>).</p>
</div>
</div>
</div>
<div class="section" id="implementation">
<h2>Implementation</h2>
<p><strong>A Pickle Subclass</strong>: joblib relies on subclassing the Python Pickler/Unpickler
<a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>. These are state machines that walk the graph of nested objects (a
dict may contain a list, that may contain…), creating a string
representation of each object encountered. The new implementation
proceeds as follows:</p>
<ul class="simple">
<li><strong>Pickling an arbitrary object</strong>: when an <tt class="docutils literal">np.ndarray</tt> object is reached,
instead of using the default pickling functions (__reduce__()), the joblib
Pickler replaces in pickle stream the ndarray with a wrapper object containing
all important array metadata (shape, dtype, flags). Then it writes the array
content in the pickle file. Note that this step breaks the pickle
compatibility. One benefit is that it enables using fast code for
copyless handling of the numpy array. For compression, we pass chunks
of the data to a compressor object (using the buffer protocol to avoid
copies).</li>
<li><strong>Unpickling from a file</strong>: when pickle reaches the array wrapper, as the
object is in the pickle stream, the file handle is at the
beginning of the array content. So at this point the Unpickler simply
constructs an array based on the metadata contained in the wrapper and then
fills the array buffer directly from the file. The object returned is the
reconstructed array, the array wrapper being dropped. A benefit is that
if the data is stored not compressed, <strong>the array can be directly memory
mapped from the storage</strong> (the mmap_mode option of <a class="reference external" href="https://pythonhosted.org/joblib/generated/joblib.load.html">joblib.load</a>).</li>
</ul>
<p>This technique allows joblib to pickle all objects in a single file but also to
have memory-efficient dump and load.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>A fast compression stream</strong>: as the pickling refactoring opens the door
to file objects usage, joblib is now able to persist data in any kind of file
object: <tt class="docutils literal">open</tt>, <tt class="docutils literal">gzip.GzipFile</tt>, <tt class="docutils literal">bz2.Bz2file</tt> and <tt class="docutils literal">lzma.LzmaFile</tt>. For
performance reason and usability, the new joblib version uses its own file
object <tt class="docutils literal">BinaryZlibFile</tt> for zlib compression. Compared to
<tt class="docutils literal">GzipFile</tt>, it disables crc computation, which bring a performance gain of 15%.</p>
<div class="topic">
<p class="topic-title">Speed penalties of on-the-fly writes</p>
<p>There’s also a small speed difference with dict/list objects between new/old
joblib when using compression.
The old version pickles the data inside a <tt class="docutils literal">io.BytesIO</tt> buffer and then
compress it in a row whereas the new version write “on the fly” compressed
chunk of pickled data to the file.
Because of this internal buffer the old implementation is not memory safe as it
indeed copy the data in memory before compressing. The small speed difference
was judged acceptable compared to this memory duplication.</p>
</div>
</div>
<div class="section" id="conclusion-and-future-work">
<h2>Conclusion and future work</h2>
<p>Memory copies were a limitation when caching on disk very large
numpy arrays, e.g arrays with a size close to the available RAM on the computer.
The problem was solved via intensive buffering and a lot of hacking on top of
pickle and numpy. Unfortunately, our strategy has poor performance with
big dictionaries or list compared to a <tt class="docutils literal">cPickle</tt>, hence try to use
numpy arrays in your internal data structures (note that something like
scipy sparse matrices works well, as it builds on arrays).</p>
<p>For the future, maybe numpy’s pickle methods could be improved and make a
better use of <a class="reference external" href="https://www.python.org/dev/peps/pep-3154/#bit-opcodes-for-large-objects">64-bit opcodes for large objects</a>
that were introduced in Python recently.</p>
<p>Pickling using file handles is a first step toward pickling in
sockets, enabling broadcasting of data between computing units
on a network. This will be priceless with <a class="reference external" href="https://github.com/joblib/joblib/pull/325">joblib’s new distributed backends</a>.</p>
<p>Other improvements will come from better compressor, making everything
faster.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The pull request was implemented by <a class="reference external" href="https://github.com/aabadie">@aabadie</a>. He thanks <a class="reference external" href="https://github.com/lesteve">@lesteve</a>, <a class="reference external" href="https://github.com/ogrisel">@ogrisel</a>
and <a class="reference external" href="https://github.com/GaelVaroquaux">@GaelVaroquaux</a> for the valuable
help, reviews and support.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>The load created by multiple files on the filesystem is
particularly detrimental for network filesystems, as it triggers
multiple requests and isn’t cache friendly.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>gzip is based on zlib with additional crc checks and a default
compression level of 3.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>A drawback of subclassing the Python Pickler/Unpickler is that it
is done for the pure-Python version, and not the “cPickle” version.
The latter is much faster when dealing with a large number of Python
objects. Once again, joblib is efficient when most of the data is
represented as numpy arrays or subclasses.</td></tr>
</tbody>
</table>
</div>
Of software and Science. Reproducible science: what, why, and how2015-12-16T00:00:00+01:002015-12-16T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-12-16:/programming/of-software-and-science-reproducible-science-what-why-and-how.html<p>At <a class="reference external" href="mloss-2015-wising-up-to-building-open-source-machine-learning.html">MLOSS 15</a> we
brainstormed on reproducible science, discussing <strong>why we care about
software in computer science</strong>. Here is a summary blending <a class="reference external" href="https://gist.github.com/GaelVaroquaux/33e7a7b297425890fefa">notes from
the discussions</a> with my
opinion.</p>
<blockquote class="epigraph">
“Without engineering, science is not more than philosophy”
—
<a class="reference external" href="https://twitter.com/GaelVaroquaux/status/619767624654786560">the community</a></blockquote>
<p><strong>How do we enable better Science? Why do we do software …</strong></p><p>At <a class="reference external" href="mloss-2015-wising-up-to-building-open-source-machine-learning.html">MLOSS 15</a> we
brainstormed on reproducible science, discussing <strong>why we care about
software in computer science</strong>. Here is a summary blending <a class="reference external" href="https://gist.github.com/GaelVaroquaux/33e7a7b297425890fefa">notes from
the discussions</a> with my
opinion.</p>
<blockquote class="epigraph">
“Without engineering, science is not more than philosophy”
—
<a class="reference external" href="https://twitter.com/GaelVaroquaux/status/619767624654786560">the community</a></blockquote>
<p><strong>How do we enable better Science? Why do we do software in science?</strong>
These are the questions that we were interested in.</p>
<div class="grey docutils container">
<strong>Improving reproducility of our scientific studies makes us more
efficient in the long run</strong> to do good science: even inside a lab, new
research efforts build upon the previous work.</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="forms-of-reproducible-science-reproduction-replication-reuse">
<h2>Forms of reproducible science: reproduction, replication, & reuse</h2>
<p><a class="reference external" href="https://politicalsciencereplication.wordpress.com/2013/02/24/is-there-a-difference-between-replication-reproduction-and-re-analysis/">The classic concepts of reproducible science</a>
are:</p>
<ul class="simple">
<li><strong>Reproducibility</strong>: being able to rerun an experiment as it was run,
for instance by reanalysing data.</li>
<li><strong>Replicability</strong>: being able to redo an experiment from scratch.</li>
</ul>
<p>The <em>reproducible science</em> movement argues sharing source code of
experiments is a need for <em>reproduction</em>.</p>
<p>For reproduction, fields like computer science (development of methods)
and biology (challenging data acquisition) have very different
constraints, with the complexity allocated differently between data and
code.</p>
<blockquote class="epigraph">
“Machine learning people use hugely complex algorithms on trivially
simple datasets. Biology does trivially simple algorithms on hugely
complex datasets.”
—
<em>an MLOSS15 attendee</em></blockquote>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>We felt that computer science needed an additional notion, complementing
replication and reproduction:</p>
<ul class="simple">
<li><strong>Reusability</strong>: applying the process to a new yet similar question.
For instance for a paper contributing data analysis method, applying it
to new data.</li>
</ul>
<div class="align-right docutils container">
Reusability is more valuable than reproducibility.</div>
<p>Reproducibility without reusability in method development may hinder the
advancement of science as it pushes people to do all the same
things, <em>eg</em> always running experiments on the same data.</p>
<p>Reusability enables results that the original investigator did not have in
mind. It implies that the experimental protocol extends further than the
exact scope of the question initially asked. For software development, it
is also harder, as it implies more robustness and flexibility.</p>
<p>Finally sharing source code is not enough: <strong>readability</strong> of the code is
necessary.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="roadblocks-to-reproducible-science">
<h2>Roadblocks to reproducible science</h2>
<div class="section" id="man-power">
<h3>Man power</h3>
<p>Reusability, readability, support of released code, all actually take a
lot of time, even though it is seldom acknowledged in talks about
reproducible science. Given a fixed man power, it is impossible to
achieve reusability and high quality for everything.</p>
</div>
<div class="section" id="computing-power">
<h3>Computing power</h3>
<p>Some numerical experiments or complex data analysis require weeks of
cluster to run. These will be much harder to reproduce. Also, rerunning
an analysis from scratch on a regular basis is a good recipe to achieve a
robust path from data to results. The more computing power is a limiting
resource, the more likely it is that a glitch is not detected.</p>
</div>
<div class="section" id="data-availability">
<h3>Data availability</h3>
<p>No access, or restricted access, to data is a show stopper for
reproducibility. Data sharing requirements are becoming common –from
funding agencies, or journals. However, privacy concerns, or confidential
information get in the way of making data public, for instance in medical
research or micro-economy. Often, these concerns serve as a pretext
to people who actually do not want to relinquish <em>control</em> <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>A related post by Deevy Bishop: <a class="reference external" href="http://deevybee.blogspot.co.uk/2015/11/whos-afraid-of-open-data.html?m=1">Who’s afraid of open data</a></td></tr>
</tbody>
</table>
</div>
<div class="section" id="incentives-problem">
<h3>Incentives problem</h3>
<p>Fancy new results are what matters for success in academia. “High impact”
journals such as Nature or Science accept papers that amaze and impress,
often with subpar inspection of the materials and methods <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>. The rate of
publication in many leading groups is incompatible with consolidation
efforts required for strong reproducibility.</p>
<p>On the other hand, it is hard to tell beforehand if a new idea is a good
one. Hence letting imagination forward to foster impossible and
improbable ideas is a good path to innovation. The underlying questions
are: What are the best community rules for the advancement of knowledge?
What do we want from the way science moves forward? Rapid publication of
many incremental ideas, <em>eg</em> at a conference, gives food for thoughts,
possibly at the sake of reproducibility.</p>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>“Science, Nature and Cell, had a higher rate of retractions” –
<a class="reference external" href="https://en.wikipedia.org/wiki/Invalid_science">Wikipedia: Invalid science</a></td></tr>
</tbody>
</table>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
</div>
<div class="section" id="how-to-improve-the-situation">
<h2>How to improve the situation</h2>
<div class="section" id="docker-containers-and-virtual-machines">
<h3>Docker, containers, and virtual machines</h3>
<p>Docker, or other virtual machine technologies, enable shipping a software
environment. It diminishes the challenges of building software and
setting up an analysis. Virtual machines are used as a way to avoid
software packaging issues. This seems to me as a plaster on a wooden leg.</p>
<div class="align-right docutils container">
Containers give easy reproduction, to the cost of hard
replication and reuse.</div>
<p>Indeed, an analysis that lives in a box can be reproduced, but can it be
understood, modified, or applied to new data? New science is likely going
to come from modifying this analysis, or combining it with other tools,
or new data. If these other tools live in a different virtual machine,
the combination will be challenging.</p>
<p>In addition, people are using containers as an excuse to avoid tackling
the need for proper documentation of requirements, and the process to set
them up. They sometimes even try justify binary blobs <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>. This is
wrong. An analysis should be runnable without requiring the stars to
align, and it should be understandable.</p>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>See also Titus Brown’s post: <a class="reference external" href="http://ivory.idyll.org/blog/2014-containers.html">The post-apocalyptic world of binary
containers</a></td></tr>
</tbody>
</table>
</div>
<div class="section" id="version-control-wear-your-seatbelt">
<h3>Version control: wear your seatbelt</h3>
<p><a class="reference external" href="https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control">Version control</a>
is like a time machine: if used with regular commits, it enables rolling
back to any point in time. For my work, it’s always been a crucial aspect
to reproducing what me or my students did a while ago. I often meet
researchers that feel they lack time to learn it. I really cannot support
this position. <a class="reference external" href="http://try.github.io">http://try.github.io</a> is an easy way to learn version
control.</p>
<p><em>Hint</em>: use a “tag” to pin-point a position in the history that you might
want to repeat, such as making a figure or the publication of an article.</p>
</div>
<div class="section" id="sotware-libraries-curated-and-maintained">
<h3>Sotware libraries, curated and maintained</h3>
<p>Consolidating an analysis pipeline, a standard visualization, or any
computational aspect of a paper into a software library is a sure way to
make the paper more reproducible. It will also make the steps reusable,
and a replication easier. If continued effort is put in the library,
chances are that computational efficiency will improve over time, thus
helping in the long run with the challenge of computing power.</p>
<div class="align-right docutils container">
Tough choices: not every variant of an analysis can be forever
reproducible.</div>
<p>Maintaining the library will ensure that results are still reproducible
on new hardware, or with evolution of the general software stack (a new
Python or Matlab release, for instance). Documentation and curated
examples will lower the bar to reuse and facilitate replication of the
original scientific results.</p>
<p>To avoid feature creep and technical debt, a library calls for focused
efforts on selecting the most important operations.</p>
</div>
<div class="section" id="datasets-serving-as-model-experiments-tractable-and-open">
<h3>Datasets, serving as model experiments, tractable and open</h3>
<p>Sometimes researchers create a toy data, with a well-posed question, that
is curated and open, small enough to be tractable yet large enough to be
relevant to the application field. This is an invaluable service to the
field. One example is the <a class="reference external" href="https://en.wikipedia.org/wiki/Netflix_Prize">netflix prize</a> in machine learning,
which led to a standard dataset. Unfortunately, the dataset was taken
down some years later due to copyright concerns. But it has been
replaced, <em>eg</em> by the <a class="reference external" href="http://grouplens.org/datasets/movielens/">movielens dataset</a>. For computer vision, a
series of datasets –<a class="reference external" href="http://www.vision.caltech.edu/Image_Datasets/Caltech101/">Caltech101</a>, <a class="reference external" href="https://www.cs.toronto.edu/~kriz/cifar.html">CIFAR</a>, <a class="reference external" href="http://www.image-net.org/">ImageNet</a>…– have led to continuous progress of the
field. In bioinformatics, standard data are regularly created, for
instance by the <a class="reference external" href="http://dreamchallenges.org/">DREAM challenges</a>.</p>
<p>These reference open datasets serve as benchmarks and therefore foster
competition. They also define a canonical experiment, helping a wider
scientific community understand the questions that they ask. Ultimately,
they result in better software tools to solve the problem at hand, as
this problem becomes a standard example and application of tools.</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Sage_Bionetworks">Sage bionetworks</a>, for
instance, is a non-profit that collects and make biomedical data
available. These people believe, as I do, that such data will lead to
better medical care.</p>
</div>
<div class="section" id="changing-incentives-setting-the-right-goals">
<h3>Changing incentives: setting the right goals</h3>
<p>Making sustainable, quality scientific work that facilitates reproduction
needs to be a clearly-visible benefit to researchers, young and senior.
Such contributions should help them get jobs and grants.</p>
<p>An unsophisticated publication count is the basis of scientific
evaluation. We need to accept publications about data, software, and
replication of prior work in high-quality journals. They need to be
strictly reviewed, to establish high standards on these contributions.
This change is happening. <a class="reference external" href="http://www.gigasciencejournal.com/">Gigascience</a>, amongst other venues, publishes
data. The <a class="reference external" href="http://jmlr.org/mloss/">MLOSS (machine learning open source software) track</a> of the JMLR (journal of machine learning
research) publishes software, with a tough review on the software quality
of the project.</p>
<div class="align-right docutils container">
Researchers should cite the software they use.</div>
<p>Yet software is still often under cited: many will use a software
implementing a method, and only cite the original paper that proposed the
method. Another remaining challenge is: how to give credit for continuing
development and maintenance.</p>
<p>Fast-paced science is probably useful even if fragile. But the difference
between a quick proof of concept and solid, reproducible and reusable
work needs to be acknowledged. It is important to select for publication
not only impressive results, but also sound reusable material and
methods. The latter are the foundation of future scientific developments,
but high-impact journals tend to focus on the former.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Related posts</strong>:</p>
<ul class="simple">
<li><a class="reference external" href="software-for-reproducible-science-lets-not-have-a-misunderstanding.html">Software for reproducible science: let’s not have a misunderstanding</a></li>
<li><a class="reference external" href="mloss-2015-wising-up-to-building-open-source-machine-learning.html">MLOSS 2015: wising up to building open-source machine learning</a></li>
</ul>
</div>
</div>
</div>
Nilearn 0.2: more powerful machine learning for neuroimaging2015-12-13T00:00:00+01:002015-12-13T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-12-13:/programming/nilearn-02-more-powerful-machine-learning-for-neuroimaging.html<div class="small sidebar">
<p class="first sidebar-title">Nilearn’s goals</p>
<p class="last">Make advanced machine learning techniques easy for neuroimaging
research.</p>
</div>
<p>After 6 months of efforts, We just released version 0.2 of <a class="reference external" href="http://nilearn.github.io">nilearn</a>, dedicated to making <strong>machine learning in
neuroimaging easier and more powerful</strong>.</p>
<p>This release integrates the features of the <a class="reference external" href="nilearn_july_2015_sprint.html">july sprint</a>, and <a class="reference external" href="http://nilearn.github.io/whats_new.html">more</a>.</p>
<div class="section" id="highlights">
<h2>Highlights</h2>
<p><strong>Better documentation …</strong></p></div><div class="small sidebar">
<p class="first sidebar-title">Nilearn’s goals</p>
<p class="last">Make advanced machine learning techniques easy for neuroimaging
research.</p>
</div>
<p>After 6 months of efforts, We just released version 0.2 of <a class="reference external" href="http://nilearn.github.io">nilearn</a>, dedicated to making <strong>machine learning in
neuroimaging easier and more powerful</strong>.</p>
<p>This release integrates the features of the <a class="reference external" href="nilearn_july_2015_sprint.html">july sprint</a>, and <a class="reference external" href="http://nilearn.github.io/whats_new.html">more</a>.</p>
<div class="section" id="highlights">
<h2>Highlights</h2>
<p><strong>Better documentation with narrative examples</strong></p>
<p>The example can now be broken down into blocks (as <a class="reference external" href="http://nilearn.github.io/auto_examples/connectivity/plot_signal_extraction.html#sphx-glr-auto-examples-connectivity-plot-signal-extraction-py">here</a>)
for a better narration (thanks to <a class="reference external" href="http://sphinx-gallery.readthedocs.org/en/latest/">sphinx-gallery</a>).</p>
<hr class="docutils" />
<div class="figure align-right">
<a class="reference external image-reference" href="http://nilearn.github.io/auto_examples/decoding/plot_mixed_gambles_space_net.html"><img alt="" src="http://nilearn.github.io/_images/sphx_glr_plot_mixed_gambles_space_net_001.png" /></a>
</div>
<p><strong>Space net: spatial regularizations in decoding</strong></p>
<p>The <a class="reference external" href="http://nilearn.github.io/decoding/space_net.html">“SpaceNet” decoder</a> does spatial
regularizations such as TV-l1 or Graph-Net to identify predictive regions
in decoding.</p>
<hr class="docutils" />
<div class="figure align-right">
<a class="reference external image-reference" href="http://nilearn.github.io/auto_examples/connectivity/plot_compare_resting_state_decomposition.html"><img alt="" src="http://nilearn.github.io/_images/sphx_glr_plot_compare_resting_state_decomposition_002.png" /></a>
</div>
<p><strong>Dictionnary learning for resting-state parcellations</strong></p>
<p>Dictionnary learning is a <a class="reference external" href="http://nilearn.github.io/connectivity/resting_state_networks.html#beyond-ica-dictionary-learning">promising alternative to ICA to learn networks</a>.</p>
<hr class="docutils" />
<div class="figure align-right">
<a class="reference external image-reference" href="http://nilearn.github.io/auto_examples/manipulating_visualizing/plot_prob_atlas.html#sphx-glr-auto-examples-manipulating-visualizing-plot-prob-atlas-py"><img alt="" src="http://nilearn.github.io/_images/sphx_glr_plot_prob_atlas_003.png" /></a>
</div>
<p><strong>Plotting sets of probabilistic maps</strong></p>
<p>With <a class="reference external" href="http://nilearn.github.io/manipulating_visualizing/plotting.html#different-plotting-functions">a simple function</a>,
you can plot outlines for multiple maps.</p>
<hr class="docutils" />
<div class="figure align-right">
<a class="reference external image-reference" href="http://nilearn.github.io/auto_examples/manipulating_visualizing/plot_extract_rois_statistical_maps.html"><img alt="" src="http://nilearn.github.io/_images/sphx_glr_plot_extract_rois_statistical_maps_003.png" /></a>
</div>
<p><strong>Separating regions out of maps</strong></p>
<p>We have a set of functions to <a class="reference external" href="http://nilearn.github.io/auto_examples/manipulating_visualizing/plot_extract_rois_statistical_maps.html">separate regions on maps</a> or <a class="reference external" href="http://nilearn.github.io/auto_examples/connectivity/plot_extract_regions_canica_maps.html">turn networks into a probabilistic parcellation</a>.</p>
<hr class="docutils" />
<p><strong>Classification on connectomes</strong></p>
<p>We now have advanced connectivity measures to do <a class="reference external" href="http://nilearn.github.io/auto_examples/connectivity/plot_connectivity_measures.html">comparisons across
connectomes for classification</a>.</p>
<hr class="docutils" />
<div class="topic">
<p class="topic-title">Thanks</p>
<p>Thanks to Alexandre Abraham who lead the effort, and <a class="reference external" href="http://nilearn.github.io/whats_new.html#contributors">all the
contributors</a>.</p>
</div>
</div>
Job offer: data crunching brain functional connectivity for biomarkers2015-12-08T00:00:00+01:002015-12-08T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-12-08:/science/job-offer-data-crunching-brain-functional-connectivity-for-biomarkers.html<p><a class="reference external" href="https://team.inria.fr/parietal/">My research group</a> is looking to fill
a <strong>post-doc position on learning biomarkers from functional
connectivity</strong>.</p>
<div class="section" id="scientific-context">
<h2>Scientific context</h2>
<p>The challenge is to use resting-state fMRI at the level of a population
to understand how intrinsic functional connectivity captures pathologies
and other cognitive phenotypes. Rest fMRI is a promising tool for …</p></div><p><a class="reference external" href="https://team.inria.fr/parietal/">My research group</a> is looking to fill
a <strong>post-doc position on learning biomarkers from functional
connectivity</strong>.</p>
<div class="section" id="scientific-context">
<h2>Scientific context</h2>
<p>The challenge is to use resting-state fMRI at the level of a population
to understand how intrinsic functional connectivity captures pathologies
and other cognitive phenotypes. Rest fMRI is a promising tool for
large-scale population analysis of brain function as it is easy to
acquire and accumulate. Scans for thousands of subjects have already been
shared, and more is to come. However, the signature of cognitions in this
modality are weak. Extracting biomarkers is a challenging data processing
and machine learning problem. This challenge is the expertise of my
research group. Medical applications cover a wider range of brain
pathologies, for which diagnosis is challenging, such as autism or
Alzheimer’s disease.</p>
<p>This project is a collaboration with the <a class="reference external" href="http://www.childmind.org/">Child Mind Institute</a>, experts on psychiatric disorders and
resting-state fMRI, as well as coordinators of the major data sharing
initiatives for rest fRMI data (eg ABIDE).</p>
</div>
<div class="section" id="objectives-of-the-project">
<h2>Objectives of the project</h2>
<p>The project hinges on processing of very large rest fMRI databases.
Important novelties of the project are:</p>
<ul class="simple">
<li>Building predictive models that can discriminate <strong>multiple
pathologies</strong> in <strong>large inhomogeneous datasets</strong>.</li>
<li>Using and improving <strong>advanced connectomics</strong> and
<strong>brain-parcellation</strong> techniques in fMRI.</li>
</ul>
<p>Expected results include the discovery of neurophenotypes for several
brain pathologies, as well as intrinsic brain structures, such as
functional parcellations or connectomes, that carry signatures of
cognition.</p>
<p>The analysis framework is based on algorithmic tools developed in Python
(crucially, leveraging scikit-learn for predictive modeling).</p>
</div>
<div class="section" id="desired-profile">
<h2>Desired profile</h2>
<p>We are looking for a post-doctoral fellow to hire in spring. The ideal
candidate would have some, but not all, of the following expertise and
interests:</p>
<ul class="simple">
<li>Experience in advanced processing of fMRI</li>
<li>General knowledge of brain structure and function</li>
<li>Good communication skills to write high-impact neuroscience publications</li>
<li>Good computing skills, in particular with Python. Cluster computing
experience is desired.</li>
</ul>
</div>
<div class="section" id="a-great-research-environment">
<h2>A great research environment</h2>
<p>The work environment is dynamic and exiting, using state-of-the-art
machine learning to answer challenging functional neuroimaging question.</p>
<p>The post-doc will be employed by <a class="reference external" href="http://www.inria.fr">INRIA</a>, the lead
computing research institute in France. We are a team of computer
scientists specialized in image processing and statistical data analysis,
integrated in one of the top French brain research centers, <a class="reference external" href="http://i2bm.cea.fr/dsv/i2bm/Pages/NeuroSpin.aspx">NeuroSpin</a>, south of Paris. We
work mostly in Python. The team includes core contributors to the
<a class="reference external" href="http://scikit-learn.org">scikit-learn project</a>, for machine learning in
Python, and the <a class="reference external" href="http://nilearn.github.io/">nilearn project</a>, for
statistical learning in NeuroImaging.</p>
<p>In addition, the post-doc will interact closely with researchers from the
<a class="reference external" href="http://www.childmind.org/">Child Mind Institute</a>, with deep expertise
in brain pathologies and in the details of the fMRI acquisitions.
Finally, he or she will have access to advanced storage and grid
computing facilities at INRIA.</p>
<p><strong>Contact information</strong>: gael dotnospam varoquaux atnotspam inria dotnospam fr</p>
</div>
MLOSS 2015: wising up to building open-source machine learning2015-11-28T00:00:00+01:002015-11-28T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-11-28:/programming/mloss-2015-wising-up-to-building-open-source-machine-learning.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p><em>The 2015 edition of the machine learning open
source software (MLOSS) workshop was full of very mature discussions
that I strive to report here.</em></p>
<p class="last"><em>I give links to the videos. Some machine-learning researchers have
great thoughts about growing communities of coders, about code as a
process and a deliverable …</em></p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p><em>The 2015 edition of the machine learning open
source software (MLOSS) workshop was full of very mature discussions
that I strive to report here.</em></p>
<p class="last"><em>I give links to the videos. Some machine-learning researchers have
great thoughts about growing communities of coders, about code as a
process and a deliverable.</em></p>
</div>
<p>I was a co-organizer of the <a class="reference external" href="https://mloss.org/workshop/icml15/">MLOSS 2015 workshop</a>, held during <a class="reference external" href="http://icml.cc/2015/">ICML 2015</a>. As I have finally figured out where the
videos are, now is a good time to summarize my impressions on the
workshop.</p>
<img alt="" src="attachments/mloss/mloss_t_shirt_white.png" style="width: 100%;" />
<div class="section" id="online-videos-of-the-talks">
<h2>Online videos of the talks</h2>
<div class="small sidebar">
<p class="first sidebar-title">Graphics & T-shirts</p>
<p>The graphics were printed on T-shirts. We ran out, but the material is
<a class="reference external" href="attachments/mloss/mloss_t_shirt_graphics.zip">here</a> for you to
print.</p>
<p class="last"><em>Anyone wants to help making an online T-shirt ordering?</em></p>
</div>
<p>The videos of all the talks are online:</p>
<ul class="simple">
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/4216268dc28148c89d8b6e4eba1ad6e51d">Python and Parallelism or Dask</a>
by <em>Matthew Rocklin</em></li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/afe6f76b3bb1452790fc8982e28112641d">Collaborative filtering via matrix decomposition in mlpack</a>
by <em>Ryan Curtin</em></li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/9cd947554ddf404b9a40ca2601e44b4c1d">BLOG: a probabilistic programming language for open-universe contingent
Bayesian networks</a>
by <em>Yi Wu</em></li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/45c3bb312a37491dbce1af25f1aeba001d">Spotlights</a>:<ul>
<li>Nilearn, machine learning for neuroimaging in Python (Alexandre
Abraham)</li>
<li>KeLP: a Kernel-based Learning Platform in Java (Simone Filice)</li>
<li>DiffSharp: Automatic Differentiation Library (Atılım Güneş Baydin)</li>
<li>The FAST toolkit for Unsupervised Learning of HMMs (José P.
González-Brenes)</li>
<li>OpenML: a Networked Science Platform for Machine Learning (Joaquin
Vanschoren)</li>
</ul>
</li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/2529ebcb20794942874d5c277c5dcc981d">Julia’s Approach to Open Source Machine Learning</a>
by <em>John Myles White</em></li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/da4f7869f07745f7bbc5a2e5f31761b61d">Do it yourself deep learning with the Caffe community</a>
by <em>Evan Shelhamer</em></li>
<li><a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/2bc15b283f324784a945d79d9a06c76c1d">From flop to success in academic software development</a>
by <em>Gaël Varoquaux</em></li>
</ul>
</div>
<div class="section" id="mloss-a-maturing-community">
<h2>MLOSS: a maturing community</h2>
<!-- Say that I was not enthousiastic, originaly, and say why (typical
flaws of academic software) -->
<p>When Antti Honkela and Cheng Soon Ong approached me to co-organize an
MLOSS workshop, I felt that it was important to do it for the sake of
open source scientific software. But it didn’t feel very enthousiastic
about the event or the talks themselves. Boy I was wrong.</p>
<div class="align-right docutils container">
Huge attendance: open-source ML software is now mainstream.</div>
<p>My first MLOSS workshop was at the ICML 2011 conference, in Haifa. The
workshop was in a tiny cramped room, with a couple of dozens of geeks,
and it felt like a clique of people on the side of the conference. This
year, we had a huge room and more than 200 people showed up.</p>
<p>I am used to talks being about a grad student or young researcher that
has whiped the code of a paper on the web, with an open license but no
vision. This year, people were presenting actual projects, with long-term
goals and the desire to solve a problem large than their latest research.
It might explain why the attendance was huge: people came because talks
might genuinely help them.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>With Cheng and Antti, we had choosen as a theme <em>“open ecosystems”</em>,
because ecosystems are the key to scaling computing and science. Between
us, imposing a theme on a workshop is something challenging, as people
submit abstracts, good or bad, and one has to compose with what one has.
However, at lot of talks mentioned how the projects slot in a wider
picture, and interact with a community. For instance, Evan attributes
part of the success of Cafe to the <a class="reference external" href="https://github.com/BVLC/caffe/wiki/Model-Zoo">“Model Zoo”</a> in which the community
contributes fitted models. At the other end of the spectrum, OpenML is a
full online project with the goal to foster collaboration and comparison.
Project developers have shown in their talk that they are very conscious
of other projects that might be used together with their’s.</p>
</div>
<div class="section" id="accepting-the-sustainability-challenges">
<h2>Accepting the sustainability challenges</h2>
<p>Over the time, I have gradually realized the importance of community
building, <em>ie</em> project management and goal setting, more than technical
virtuosity. Historically, the scientific culture of code has put the
emphasis on the genius ideas behind the code, and the craftsmanship of
the implementation, to the cost of sustainability.</p>
<div class="align-right docutils container">
Alone, I go fast. Together, we go far.</div>
<p>I was surprised to see that the MLOSS community was growing very aware of
mechanisms of long-term project life, in particular the human factors.</p>
<p>I was asked by my coorganizers to give <a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/2bc15b283f324784a945d79d9a06c76c1d">a talk on factors of success of
open source scientific software</a>.
I touched upon <strong>software engineering</strong>, <strong>project vision</strong>,
<strong>licensing</strong>, <strong>governance</strong>, <strong>community building</strong>. All these topics
deemed <em>“non scientific”</em> and thus so often despised and left out. I was
astonished to find out that the talks before me were giving very good
advice on these. I found that I only had to summarize and comment what
had been said before. This evolution of the scientific community makes me
very hopeful for the future.</p>
<blockquote class="epigraph">
<p>Every line of code you write is dept. You should be ashamed of every line
of code you have written. […]</p>
<p>You have a supply of labor. These are the people who are contributors
[…].
The people who are users and not contributors are actually a source of
demand […] they mostly consume sources of labor rather than produce it.
—
<a class="reference external" href="http://k4webcast.mediasite.com/Mediasite/Play/2529ebcb20794942874d5c277c5dcc981d">John Myles White</a></p>
</blockquote>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Thanks to our sponsors</strong></p>
<p><a class="reference external" href="http://www.facebook.com">Facebook</a> and <a class="reference external" href="http://www.continuum.io">continuum</a> sponsored the trip for our keynote
speakers. Thank you very much, the keynotes were great!</p>
<p>The <a class="reference external" href="http://www.datascience-paris-saclay.fr/">Paris-Saclay Center for Data Science (CDS)</a> gave us our main operating
fund, which is critical for organizing an event. In general, I must
say that the CDS has been hugely supportive of open source data
science in the Paris area, having a significant impact on training as
well as development.</p>
<p>And also, I must acknowledge support from <a class="reference external" href="http://http://www.inria.fr/">Inria</a> for the accounting and administration
of the event.</p>
<p>Finally, <strong>our reviewers were amazing</strong>. Most of them reviewed the
project, ie its code, its documentation, its support. They arose above
the typical petty fights that we see in academia and focused on what
the project was bringing to the scientific community. Often there
reviews were longer and with more information than the abstract
submitted.</p>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Related posts</strong>:</p>
<ul class="simple">
<li><a class="reference external" href="software-for-reproducible-science-lets-not-have-a-misunderstanding.html">Software for reproducible science: let’s not have a misunderstanding</a></li>
<li><a class="reference external" href="../science/publishing-scientific-software-matters.html">Publishing scientific software matters</a></li>
</ul>
</div>
</div>
Nilearn sprint: hacking neuroimaging machine learning2015-08-04T00:00:00+02:002015-08-04T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2015-08-04:/programming/nilearn-sprint-hacking-neuroimaging-machine-learning.html<p>A couple of weeks ago, we had in Paris the second international <a class="reference external" href="http://nilearn.github.io">nilearn</a> sprint, dedicated to making <strong>machine learning
in neuroimaging easier and more powerful</strong>.</p>
<p>It was such a fantastic experience, as nilearn is really shaping up as a
simple yet powerful tool, and there is a lot of enthusiasm …</p><p>A couple of weeks ago, we had in Paris the second international <a class="reference external" href="http://nilearn.github.io">nilearn</a> sprint, dedicated to making <strong>machine learning
in neuroimaging easier and more powerful</strong>.</p>
<p>It was such a fantastic experience, as nilearn is really shaping up as a
simple yet powerful tool, and there is a lot of enthusiasm. For me, this
sprint is a turning point, as I could see people other than the original
core team (that spanned out of <a class="reference external" href="https://team.inria.fr/parietal/">our research team</a>) excited about the project’s future.
Thank you to all who came:</p>
<ul class="columns simple">
<li>Ahmed Kanaan</li>
<li>Andres Hoyos Idrobo</li>
<li>Alexandre Abraham</li>
<li>Arthur Mensch</li>
<li>Ben Cipolli (remote)</li>
<li>Bertrand Thirion</li>
<li>Chris Filo Gorgolewski</li>
<li>Danilo Bzdok</li>
<li>Elvis Dohmatob</li>
<li>Julia Hutenburg</li>
<li>Kamalaker Dadi</li>
<li>Loic Esteve</li>
<li>Martin Perez</li>
<li>Michael Hanke</li>
<li>Oscar Nájera, working on
<a class="reference external" href="http://sphinx-gallery.readthedocs.org/">sphinx-gallery</a></li>
</ul>
<img alt="" src="attachments/nilearn_july_2015_sprint/nilearn_sprint.jpg" style="width: 100%;" />
<p>The sprint was a joint sprint with the <a class="reference external" href="http://martinos.org/mne/stable/mne-python.html">MNE-Python</a> team, that makes MEG
processing awesome. We also need to thank <a class="reference external" href="http://alexandre.gramfort.net">Alex Gramfort</a>, who did most of the work to set up the
sprint, as well as <a class="reference external" href="https://www.universite-paris-saclay.fr/en/research/project/lidex-neurosaclay">NeuroSaclay</a>
for funding, and <a class="reference external" href="http://lapaillasse.org/">La paillasse</a>, <a class="reference external" href="http://www.telecom-paristech.fr">Telecom</a>, and <a class="reference external" href="http://www.inria.fr/en/centre/saclay">INRIA</a> for hosting.</p>
<div class="section" id="highlights-of-the-sprints-results">
<h2>Highlights of the sprints results</h2>
<p><strong>Plotting of multiple maps</strong></p>
<blockquote>
<a class="reference external image-reference" href="https://circle-artifacts.com/gh/nilearn/nilearn/128/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/connectivity/plot_canica_resting_state.html"><img alt="" class="align-right" src="attachments/nilearn_july_2015_sprint/plot_canica_resting_state_001.png" style="width: 200px;" /></a>
<p>A function to visualize overlays of various maps, eg for a
probabilistic atlas, with defaults that try to adapt to the number of
maps (see the <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/128/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/manipulating_visualizing/plot_prob_atlas.html">example</a>).
It’s very useful for example for <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/128/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/connectivity/plot_canica_resting_state.html">easy visualizing of ICA components</a>.</p>
</blockquote>
<p><strong>Sign of activation in glass brain</strong></p>
<blockquote>
<a class="reference external image-reference" href="https://circle-artifacts.com/gh/nilearn/nilearn/287/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/manipulating_visualizing/plot_demo_glass_brain_extensive.html"><img alt="" class="align-right" src="attachments/nilearn_july_2015_sprint/plot_demo_glass_brain_extensive_005.png" style="width: 200px;" /></a>
<p>Our glass brain plotting was greatly improved adding amongst other
things the option to capture the sign of the activation in the color
(see this <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/287/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/manipulating_visualizing/plot_demo_glass_brain_extensive.html">example</a>).</p>
</blockquote>
<p><strong>Spatially-regularized decoder</strong></p>
<blockquote>
<a class="reference external image-reference" href="https://circle-artifacts.com/gh/nilearn/nilearn/287/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/decoding/plot_haxby_space_net.html"><img alt="" class="align-right" src="attachments/nilearn_july_2015_sprint/plot_haxby_space_net_002.png" style="width: 200px;" /></a>
<p>Decoders based on GraphNet and total variation have finally landed in
nilearn. This has required a lot of work to get fast convergence and
robust parameter selection. At the end of the day, it is much slower
than an SVM, but the maps look splendid
(see this <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/287/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/decoding/plot_haxby_space_net.html">example</a>).</p>
</blockquote>
<p><strong>Sparse dictionary learning</strong></p>
<blockquote>
<a class="reference external image-reference" href="https://circle-artifacts.com/gh/nilearn/nilearn/282/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/connectivity/plot_dict_learning_resting_state.html"><img alt="" class="align-right" src="attachments/nilearn_july_2015_sprint/plot_dict_learning_resting_state_001.png" style="width: 200px;" /></a>
<p>We have almost merged sparse dictionnary learning as a alternative to ICA.
Experience shows that on resting-state data, it gives more contrasted
segmentation of networks
(see this <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/282/artifacts/0/home/ubuntu/nilearn/doc/_build/html/auto_examples/connectivity/plot_dict_learning_resting_state.html">example</a>).</p>
</blockquote>
<p><strong>New installation docs</strong></p>
<blockquote>
New webpage layout using tabs to display only the installation
instruction relevant to the OS of the user (see <a class="reference external" href="https://circle-artifacts.com/gh/nilearn/nilearn/287/artifacts/0/home/ubuntu/nilearn/doc/_build/html/introduction.html#installation">here</a>).
The results are more compact and more clear instructions, that I hope
will make our users’ life easier.</blockquote>
<p><strong>CircleCI integration</strong></p>
<blockquote>
We now use <a class="reference external" href="https://circleci.com/gh/nilearn/nilearn">CircleCI</a> to
run the examples and build the docs. This is challenging because our
examples are real cases of neuroimaging data analysis, and thus require
heavy datasets and computing horse power.</blockquote>
<p><strong>Neurodebian packaging</strong></p>
<blockquote>
There are now <a class="reference external" href="http://neuro.debian.net/pkgs/python-nilearn.html">neurodebian packages</a> for nilearn.</blockquote>
<p>And much more!</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">Features listed above are <strong>not</strong> in the released version of nilearn.
You need to wait a month or so.</p>
</div>
</div>
Software for reproducible science: let’s not have a misunderstanding2015-05-18T00:00:00+02:002015-05-18T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2015-05-18:/programming/software-for-reproducible-science-lets-not-have-a-misunderstanding.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p><strong>tl;dr:</strong> <em>Reproducibilty is a noble cause and scientific
software a promising vessel. But excess of reproducibility can be at
odds with the housekeeping required for good software engineering.
Code that “just works” should not be taken for granted.</em></p>
<p class="last"><em>This post advocates for a progressive consolidation effort of
scientific …</em></p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p><strong>tl;dr:</strong> <em>Reproducibilty is a noble cause and scientific
software a promising vessel. But excess of reproducibility can be at
odds with the housekeeping required for good software engineering.
Code that “just works” should not be taken for granted.</em></p>
<p class="last"><em>This post advocates for a progressive consolidation effort of
scientific code, rather than putting too high a bar on code release.</em></p>
</div>
<p><a class="reference external" href="http://ivory.idyll.org/blog/">Titus Brown</a> recently shared <a class="reference external" href="http://ivory.idyll.org/blog/2015-how-should-we-think-about-research-software.html">an
interesting war story</a>
in which a reviewer refuses to review a paper until he can run the code
on his own files. Titus’s comment boils down to:</p>
<blockquote>
<blockquote class="epigraph">
<a class="reference external" href="http://ivory.idyll.org/blog/2015-how-should-we-think-about-research-software.html">“Please destroy this software after publication”</a>.</blockquote>
</blockquote>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Reproducible science: Does the emperor have clothes?</p>
</div>
<p>In other words, code for a publication is often not reusable. This
point of view is very interesting from someone like Titus, who is a
<a class="reference external" href="http://ivory.idyll.org/blog/a-conversation-on-reproducibility.html">vocal proponent</a> of
reproducible science. His words triggered some surprises, which led Titus
to wonder if <a class="reference external" href="http://ivory.idyll.org/blog/2015-we-live-in-a-bubble.html">some of the reproducible science crowd folks live in a
bubble</a>. I
was happy to see <a class="reference external" href="https://twitter.com/ctitusbrown/status/589171853031186434">the discussion</a> unroll, as
I think that there is a strong risk of creating a bubble around
reproducible science. Such a bubble will backfire.</p>
<!-- Let me share my point of view on software for reproducible science. -->
<div class="section" id="replication-is-a-must-for-science-and-society">
<h2>Replication is a must for science and society</h2>
<p>Science advances by accumulating knowledge built upon
observations. It’s easy to forget that these observations, and the
corresponding paradigmatic conclusions, are not always as simple to
establish as the fact that hot air rises: <strong>replicating many times the
scientific process transforms an evidence into a truth</strong>.</p>
<p>One striking example of scientific replication is <a class="reference external" href="http://www.nature.com/news/first-results-from-psychology-s-largest-reproducibility-test-1.17433">the on-going effort in
psychology</a>
to replay the evidence behind well-accepted findings central to
current line of thoughts in psychological sciences. It implies setting up
the experiments accordingly to the seminal publications, acquiring the
data, and processing it to come up to the same conclusions. Surprisingly,
not everything that was taken for granted holds.</p>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Findings later discredited backed economic policy</p>
</div>
<p>Another example, with massive consequences on Joe Average’s everyday, is
the failed replication of Reinhart and Rogoff’s <a class="reference external" href="http://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt">“Growth in a Time of
Debt”</a>
publication. The original paper, published in 2010 in the American
Economic Review, claimed empirical findings linking important public debt
to failure of GDP growth. In a context of economical crisis, it was used
by policy makers as a justification for restricted public spending.
However, while pursuing a mere homework assignment to replicate these
findings, <a class="reference external" href="http://www.bbc.com/news/magazine-22223190">a student uncovered methodological flaws with the paper</a>. Understanding the
<a class="reference external" href="http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems">limitations</a>
of the original study took a while, and <strong>discredited the academic
backing to the economical doctrine of austerity</strong>. Critically, the
analysis of the publication was possible only because Reinhart and Rogoff
<strong>released their spreadsheet, with data and analysis details</strong>.</p>
</div>
<div class="section" id="sharing-code-can-make-science-reproducible">
<h2>Sharing code can make science reproducible</h2>
<p>A great example of sharing code to make a publication reproducible is the
recent paper on <a class="reference external" href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0126255">orthogonalization of regressors in fMRI models</a>,
by Mumford, Poline and Poldrack. The paper is a didactic refutation
of non-justified data processing practices. The authors made their
point much stronger by giving <a class="reference external" href="http://nbviewer.ipython.org/github/jmumford/orthogonalizaton_ipynb/blob/master/orthogonalization.ipynb">an IPython notebook</a>
to reproduce their figures. The recipe works perfectly here, because the
ideas underlying the publication are simple and can be illustrated on
synthetic data with relatively inexpensive computation. A short IPython
notebook is all it takes to convince the reader.</p>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Sharing complex code… chances are it won’t run on new data.</p>
</div>
<p>At the other end of the spectrum, a complex analysis pipeline will not be
as easy to share. For instance, a feat of strength such as Miyawaki <em>et
al</em>’s <a class="reference external" href="http://www.cell.com/neuron/abstract/S0896-6273%2808%2900958-6">visual image
reconstruction from brain activity</a>
requires complex statistical signal processing to extract weak
signatures. Miyawaki <em>et al</em> shared the data. They might share the code, but
it would be a large chunk of code, probably fragile to changes in the
environment (Matlab version, OS…). Chances are that it wouldn’t run on
new data. This is the scenario that prompted Titus’s words:</p>
<blockquote>
<blockquote class="epigraph">
<a class="reference external" href="http://ivory.idyll.org/blog/2015-how-should-we-think-about-research-software.html">“Please destroy this software after publication”</a>.</blockquote>
</blockquote>
<p>I have good news: you can reproduce Miyawaki’s work with <a class="reference external" href="http://nilearn.github.io/auto_examples/decoding/plot_miyawaki_reconstruction.html">an example</a>
in <a class="reference external" href="http://nilearn.github.io">nilearn</a>, a library for
machine learning on brain images. The example itself is concise,
readable and it reliably produces figures close to that of the paper.</p>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Maintained libraries make feats of strength routinely
reproducible.</p>
</div>
<p>This easy replication is only possible because <strong>the corresponding code
leverages a set of libraries that encapsulate the main steps of the
analysis</strong>, mainly <a class="reference external" href="http://scikit-learn.org/stable/">scikit-learn</a> and
<a class="reference external" href="http://nilearn.github.io">nilearn</a> here. These libraries are
<a class="reference external" href="https://travis-ci.org/nilearn/nilearn">tested</a>, <a class="reference external" href="https://github.com/nilearn/nilearn/issues?q=is%3Aissue+is%3Aclosed">maintained</a>
and <a class="reference external" href="http://gael-varoquaux.info/programming/scikit-learn-015-release-highlights.html">released</a>.
They enable us to go from a feat of strength to routine replication.</p>
<!-- * An example of non-reproducible research (my ICML paper) -->
<!-- Can research be up to the software engineering challenge? -->
</div>
<div class="section" id="reproducibility-is-not-sustainable-for-everything">
<h2>Reproducibility is not sustainable for everything</h2>
<!-- Things are not always that easy
It's not you, it's me
Nobody said it was easy
Living up to the promise? -->
<blockquote class="epigraph">
Thinking is easy, acting is difficult
— <em>Goethe</em></blockquote>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Keeping a physics apparatus running for replication years later?</p>
</div>
<p>I started my scientific career doing physics, and fairly <a class="reference external" href="http://gael-varoquaux.info/science/general-relativity-quantum-physics-freely-falling-planes-and-bayesian-statistics.html">“heavy” physics</a>:
vacuum systems, lasers, free-falling airplanes. In such settings, the
cost of maintaining an experiment is apparent to the layman. No-one is
expected to keep an apparatus running for replication years later. The
pinnacle of reproducible research is when the work becomes doable in a
students lab. Such progress is often supported by improved
technology, driven by wider applications of the findings.</p>
<p>However, not every experiment will give rise to a students lab.
Replicating the others will not be easy. Even if the instruments are
still around the lab, they will require setting up, adjusting and wiring.
And chances are that connectors or cables will be missing.</p>
<p>Software is no different. Storing and sharing it is cheaper. But
technology evolves very fast. Every setup is different. Code for a
scientific paper has seldom been built for easy maintenance: lack of
tests, profusion of exotic dependencies, inexistent documentation.
Robustness, portability, isolation, would be desirable, but it is
difficult and costly.</p>
<p>Software developers know that understanding the constraints to design a
good program requires writing a prototype. <strong>Code for a scientific paper
is very much a prototype</strong>: it’s a first version of an idea, that proves
its feasibility. Common sense in software engineering says that
<a class="reference external" href="http://blog.codinghorror.com/the-prototype-pitfall/">prototypes are designed to be thrown away</a>. Prototype code
is fragile. It’s untested, probably buggy for certain usage. Releasing
prototypes amounts to distributing semi-functioning code. This is the
case for most code accompanying a publication, and it is to be expected
given the very nature of research: exploration and prototyping <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
<!-- Quality scientific software require making choices -->
<!-- Doing less, better -->
<!-- Quality scientific software, only for a happy few -->
</div>
<div class="section" id="no-success-without-quality">
<h2>No success without quality, …</h2>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Highly-reliable is more useful than state-of-the-art.</p>
</div>
<p>My experience with scientific code has taught me that success require
quality. Having a good implementation of simple, well-known, methods
seems to matter more than doing something fancy. This is what the
success of scikit-learn has taught us: we are really providing classic
“old” machine learning methods, but with a good API, good docs,
computational performance, and stable numerics controlled by stringent
tests. There exists plenty of more sophisticated machine-learning
methods, including some that I have developed specifically for my data.
Yet, I find myself advising my co-workers to use the methods in
scikit-learn, because I know that the implementation is reliable and that
they will be able to use them <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>.</p>
<p>This quality is indeed central to doing science with code. What good is a
data analysis pipeline if it crashes when I fiddle with the data? How can
I draw conclusions from simulations if I cannot change their parameters?
As soon as I need trust in code supporting a scientific
finding, I find myself tinkering with its input, and often breaking it.
Good scientific code is code that can be reused, that can lead to
large-scale experiments validating its underlying assumptions.</p>
<div class="figure align-right">
<a class="reference external image-reference" href="https://twitter.com/divineomega/status/576165762911608833"><img alt="" src="../programming/attachments/sqlite_code.png" /></a>
<p class="caption">Sqlite is so much used that its developers have been woken up at
night by users.</p>
</div>
<p>You might say that I am putting the bar too high; that slightly buggy
code is more useful than no code. But I frown at the idea of releasing
code for which I am unable to do proper quality assurance. I may have
done too much of that in the past. And because I am a prolific coder, many
people are using code that has been through my hands. My mailbox looks
like a battlefield, and when I go the coffee machine I find myself
answering questions.</p>
<!-- Pour vivre heureux, vivons cachés.
http://en.wikipedia.org/wiki/Jean-Pierre_Claris_de_Florian -->
</div>
<div class="section" id="and-making-difficult-choices">
<h2>… and making difficult choices</h2>
<!-- diminishing returns -->
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Craftsmanship is about trade-offs</p>
</div>
<p>Achieving quality requires making choices. Not only because time
is limited, but also because the difficulty to maintain and improve a
codebase increases much quicker than the numbers of features <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>. This
phenomena is actually frightening to watch: adding a feature in
scikit-learn these days is much much harder than what it used to be in
the early days. Interactions between features is a killer: when you
modify something, something else unrelated breaks. For a given
functionality, <strong>nothing makes the code more incomprehensible than
cyclomatic complexity</strong>: the multiplicity of branching, if/then clauses,
for loops. This complexity naturally appears when supporting different
input types, or minor variants of a same method.</p>
<p>The consequence is that ensuring quality for many variants of a method is
prohibitory. This limit is a real problem for reproducible
science, as science builds upon comparing and opposing models. However,
ignoring it simply leads to code that fails doing what it claims to do.
What this is telling us, is that if we are really trying to do long-term
reproducibility, we <strong>need to identify successful and important research
and focus our efforts on it</strong>.</p>
<p>If you agree with my earlier point that the code of a publication is
a prototype, this iterative process seems natural. Various ideas
can be thought of as competing prototypes. Some will not lead to
publication at all, while others will end up having a high impact.
Knowing before-hand is impossible. Focusing too early on achieving high
quality is counter productive. What matters is <strong>progressively
consolidating the code</strong>.</p>
<!-- XXX rephrase the above to avoid 'what matters'? -->
<!-- I am sorry to say that my publications are not based on code with 90% test coverage. -->
<!-- say that my methods in machine learning will probably never make it to
scikit-learn -->
</div>
<div class="section" id="reproducible-science-a-rich-trade-off-space">
<h2>Reproducible science, a rich trade-off space</h2>
<div class="admonition align-right note">
<p class="first admonition-title">Note</p>
<p class="last">Verbatim replication or reuse?</p>
</div>
<p>Does Reinhart and Rogoff’s <em>“Growth in a Time of Debt”</em> paper face the
same challenges as the manuscript under review by Titus? One is
describing mechanisms while the other is introducing a method. The code
of the former is probably much simpler than that of the latter. Different
publications come with different goals and code that is more or less easy
to share. For verbatim replication of the analysis of a paper, a simple
IPython notebook without tests or API is enough. To go beyond requires
applying the analysis to different problems or data: reuse. Reuse is
very difficult and cannot be a requirement for all publications.</p>
<!-- As someone who spends a lot of time on method development, I think a lot
in terms of code reuse. On the contrary, -->
<p>Conventional wisdom in academia is that science builds upon ideas and
concepts rather than methods and code. Galileo is known for his
contribution to our understanding of the cosmos. Yet, methods
development underpins science. Galileo is also the inventor of the
telescope, which was a huge technical achievement. He needed to develop
it to back his cosmological theories. Today, Galileo’s measurements are
easy to reproduce because telescopes are readily-available as consumer
products.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<blockquote class="epigraph">
Standing on the shoulders of giants —
<em>Isaac Newton, on software libraries</em></blockquote>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="topic">
<p class="topic-title"><strong>Related posts</strong>:</p>
<ul class="simple">
<li><a class="reference external" href="../science/publishing_scientific_software_matters.html">Publishing scientific software matters</a></li>
<li><a class="reference external" href="views_on_scientific_computing.html">Personal views on scientific computing</a></li>
</ul>
</div>
<!-- With great powers come great responsibility -->
<!-- Some publications, including computational ones, strive to contribute an idea. -->
<!-- The way I understand Titus's
phrase *"Please destroy this software after publication"* is that some
methods publication -->
<!-- Is the output of a paper the idea, or the code? It depends? (example of
the ICML) -->
<!-- Different code complexity, different trade-off (loops back to the point
above with Poldrack) -->
<!-- XXX: need to point to the donoho paper and cite it -->
<!-- Recommendations (in a separate blog post?):
* What the difficulties are (evolving APIs, plus configuration problems)
(skip this point?)
* don't publish method work on non open data (very restrictive, I have
been criticized for working on 'old', 'uninteresting' data). -->
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>To make my point very clear, releasing buggy untested code is not
a good thing. However, it is not possible to ask for all research
papers to come with industial-quality code. I am trying here to push
for a collective, reasoned, undertaking of consolidation.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Theory tells us that there is there is no universal machine
learning algorithm. Given a specific machine-learning application, it
is always possible to devise a custom strategy that out-performs a
generic one. However, <a class="reference external" href="http://jmlr.org/papers/volume15/delgado14a/delgado14a.pdf">do we need hundreds of classifiers to solve
real world classification problems?</a>
Empirical results <a class="reference external" href="http://jmlr.org/papers/volume15/delgado14a/delgado14a.pdf">[Delgado 2014]</a> show
that most of the benefits can be achieved with a small number of
strategies. Is it desirable and sustainable to distribute and keep
alive the code of every machine learning paper?</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Empirical studies on the workload for programmers to achieve a
given task showed that 25 percent increase in problem complexity results in
a 100 percent increase in programming complexity: <a class="reference external" href="http://ieeexplore.ieee.org/Xplore/login.jsp?url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F32%2F35909%2F01702600.pdf%3Farnumber%3D1702600&authDecision=-203">An Experiment on
Unit increase in Problem Complexity, Woodfield 1979</a>.</td></tr>
</tbody>
</table>
<p class="small">I need to thank my colleague <a class="reference external" href="http://multiplecomparisons.blogspot.fr">Chris Filo Gorgolewski</a> and my sister <a class="reference external" href="http://cbio.ensmp.fr/~nvaroquaux/">Nelle
Varoquaux</a> for their
feedback on this note.</p>
</div>
MLOSS: machine learning open source software workshop @ ICML 20152015-04-23T00:00:00+02:002015-04-23T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2015-04-23:/programming/mloss-machine-learning-open-source-software-workshop-icml-2015.html<div class="admonition note">
<p class="first admonition-title">Note</p>
<p><em>This year again we will have an exciting workshop on the
leading-edge machine-learning open-source software. This subject is
central to many, because software is how we propagate, reuse, and
apply progress in machine learning.</em></p>
<p class="last"><strong>Want to present a project? The deadline for the call for papers is
Apr 28th …</strong></p></div><div class="admonition note">
<p class="first admonition-title">Note</p>
<p><em>This year again we will have an exciting workshop on the
leading-edge machine-learning open-source software. This subject is
central to many, because software is how we propagate, reuse, and
apply progress in machine learning.</em></p>
<p class="last"><strong>Want to present a project? The deadline for the call for papers is
Apr 28th, in a few days</strong>
: <a class="reference external" href="http://mloss.org/workshop/icml15/">http://mloss.org/workshop/icml15/</a></p>
</div>
<p>The workshop will be help at the <a class="reference external" href="http://icml.cc/2015/">ICML conference</a>, in Lille France, on July 10th. ICML
–International Conference in Machine Learning– is the leading venue for
academic research in machine learning. It’s a fantastic place to hold
such a workshop, as the actors of theoretical progress are all around.
Software is the bridge that brings this progress beyond papers.</p>
<p>There is a <a class="reference external" href="http://mloss.org/workshop/">long tradition</a> of MLOSS
workshop, with one every year and a half. Last time, at NIPS 2013, I
could feel a bit of a turning point, as people started feeling that
different software slotted together, to create an efficient and
state-of-the art working environment. For this reason, we have entitled
this year’s workshop ‘open ecosystems’, stressing that contributions in
the scope of the workshop, that build a thriving work environment, are
not only machine learning software, but also better statistics or
numerical tools.</p>
<p>We have two keynotes with important contributions to such ecosystems:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.johnmyleswhite.com/">John Myles White</a> (Facebook), lead
developer of Julia statistics and machine learning: “Julia for machine
learning: high-level syntax with compiled-code speed”</li>
<li><a class="reference external" href="http://matthewrocklin.com">Matthew Rocklin</a> (Continuum Analytics),
developer of Python computational tools, in particular Blaze (confirmed):
“Blaze, a modern numerical engine with out-of-core and out-of-order
computations”.</li>
</ul>
<p>There will be also a practical presentation on how to set up an
open-source project, discussing hosting, community development, quality
assurance, license choice, by yours truly.</p>
Job offer: working on open source data processing in Python2015-04-02T00:00:00+02:002015-04-02T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2015-04-02:/programming/job-offer-working-on-open-source-data-processing-in-python.html<p>We, <a class="reference external" href="https://team.inria.fr/parietal/">Parietal team</a> at <a class="reference external" href="http://www.inria.fr/">INRIA</a>, are recruiting software developers to work on
open source machine learning and neuroimaging software in Python.</p>
<p>In general, we are looking for people who:</p>
<blockquote>
<ul class="simple">
<li>have a mathematical mindset,</li>
<li>are curious about data (ie like looking at data and understanding it)</li>
<li>have an affinity for problem-solving …</li></ul></blockquote><p>We, <a class="reference external" href="https://team.inria.fr/parietal/">Parietal team</a> at <a class="reference external" href="http://www.inria.fr/">INRIA</a>, are recruiting software developers to work on
open source machine learning and neuroimaging software in Python.</p>
<p>In general, we are looking for people who:</p>
<blockquote>
<ul class="simple">
<li>have a mathematical mindset,</li>
<li>are curious about data (ie like looking at data and understanding it)</li>
<li>have an affinity for problem-solving tradeoffs</li>
<li>love high-quality code</li>
<li>worry about users</li>
<li>are good scientific Python coders,</li>
<li>enjoy interacting with a community of developers</li>
</ul>
</blockquote>
<p>We welcome candidates people without all the skills, but are strongly
motivated to acquire them. Prior open-source experience is a big plus.</p>
<p>One example of such position with application in Neuroimaging is:
<a class="reference external" href="http://gael-varoquaux.info/programming/hiring-a-programmer-for-a-brain-imaging-machine-learning-library.html">http://gael-varoquaux.info/programming/hiring-a-programmer-for-a-brain-imaging-machine-learning-library.html</a>
Which was opened a year ago and has now resulted in nilearn:
<a class="reference external" href="http://nilearn.github.io/">http://nilearn.github.io/</a></p>
<p>Other positions may be more focused on general machine learning or
computing tools such as scikit-learn and joblib, which are reference
open-source libraries for data processing in Python.</p>
<p>We are a tightly knit team, with a high degree of programming, data
analysis and neuroimaging skills.</p>
<p>Please contact me and Olivier Grisel if you are interested,</p>
Euroscipy 2015: Call for paper2015-03-28T00:00:00+01:002015-03-28T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-03-28:/programming/euroscipy-2015-call-for-paper.html<p>EuroScipy 2015, the annual conference on Python in science will take
place in Cambridge, UK on 26-30 August 2015. The conference features two
days of tutorials followed by two days of scientific talks & posters and
an extra day dedicated to developer sprints. It is the major event in
Europe in …</p><p>EuroScipy 2015, the annual conference on Python in science will take
place in Cambridge, UK on 26-30 August 2015. The conference features two
days of tutorials followed by two days of scientific talks & posters and
an extra day dedicated to developer sprints. It is the major event in
Europe in the field of technical/scientific computing within the Python
ecosystem. Scientists, PhD’s, students, data scientists, analysts, and
quants from more than 20 countries attended the conference last year.</p>
<p>The topics presented at EuroSciPy are very diverse, with a focus on advanced
software engineering and original uses of Python and its scientific libraries,
either in theoretical or experimental research, from both academia and the
industry.</p>
<p>Submissions for posters, talks & tutorials (beginner and advanced) are welcome
on our website at <a class="reference external" href="http://www.euroscipy.org/2015/">http://www.euroscipy.org/2015/</a>
Sprint proposals should be addressed directly to the organisation at
<em>euroscipy-org@python.org</em></p>
<p><strong>Important dates</strong>:</p>
<ul class="simple">
<li><em>Apr 30, 2015</em> Talk and tutorials submission deadline</li>
<li><em>May 1, 2015</em> Registration opens</li>
<li><em>May 30, 2015</em> Final program announced</li>
<li><em>Jun 15, 2015</em> Early-bird registration ends</li>
<li><em>Aug 26-27, 2015</em> Tutorials</li>
<li><em>Aug 28-29, 2015</em> Main conference</li>
<li><em>Aug 30, 2015</em> Sprints</li>
</ul>
<p>We look forward to an exciting conference and hope to see you in Cambridge</p>
<p>The EuroSciPy 2015 Team - <a class="reference external" href="http://ww.euroscipy.org/2015/">http://ww.euroscipy.org/2015/</a></p>
PRNI 2016: call for organization2015-01-01T00:00:00+01:002015-01-01T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2015-01-01:/programming/prni-2016-call-for-organization.html<p class="first last">The steering committee of PRNI (Pattern Recognition for NeuroImaging) is opening a call for bid to organize the conference in June 2016, in Europe</p>
<p><a class="reference external" href="http://www.prni.org">PRNI (Pattern Recognition for NeuroImaging)</a> is
an IEEE conference about applying pattern recognition and machine
learning to brain imaging. It is a mid-sized conference (about 150
attendee), and is a satellite of OHBM (the annual “Human Brain Mapping”
meeting).</p>
<p>The steering committee is calling for bids to organize the conference in
June 2016, in Europe, as a satellite the OHBM meeting in Geneva.</p>
New website2014-10-09T00:00:00+02:002014-10-09T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-10-09:/misc/new-website.html<p>I am moving my website to a new design, relying on <a class="reference external" href="http://blog.getpelican.com/">Pelican</a> and more modern CSS.</p>
<p>So far, I had been using <a class="reference external" href="http://www.voidspace.org.uk/python/rest2web/">rest2web</a> to generate the static
part of the website, and a local install of wordpress for the blog. I
wasn’t doing good on keeping the wordpress install …</p><p>I am moving my website to a new design, relying on <a class="reference external" href="http://blog.getpelican.com/">Pelican</a> and more modern CSS.</p>
<p>So far, I had been using <a class="reference external" href="http://www.voidspace.org.uk/python/rest2web/">rest2web</a> to generate the static
part of the website, and a local install of wordpress for the blog. I
wasn’t doing good on keeping the wordpress install up to date, and I
eventually got hacked. It was hurting my desire of simplicity to need a
dynamic website. The combination of <a class="reference external" href="http://blog.getpelican.com/">Pelican</a> for my content, and <a class="reference external" href="https://disqus.com/">Disqus</a> suits very well my need, as it enables me to have
a simpler website, and still have blog posts and discussions.</p>
<p>I also took the opportunity to clean up the website, remove some old
content, and move my travel pictures to
<a class="reference external" href="https://www.flickr.com/photos/gaelvaroquaux/">flickr</a>.</p>
<div class="section" id="technical-choices">
<h2>Technical choices</h2>
<ul>
<li><p class="first"><a class="reference external" href="http://blog.getpelican.com/">Pelican</a> for the core engine. I like
the fact that it generates a static blog, that it uses restructured
text to store the content, and <a class="reference external" href="http://jinja.pocoo.org">jinja</a> as a
templating engine.</p>
<p>One interesting aspect of redoing my website with a more modern content
managment system was that I could lay out the information based on tags
and categories, rather than the old way of having a tree of nested
topics. This is much more flexible because one article is likely to
fall in many topics. Modern information organization is moving away
from the notion of path used to access to an entry, to the notion of
set of properties (tags here).</p>
</li>
<li><p class="first"><a class="reference external" href="http://purecss.io">Pure CSS</a> as a CSS base layer. I chose to use
Pure CSS rather than <a class="reference external" href="http://getbootstrap.com/">Bootstrap</a> as it is a
pure CSS framework (no javascript) and it is much lighter. I find that
Bootstrap websites can easily slow down browsing (due to download size
and javascript). Bootstrap also does play very well with html documents
in which ones doesn’t control the class tags, as those generated from
restructured text. But that’s true of most web front-end frameworks.
Another option was <a class="reference external" href="http://foundation.zurb.com/">Foundation</a>. I
didn’t explore it in details, but it looked like an interesting
tradeoff between Pure, which is very bare bones, and Bootstrap, the
heavy lifter. I chose to go for the most lightweight option, because I
had simple needs.</p>
<p>A result of using more modern CSS is that the website should look good
on any screen size, from very large screens to mobile phones.</p>
</li>
</ul>
</div>
Improving your programming style in Python2014-09-29T00:00:00+02:002014-09-29T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-09-29:/programming/improving-your-programming-style-in-python.html<p class="first last">Some references on software development techniques and patterns to help write better code.</p>
<p>Here are some references on software development techniques and patterns
to help write better code. They are intended for the casual programmer,
and certainly not an advanced developer.</p>
<p>They are listed in order of difficulty.</p>
<div class="section" id="software-carpentry">
<h2>Software carpentry</h2>
<p><a class="reference external" href="http://swc.scipy.org">http://swc.scipy.org</a>.</p>
<p>These are the original notes from Greg Wilson’s course on software
engineering at the university of Toronto. This course is specifically
intended for scientists, but not computer science students. It is very
basic and does not cover design issues.</p>
</div>
<div class="section" id="a-tutorial-introduction-to-python">
<h2>A tutorial introduction to Python</h2>
<p><a class="reference external" href="http://www.informit.com/articles/article.asp?p=23100&seqNum=3&rl=1">http://www.informit.com/articles/article.asp?p=23100&seqNum=3&rl=1</a>.</p>
<p>This tutorial is easier to follow than <a class="reference external" href="http://www.python.org/doc/">Guido’s tutorial</a>, thought it does not go as much in depth.</p>
</div>
<div class="section" id="python-essential-reference">
<h2>Python Essential Reference</h2>
<p><a class="reference external" href="http://www.informit.com/articles/article.asp?p=453682&rl=1">http://www.informit.com/articles/article.asp?p=453682&rl=1</a></p>
<p><a class="reference external" href="http://www.informit.com/articles/article.asp?p=459269&rl=1">http://www.informit.com/articles/article.asp?p=459269&rl=1</a></p>
<p>These are two chapters out of David Beazley’s excellent book <a class="reference external" href="http://www.amazon.com/Python-Essential-Reference-David-Beazley/dp/0735710910">Python
Essential Reference</a>.
They allow to understand more deeply how python works. I strongly recommend
this book to anybody serious about python.</p>
</div>
<div class="section" id="an-introduction-to-regular-expressions">
<h2>An Introduction to Regular Expressions</h2>
<p><a class="reference external" href="http://www.informit.com/articles/article.asp?p=20454&rl=1">http://www.informit.com/articles/article.asp?p=20454&rl=1</a></p>
<p>If you are going to do any sort of text manipulation, you absolutely need
to know how to use regular expressions: powerful search and replace patterns.</p>
</div>
<div class="section" id="software-design-for-maintainability">
<h2>Software design for maintainability</h2>
<p><a class="reference external" href="./software-design-for-maintainability.html">My own post</a></p>
<p>A case of shameless plug: this is a post that I wrote a few years ago. I
think that it is still relevant.</p>
</div>
<div class="section" id="writing-a-graphical-application-for-scientific-programming-using-traitsui">
<h2>Writing a graphical application for scientific programming using TraitsUI</h2>
<p><a class="reference external" href="http://gael-varoquaux.info/computers/traits_tutorial/index.html">http://gael-varoquaux.info/computers/traits_tutorial/index.html</a></p>
<p>Building interactive graphical application is a difficult problem. I have
found that the traitsUI module provides a great answer to this problem.
This is a tutorial intended for the non programmer.</p>
</div>
<div class="section" id="an-introduction-to-python-iterators">
<h2>An introduction to Python iterators</h2>
<p><a class="reference external" href="http://www.informit.com/articles/article.asp?p=26895&rl=1">http://www.informit.com/articles/article.asp?p=26895&rl=1</a></p>
<p>This article may not be terribly easy to follow, but iterator are a
great feature of Python, so this is definitely worth reading.</p>
</div>
<div class="section" id="functional-programming">
<h2>Functional programming</h2>
<p><a class="reference external" href="http://www.ibm.com/developerworks/linux/library/l-prog.html?open&l=766,t=gr,p=PrmgPyth">http://www.ibm.com/developerworks/linux/library/l-prog.html?open&l=766,t=gr,p=PrmgPyth</a></p>
<p>Functional programming is a programming style where mathematical
functions are successively applied to immutable objects to go from the
inputs of the program to its outputs in a succession of transformation.
It is appreciated by some because it is easy to analyze and prove.
In certain cases it can be very readable.</p>
</div>
<div class="section" id="patterns-in-python">
<h2>Patterns in Python</h2>
<p><a class="reference external" href="http://www.suttoncourtenay.org.uk/duncan/accu/pythonpatterns.html">http://www.suttoncourtenay.org.uk/duncan/accu/pythonpatterns.html</a>.</p>
<p>This document exposes a few design patterns in Python. Design patterns
are solutions to recurring development problems using object oriented
programming. I suggest this reading only if you are familiar with OOP.</p>
</div>
<div class="section" id="idiomatic-python">
<h2>Idiomatic Python</h2>
<ul>
<li><p class="first">Jeff Knupp’s post, a summary of his book:</p>
<p><a class="reference external" href="http://www.jeffknupp.com/blog/2012/10/04/writing-idiomatic-python/">http://www.jeffknupp.com/blog/2012/10/04/writing-idiomatic-python/</a></p>
</li>
<li><p class="first">The <a class="reference external" href="https://scipy-lectures.github.io">scipy-lectures</a> chapter on
advanced Python:</p>
<p><a class="reference external" href="https://scipy-lectures.github.io/advanced/advanced_python/index.html">https://scipy-lectures.github.io/advanced/advanced_python/index.html</a></p>
</li>
</ul>
</div>
<div class="section" id="general-object-oriented-programming-advice">
<h2>General Object-Oriented programming advice</h2>
<p>Designing Object-oriented code actually requires some care: when you are
building your set of abstractions, you are designing the world in which
you are going to be condemned to living (or actually coding). I would
advice people to keep things as simple as possible, and follow the SOLID
principles:</p>
<p><a class="reference external" href="http://mmiika.wordpress.com/oo-design-principles/">http://mmiika.wordpress.com/oo-design-principles/</a></p>
</div>
<div class="section" id="using-decorators-to-do-meta-programming-in-python">
<h2>Using decorators to do meta-programming in Python</h2>
<p><a class="reference external" href="http://www-128.ibm.com/developerworks/linux/library/l-cpdecor.html">http://www-128.ibm.com/developerworks/linux/library/l-cpdecor.html</a>.</p>
<p>A very beautiful article for the advanced python user. Meta-programming
is a programming technique that involves changing the program at the
run-time. This allows to add new abstractions to the code the
programmer writes, thus creating a “meta-language”. This article shows
this very well.</p>
</div>
<div class="section" id="a-primer-on-python-metaclass-programming">
<h2>A Primer on Python Metaclass Programming</h2>
<p><a class="reference external" href="http://www.onlamp.com/lpt/a/3388">http://www.onlamp.com/lpt/a/3388</a></p>
<p>Metaclasses allow to define new style of objects, that can have different
calling, creation or inheritance rules. This is way over my head, but I
am referencing it here for the record.</p>
</div>
<div class="section" id="iterators-in-python">
<h2>Iterators in Python</h2>
<p><a class="reference external" href="https://docs.python.org/2/library/itertools.html#recipes">https://docs.python.org/2/library/itertools.html#recipes</a></p>
<p>Learn to use the itertools (but don’t abuse them)!</p>
<p>Related to the producer/consumer problem with iterators, see:</p>
<p><a class="reference external" href="http://www.oluyede.org/blog/2007/04/09/producerconsumer-in-python/">http://www.oluyede.org/blog/2007/04/09/producerconsumer-in-python/</a></p>
<!-- vim:spell:spelllang=en_us ft=rst -->
</div>
Hiring an engineer to mine large functional-connectivity databases2014-09-20T00:00:00+02:002014-09-20T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-09-20:/programming/hiring-an-engineer-to-mine-large-functional-connectivity-databases.html<p><strong>Work with us to leverage leading-edge machine learning for
neuroimaging</strong></p>
<p>At <a class="reference external" href="https://team.inria.fr/parietal">Parietal</a>, my research team,
we work on improving the way brain images are analyzed, for medical
diagnostic purposes, or to understand the brain better. We develop
new machine-learning tools and investigate new methodologies for
for quantifying brain function from …</p><p><strong>Work with us to leverage leading-edge machine learning for
neuroimaging</strong></p>
<p>At <a class="reference external" href="https://team.inria.fr/parietal">Parietal</a>, my research team,
we work on improving the way brain images are analyzed, for medical
diagnostic purposes, or to understand the brain better. We develop
new machine-learning tools and investigate new methodologies for
for quantifying brain function from MRI scans.</p>
<p>One of our important alley of contributions is in deciphering “functional
connectivity”: analysis the correlation of brain activity to measure
interactions across the brain. This direction of research is exciting
because it can be used to probe the neural-support of <em>functional</em>
deficits in incapacitated patients, and thus lead to new biomarkers on
functional pathologies, such as autism. Indeed, functional connectivity
can be computed without resorting to complicated cognitive tasks, unlike
most functional imaging approaches. The flip side is that exploiting such
“resting-state” signal requires advanced multivariate statistics tools,
something at which the Parietal team excels.</p>
<p>For such multivariate processing of brain imaging data, Parietal has an
ecosystem of <a class="reference external" href="https://team.inria.fr/parietal/software">leading-edge high-quality tools</a>. In particular we have built
the foundations of the most successful Python machine learning library,
<a class="reference external" href="http://scikit-learn">scikit-learn</a>, and we are growing a dedicate
software, <a class="reference external" href="http://nilearn.github.io/">nilearn</a>, that leverages
machine-learning for neuroimaging. To support this ecosystem, we have
dedicated top-notch programmers, lead by the well-known
<a class="reference external" href="http://ogrisel.com/">Olivier Grisel</a>.</p>
<p>We are looking for a data-processing engineer to join our team and work
on <strong>applying our tools on very large neuroimaging databases to
learn specific biomarkers of pathologies</strong>. For this, the work will be
shared with the <a class="reference external" href="http://www.cati-neuroimaging.com/">CATI</a>, the Fench
platform for multicentric neuroimaging studies, located in the same
building as us. The general context of the job is the <a class="reference external" href="https://team.inria.fr/parietal/research/spatial_patterns/niconnect/">NiConnect</a>
project, a multi-organisational research project that I lead and
that focuses on improving diagnostic tools on resting-state functional
connectivity. We have access to unique algorithms and datasets, before
they are published. What we are now missing between those two, and that
link could be you.</p>
<p>If you want more details, they can be found on the <a class="reference external" href="https://team.inria.fr/parietal/job-offers">job offer</a>. This post is to motivate
the job in a personal way, that I cannot give in an official posting.</p>
<div class="section" id="why-take-this-job">
<h2>Why take this job?</h2>
<p>I don’t expect some to take this job only because it pays the bill. To be
clear, the kind of person I am looking for has no difficulties finding a
job elsewhere. So, if you are that person, why would you take the job?</p>
<ul class="simple">
<li>To join <a class="reference external" href="https://team.inria.fr/parietal/team-members/">a great team</a>
with many experts, focused on finding elegant solutions to hard
problems at the intersection of machine learning, cognitive science,
and software. Choose to work with great people, knowledgeable,
passionate, and <a class="reference external" href="https://team.inria.fr/parietal/inria-winter-party-2014/">fun</a>.</li>
<li>To work on interesting problems, that matter. They are interesting
because they are challenging but we have the skills to solve them. They
matter because they can make brain research better.</li>
<li>To learn. NeuroImaging + Machine learning is a quickly growing topic.
If you come from a NeuroImaging background and want to add to your CV
an actual expertise in machine learning for NeuroImaging. This is the
place to be.</li>
</ul>
</div>
<div class="section" id="what-would-make-me-excited-in-a-resume">
<h2>What would make me excited in a resume?</h2>
<ul class="simple">
<li>A genuine experience in neuroimaging data processing, especially large
databases.</li>
<li>Talent with computers and ideally some Python experience.</li>
<li>The unlikely combination of research training (graduate or
undergraduate) and experience in a non academic setting.</li>
<li>A problem-solving mindset.</li>
<li>A good ability to write about neuroimaging and data processing in
English: who knows, if everything goes to plan, you could very well be
publishing about new biomarkers.</li>
</ul>
<hr class="docutils" />
<p>Now if you are interested and feel up for the challenge, read the real
<a class="reference external" href="https://team.inria.fr/parietal/job-offers">job offer</a>, and send me
your resume.</p>
</div>
Scikit-learn 2014 sprint: a report2014-07-25T00:00:00+02:002014-07-25T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-07-25:/programming/scikit-learn-2014-sprint-a-report.html<p>A week ago, the 2014 edition of the
<a class="reference external" href="http://scikit-learn.org">scikit-learn</a> sprint was held in Paris.
This was the third time that we held an internation sprint and it was
hugely productive, and great fun, as always.</p>
<div class="section" id="great-people-and-great-venues">
<h2>Great people and great venues</h2>
<img alt="" class="align-center" src="https://pbs.twimg.com/media/BsqD4BeCQAEnT6w.jpg" style="width: 65%;" />
<p>We had a mix of core contributors and newcomers, which …</p></div><p>A week ago, the 2014 edition of the
<a class="reference external" href="http://scikit-learn.org">scikit-learn</a> sprint was held in Paris.
This was the third time that we held an internation sprint and it was
hugely productive, and great fun, as always.</p>
<div class="section" id="great-people-and-great-venues">
<h2>Great people and great venues</h2>
<img alt="" class="align-center" src="https://pbs.twimg.com/media/BsqD4BeCQAEnT6w.jpg" style="width: 65%;" />
<p>We had a mix of core contributors and newcomers, which is a great
combination, as it enables us to be productive, but also to foster the
new generation of core developers. Were present:</p>
<ul class="simple">
<li>Laurent Direr</li>
<li>Michael Eickenberg</li>
<li>Loic Esteve</li>
<li>Alexandre Gramfort</li>
<li>Olivier Grisel</li>
<li>Arnaud Joly</li>
<li>Kyle Kastner</li>
<li>Manoj Kumar</li>
<li>Balazs Kegl</li>
<li>Nicolas Le Roux</li>
<li>Andreas Mueller</li>
<li>Vlad Niculae</li>
<li>Fabian Pedregosa</li>
<li>Amir Sani</li>
<li>Danny Sullivan</li>
<li>Gabriel Synnaeve</li>
<li>Roland Thiolliere</li>
<li>Gael Varoquaux</li>
</ul>
<img alt="" class="align-center" src="https://pbs.twimg.com/media/BsqRedvCEAE5Opw.jpg" style="width: 65%;" />
<p>As the sprint extended through a French bank holiday and the week end,
we were hosted in a variety of venues:</p>
<ul class="simple">
<li><a class="reference external" href="http://lapaillasse.org">La paillasse</a>, a Paris bio-hacker space</li>
<li><a class="reference external" href="http://www.inria.fr">INRIA</a>, the French computer-science national
research, and the place where I work :)</li>
<li><a class="reference external" href="http://www.criteo.com">Criteo</a>, a French company doing word-wide
add-banner placement. The venue there was absolutely gorgeous, with a
beautiful terrace on the roofs of Paris. And they even had a social
event with free drinks one evening.</li>
<li><a class="reference external" href="http://www.tinyclues.com">Tinyclues</a>, a French startup mining
e-commerce data.</li>
</ul>
<p>I must say that we were treated like kings during the whole stay; each
host welcoming us as well they could. Thank you to all of our hosts!</p>
</div>
<div class="section" id="sponsored-by-the-digicosm-labex">
<h2>Sponsored by the Digicosm Labex</h2>
<p>Beyond our hosts, we need to thank the <a class="reference external" href="https://digicosme.lri.fr/tiki-index.php">Digicosme Labex</a>.
Digicosm gave us funding that covered some of the lunches, accomodations,
and travel expenses to bring in our contributors from abroad.</p>
</div>
<div class="section" id="achievements-during-the-sprint">
<h2>Achievements during the sprint</h2>
<p>The first day of the sprint was dedicated to polishing the <a class="reference external" href="http://www.scikit-learn.org/stable/whats_new.html">0.15
release</a>, which
was finally released on the morning of the second day, after 10 months
of development.</p>
<p>A large part of the efforts of the sprint were dedicated to improving
the coding base, rather than directly adding new features. Some files
were reorganized. The input validation code was cleaned up (opening the
way for better support of pandas structures in scikit-learn). We hunted
dead code, deprecation warnings, numerical instabilities and tests
randomly failing. We made the test suite faster, and refactored our
common tests that scan all the model.</p>
<p>Some work of our GSOC student, Manoj Kumar, was merged, making some
linear models faster.</p>
<p>Our <a class="reference external" href="http:/scikit-learn.org/dev">online documentation</a> was improve
with the <a class="reference external" href="http://scikit-learn.org/stable/modules/classes.html">API
documentation</a>
pointing to examples and source code.</p>
<p>Still work in progress:</p>
<ul class="simple">
<li>Faster stochastic gradient descent (with AdaGrad, ASGD, and one day
SAG)</li>
<li>Calibration of probabilities for models that do not have a
‘predict_proba’ method</li>
<li>Warm restart in random forests to add more estimators to an existing
ensemble.</li>
<li>Infomax ICA algorithm.</li>
</ul>
</div>
Scikit-learn 0.15 release: highlights2014-07-15T00:00:00+02:002014-07-15T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-07-15:/programming/scikit-learn-015-release-highlights.html<p>We have just released the 0.15 version of scikit-learn. Hurray!! Thanks
to all
<a class="reference external" href="http://scikit-learn.org/stable/whats_new.html#people">involved</a>.</p>
<div class="section" id="a-long-development-stretch">
<h2>A long development stretch</h2>
<p>It’s been a while since the <a class="reference external" href="http://gael-varoquaux.info/programming/scikit-learn-014-release-features-and-benchmarks.html">last release of
scikit-learn</a>. So a lot has
happened. Exactly 2611 commits according my count. Quite clearly, we
have more and more existing code …</p></div><p>We have just released the 0.15 version of scikit-learn. Hurray!! Thanks
to all
<a class="reference external" href="http://scikit-learn.org/stable/whats_new.html#people">involved</a>.</p>
<div class="section" id="a-long-development-stretch">
<h2>A long development stretch</h2>
<p>It’s been a while since the <a class="reference external" href="http://gael-varoquaux.info/programming/scikit-learn-014-release-features-and-benchmarks.html">last release of
scikit-learn</a>. So a lot has
happened. Exactly 2611 commits according my count. Quite clearly, we
have more and more existing code, more and more features to support.
This means that when we modify an algorithm, for instance to make it
faster, something else might break due to numerical instability, or
exploring some obscure option. The good news is that we have tight
continuous integration, mostly thanks to
<a class="reference external" href="https://travis-ci.org/scikit-learn/scikit-learn">travis</a> (but
Windows continuous integration is on its way), and we keep growing our
test suite. Thus while it is getting harder and harder to change
something in scikit-learn, scikit-learn is also becoming more and more
robust.</p>
</div>
<div class="section" id="highlights">
<h2>Highlights</h2>
<a class="reference external image-reference" href="https://twitter.com/t3kcit/status/434378452901187584"><img alt="" src="https://pbs.twimg.com/media/Bgc45seCUAAbze1.png" /></a>
<p><strong>Quality</strong> — Looking at the commit log, there has been a huge amount of
work to <a class="reference external" href="http://scikit-learn.org/stable/whats_new.html#id7">fix minor annoying
issues</a>.</p>
<p><strong>Speed</strong> — There has been a huge effort put in making many parts of
scikit-learn faster. Little details all over the codebase. We do hope
that you’ll find that your applications run faster. For instance, we
find that the worst case speed of Ward clustering is 1.5 times faster in
0.15 than 0.14. K-means clustering is often 1.1 times faster. KNN, when
used in brute-force mode, got faster by a factor of 2 or 3.</p>
<p><strong>Random Forest and various tree methods</strong> — The random forest and
various tree methods are much much faster, use parallel computing much
better, and use less memory. For instance, the picture on the right
shows the scikit-learn random forest running in parallel on a fat Amazon
node, and nicely using all the CPUs with little RAM usage.</p>
<p><strong>Hierarchical aglomerative clustering</strong> — <a class="reference external" href="http://scikit-learn.org/dev/modules/clustering.html#different-linkage-type-ward-complete-and-average-linkage">Complete linkage and average
linkage clustering have been
added</a>.
The benefit of these approach compared to the existing Ward clustering
is that they can take <a class="reference external" href="http://scikit-learn.org/stable/modules/clustering.html#varying-the-metric">an arbitrary distance
matrix</a>.</p>
<p><strong>Robust linear models</strong> — Scikit-learn now includes
<a class="reference external" href="http://scikit-learn.org/0.15/modules/linear_model.html#robustness-to-outliers-ransac">RANSAC</a>
for robust linear regression.</p>
<p><strong>HMM are deprecated</strong> — We have been discussing for a long time removing
HMMs, that do not fit in the focus of scikit-learn on predictive
modeling. We have created a separate
<a class="reference external" href="https://github.com/hmmlearn/hmmlearn">hmmlearn</a> repository for the
HMM code. It is looking for maintainers.</p>
<p><strong>And much more</strong> — plenty of <a class="reference external" href="http://scikit-learn.org/stable/whats_new.html">“minor
things”</a>, such as
better support for sparse data, better support for multi-label data…</p>
</div>
Google summer of code projects for scikit-learn2014-04-23T00:00:00+02:002014-04-23T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2014-04-23:/programming/google-summer-of-code-projects-for-scikit-learn.html<p>I’d like to welcome the four students that were accepted for the GSoC
this year:</p>
<ul class="simple">
<li>Issam: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/issamou/5733935958982656">Extending Neural networks</a></li>
<li>Hamzeh: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/hamsal/5709068098338816">Sparse Support for Ensemble Methods</a></li>
<li>Manoj: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/manojkumar/5673522948997120">Making Linear models faster</a></li>
<li>Maheshakya: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/maheshakya/5754903989321728">Locality Sensitive Hashing</a></li>
</ul>
<p>Welcome to all of you. Your submissions were excellent, and you
demonstrated a good will …</p><p>I’d like to welcome the four students that were accepted for the GSoC
this year:</p>
<ul class="simple">
<li>Issam: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/issamou/5733935958982656">Extending Neural networks</a></li>
<li>Hamzeh: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/hamsal/5709068098338816">Sparse Support for Ensemble Methods</a></li>
<li>Manoj: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/manojkumar/5673522948997120">Making Linear models faster</a></li>
<li>Maheshakya: <a class="reference external" href="http://www.google-melange.com/gsoc/project/details/google/gsoc2014/maheshakya/5754903989321728">Locality Sensitive Hashing</a></li>
</ul>
<p>Welcome to all of you. Your submissions were excellent, and you
demonstrated a good will to integrate in the project, with its social and
coding dynamics. It is a privilege to work with you.</p>
<p>I’d also like to thank all the mentors, Alex, Arnaud, Daniel, James,
Jaidev, Olivier, Robert and Vlad. It is a lot of work to mentor and
mentors are not only making it possible for great code to enter
scikit-learn, but also shaping a future generation of scikit-learn
contributors.</p>
Hiring a programmer for a brain imaging machine-learning library2014-02-12T00:00:00+01:002014-02-12T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2014-02-12:/programming/hiring-a-programmer-for-a-brain-imaging-machine-learning-library.html<p><strong>Work with us on putting machine learning in the hand of cognitive
scientists</strong></p>
<p>Parietal is a research team that creates advanced data analysis to mine
functional brain images and solve medical and cognitive science problems.
Our day to day work is to write machine-learning and statistics code to
understand and …</p><p><strong>Work with us on putting machine learning in the hand of cognitive
scientists</strong></p>
<p>Parietal is a research team that creates advanced data analysis to mine
functional brain images and solve medical and cognitive science problems.
Our day to day work is to write machine-learning and statistics code to
understand and use better images of brain function (most often fMRI). Our
purpose is to be useful to the NeuroImaging community, mostly medical and
cognitive science researched, to understand brain function better. What
is limiting us in this respect is that to reach end users we need to turn
our algorithms in usable software.</p>
<p>This is why Parietal has a long tradition of investing in building an
ecosystem of <a class="reference external" href="https://team.inria.fr/parietal/software">high-quality libraries and tools</a>: we build, layer by layer, an
environment in which we can do our research, and with which we hope to
one day reach the user. We choose Python, as a high-level general purpose
language with which we can do scientific computing, and, one day, GUIs,
or web servers. We contribute to the scipy ecosystem; we have built the
foundations of the most successful Python machine learning library,
<a class="reference external" href="http://scikit-learn">scikit-learn</a>. We are invested in the
<a class="reference external" href="http://nipy.org">neuroimaging in Python ecosystem</a>. Our students, our
team members, send patches to scientific Python projects, teach courses
on how to use them, speak at conferences.</p>
<p>But to go all the way, we need support from people who do software as
there sole goal. To put the finishing touch on the quality of our
end-user libraries, we need full-time programmers. In an academic
setting, they can be hard to justify, but we have always had dedicate
top-notch engineers at Parietal, our latest hire being the well-known
<a class="reference external" href="http://ogrisel.com/">Olivier Grisel</a>. This is where <strong>you</strong> can come
in.</p>
<p>The <a class="reference external" href="https://team.inria.fr/parietal/research/spatial_patterns/niconnect/">NiConnect</a>
is a specific research project in which we are developing leading
algorithmic tools. For this project, we have funding for a full-time
programmer. Someone that will help us make from our understand of how to
process brain images, a software tool that an cognitive science
researcher can use. We have started work on such a software, in the
<a class="reference external" href="http://nilearn.github.io/">nilearn</a> project. What we need is someone
who drives the project, and makes sure that the piece fit in together
well. That the code to solve the user’s problem is not our research code,
but a clean and lean library, just like scikit-learn is an elegant
answer to day-to-day machine learning tasks.</p>
<p>If you want more details, they can be found on the <a class="reference external" href="https://team.inria.fr/parietal/job-offers">job offer</a>. This post is to motivate
the job in a personal, that I cannot give in an official posting.</p>
<div class="section" id="why-take-this-job">
<h2>Why take this job?</h2>
<p>I don’t expect some to take this job only because it pays the bill. To be
clear, the kind of person I am looking for has no difficulties finding a
well-payed job elsewhere. So, if you are that person, why would you take
the job.</p>
<ul class="simple">
<li>To join <a class="reference external" href="https://team.inria.fr/parietal/team-members/">a great team</a>
that is focused on finding elegant solutions to hard problems at the
intersection of machine learning, cognitive science, and software.
Choose to work with great people, knowledgeable, passionate, and <a class="reference external" href="https://team.inria.fr/parietal/inria-winter-party-2014/">fun</a>.</li>
<li>To work on interesting problems, that matter. They are interesting
because they are challenging but we have the skills to solve them. They
matter because these skills need to be used to make brain research
better.</li>
<li>To have a boss (<a class="reference external" href="https://github.com/GaelVaroquaux">me</a>) that
actually codes and gives you feedback on your code.</li>
<li>To learn. Data science + Python is <em>the</em> combination of skills to have.
We have a at Parietal a unique expertise in these. And add to it fine
understanding of algorithms, high performance computing, statistics,
and software quality. You have the perfect lines on a CV.</li>
</ul>
</div>
<div class="section" id="what-would-make-me-excited-in-a-resume">
<h2>What would make me excited in a resume?</h2>
<ul class="simple">
<li>Open source contributions (there is no better coding CV than a github
account).</li>
<li>Experience in agile-like situations</li>
<li>A passion for code quality</li>
<li>Good Python experience</li>
<li>The unlikely combination of research-like training (eg undergraduate)
and experience in a non academic and non scientific setting (say web
development).</li>
<li>To know that you care about user experience, about understanding and
solving the user’s problems.</li>
</ul>
<hr class="docutils" />
<p>Now if you are interested and feel up for the challenge, read the real
<a class="reference external" href="https://team.inria.fr/parietal/job-offers">job offer</a>, and send me
your resume.</p>
</div>
Publishing scientific software matters2013-09-19T00:00:00+02:002013-09-19T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2013-09-19:/science/publishing-scientific-software-matters.html<p class="light">Christophe Pradal, Hans Peter Langtangen, and myself recently edited
<a class="reference external" href="http://www.sciencedirect.com/science/journal/18777503/4/5">a version</a> of the
Journal of Computational Science on scientific software, in
particular those written in Python. We wrote <a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1877750313000938">an editorial</a>
defending writing and publishing open source scientific software that
I wish to summarize here. The <a class="reference external" href="http://hal.inria.fr/hal-00858663/en">full text preprint</a> is openly …</p><p class="light">Christophe Pradal, Hans Peter Langtangen, and myself recently edited
<a class="reference external" href="http://www.sciencedirect.com/science/journal/18777503/4/5">a version</a> of the
Journal of Computational Science on scientific software, in
particular those written in Python. We wrote <a class="reference external" href="http://www.sciencedirect.com/science/article/pii/S1877750313000938">an editorial</a>
defending writing and publishing open source scientific software that
I wish to summarize here. The <a class="reference external" href="http://hal.inria.fr/hal-00858663/en">full text preprint</a> is openly available in <a class="reference external" href="http://gael-varoquaux.info/publications.html">my
publications list</a> as always. It
includes, amongst other things, references.</p>
<p><strong>Software is a central part of modern scientific discovery.</strong> Software turns a
theoretical model into quantitative predictions; software controls an
experiment; and software extracts from raw data evidence supporting or
rejecting a theory. As of today, scientific publications seldom discuss
software in depth, maybe because it is both highly technical and a recent
addition to scientific tools. But times are changing. More and more scientific
investigators are developing software and it is important to establish norms
for publication of this work. Producing scientific software is an important
part of the landscape of research activities. Very visible scientific software
is found in products developed by private companies, such as Mathwork’s Matlab
or Wolfram’s Mathematica, but let us not forget that these build upon code
written by and for academics. Scientists writing software contribute to the
advancement of Science via several factors.</p>
<p>First, software developed in one field, if written in a sufficiently general
way, can often be applied to advance a different field if the underlying
mathematics is common. <strong>Modern scientific software development has a strong
emphasis on generality and reusability by taking advantage of the general
properties of the mathematical structures in the problem.</strong> This feature of
modern software help close the gap between fields and accelerate scientific
discovery through packaging mathematical theories in a directly applicable way.</p>
<p>Second, <strong>the public availability of code is a corner stone of the
scientific method</strong>, as it is a requirement to reproducing scientific
results: “<em>if it’s not open and verifiable by others, it’s not science,
or engineering, or whatever it is you call what we do.</em>” (V. Stodden,
<em>The scientific method in practice</em>). Emphasizing code to an extreme,
Buckheit and Donoho have challenged the traditional view that a
publication was the valuable outcome of scientific research: “<em>an article
about computational science in a scientific publication is not the
scholarship itself, it is merely advertising of the scholarship. The
actual scholarship is the complete software development environment
[…]</em>”.</p>
<p>It is important to keep in mind that <strong>going beyond replication of
results requires reusable software tools</strong>: code that is portable, comes
with documentation, and, most of all, is maintained throughout the years.
Indeed, <strong>software development is a major undertaking that must build
upon best practices and a quality process</strong>. Reversing Buckheit and
Donoho’s argument, publications about scientific software play an increasingly
important part in the scientific methodology. First, in the publish-or-perish
academic culture, such publications give an incentive to software production
and maintenance, because good software can lead to highly-cited papers. Second,
<strong>the publication and review process are the de facto standards of
ensuring quality in the scientific world. As software is becoming increasingly
more central to the scientific discovery process, it must be subject to these
standards</strong>. We have found that writing an article on software leads the
authors to better clarify the project vision, technically and scientifically,
the prior art, and the contributions. Last but not least, scientists publishing
new results based on a particular software need an informed analysis of the
validity of that software. Unfortunately, much of the current practice for
adopting research software relies on ease of use of the package and reputation
of the authors.</p>
<p>[…]</p>
<p>Today, software is to scientific research what Galileo’s telescope was to
astronomy: a tool, combining science and engineering. It lies outside the
central field of principal competence among the researchers that rely on it.
Like the telescope, it also builds upon scientific progress and shapes our
scientific vision. Galileo’s telescope was a leap forward in optics, a field of
investigation that is now well established, with its own high-impact journals
and scholarly associations. Similarly, we hope that visibility and recognition
of scientific software development will grow.</p>
Scikit-learn 0.14 release: features and benchmarks2013-08-08T00:00:00+02:002013-08-08T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2013-08-08:/programming/scikit-learn-014-release-features-and-benchmarks.html<p>I have tagged and released the scikit-learn 0.14 release yesterday
evening, after more than 6 months of heavy development from the team. I
would like to give a quick overview of the highlights of this release in
terms of features but also in term of performance. Indeed, the
scikit-learn …</p><p>I have tagged and released the scikit-learn 0.14 release yesterday
evening, after more than 6 months of heavy development from the team. I
would like to give a quick overview of the highlights of this release in
terms of features but also in term of performance. Indeed, the
scikit-learn developers believe that <strong>performance matters</strong> and strive
to be fast and efficient on fairly datasets.</p>
<p>I will show in this article on a couple of benchmarks that we have
significant performance improvement and are competitive with the faster
libraries such as the proprietary WiseRF.</p>
<div class="section" id="prohiminent-new-features">
<h2>Prohiminent new features</h2>
<p>Most of the new features of the upcoming release have been mentionned
more in details on <a class="reference external" href="http://peekaboo-vision.blogspot.de/2013/07/scikit-learn-sprint-and-014-release.html">Andy Mueller’s
blog</a>.
I am just giving a quick list here for completness (see also the <a class="reference external" href="http://scikit-learn.org/stable/whats_new.html">full
list of changes</a>):</p>
<p><strong>Major new estimators</strong>:</p>
<ul class="simple">
<li><strong>AdaBoost</strong> (by <a class="reference external" href="http://noel.dawe.me">Noel Dawe</a> and <a class="reference external" href="http://www.montefiore.ulg.ac.be/~glouppe/">Gilles
Louppe</a>): the classic
boosting algorithm. This implementation can be applied to any
estimator, but uses trees by default.
AdaBoost is a learning strategy that builds from simple learning
strategies by focussing successively on samples that are not well
predicted. Typically, the simple learners (called <em>weak learners</em>)
can be rules as simple as taking simple thresholds of observed
quantities (this will form <em>decision stumps</em>).
<a class="reference external" href="http://scikit-learn.org/stable/modules/ensemble.html#AdaBoost">Documentation</a>
—
<a class="reference external" href="http://scikit-learn.org/stable/auto_examples/ensemble/plot_adaboost_twoclass.html">Example</a></li>
<li><strong>Biclustering</strong> (by <a class="reference external" href="http://www.kemaleren.com">Kemal Eren</a>):
clustering rows and columns of the data matrices.
Suppose you have access to the shopping list of many consumers,
biclustering would consists is grouping both consumers and product
they bought to come up with stories such as “geeks buy computers and
phones”, where “geeks” would be a group of consumers and “computers”
and “phones” would be groups of products.
<a class="reference external" href="http://scikit-learn.org/stable/modules/biclustering.html">Documentation</a>
—
<a class="reference external" href="http://scikit-learn.org/stable/auto_examples/bicluster/plot_spectral_biclustering.html">Example</a></li>
<li><strong>Missing value imputation</strong> (by <a class="reference external" href="http://nicolastr.com/">Nicolas
Tresegnie</a>): simple transformer filling
missing data with means or medians.
If your data-acquisition has failures, human or material, you can
easily end up with some descriptors missing for some observations. It
would be a pitty to throw away either those observations, or some
descriptors. “Imputation” fills in the blanks with simple strategies.
<a class="reference external" href="http://scikit-learn.org/stable/modules/preprocessing.html#imputation-of-missing-values">Documentation</a>
—
<a class="reference external" href="http://scikit-learn.org/stable/auto_examples/imputation.html">Example</a></li>
<li><strong>RBMs (Restricted Boltzmann Machines)</strong> (by <a class="reference external" href="http://ynd.github.io/">Yann
Dauphin</a>): a neural network model useful
for unsupervised learning of features.
Restricted Boltzmann machines learn a set of hidden (latent) factors
that have, for each observation, a probability to be activated or
not. These activations are found so that they explain the data well,
when combined across all the hidden factors with connection weights.
Typically, they form a new feature set that can be useful in a
prediction task.
<a class="reference external" href="http://scikit-learn.org/stable/modules/neural_networks.html#restricted-boltzmann-Machines">Documentation</a>
—
<a class="reference external" href="http://scikit-learn.org/stable/auto_examples/plot_rbm_logistic_classification.html">Example</a></li>
<li><strong>RandomizedSearchCV</strong> (by <a class="reference external" href="http://peekaboo-vision.blogspot.com">Andreas
Mueller</a>): setting
meta-parameters on estimators using a randomized parameter
exploration rather than a grid, as in a grid-search.
A CV (cross-validated) meta-estimator sets parameters of an
estimator by maximizing their cross-validated prediction scores. This
entails fitting the estimator for each parameter value tried. The
randomized-search explores the parameter space randomly, avoiding the
exponential growth in number of points to fit of the grid search.
<a class="reference external" href="http://scikit-learn.org/stable/modules/grid_search.html#randomized-parameter-optimization">Documentation</a>
—
<a class="reference external" href="http://scikit-learn.org/stable/auto_examples/randomized_search.html">Example</a></li>
</ul>
<p><strong>Infrastucture work</strong>:</p>
<ul class="simple">
<li><strong>New wesbite</strong> (mostly by <a class="reference external" href="http://www.montefiore.ulg.ac.be/~glouppe/">Gilles
Louppe</a>, <a class="reference external" href="https://github.com/nellev">Nelle
Varoquaux</a>, Vincent Michel and <a class="reference external" href="http://peekaboo-vision.blogspot.com">Andreas
Mueller</a>). The redesign of
the website had two objectives: <em>i)</em> unclutter the pages to help
prioritize information, <em>ii)</em> make it easier for users to find the
stable documentation, if they follow an external link to a
documentation of previous releases. I think that it also looks
prettier <em>:)</em>.</li>
<li><strong>Python 3 support</strong> (<a class="reference external" href="https://github.com/justinvf">Justin
Vincent</a>, <a class="reference external" href="https://github.com/larsmans">Lars
Buitinck</a>, <a class="reference external" href="https://github.com/smoitra87">Subhodeep
Moitra</a> and <a class="reference external" href="http://twitter.com/ogrisel">Olivier
Grisel</a>). As a side note, under Python
3.3, on Windows, we have found that <em>np.load</em> can trigger segfaults,
which means our test suite crashes. The tests not relying on
<em>np.load</em> pass.</li>
</ul>
</div>
<div class="section" id="major-api-changes">
<h2>Major API changes</h2>
<ul class="simple">
<li><strong>The scoring parameter</strong> One of the benefits of scikit-learn over
other learning packages is that it can set parameters to maximizing a
prediction score. However, the prediction that one would want to
optimize might depend on the application. Also, some scores can only
be computed with specific estimators, for instance because they
require probabilistic prediction. <a class="reference external" href="http://peekaboo-vision.blogspot.com">Andreas
Mueller</a> and <a class="reference external" href="https://github.com/larsmans">Lars
Buitinck</a> came up with <a class="reference external" href="http://scikit-learn.org/dev/modules/model_evaluation.html#the-scoring-parameter-defining-model-evaluation-rules">a new
API</a>
to specifies the scoring strategy that is versatile and hides
complexity from the user. This replaces the <em>score_func</em> argument.</li>
<li><strong>*sklearn.test()*</strong> is deprecated and will not run the test suite.
Please use <em>nosetests sklearn</em> from the command line.</li>
</ul>
<p>The full list of API changes can be found on the <a class="reference external" href="http://scikit-learn.org/stable/whats_new.html">change
log</a>.</p>
</div>
<div class="section" id="performance-improvements">
<h2>Performance improvements</h2>
<p>Many part of the codebase got speed-ups, with a focus on making
<strong>scikit-learn more scalable for bigger data</strong>.</p>
<ul class="simple">
<li>The trees (random forests and extra-trees) were massively sped up by
<a class="reference external" href="http://www.montefiore.ulg.ac.be/~glouppe/">Gilles Louppe</a>,
bringing them to par with the fastest libraries (see benchmarks
below)</li>
<li><a class="reference external" href="http://www.astro.washington.edu/users/vanderplas/">Jake
Vanderplas</a>
improved the BallTree and implemented fast KDTrees for
nearest-neighbor search (benchmarks below).</li>
<li><a class="reference external" href="https://github.com/cleverless">“cleverless”</a> made the DBSCAN
implementation scale to a large number of samples by relying on
KDTree and BallTree for neighbor search.</li>
<li>KMeans much faster on sparse data (<a class="reference external" href="https://github.com/larsmans">Lars
Buitinck</a>)</li>
<li>For text vectorization: much faster CountVectorizer and
TfidVectorizer with less memory consumption (Jochen Wersdorfer and
Roman Sinayev)</li>
<li>Out-of-core learning for discrete naive Bayes classifiers by <a class="reference external" href="http://twitter.com/ogrisel">Olivier
Grisel</a>. Estimators that implement a
<em>partial_fit</em> method can be used to fit the model with an
out-of-core strategy, as illustrated by the <a class="reference external" href="http://scikit-learn.org/dev/auto_examples/applications/plot_out_of_core_classification.html">out-of-core
classification
example</a>.
These settings are well suited to very big data.</li>
<li>FastICA: less memory consumptions and slightly faster code (<a class="reference external" href="https://github.com/dengemann">Denis
Engemann</a> and <a class="reference external" href="http://alexandre.gramfort.net">Alexandre
Gramfort</a>)</li>
<li>Faster IsotonicRegression (<a class="reference external" href="https://github.com/nellev">Nelle
Varoquaux</a>)</li>
<li>OrthogonalMatchingPursuitCV by <a class="reference external" href="http://alexandre.gramfort.net">Alexandre
Gramfort</a> and <a class="reference external" href="http://vene.ro">Vlad
Niculae</a>: while strictly-speaking not a speedup of
a existing estimator, this new estimator means that OMP parameters
can be set much faster.</li>
</ul>
</div>
<div class="section" id="we-are-faster-lies-damn-lies-and-benchmarks">
<h2>We are faster: lies, damn lies and benchmarks</h2>
<blockquote class="epigraph">
<p>“There are three kinds of lies: lies, damned lies and statistics.” —</p>
<p><em>Mark Twain’s Own Autobiography: The Chapters from the North
American Review</em></p>
</blockquote>
<p>I claim we have gotten faster at certain things. Other libraries, such
as <a class="reference external" href="http://docs.wise.io/">WiseRf</a>, have performance claims compared
to us. It turns out that benching statistical learning code is very
hard, because speed depends a lot on the properties of the data.</p>
<div class="section" id="fast-neighbor-searches-good-kdtrees-beat-balltrees">
<h3>Fast neighbor searches: good KDTrees beat BallTrees</h3>
<p>A good example of interplay between properties of the data and
computational speed is the nearest neighbor search. In general, finding
the nearest neighbor to a point out of <em>n</em> other points will cost you
<em>n</em> operations, as you have to compute the distance to each of these
points. However, building a tree-like data structure ahead of time can
make this query cost only <em>log n</em>. If these points are in 1D, <em>ie</em>
simple scalars, this would be achieve by sorting them. In higher
dimensions that can be achieved by building a <em>KDTree</em>, made of planes
dividing the space in half-spaces, or a <em>BallTree</em>, made of nested
balls.</p>
<div class="figure align-center">
<img alt="" src="http://www.astroml.org/_images/fig_kdtree_example_1.png" style="width: 60%;" />
<p class="caption"><strong>KD Tree</strong> Image from <a class="reference external" href="http://www.astroml.org/index.html">AstroML’s documentation</a></p>
</div>
<div class="figure align-center">
<img alt="" src="http://www.astroml.org/_images/fig_balltree_example_1.png" style="width: 60%;" />
<p class="caption"><strong>Ball tree</strong> Image from <a class="reference external" href="http://www.astroml.org/index.html">AstroML’s documentation</a></p>
</div>
<p>Popular wisdom in machine learning is that in high dimensions, BallTrees
scale better than KDTrees. This is explained by the fact that as the
dimensionality grows, the number of planes required to break up the
space grows too. On the contrary, if the data has structure, BallTrees
can more efficiently cover this structure. I have benched scikit-learn’s
KDTree and BallTree, as well as scipy’s KDTree, which employs a simpler
tree-building strategy, on a variety of datasets, both real-life and
artificial. Below if a summary plot giving relative performance of
neighbor search</p>
<div class="figure align-center">
<img alt="" src="https://gael-varoquaux.info/programming/attachments/sklearn_0.14.X_speed/nn_trees.png" style="width: 60%;" />
<p class="caption"><em>n</em> is the number of data points, and <em>p</em> the dimensionality.</p>
</div>
<p>We can see that no approach win on all counts. That said, it came to a
surprise to me to see that even in high dimension, <strong>scikit-learn’s
KDTree outperformed the BallTrees</strong>. This is explained because these
datasets do not display a heavily-structured low ambient dimension. On
highly-structured synthetic data, the benefit of BallTree can clearly
stand out, as shown by Jake
<a class="reference external" href="http://jakevdp.github.io/blog/2013/04/29/benchmarking-nearest-neighbor-searches-in-python">here</a>.
However, on most dataset people encounter, it seems that this is not the
case. Note also that <strong>scikit-learn’s KDTree tend to scale better in
high dimension than scipy’s</strong>. This is due to the more elaborate choice
of cutting planes. Note that it also has a cost, and may backfire, as on
some datasets scikit-learn is slower than scipy.</p>
<p>Overall, the new KDTree in scikit-learn seem to be giving an excellent
compromise. Congratulations
<a class="reference external" href="http://www.astro.washington.edu/users/vanderplas/">Jake</a>!</p>
</div>
<div class="section" id="dbscan-is-faster-with-trees">
<h3>DBSCAN is faster with trees</h3>
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/clustering.html#dbscan">DBSCAN</a>
is a clustering algorithm that relies heavily on the local neighborhood
structure. The implementation in scikit-learn 0.13 computed the complete
<em>n</em> by <em>n</em> matrix of distance between observations, which means that if
you had a lot of data, you would blow your memory. In the 0.14 release,
DBSCAN uses the BallTree, and as a result scales to much larger datasets
and brings speed benefits. Here is a comparison between 0.13 and 0.14
implementations (I couldn’t put data as large as I wanted because the
0.13 code would blow):</p>
<table border="1" class="docutils">
<colgroup>
<col width="53%" />
<col width="23%" />
<col width="24%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Dataset</th>
<th class="head">time with 0.13</th>
<th class="head">time with 0.14</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>“lfw”: 13233 samples, 5 features</td>
<td>6.57 seconds</td>
<td>3.59 seconds</td>
</tr>
<tr><td>“make_blobs”: 30000, with 10 features</td>
<td>33.50 seconds</td>
<td>12.87 seconds</td>
</tr>
</tbody>
</table>
<p>Importantly, the scaling is different: while the 0.13 code scales as <em>n
^ 2</em>, the 0.14 code scales as <em>n log n</em>. This means that the benefit is
bigger for large dataset.</p>
</div>
<div class="section" id="scikit-learn-0-14-s-random-forests-are-fast">
<h3>Scikit-learn 0.14’s random forests are fast</h3>
<p><a class="reference external" href="http://www.montefiore.ulg.ac.be/~glouppe/">Gilles Louppe</a> has made
the random forests significantly faster in the 0.14 release. Let us
bench them in comparison with WiseIO’s
<a class="reference external" href="http://docs.wise.io/">WiseRf</a>, a proprietary package that only does
random forest and for which the main selling point is that it is
significantly than scikit-learn. However, let us also bench
<a class="reference external" href="http://scikit-learn.org/stable/modules/ensemble.html#extremely-randomized-trees">ExtraTrees</a>,
a tree-based model that is very similar to random forests, but that in
our experience can be implemented a bit faster, and tends to work
better.</p>
<p><strong>On the digits dataset (1797 samples, 641 features):</strong></p>
<table border="1" class="docutils">
<colgroup>
<col width="33%" />
<col width="19%" />
<col width="17%" />
<col width="31%" />
</colgroup>
<tbody valign="top">
<tr><td>Forest implementation</td>
<td>train time</td>
<td>test time</td>
<td>prediction accuracy</td>
</tr>
<tr><td>Sklearn ExtraTrees</td>
<td>2.641s</td>
<td>0.082s</td>
<td>0.986</td>
</tr>
<tr><td>Sklearn RandomForest</td>
<td>5.074s</td>
<td>0.088s</td>
<td>0.981</td>
</tr>
<tr><td>WiseRF</td>
<td>5.665s</td>
<td>0.108s</td>
<td>0.979</td>
</tr>
</tbody>
</table>
<p>So we see that on a mid-sized dataset, scikit-learn is faster than
WiseRF, and ExtraTrees is twice as fast as RandomForest, for better
results.</p>
<p><strong>On the MNIST dataset (70000 samples, 784 features):</strong></p>
<table border="1" class="docutils">
<colgroup>
<col width="33%" />
<col width="19%" />
<col width="17%" />
<col width="31%" />
</colgroup>
<tbody valign="top">
<tr><td>Forest implementation</td>
<td>train time</td>
<td>test time</td>
<td>prediction accuracy</td>
</tr>
<tr><td>Sklearn ExtraTrees</td>
<td>1378.141s</td>
<td>4.768s</td>
<td>0.976</td>
</tr>
<tr><td>Sklearn RandomForest</td>
<td>1639.866s</td>
<td>4.132s</td>
<td>0.972</td>
</tr>
<tr><td>WiseRF</td>
<td>1102.465s</td>
<td>14.542s</td>
<td>0.972</td>
</tr>
</tbody>
</table>
<p>On a big dataset, WiseRF takes the lead, but not by a large factor.</p>
<p><strong>Using 2 CPUs (n_jobs=2) on the digits dataset:</strong></p>
<table border="1" class="docutils">
<colgroup>
<col width="33%" />
<col width="19%" />
<col width="17%" />
<col width="31%" />
</colgroup>
<tbody valign="top">
<tr><td>Forest implementation</td>
<td>train time</td>
<td>test time</td>
<td>prediction accuracy</td>
</tr>
<tr><td>Sklearn ExtraTrees</td>
<td>4.874s</td>
<td>1.478s</td>
<td>0.986</td>
</tr>
<tr><td>Sklearn RandomForest</td>
<td>5.716s</td>
<td>1.349s</td>
<td>0.978</td>
</tr>
<tr><td>WiseRF</td>
<td>3.264s</td>
<td>0.104s</td>
<td>0.979</td>
</tr>
</tbody>
</table>
<p>Both scikit-learn and WiseRF can use several CPUs. However, the Python
parallel execution model via multiple processes has an overhead in term
of computing time and of memory usage. The internals of WiseRF are coded
in C++, and thus it is not limited by this overhead. Also, because of
the memory duplication with multiples processes in scikit-learn, I could
not run it on MNIST with 2 jobs. Next release will address these issues,
partly by using memmapped arrays to share memory between processes.</p>
</div>
</div>
<div class="section" id="we-make-good-use-of-funding-the-paris-sprint">
<h2>We make good use of funding: the Paris sprint</h2>
<p>A couple of weeks ago, we had a coding sprint in Paris. We were able to
bring in a lot of core developers from all over Europe thanks to our
sponsors: <a class="reference external" href="http://www.frs-fnrs.be/%20">FNRS</a>,
<a class="reference external" href="http://www.afpy.org">AFPy</a>, <a class="reference external" href="http://www.telecom-paristech.fr/">Telecom
Paristech</a>, and <a class="reference external" href="http://www.svi.cnrs-bellevue.fr">Saint-Gobain
Recherche</a>. The total budget,
including accommodation and travel, was a couple thousand euros, thanks
to <a class="reference external" href="http://www.telecom-paristech.fr/">Telecom Paristech</a> and
<a class="reference external" href="http://www.tinyclues.com">tinyclues</a> helping us with accommodation
and hosting the sprint.</p>
<p>The productivity of such a sprint is huge, both because we get together
and work efficiently, but also because we get back home and keep working
(I have been sleep deprived because of late-night hacking ever since the
sprint). As an illustration, here is the diagram of commits as can be
seen on Github. The huge spike correspond to the second international
sprint: Paris 2013.</p>
<div class="figure align-center">
<img alt="" src="https://gael-varoquaux.info/programming/attachments/sklearn_0.14.X_speed/commit_graph.png" style="width: 100%;" />
</div>
<p><strong>We now have a “donate” button</strong> on the
<a class="reference external" href="http://scikit-learn.org/stable">website</a>. I can assure you that
your donations are well spent and turned into code.</p>
</div>
RIP John Hunter: the loss of a great man2012-08-30T10:21:00+02:002012-08-30T10:21:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2012-08-30:/programming/rip-john-hunter-the-loss-of-a-great-man.html<p>John Hunter, the author of <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> passed away yesterday after a
short battle against cancer. John gave the keynote at the scipy 2012
conference a few weeks ago, and was diagnosed with cancer just on his
return from the conference. It is a shock to me that that a friend …</p><p>John Hunter, the author of <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> passed away yesterday after a
short battle against cancer. John gave the keynote at the scipy 2012
conference a few weeks ago, and was diagnosed with cancer just on his
return from the conference. It is a shock to me that that a friend can
disappear so quickly. Please read the <a class="reference external" href="https://groups.google.com/forum/#!msg/pydata/FpwXp3sX6N8/mxopkZ1PkBQJ">announcement</a> of <a class="reference external" href="http://fperez.org/">Fernando
Perez</a>, who supported John in the last weeks to learn more about John.</p>
<div class="section" id="a-man-who-gave-a-lot-not-asking-for-anything-in-return">
<h2>A man who gave a lot, not asking for anything in return</h2>
<p>Many have benefited from the silent efforts of John, and are not fully
aware of how he generously invested his time and talent for the benefit
of others. Matplotlib, the Python plotting library that he created in
2002, has propelled Python as a major tool for scientific research and
engineering. The impact of John’s efforts go well beyond Matplotlib.
Early on, John had the vision of Python as a interactive scientific
environment. He promoted this vision pairing with Fernando Perez to
develop the fantastic <a class="reference external" href="http://ipython.org/">ipython</a>/<a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> tandem, solving many
technical challenges. But he also invested a lot of energy in teaching
workshops that helped change the way people compute, as well as writing
didactic documentation and articles. He was a friendly, active, leader
of an online community, open and helpful to newcomers.</p>
<p>As Travis Oliphant said on John’s numfocus <a class="reference external" href="http://numfocus.org/johnhunter/">memorial webpage</a>:</p>
<blockquote class="epigraph">
Those who contribute much to open source, as John did, do so at the
expense of something - often it is time with family.</blockquote>
<p>I cannot stress how true this is. The entire open source software, that
nowadays supports our economy, our education, and our research, is built
on the shoulders of a fairly small number of generous people that spend
their energy in making better software, rather than personal wealth.</p>
<p>John was a humble man. He did not have a blog, or a twitter account, did
not seek fame or money. For this reason I feel that his contributions
are unknown and undervalued by many. In my eyes, he is an unknown
soldier of our modern times. I hope that I am not being too emphatic,
but this is how I feel.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p><strong>John passed away at 44, leaving behind a wife and 3 daughters. Please
do consider supporting them:</strong></p>
<blockquote class="last">
<a class="reference external" href="http://numfocus.org/johnhunter">http://numfocus.org/johnhunter</a></blockquote>
</div>
</div>
A journal promoting high-quality research code: dream and reality2012-06-04T21:39:00+02:002012-06-04T21:39:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2012-06-04:/programming/a-journal-promoting-high-quality-research-code-dream-and-reality.html<p><a class="reference external" href="http://www.openresearchcomputation.com/">Open research computation (ORC)</a> was an attempt to create a scientific
publication promoting <strong>high-quality and open source scientific code</strong>.
The project went public in falls 2010, but last month, facing the low
volume of submission, the editorial board <a class="reference external" href="http://blogs.openaccesscentral.com/blogs/bmcblog/entry/open_research_computation_thematic_series">chose to reorient it</a> as a
special track of an existing journal …</p><p><a class="reference external" href="http://www.openresearchcomputation.com/">Open research computation (ORC)</a> was an attempt to create a scientific
publication promoting <strong>high-quality and open source scientific code</strong>.
The project went public in falls 2010, but last month, facing the low
volume of submission, the editorial board <a class="reference external" href="http://blogs.openaccesscentral.com/blogs/bmcblog/entry/open_research_computation_thematic_series">chose to reorient it</a> as a
special track of an existing journal.</p>
<p>The challenges that we face are discussed in our editorial:</p>
<blockquote>
<a class="reference external" href="http://www.scfbm.org/content/7/1/2/abstract">Changing computational research. The challenges ahead.</a> C Neylon,
J Aerts, CT Brown, D Lemire, J Millman, P Murray-Rust, F Perez, N
Saunders, A Smith, G Varoquaux and E Willighagen, <em>Source Code for
Biology and Medicine</em> 2012, 7:20</blockquote>
<p>Here is my own personal take on the rise and fall of this ideal.</p>
<div class="section" id="my-story-with-orc">
<h2>My story with ORC</h2>
<img alt="" class="align-right" src="http://www.rcac.net.au/images/Publications1.jpg" style="width: 40%;" />
<p><strong>From pipe dream to journal -</strong> My involvement with ORC started long
before there was such a thing as ORC. In falls 2008, I had a discussion
with a friend working in the publication industry, telling her how I
believed that the publication system is broken, because it promotes new
results without any interest on whether these can be exported outside
the lab that produced them: <strong>it is currently easier to publish a minor
but novel result than a tool enabling the routine reproduction of
previous results</strong>. This seemed particularly marked in the scientific
software world, as software tools are becoming central to the scientific
workflow, and cost nothing to duplicate when produced under open-source
license. To my surprise, she took me seriously, and asked me to write my
ideas down in an email that she would forward to her colleagues in the
publication industry.</p>
<p>Looking back at the email that I send, my concerns were, back then, to
promote:</p>
<ul class="simple">
<li>quality and openness of scientific software</li>
<li>basic tools shared across communities</li>
<li>recognition of software development as a challenging and worthwhile
task in academic research</li>
</ul>
<p><strong>Shaping the idea -</strong>In the year that followed, I had a few
discussions with staff from <a class="reference external" href="http://www.biomedcentral.com">BioMedCentral</a>, an open-access publisher
in biology and medicine that was looking into expending in the physics
and math related fields. Eventually, my contact there told me that they
had other similar requests and were launching a journal that would be
lead by Cameron Neylon, a British biophysicist and strong advocate of
openness and reproducibility in science. This was the start of ORC, and
for me the chance to meet other people sharing my concerns, some new and
some <a class="reference external" href="http://fperez.org/">already</a> <a class="reference external" href="http://jarrodmillman.com/">old</a> <a class="reference external" href="http://ivory.idyll.org">friends</a>.</p>
<div class="figure align-right">
<img alt="" src="http://www.salinafbc.com/Websites/fbcsalina/images/nerd_computer.gif" style="width: 230px;" />
<p class="caption">ORC editor</p>
</div>
<div class="figure align-left">
<img alt="" src="http://researchsupportgroup.files.wordpress.com/2011/11/kayla1.jpg" style="width: 150px;" />
<p class="caption">Conventional editor</p>
</div>
<hr class="docutils" />
<p><strong>Setting up the journal -</strong>BioMedCentral was instrumental in setting
up the journal project. I quickly learned that, no surprises, a journal
is a product, like anything else, and it must find customers. Here, as
we were launching an open access journal, the customers were authors.
This is where a journal faces a chicken and egg problem: to be
recognised it needs high-visibility publications, but authors will
submit only to journals that they know. The main tool to overcome this
challenge are communication and advocacy. I then realized that these
really weren’t my strong points. Cameron Neylon absolutely shined on
this side, with very enthusiastic <a class="reference external" href="http://cameronneylon.net/blog/open-research-computation-an-ordinary-journal-with-extraordinary-aims/">communications</a> and an incredibly
active <a class="reference external" href="https://twitter.com/#!/CameronNeylon">twitter account</a>. On my side, I am a slow writer, and I tend to
speak Python code better than English language, which is not a strong
asset to be a journal editor.</p>
<p><strong>Wild editorial discussions -</strong> The discussions in the editorial board
really thrilled me because they were centered on how to set standards to
improve the quality of code published. Looking in my mailbox, I see
discussions about code repositories, software testing, documentation or
licensing issues. This is not that surprising, given that a lot of the
editors where actually contributors to major software projects. It made
me very happy, as I have the feeling that, so far, most committees or
decision makers are clueless about software.</p>
</div>
<div class="section" id="sand-in-the-gears-the-lack-of-uptake">
<h2>Sand in the gears: the lack of uptake</h2>
<p><strong>A false start -</strong>So ORC was launched late 2010 and we had fantastic
feedback. I had the feeling that people were <a class="reference external" href="http://neuralensemble.blogspot.fr/2010/12/open-research-computation-new-journal.html">genuinely</a> <a class="reference external" href="https://twitter.com/vaguery/status/15402390589018112">excited</a>
about our program: changing the way computational science worked from
the inside, through the review process. The idea was that we had opened
a pre-submission call, and were waiting for a few good papers to be
submitted to launch the journal. However, it turned out that the papers
were slow to come. It took me a while to realize that there was
something wrong. But slowly we had to face the truth: many people were
excited about the journal, but most were sending their papers elsewhere.</p>
<p><strong>What went wrong? -</strong>If I really knew what went wrong, I would
probably not be discussing it here, but rather changing the world.
However, I can come up with a few guesses:</p>
<ul class="simple">
<li><strong>Working across communities is harder.</strong> From the beginning we had
wanted to position the journal across communities, in order to foster
the sharing of tools for a greater good. The challenge is that a
central role of publication is nowadays to provide recognition. It is
much easier to achieve recognition in a given community than across
communities, and authors always preferred submitting their work to a
non-software oriented journal in their field. We couldn’t fight
together the battle for software quality and the battle for
inter-community work.</li>
<li><strong>Setting the bar too high.</strong> Many felt that the submission
requirements that where too demanding, as expressed on a NeuroImaging
forumn to quote a researcher: <a class="reference external" href="http://www.nitrc.org/forum/message.php?msg_id=3674">“I think it’s setting the bar
unrealistically high for most neuroimaging software”</a>. While we had
originally shot for a very high test coverage (probably too high), we
had scaled it back quickly, simply stressing that editors and
reviewers would be looking closely at test coverage, documentation
and ease of installation. That said, the average researcher did not
share our ideals of raising the quality of scientific software.
Trying to ask only for excellent publications in a new and unproven
journal was probably unrealistic.</li>
<li><strong>Editors not willing to game the system.</strong> I have watched a few
journal launches, and it seems to me that a common trick is to line
up articles that are created by the editors and their friends
specifically for the new journal. People come up with <em>opinion
papers</em>, <em>reviews</em>, <em>commentaries</em> that only serve to generate an
identity to the journal. This did not happen for ORC, and I believe
that it is because <a class="reference external" href="http://cameronneylon.net/blog/open-research-computation-an-ordinary-journal-with-extraordinary-aims">the editors themselves</a> were not huge fans of
the low signal-to-noise ratio in modern scientific publishing
practice.</li>
</ul>
</div>
<div class="section" id="the-times-they-are-a-changing">
<h2>The times they are a changing</h2>
<img alt="" class="align-right" src="http://www.pictures88.com/p/success/success_005.jpg" style="width: 35%;" />
<p><strong>ORC is dead, long live ORC -</strong> We did get a few submissions. ORC is
not coming to an end, it is morphing into a special thematic series in
<a class="reference external" href="http://www.scfbm.org/">source code for biology and medicine</a>. This solution is not completely
satisfactory, as it pushes what should have been a forum for exposing
good practices and good software into a smaller community. But at least
there is now a venue in which people can publish a paper about software
that they have been improving and maintaining, and not only about a new
algorithm.</p>
<p><strong>Changing practices across the board -</strong> Among the reasons for which we
had a hard time making a breakthrough, is that authors where sending
their software papers to other journals, in particular journals not
specialized on software. While these papers are not getting the
attention of a review and editorial team expert on software development,
as we are setting up with ORC, this is still a good thing. Indeed it
shows that the times are changing and that recognition of software as a
scientific work is improving. I have been impressed to see that many
high profile journals have changed their editorial policies to
specifically accept software papers, or have create tracks dedicated to
software.</p>
<p>Software is being slowly recognized as a pillar of modern scientific
research. We need to keep pushing to make sure that quality standards
are set and that the open-source scientific software grows into a mature
ecosystem focused on problem solving.</p>
</div>
Update on scikit-learn: recent developments for machine learning in Python2012-05-09T00:12:00+02:002012-05-09T00:12:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2012-05-09:/programming/update-on-scikit-learn-recent-developments-for-machine-learning-in-python.html<p>Yesterday, we released version 0.11 of the <a class="reference external" href="http://scikit-learn">scikit-learn</a> toolkit for
machine learning in Python, and there was much rejoincing.</p>
<div class="section" id="major-features-gained-in-the-last-releases">
<h2>Major features gained in the last releases</h2>
<p>In the last 6 months, there have been many things happening with the
scikit-learn. While I do not whish to give an exhaustive …</p></div><p>Yesterday, we released version 0.11 of the <a class="reference external" href="http://scikit-learn">scikit-learn</a> toolkit for
machine learning in Python, and there was much rejoincing.</p>
<div class="section" id="major-features-gained-in-the-last-releases">
<h2>Major features gained in the last releases</h2>
<p>In the last 6 months, there have been many things happening with the
scikit-learn. While I do not whish to give an exhaustive summary of
features added (it can be found <a class="reference external" href="http://scikit-learn.org/stable/whats_new.html">here</a>), let me list a few of the
additions that I personnally find exciting.</p>
<div class="section" id="non-linear-prediction-models">
<h3>Non-linear prediction models</h3>
<p>For complex prediction problems where there is no simple model
available, as in computer vision, non-linear models are handy. A good
example of such models are those based on decisions trees and model
averaging. For instance random forests are used in the Kinect to locate
body parts. As they are intrinsically complex, they may need a large
amount of training data. For this reason, they have been implemented in
the scikit-learn with special attention to computational efficiency.</p>
<ul class="simple">
<li><a class="reference external" href="http://scikit-learn.org/stable/modules/ensemble.html#forests-of-randomized-trees">Randomized Forests and extra-trees</a></li>
<li><a class="reference external" href="http://scikit-learn.org/stable/modules/ensemble.html#gradient-tree-boosting">Gradient boosted regression trees</a></li>
</ul>
</div>
<div class="section" id="dealing-with-unlabeled-instances">
<h3>Dealing with unlabeled instances</h3>
<p>It is often easy to gather unlabeled observations than labeled
observation. While prediction of a quantity of interest is then harder
or simply impossible, mining this data can be useful.</p>
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/label_propagation.html">Semi-supervised learning</a>: using unlabeled observations together with
labeled ones for better prediction.</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/outlier_detection.html">Outlier/novelty detection</a>: detect deviant observations.</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/manifold.html">Manifold learning</a>: discover a non-linear low-dimensional structure in
the data.</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/clustering.html">Clustering</a> with <a class="reference external" href="http://scikit-learn.org/stable/modules/clustering.html#mini-batch-k-means">an algorithm</a> that can scale to really large
datasets using an online approach: fitting small portions of the data on
after the other (Mini-batch k-means).</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/decomposition.html#dictionarylearning">Dictionary learning</a>: learning patterns in the data that represent it
sparsely: each observation is a combination of a small number patterns.</p>
</div>
<div class="section" id="sparse-models-when-very-few-descriptors-are-relevant">
<h3>Sparse models: when very few descriptors are relevant</h3>
<p>In general, finding which descriptors are useful when there are many of
them is like find a needle in a haystack: it is a very hard problem.
However, you know that only a few of these descriptors actually carry
information, you are in a so-called <em>sparse</em> problem, for specific
approaches can work well.</p>
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/linear_model.html#orthogonal-matching-pursuit-omp">Orthogonal matching pursuit</a>: a greedy and fast algorithm for very
sparse linear models</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/feature_selection.html#randomized-sparse-models">Randomized sparsity (randomized Lasso)</a>: selecting the relevant
descriptors in noisy high-dimensional observations</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.org/stable/modules/generated/sklearn.covariance.GraphLasso.html#sklearn.covariance.GraphLasso">Sparse inverse covariance</a>: learning graphs of connectivity from
correlations in the data</p>
<div class="section" id="getting-developpers-together-the-granada-sprint">
<h4>Getting developpers together: the Granada sprint</h4>
<p>
<object width="400" height="300" align="right">
<embed type="application/x-shockwave-flash" src="http://www.flickr.com/apps/slideshow/show.swf?v=109615" allowfullscreen="true" flashvars="offsite=true⟨=en-us&page_show_url=%2Fsearch%2Fshow%2F%3Fq%3Dscikit-learn%26m%3Dtags%26w%3D66885349%2540N03&page_show_back_url=%2Fsearch%2F%3Fq%3Dscikit-learn%26m%3Dtags%26w%3D66885349%2540N03&method=flickr.photos.search&api_params_str=&api_tags=scikit-learn&api_tag_mode=bool&api_user_id=66885349%40N03&api_safe_search=3&api_content_type=7&api_media=all&api_sort=date-posted-desc&jump_to=&start_index=0" width="400" height="300">
</embed>
</object>
</p><p>Of course, such developments happen only because we have a great team of
<a class="reference external" href="https://github.com/scikit-learn/scikit-learn/graphs/contributors">dedicated coders</a>.</p>
<p>Getting along and working together is a critical part of the project. In
December 2011, we held the first international <a class="reference external" href="http://scikit-learn">scikit-learn</a> sprint in
Granada, on the side of the <a class="reference external" href="http://nips.cc">NIPS conference</a>. That was a while ago,
and I haven’t found time to blog about it, maybe because I was too busy
merging in the code produced :). Here is a small report from my point of
view. Better late than never.</p>
</div>
</div>
</div>
<div class="section" id="participants-from-all-over-the-globe">
<h2>Participants from all over the globe</h2>
<p>This sprint was a big deal for us, because for the first time, thanks to
sponsor money, we were able to fly contributors from overseas and meet
the team in person. For the first time I was able to see the faces
behind many of the fantastic people that I knew only from the mailing
list.</p>
<p>I really think that we must thank our sponsors, <strong>Google</strong> and
<strong>tinyclues</strong>, but also The PSF, that is in particular Jesse Noller but
especially <strong>Steve Holden</strong>, whose help was absolutely instrumental in
getting sponsor money. This money is what made it possible to unite a
good fraction of the team, and it opened the door to great moments of
coding, and more.</p>
</div>
<div class="section" id="producing-code-lines-and-friendship">
<h2>Producing code lines and friendship</h2>
<p>An important aspect of the sprint for me was that I really felt the team
being united. Granada is a great city and we spent fantastic moments
together. Now when I review code, I can often put a face on the author
of that code and remember a walk below the Alhambra or an evening in a
bar. I am sure it helps reviewing code!</p>
</div>
<div class="section" id="was-it-worth-the-money">
<h2>Was it worth the money?</h2>
<img alt="" src="attachments/skl_activity.png" style="width: 90%;" />
<p>I really appreciate that the sponsors did not ask for specific returns on
investment beyond acknowledgments, but I think that it is useful for us
to ask the question: was it worth the money? After all, we got around
$5000, and that’s a lot of money. First of all, as a side effect of the
sprint, people who had invested a huge amount of time in a machine
learning toolkit without asking anything in return got help to go to a
major machine learning conference.</p>
<p>But was there a return over investment in terms of code? If you look at
the number of lines of code modified weekly (figure on the right), there
is a big spike in December 2011. That’s our sprint! Importantly, if you
look at the months following the sprint, there still is a lot of activity
in the months following the sprint. This is actually unusual, as the
active developments happen more in the summer break than during the
winter, as our developpers are busy working on papers or teaching.</p>
<p>The explaination is simple: we where thrilled by the sprint. Overall, it
was incredibly beneficial to the project. I am looking forward to the
next ones.</p>
</div>
3 Google summer of code for scikit-learn and more…2012-04-23T22:25:00+02:002012-04-23T22:25:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2012-04-23:/programming/3-google-summer-of-code-for-scikit-learn-and-more.html<p>The <a class="reference external" href="http://scikit-learn.org">scikit-learn</a> got 3 students accepted for the Google summer of
code.</p>
<ul class="simple">
<li><a class="reference external" href="http://ibayer.blogspot.fr/">Imanuel Bayer</a> will work on making our sparse linear models, for
regression and classification, faster. His proposal <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/ibayer/11001">Optimizing
sparse linear models using coordinate descent and strong rules</a>.</li>
<li><a class="reference external" href="http://www.davidmarek.cz/">David Marek</a> will implement multi-layer perceptrons for the scikit.
His proposal …</li></ul><p>The <a class="reference external" href="http://scikit-learn.org">scikit-learn</a> got 3 students accepted for the Google summer of
code.</p>
<ul class="simple">
<li><a class="reference external" href="http://ibayer.blogspot.fr/">Imanuel Bayer</a> will work on making our sparse linear models, for
regression and classification, faster. His proposal <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/ibayer/11001">Optimizing
sparse linear models using coordinate descent and strong rules</a>.</li>
<li><a class="reference external" href="http://www.davidmarek.cz/">David Marek</a> will implement multi-layer perceptrons for the scikit.
His proposal: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/h4wk_cz/24001">Multilayer Perceptron</a></li>
<li><a class="reference external" href="http://blog.vene.ro/">Vlad Niculae</a> will work on speeding up the library in general,
catching all the low hanging fruits, and the ones a bit higher. His
proposal: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/vladn/26002">Need for scikit-learn speed</a></li>
</ul>
<p>In addition, other related projects have exciting projects, for instance
<a class="reference external" href="http://statsmodels.sourceforge.net/">**statsmodels**</a>:</p>
<ul class="simple">
<li>Divyanshu Bandil: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/divyanshu/34002">Extension of Linear to Non Linear Models in
Statsmodels Python module</a></li>
<li>Alexandre Crayssac: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/alexandreyc/8001">estimating system of equations</a></li>
<li>Justin Grana: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/j_grana/8001">empirical Likelihood in Statsmodels</a></li>
<li>Georgi Panterov: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/gpanterov/7001">nonparametric estimation</a></li>
</ul>
<p>and <a class="reference external" href="http://www.cython.org">Cython</a>:</p>
<ul class="simple">
<li>Philip Herron: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/redbrain1123/28002">pxd generation using gcc-python-plugin</a></li>
<li>Mark Florisson: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/markflorisson88/30002">Fast Numerical Computing with Cython</a></li>
</ul>
<p>finally, in <a class="reference external" href="http://pandas.pydata.org/">Pandas</a>:</p>
<ul class="simple">
<li>Vytautas Jancauskas: <a class="reference external" href="http://www.google-melange.com/gsoc/project/google/gsoc2012/bucket_brigade/42002">Plots in pandas</a></li>
</ul>
<p>Congratulations to all of the students. This is going to be an exciting
summer.</p>
The problems of low statistical power and publication bias2012-04-14T16:16:00+02:002012-04-14T16:16:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2012-04-14:/science/the-problems-of-low-statistical-power-and-publication-bias.html<img alt="" class="align-right" src="http://idoubtit.files.wordpress.com/2010/12/coldfusion.jpg" style="width: 30%;" />
<p>Lately, I have been a mood of scientific scepticism: I have the feeling
that the worldwide academic system is more and more failing to produce
useful research. Christophe Lalanne’s <a class="reference external" href="https://twitter.com/#!/chlalanne">twitter feed</a> lead me to an
interesting article in a non-mainstream journal: <a class="reference external" href="http://beheco.oxfordjournals.org/content/15/6/1044.short">A farewell to
Bonferroni: the problems of low …</a></p><img alt="" class="align-right" src="http://idoubtit.files.wordpress.com/2010/12/coldfusion.jpg" style="width: 30%;" />
<p>Lately, I have been a mood of scientific scepticism: I have the feeling
that the worldwide academic system is more and more failing to produce
useful research. Christophe Lalanne’s <a class="reference external" href="https://twitter.com/#!/chlalanne">twitter feed</a> lead me to an
interesting article in a non-mainstream journal: <a class="reference external" href="http://beheco.oxfordjournals.org/content/15/6/1044.short">A farewell to
Bonferroni: the problems of low statistical power and publication
bias</a>, by Shinichi Nakagawa.</p>
<p>Each study performed has a probability of being wrong. Thus performing
many studies will lead to some wrong conclusions by chance. This is
known in statistics as the <a class="reference external" href="http://en.wikipedia.org/wiki/Multiple_comparisons">multiple comparisons</a> problem. When a
working hypothesis is not verified empirically in a study, this null
finding is seldom reported, leading to what is called <em>publication
bias</em>: <strong>discoveries are further studied; negative results are usually
ignored</strong> (Y. Benjamini). Because only <em>discoveries</em>, called
<em>detections</em> in statistical terms, are reported, <strong>published results
contain more false detections than the individual experiments and very
little false negatives</strong>. Arguably, the original investigators have
corrected using the understanding that they gained the experiments
performed and account in a <em>post-hoc analysis</em> for the fact that some of
their working hypothesis could not have been correct. Such a correction
can work only in a field where there is a good mechanistic
understanding, or models, such as physics, but in my opinion not in life
and social sciences.</p>
<p>Let me quote some relevant extracts of <a class="reference external" href="http://beheco.oxfordjournals.org/content/15/6/1044.short">the article</a>, as you may never
have access to it thanks to the way scientific publishing works:</p>
<blockquote class="epigraph">
<p>Recently, Jennions and Moller (2003) carried out a meta-analysis
on statistical power in the field of behavioral ecology and animal
behavior, reviewing 10 leading journals including Behavioral
Ecology. Their results showed dismayingly low average statistical
power (note that a meta-analytic review of statistical power is
different from post hoc power analysis as criticized in Hoenig and
Heisey, 2001). The statistical power of a null hypothesis (Ho)
significance test is the probability that the test will reject Ho
when a research hypothesis (Ha) is true.</p>
<p>…</p>
<p>The meta-analysis on statistical power by Jennions and Moller
(2003) revealed that, in the field of behavioral ecology and animal
behavior, statistical power of less than 20% to detect a small
effect and power of less than 50% to detect a medium effect existed.
This means, for example, that the average behavioral scientist
performing a statistical test has a greater probability of making a
Type II error (or beta) (<em>i.e.</em>, not rejecting Ho when Ho is false;
note that statistical power is equals to 1 - beta) than if they had
flipped a coin, when an experiment effect is of medium size.</p>
<p>…</p>
<p>Imagine that we conduct a study where we measure as many relevant
variables as possible, 10 variables, for example. We find only two
variables statistically significant. Then, what should we do? We
could decide to write a paper highlighting these two variables (and
not reporting the other eight at all) as if we had hypotheses about
the two significant variables in the first place. Subsequently, our
paper would be published. Alternatively, we could write a paper
including all 10 variables. When the paper is reviewed, referees
might tell us that there were no significant results if we had
“appropriately” employed Bonferroni corrections, so that our study
would not be advisable for publication. However, the latter paper is
scientifically more important than the former paper. For example, if
one wants to conduct a meta-analysis to investigate an overall
effect in a specific area of study, the latter paper is five times
more informative than the former paper. In the long term,
statistical significance of particular tests may be of trivial
importance (if not always), although, in the short term, it makes
papers publishable. Bonferroni procedures may, in part, be
preventing the accumulation of knowledge in the field of behavioral
ecology and animal behavior, thus hindering the progress of the
field as science.</p>
</blockquote>
<img alt="" class="align-right" src="http://farm6.staticflickr.com/5206/5330056727_a98c97c3c5.jpg" style="width: 50%;" />
<p>Some of the concerns raised here are partly a criticism of Bonferoni
corrections, <em>i.e.</em> in technical terms correcting for <a class="reference external" href="http://en.wikipedia.org/wiki/Familywise_error_rate">family-wise error
rate (FWER)</a>. It is actually the message that the author wants to
convey in his paper. Proponents of controling for <a class="reference external" href="http://en.wikipedia.org/wiki/False_discovery_rate">false discovery rate
(FDR)</a> argue that an investigator shouldn’t be penalized for asking
more questions, and the fraction of errors in the answers should be
controlled, rather than the absolute value. That said, FDR, while
useful, does not answer the problems of publication bias.</p>
Want features? Just code2012-03-08T22:46:00+01:002012-03-08T22:46:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2012-03-08:/programming/want-features-just-code.html<p>Somebody just sent an email on a user’s mailing list for an open-source
scientific package entitled <strong>“Feature foo: why is package bar
not up to the task?”</strong>. To quote him:</p>
<blockquote class="epigraph">
Is there ANY plan for having such a module in <em>package bar</em>?? I
think (personally) that this is a …</blockquote><p>Somebody just sent an email on a user’s mailing list for an open-source
scientific package entitled <strong>“Feature foo: why is package bar
not up to the task?”</strong>. To quote him:</p>
<blockquote class="epigraph">
Is there ANY plan for having such a module in <em>package bar</em>?? I
think (personally) that this is a MUST DO. This is typically the
type of routines that I hear people use in e.g., idl etc. If this
could be an optimised, fast (and easy to use) routine, all the
better.</blockquote>
<p>As some one who spends a fair amount of time working on open
source software I hear such remarks quite often. I am finding it harder
and harder not to react negatively to these emails. Now I cannot
consider myself as a contributor to <em>package bar</em>, and thus I can claim
that I am not taking your comment personally.</p>
<p>Why aren’t package not up to the task? Will, the answer is quite
simple: because they are developed by volunteers that do it on their
spare time, late at night too often, or companies that put some of their
benefits in open source rather in locking down a market. 90% of the time
the reason the feature isn’t as good as you would want it is because of
lack of time.</p>
<p>I personally find that suggesting that somebody else should put more
of the time and money they are already giving away in improving a
feature that you need is almost insulting.</p>
<p>I am aware that people do not realize how small the group of people
that develop and maintain their toys is. Borrowing the figure below from
<a class="reference external" href="http://www.euroscipy.org/file/6459?vid=download">Fernando Perez’s talk at Euroscipy</a>, the number of people that do 90%
of the grunt work to get the core scientific Python ecosystem going is
around two handfuls:</p>
<img alt="" src="attachments/fperez_euroscipy_2011_contributors.jpg" style="width: 70%;" />
<p>I’d like to think that this recruitment problem is a lack of skill set:
users that have the ability to contribute are just too rare. This is not
entirely true, there are scores of skilled people on the mailing lists.
The poster himself mentioned his email that he was developing a package.
I personally started contribution not knowing anything about software
development. I struggled, I did the grunt work like maintaining wikis,
answer questions on mailing list, and writing documentation. These
easier tasks were useful to the community, I think, but must
importantly, they taught me a lot because I was investing energy in
them.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last"><strong>If people want things to improve, they will have more successes
sending in pull requests than messages on mailing list that sound
condescending to my ears.</strong></p>
</div>
<p>I hope that I haven’t overreacted too badly :), that email turned me on.
That said, I am not sure that people realize how much they owe to the
open source developers breaking their backs on the packages they use.</p>
<img alt="" src="attachments/fperez_euroscipy_2011_i_want_you.jpg" style="width: 50%;" />
<p>All credit for images goes to <a class="reference external" href="http://fperez.org/">Fernando Perez</a></p>
Book review: NumPy 1.5 Beginner’s guide2012-01-10T08:57:00+01:002012-01-10T08:57:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2012-01-10:/programming/book-review-numpy-15-beginners-guide.html<p>Packt publishing sent me a copy of <a class="reference external" href="http://www.packtpub.com/numpy-1-5-using-real-world-examples-beginners-guide/Book">NumPy 1.5 Beginner’s guide</a> by Ivan
Idris.</p>
<p>The book actually covers more than only <a class="reference external" href="http://numpy.scipy.org/">numpy</a>: it is a full
introduction to numerical computing with Python. The <a class="reference external" href="http://www.packtpub.com/toc/numpy-15-beginners-guide-table-contents">table of
contents</a> is the following:</p>
<ul class="simple">
<li>NumPy Quick Start</li>
<li>Beginning with NumPy Fundamentals</li>
<li>Get into …</li></ul><p>Packt publishing sent me a copy of <a class="reference external" href="http://www.packtpub.com/numpy-1-5-using-real-world-examples-beginners-guide/Book">NumPy 1.5 Beginner’s guide</a> by Ivan
Idris.</p>
<p>The book actually covers more than only <a class="reference external" href="http://numpy.scipy.org/">numpy</a>: it is a full
introduction to numerical computing with Python. The <a class="reference external" href="http://www.packtpub.com/toc/numpy-15-beginners-guide-table-contents">table of
contents</a> is the following:</p>
<ul class="simple">
<li>NumPy Quick Start</li>
<li>Beginning with NumPy Fundamentals</li>
<li>Get into Terms with Commonly Used Functions</li>
<li>Convenience Functions for Your Convenience</li>
<li>Working with Matrices and ufuncs</li>
<li>Move Further with NumPy Modules</li>
<li>Peeking Into Special Routines</li>
<li>Assure Quality with Testing</li>
<li>Plotting with Matplotlib</li>
<li>When NumPy is Not Enough: SciPy and Beyond</li>
</ul>
<p>The book is easy to read, as it requires no specific expertise other
than knowing basic Python programming. It is full of examples and
exercises, which is really great for learning. I find the style of the
author, Ivan Idris, particularly amusing and relaxing, engaging the
reader with questions, challenges, or even jokes (<em>“Have a go hero”</em>).</p>
<p>With regards to the formatting and the print, the book is written in
large fonts, with sectioning information, tips and exercises clearly
standing out.</p>
<p>It is full of practical information, such as how to install the
software, or where to get help. Finally, One thing that I appreciated,
is that the examples are typed in <a class="reference external" href="http://ipython.org/">IPython</a>. Each time I teach, I like
to use IPython, because it is full of features to help plotting,
debugging and profiling numerical code. The book even has a little
introduction to some useful IPython features.</p>
<p>After an introduction to the work flow, the book explores array
manipulation such as creation or reshaping, followed by some simple
numerics and the battery of array-based operations on functions and
polynomials. Then it presents linear algebra and signal processing
basics (FFT). It also covers the financial functions that are present in
numpy and mentions testing, which is very important to achieve quality
code. The book finishes with matplotlib and scipy, two modules that are
important to know to go further.</p>
<p>The examples are mostly drawn from statistics or financial applications,
such as computing running averages on stock quotes. Basic math
explanations, such as the definition of the Moore-Penrose
pseudo-inverse, are given when needed.</p>
<p>To conclude, I enjoyed this book and I think that it is a nice addition
to my library. It answers exactly it’s title: it is well-suited for
beginners wanting to learn numpy. On the other hand, I would not
recommend it as a reference material, or as a book to learn more general
scientific or numerical computing with Python.</p>
Joblib beta release: fast compressed persistence + Python 32012-01-07T19:27:00+01:002012-01-07T19:27:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2012-01-07:/programming/joblib-beta-release-fast-compressed-persistence-python-3.html<div class="section" id="joblib-0-6-better-i-o-and-python-3-support">
<h2>Joblib 0.6: better I/O and Python 3 support</h2>
<p>Happy new year, every one. I have just released <a class="reference external" href="http://en.wikipedia.org/wiki/Out-of-core_algorithm">Joblib</a> 0.6.0 beta.
The highlights of the 0.6 release are a reworked enhanced pickler, and
Python 3 support.</p>
<p>Many thanks go to the contributors to the 0.5 …</p></div><div class="section" id="joblib-0-6-better-i-o-and-python-3-support">
<h2>Joblib 0.6: better I/O and Python 3 support</h2>
<p>Happy new year, every one. I have just released <a class="reference external" href="http://en.wikipedia.org/wiki/Out-of-core_algorithm">Joblib</a> 0.6.0 beta.
The highlights of the 0.6 release are a reworked enhanced pickler, and
Python 3 support.</p>
<p>Many thanks go to the contributors to the 0.5.X series (Fabian
Pedregosa, Yaroslav Halchenko, Kenneth C. Arnold, Alexandre Gramfort,
Lars Buitinck, Bala Subrahmanyam Varanasi, Olivier Grisel, Ralf Gommers,
Juan Manuel Caicedo Carvajal, and myself). In particular Fabian made
sure that Joblib worked under Python 3.</p>
<p>In this blog post, I’d like to discuss a bit more the compressed
persistence engine, as it illustrates well key factors in implementing
and using compressed serialization.</p>
</div>
<div class="section" id="fast-compressed-persistence">
<h2>Fast compressed persistence</h2>
<p>One of the key components of joblib is it’s ability to persist arbitrary
Python objects, and read them back very quickly. It is particularly
efficient for <strong>containers that do their heavy lifting with numpy
arrays</strong>. The trick to achieving great speed has been to save in
separate files the numpy arrays, and load them via <strong>memmapping</strong>.</p>
<p>However, one drawback of joblib, is that the caching mechanism may end
up using a lot of disk space. As a result, there is strong interest in
having <strong>compressed storage</strong>, provided it doesn’t slow down the library
too much. Another use case that I have in mind for fast compressed
persistence, is implementing <a class="reference external" href="http://en.wikipedia.org/wiki/Out-of-core_algorithm">out of core computation</a>.</p>
<p>There are some great compressed I/O libraries for Python, for instance
<a class="reference external" href="http://pytables.github.com/index.html">Pytables</a>. You may wonder why the need to code yet another one. The
answer is that joblib is <strong>pure Python, depending only on the standard
library</strong> (numpy is optional), but also that the goal here is
<strong>black-box persistence of arbitrary objects</strong>.</p>
<div class="section" id="comparing-i-o-speed-and-compression-to-other-libraries">
<h3>Comparing I/O speed and compression to other libraries</h3>
<p>Implementing efficient compressed storage was a bit of a struggle and I
learned a lot. Rather than going into the details straight away, let me
first discuss a few benchmarks of the resulting code. Benching such
feature is very hard, first because you are fighting with the disk
cache, second because they performances depends very much on the data at
hand (some data compress better than others), last because they are
three interesting metrics: disk space used, write speed, and read speed.</p>
<p><strong>Dataset used</strong> - I chose to compare the different strategies on some
datasets that I work with, namely the probabilistic brain atlases MNI
1mm (62Mb uncompressed) and Juelich 2mm (105Mb uncompressed). Whether
the data is represented as a Fortran-ordered array, or a C-ordered array
is important for the I/O performance. This data is normally stored to
disk compressed using the domain-specific Nifti format (<em>.nii</em> files),
accessed in Python with the <a class="reference external" href="http://nipy.sourceforge.net/nibabel/">Nibabel</a> library.</p>
<p><strong>Libraries used</strong> - I benched different compression strategies in
joblib against Nibabel’s Nifti I/O, compressed or not, and against using
Pytables to store the data buffer (without the meta-informations).
Pytables exposed a variety of compression strategies, with different
speed compromises. In addition, I benched numpy’s builtin
<em>save_compressed</em>.</p>
<p>I would like to stress that I am comparing a general purpose persistence
engine (joblib) to specific I/O libraries either optimized for the data
(Nifti), or requiring some massaging to enable persistence (pytables).</p>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/disk.png" style="width: 66%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/write.png" style="width: 66%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/read.png" style="width: 66%;" />
<p><em>Comparing to other libraries</em></p>
<p>Actual numbers can be found <a class="reference external" href="attachments/joblib_rel_0.6_speed/results_nii.csv">here</a>.</p>
<p><strong>Take home messages</strong> - The graphs are not crystal-clear, but a few
tendencies appear:</p>
<ul class="simple">
<li>Pytables with LZO or blosc compression is the king of the hill for
read and write speed.</li>
<li>I/O of compressed data is often faster than with uncompressed data
for a good compression algorithm.</li>
<li>Joblib with Zlib compression level 1 performs honorably in terms of
speed with only the Python standard library and no compiled code.</li>
<li>Read time of memmapping (with nibabel or joblib) is negligeable (it
is tiny on the graphs), however the loading time appears when you
start accessing the data.</li>
<li>Passing in arrays with a memory layout (Fortran versus C order) that
the I/O library doesn’t expect can really slow down writing.</li>
<li>Compressing with Zlib compression-level 1 gets you most of the disk
space gains for a reasonable cost in write/read speed.</li>
<li>Compressing with Zlib compression-level 9 (not shown on the figures)
doesn’t buy you much in disk space, but costs a lot in writing time.</li>
</ul>
</div>
<div class="section" id="benching-datasets-richer-than-pure-arrays">
<h3>Benching datasets richer than pure arrays</h3>
<p>The datasets used so far are pretty much composed of one big array, a 4D
smooth spatial map. I wanted to test on more datasets, to see how the
performances varied with data type and richness. For this, I used the
datasets of the <a class="reference external" href="http://scikit-learn.org">scikit-learn</a>, real life data of various nature,
described <a class="reference external" href="http://scikit-learn.org/stable/datasets/index.html">here</a>:</p>
<ul class="simple">
<li><strong>20 news</strong> - 20 usenet news group: this data mainly consists of
text, and not numpy arrays.</li>
<li><strong>LFW people</strong> - Labeled faces in the wild, many pictures of
different people’s face.</li>
<li><strong>LFW pairs</strong> - Labeled faces in the wild, pairs of pictures for each
individual. This is a high entropy dataset, it does not have much
redundant information.</li>
<li><strong>Olivetti</strong> - Olivetti dataset: centered pictures of faces.</li>
<li><strong>Juelich(F)</strong> - Our previous Juelich atlas</li>
<li><strong>Big people</strong> - The LFW people dataset, but repeated 4 times, to put
a strain on memory resources.</li>
<li><strong>MNI(F)</strong> - Our previous MNI atlas</li>
<li><strong>Species</strong> - Occurence of species measured in latin America, with a
lot of missing data.</li>
</ul>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/joblib_disk.png" style="width: 50%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/joblib_write.png" style="width: 50%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<img alt="" class="align-center" src="attachments/joblib_rel_0.6_speed/joblib_read.png" style="width: 50%;" />
<p>Actual numbers can be found
<a class="reference external" href="attachments/joblib_rel_0.6_speed/joblib_results.csv">here</a>.</p>
<p><strong>What this tells us</strong> - The main message from these benchmarks is that
datasets with redundant information, i.e. that compress well, give fast
I/O. This is not surprising. In particular, good compression can give
good I/O on text (20 news). Another result, more of a sanity check, is
that compressed I/O on big data (Big people, ) works as well as on
smaller data. Earlier code would start to swap. Finally, I conclude from
these graphs, that compression levels from 1 to 3 buy you most of the
gains for reasonable costs, and that going up to 9 is not recommended,
unless you know that your data can be compressed a lot (species).</p>
</div>
<div class="section" id="lessons-learned">
<h3>Lessons learned</h3>
<p>I’ll keep this paragraph short, because the information is really in
<a class="reference external" href="https://github.com/joblib/joblib/blob/0.5.X/joblib/numpy_pickle.py">joblib’s code and comments</a>. Don’t hesitate to have a look, it’s
BSD-licenced, so you are free to borrow what you please.</p>
<ol class="arabic simple">
<li>Memory copies, of arrays, but also of strings and byte streams can
really slow you down with big data.</li>
<li>To avoid copies with numpy arrays, fully embrace numpy’s strided
memory model. For instance, you do not need to save arrays in C
order, if they are given to you in a different order. Accessing the
memory in the wrong striding direction explains the poor write
performance of pytables on Fortran-ordered Juelich.</li>
<li>When dealing with the file system, the OS makes so much magic (e.g.
prefetching) that clever hacks tend not to work: always benchmark.</li>
<li>Depending on the size of the data, it may be more efficient to store
subsets in different files: it introduces ‘chunk’ that avoid filling
in the memory too much (parameter <em>cache_size</em> in joblib’s code). In
addition, data of a same nature tends to compress better.</li>
<li>The I/O stream or file object interfaces are abstractions that can
hide the data movement and the creation of large temporaries. After
experiments with GZipFile and StringIO/BytesIO I found it more
efficient to fall back to passing around big buffer object, numpy
arrays, or strings.</li>
<li>For reasons 4 and 5, I ended up avoiding the gzip module: raw access
to the zlib with buffers gives more control. This explains a good
part of the differences in read speed for pure arrays with numpy’s
<em>save_compressed</em>.</li>
</ol>
<p>One of my conclusions for joblib, is that I’ll probably use Pytables as
an optional backend for persistence in a future release.</p>
</div>
<div class="section" id="details-on-the-benchmarks">
<h3>Details on the benchmarks</h3>
<p>These benchmarks where run on a Dell Lattitude D630 laptop. That’s a
dual-core Intel Core2 Duo box, with 2M of CPU cache.</p>
<p>The code for the benchmarks below can be found on <a class="reference external" href="https://gist.github.com/1551250">a gist</a>.</p>
</div>
<div class="section" id="thanks">
<h3>Thanks</h3>
<p>I’d like to that Francesc Alted for very useful feedback he gave on this
topics. In particular, the <a class="reference external" href="http://sourceforge.net/mailarchive/message.php?msg_id=28609087">following thread</a> on the pytables
mailing-list may be of interest to the reader.</p>
</div>
</div>
Scikit-learn NIPS 2011 sprint: international thanks to our sponsors2011-11-18T14:47:00+01:002011-11-18T14:47:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-11-18:/programming/scikit-learn-nips-2011-sprint-international-thanks-to-our-sponsors.html<p><strong>The NIPS conference: time for a sprint.</strong> The <a class="reference external" href="http://nips.cc/">NIPS conference</a>, one
of the major conferences in machine learning, is hosted in Granada this
year. I believe that it is the first time that it is hosted in Europe.
As many of the <a class="reference external" href="http://scikit-learn.org">scikit-learn</a> developers are part of the wider NIPS …</p><p><strong>The NIPS conference: time for a sprint.</strong> The <a class="reference external" href="http://nips.cc/">NIPS conference</a>, one
of the major conferences in machine learning, is hosted in Granada this
year. I believe that it is the first time that it is hosted in Europe.
As many of the <a class="reference external" href="http://scikit-learn.org">scikit-learn</a> developers are part of the wider NIPS
community, but also many live in Europe, we jumped on the occasion to
organize a truly international sprint: the <a class="reference external" href="http://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events">NIPS 2011 scikit-learn
sprint</a>.</p>
<p><strong>Finding money.</strong> As often with open source development, a lot of our
contributors are young people, investing their free time outside of any
request from their hierarchy. In such a situation, it can be hard to
find travel money. So we started looking for sponsors. We needed to find
a decent sum of money, as we were flying people in from places such as
the West coast of the US, or even Japan. The good news is that we found
money, and between supervisors pitching in, universities giving travel
grants, and our generous sponsors, there will be an impressive list of
contributors from all over the world at the sprint.</p>
<p><strong>Thanks to our sponsors.</strong> The first people that we need to thank are
Google, who gave us a sizable sponsorship, and the <a class="reference external" href="http://www.python.org/psf/">PSF</a>, who made
Google’s sponsorship possible through their accounting and sprints
programs. We also need to thanks our other sponsors, namely
<a class="reference external" href="http://www.tinyclues.com/">Tinyclues</a>. Thanks to these sponsors, and additional investment from
many universities and research group, we have been able to gather a
total of 12 contributors in Granada, a handful coming from overseas.
Also, we are indebted to the <a class="reference external" href="http://www.ugr.es/">University of Granada</a>, and the Gnu/Linux
Granada Group (GGG), who are providing hosting for the sprint, as well
as Régine Bricquet, from INRIA, who did a lot of the trip planing for
the sponsored people.</p>
<p>I am very much looking forward to the sprint. It will be the first time
that meet in real life many of the contributors, and judging by the
warmness of the on-line exchanges, it will be a great moment. Besides,
Granada is known to be a lively and historical city.</p>
<p>If you are around and want to join us, to work on Python in machine
learning, send us a mail on the <a class="reference external" href="https://lists.sourceforge.net/lists/listinfo/scikit-learn-general">mailing list</a>.</p>
Cython example of exposing C-computed arrays in Python without data copies2011-09-28T23:42:00+02:002011-09-28T23:42:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-09-28:/programming/cython-example-of-exposing-c-computed-arrays-in-python-without-data-copies.html<p>Some advice on passing arrays from C to Python avoiding copies. I use
Cython as I have found the code to be more maintainable than hand-written
Python C-API code.</p>
<p>I found out that there was no self-contained example of creating numpy
arrays from existing data in Cython. Thus I created …</p><p>Some advice on passing arrays from C to Python avoiding copies. I use
Cython as I have found the code to be more maintainable than hand-written
Python C-API code.</p>
<p>I found out that there was no self-contained example of creating numpy
arrays from existing data in Cython. Thus I created my own. The full code
with readme build and demo scripts is available on a <a class="reference external" href="https://gist.github.com/1249305">gist</a>. Here I only
give an executive summary.</p>
<p>The core functionality is implemented by the
<a class="reference external" href="http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html#PyArray_SimpleNewFromData">PyArray_SimpleNewFromData</a> function of the C API of numpy that can
create an ndarray from a pointer to the data, a simple data type, and
the shape of the data. The Cython file just builds around that function:</p>
<p>
<script src="https://gist.github.com/1249305.js?file=cython_wrapper.pyx"></script>
</p>Python at scientific conferences2011-09-11T15:52:00+02:002011-09-11T15:52:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-09-11:/programming/python-at-scientific-conferences.html<p>Top notch scientific conferences are starting to add Python tracks to
their program. This is good news. Indeed, it scientific Python
conferences (namely <a class="reference external" href="http://conference.scipy.org/scipy2011/">Scipy</a>, <a class="reference external" href="http://www.euroscipy.org/">EuroSciPy</a> and <a class="reference external" href="http://scipy.in/scipyin/2011/">Scipy India</a>) are doing
great to get together people who have already heard about Python for
science, but we need to reach out to …</p><p>Top notch scientific conferences are starting to add Python tracks to
their program. This is good news. Indeed, it scientific Python
conferences (namely <a class="reference external" href="http://conference.scipy.org/scipy2011/">Scipy</a>, <a class="reference external" href="http://www.euroscipy.org/">EuroSciPy</a> and <a class="reference external" href="http://scipy.in/scipyin/2011/">Scipy India</a>) are doing
great to get together people who have already heard about Python for
science, but we need to reach out to specific Python communities to
maximize impact.</p>
<div class="section" id="esco-2012-european-seminar-on-coupled-problems">
<h2>ESCO 2012 - European Seminar on Coupled Problems</h2>
<p><a class="reference external" href="http://esco2012.femhub.com/">ESCO 2012</a> is the 3rd event in a series of interdisciplineary meetings
dedicated to computational science challenges in multi-physics and PDEs.</p>
<p>I was invited as ESCO last year. It was an aboslute pleasure, because it
is a small conference that is very focused on discussions. I learned a
lot and could sit down with people who code top notch PDE libraries such
as FEniCS and have technical discussions. Besides, it is hosted in the
historical brewery where the Pilsner was invented. Plenty of great beer.</p>
<p><strong>Application areas</strong> Theoretical results as well as applications are
welcome. Application areas include, but are not limited to:
Computational electromagnetics, Civil engineering, Nuclear engineering,
Mechanical engineering, Computational fluid dynamics, Computational
geophysics, Geomechanics and rock mechanics, Computational hydrology,
Subsurface modeling, Biomechanics, Computational chemistry, Climate and
weather modeling, Wave propagation, Acoustics, Stochastic differential
equations, and Uncertainty quantification.</p>
<p><strong>Minisymposia</strong></p>
<ul class="simple">
<li>Multiphysics and Multiscale Problems in Civil Engineering</li>
<li>Modern Numerical Methods for ODE</li>
<li>Porous Media Hydrodynamics</li>
<li>Nuclear Fuel Recycling Simulations</li>
<li>Adaptive Methods for Eigenproblems</li>
<li>Discontinuous Galerkin Methods for Electromagnetics</li>
<li>Undergraduate Projects in Technical Computing</li>
</ul>
<p><strong>Software afternoon</strong> Important part of each ESCO conference is a
software afternoon featuring software projects by participants.
Presented can be any computational software that has reached certain
level of maturity, i.e., it is used outside of the author’s institution,
and it has a web page and a user documentation. If you would like to
present your software project, let us know soon.</p>
<p><strong>Proceedings</strong> For each ESCO we strive to reserve a special issue of an
international journal with impact factor. Proceedings of ESCO 2008
appeared in Math. Comput. Simul., proceedings of ESCO 2010 in CiCP and
Appl. Math. Comput. Proceedings of ESCO 2012 will appear in Computing.</p>
<p><strong>Important Dates</strong></p>
<ul class="simple">
<li>December 15, 2011: Abstract submission deadline.</li>
<li>December 15, 2011: Minisymposia proposals.</li>
<li>January 15, 2012: Notification of acceptance.</li>
</ul>
</div>
<div class="section" id="pyhpc-python-for-high-performance-computing">
<h2>PyHPC: Python for High performance computing</h2>
<p>If you are doing super computing, <a class="reference external" href="http://sc11.supercomputing.org/">SC11, the Super Computing
conference</a> is <em>the</em> reference conference. This year there will a
workshop on high performance computing with Python: <a class="reference external" href="http://www.dlr.de/sc/desktopdefault.aspx/tabid-1183/1638_read-31733/">PyHPC</a>.</p>
<p>At the scipy conference, I was having a discussion with some of the
attendees on how people often still do process management and I/O with
Fortran in the big computing environment. This is counter productive.
However, has success stories of supercomputing folks using high-level
languages are not advertized, this is bound to stay. Come and tell us
how you use Python for high performance computing!</p>
<p><strong>Topics</strong></p>
<ul class="simple">
<li>Python-based scientific applications and libraries</li>
<li>High performance computing</li>
<li>Parallel Python-based programming languages</li>
<li>Scientific visualization</li>
<li>Scientific computing education</li>
<li>Python performance and language issues</li>
<li>Problem solving environments with Python</li>
<li>Performance analysis tools for Python application</li>
</ul>
<p><strong>Papers</strong> We invite you to submit a paper of up to 10 pages via the
submission site. Authors are encouraged to use IEEE two column format.</p>
<p><strong>Important Dates</strong></p>
<ul class="simple">
<li>Full paper submission: September 19, 2011</li>
<li>Notification of acceptance: October 7, 2011</li>
<li>Camera-ready papers: October 31, 2011</li>
</ul>
</div>
Conference posters2011-09-05T04:15:00+02:002011-09-05T04:15:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-09-05:/science/conference-posters.html<p>At the request of a friend, I am putting up some of the posters that I
recently presented at conferences.</p>
<img alt="" class="align-left" src="attachments/scientific_posters/poster_nips.png" style="width: 30%;" />
<p><strong>Large-scale functional-connectivity graphical models for individual
subjects using population prior.</strong></p>
<p>This is a poster for <a class="reference external" href="http://hal.inria.fr/inria-00512451/en">our NIPS work</a></p>
<p><a class="reference external" href="attachments/scientific_posters/poster_nips.pdf">PDF</a></p>
<hr class="docutils" />
<img alt="" class="align-left" src="attachments/scientific_posters/poster_ipmi.png" style="width: 30%;" />
<p><strong>Multi-subject dictionary learning to segment an atlas of brain
spontaneous activity …</strong></p><p>At the request of a friend, I am putting up some of the posters that I
recently presented at conferences.</p>
<img alt="" class="align-left" src="attachments/scientific_posters/poster_nips.png" style="width: 30%;" />
<p><strong>Large-scale functional-connectivity graphical models for individual
subjects using population prior.</strong></p>
<p>This is a poster for <a class="reference external" href="http://hal.inria.fr/inria-00512451/en">our NIPS work</a></p>
<p><a class="reference external" href="attachments/scientific_posters/poster_nips.pdf">PDF</a></p>
<hr class="docutils" />
<img alt="" class="align-left" src="attachments/scientific_posters/poster_ipmi.png" style="width: 30%;" />
<p><strong>Multi-subject dictionary learning to segment an atlas of brain
spontaneous activity.</strong></p>
<p>This is a poster for <a class="reference external" href="http://hal.inria.fr/inria-00588898/en">our IPMI work</a></p>
<p><a class="reference external" href="attachments/scientific_posters/poster_ipmi.png">PDF</a></p>
<hr class="docutils" />
<img alt="" class="align-left" src="attachments/scientific_posters/poster_mayavi.png" style="width: 30%;" />
<p><strong>Mayavi for 3D visualization of neuroimaging data: powerful scripting
and reusable components in Python.</strong></p>
<p><a class="reference external" href="attachments/scientific_posters/poster_mayavi.pdf">PDF</a></p>
<hr class="docutils" />
<img alt="" class="align-left" src="attachments/scientific_posters/poster_scikit.png" style="width: 30%;" />
<p><strong>Machine learning for fMRI in Python: inverse inference with
scikit-learn.</strong></p>
<p><a class="reference external" href="attachments/scientific_posters/poster_scikit.pdf">PDF</a></p>
Hiring a junior developer on the scikit-learn2011-09-03T07:26:00+02:002011-09-03T07:26:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-09-03:/programming/hiring-a-junior-developer-on-the-scikit-learn.html<p>Once again, we are looking for a junior developer to work on the
scikit-learn. Below is the official job posting. As a personal remark, I
would like to stress that this is a unique opportunity to be payed for
two years to work on learning and improving the scientific Python …</p><p>Once again, we are looking for a junior developer to work on the
scikit-learn. Below is the official job posting. As a personal remark, I
would like to stress that this is a unique opportunity to be payed for
two years to work on learning and improving the scientific Python
toolstack.</p>
<hr class="docutils" />
<div class="section" id="job-description">
<h2>Job Description</h2>
<p>INRIA is looking to hire a young graduate on a 2-year position to help
with the community-driven development of the open source machine
learning in Python library, scikit-learn. The scikit-learn is one of the
majormajor machine-learning libraries in Python. It aims to be
state-of-the-art on mid-size to large datasets by harnessing the power
of the scientific Python toolstack.</p>
<p>Speaking French is not a requirement, as it is an international team.</p>
<div class="section" id="requirements">
<h3>Requirements</h3>
<ul class="simple">
<li>Programming skills in Python and C/C++</li>
<li>Understanding of quality assurance in software development:
test-driven programming, version control, technical documentation.</li>
<li>Some knowledge of Linux/Unix</li>
<li>Software design skills</li>
<li>Knowledge of open-source development and community-driven
environments</li>
<li>Good technical English level</li>
<li>An experience in statistical learning or a mathematical-oriented
mindset is a plus</li>
<li>We can only hire a young-graduate that has received a masters or
equivalent degree at most a year ago.</li>
</ul>
</div>
</div>
<div class="section" id="about-inria">
<h2>About INRIA</h2>
<p>INRIA is the French computer science research institute. It recognized
word-wide as one of the leading research institutions and has a strong
expertise in machine learning. You will be working in the <a class="reference external" href="https://parietal.saclay.inria.fr">Parietal
team</a> that makes a heavy use of Python for brain imaging analysis.</p>
<p>Parietal is a small research team (around 10 people) with an excellent
technical knowledge of scientific and numerical computing in Python as
well as a fine understanding of algorithmic issues in machine learning
and statistics. Parietal is committed to investing in scikit-learn.</p>
<p>Working at Parietal is a unique opportunity to improve your skills in
machine learning and numerical computing in Python. In addition, working
full time on the scikit-learn, a very active open-source project, will
give you premium experience of open source community management and
collaborative project development.</p>
</div>
<div class="section" id="contact-info">
<h2>Contact Info:</h2>
<ul class="simple">
<li><strong>Technical Contact</strong>: Bertand Thirion</li>
<li><strong>E-mail contact</strong>: bertrand dotnospam thirion atnospam inria
dotnospam fr</li>
<li><strong>HR Contact</strong>: Marie Domingues</li>
<li><strong>E-mail Contact</strong>: marie dotnospam domingues atnospam inria
dotnospam fr</li>
<li><strong>No telecommuting</strong></li>
</ul>
</div>
My conference travels: Scipy 2011 and HBM 20112011-07-23T23:45:00+02:002011-07-23T23:45:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-07-23:/science/my-conference-travels-scipy-2011-and-hbm-2011.html<div class="section" id="the-scipy-2011-conference-in-austin">
<h2>The Scipy 2011 conference in Austin</h2>
<p>Last week, I was at the Scipy conference in Austin. It was really great
to see old friends, and Austin is such a nice place.</p>
<img alt="" class="align-center" src="http://farm7.static.flickr.com/6143/5931239349_13c78bbef5_m.jpg" style="width: 50%;" />
<p>The Scipy conference was held in <a class="reference external" href="http://www.meetattexas.com/">UT Austin’s conference center</a>, which
is a fantastic venue. This is the …</p></div><div class="section" id="the-scipy-2011-conference-in-austin">
<h2>The Scipy 2011 conference in Austin</h2>
<p>Last week, I was at the Scipy conference in Austin. It was really great
to see old friends, and Austin is such a nice place.</p>
<img alt="" class="align-center" src="http://farm7.static.flickr.com/6143/5931239349_13c78bbef5_m.jpg" style="width: 50%;" />
<p>The Scipy conference was held in <a class="reference external" href="http://www.meetattexas.com/">UT Austin’s conference center</a>, which
is a fantastic venue. This is the first geek’s conference I have been at
where the wireless network worked flawlessly with a good bandwidth, even
thought 200 geeks were pounding on it. As a tutorial presenter, this was
incredibly useful.</p>
<div class="section" id="conference-highlight">
<h3>Conference highlight</h3>
<p>Here is a short list of what I <em>felt</em> were the big trends and highlights
of the conference. This is obviously biased by my own interests. I am
not listing parallel computing, as it is clearly an important area of
progress and debates, but it has been the case for the last few years.</p>
<div class="section" id="eric-jone-s-keynote">
<h4>Eric Jone’s keynote</h4>
<p>Of course Eric’s keynote was excellent. Eric is a great speaker and
always has good insights on how to run a team and a project. This year
he shared (some) of his tricks in making Enthought deliver on software
projects: <em>“What Matters in Scientific Software Projects? 10 Years of
Success and Failure Distilled”</em>. The video is not yet online,
unfortunately. Grab it when you can.</p>
</div>
<div class="section" id="hilary-mason-s-keynote">
<h4>Hilary Mason’s keynote</h4>
<p>Hilary is an applied data geek, just what I like! She gave an
interesting <a class="reference external" href="http://conference.scipy.org/scipy2011/slides/mason_awesome.pdf">keynote</a> on how <a class="reference external" href="https://bitly.com/">bitly</a> (an URL-shortening startup, for
those living under a rock) mines the requests on the URLs that the serve
to do things like ranking or phishing attempts detection. Of course, I
couldn’t resist asking what tools they used, thinking that she would
reply R. She mentioned that they did do some roll-their-own, but she
mentioned <a class="reference external" href="https://mlpy.fbk.eu/">mlpy</a> and <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>, with a mention that it was very
nice, at which point I believe that I blushed. She stressed that R was
hard to use and production and raised the point that most often academic
software doesn’t pan out in these settings (I hope that I am not
distorting her thoughts too much).</p>
</div>
<div class="section" id="statistics-and-learning">
<h4>Statistics and learning</h4>
<p>I had the feeling that statistics and data mining played a big role at
scipy this year. Maybe it is because I am more tuned to these questions
nowadays, but some signs do not lie. There was a special session on
Python in data sciences, a panel discussion on Python in finance and
<a class="reference external" href="http://conference.scipy.org/scipy2011/slides/cron_gpustats.pdf">many</a>
<a class="reference external" href="http://conference.scipy.org/scipy2011/slides/refsdal_sherpa.zip">many</a>
<a class="reference external" href="http://conference.scipy.org/scipy2011/slides/mckinney_time_series.pdf">statistics</a> and <a class="reference external" href="http://conference.scipy.org/scipy2011/slides/determan_vision_spreadsheet.pdf">data</a> <a class="reference external" href="http://conference.scipy.org/scipy2011/slides/caraciolo_crab_recommendation.pdf">related</a> talks, as well as two tutorials and
a keynote.</p>
<p>In addition, on a personal basis it was really great to meet part of the
team behind <a class="reference external" href="http://statsmodels.sourceforge.net/">scikits.statmodels</a>. We had plenty of very interesting
discussions and they really help me understand the way that some
statisticians abord data: very differently than me, because they have
fairly little data, and can afford to inspect reports and graphs,
whereas I rely more on automated decision rules.</p>
</div>
<div class="section" id="ipython">
<h4>IPython</h4>
<p><a class="reference external" href="http://twitter.com/#!/minrk">Min</a> gave <a class="reference external" href="http://minrk.github.com/scipy-tutorial-2011/">an excellent tutorial</a> on how to do parallel computing
using IPython. These guys have certainly done an excellent job to make
cluster-level programming in Python easier. While they don’t play yet
terribly well with the restrictive job-queue policy of the clusters to
which I have access, they have all the right low-level tools to address
these issues and Min told me that they will be working on this next
year.</p>
<p><a class="reference external" href="http://fperez.org/">Fernando</a> gave <a class="reference external" href="http://conference.scipy.org/scipy2011/slides/perez_ipython.pdf">an impressive talk</a> on the new developments of
IPython. In particular, the new Qt-based terminal is <em>`really cool`_</em>
and there is a web frontend in the works.</p>
</div>
<div class="section" id="cluster-computing-as-facility">
<h4>Cluster computing as facility</h4>
<p>While I mention cluster computing, I must confess that I have always
stayed away from this beast: I find it a time sink, and I find that I
get more science done without it. This is why I really like the
presentation of the <a class="reference external" href="http://www.picloud.com/">PiCould</a> guys on, … cluster computing! The
reason I liked it, is that they start from the principle that your time
is more important than CPU time. I hear so much about <em>bigger better
faster more</em> high-performance computing when researchers forget to
address the biggest issue:</p>
<blockquote class="epigraph">
… a whole generation of researchers turned into system
administrators by the demands of computing - Dan Reed, VP Microsoft</blockquote>
</div>
<div class="section" id="abstract-code-manipulation-for-numerical-computation">
<h4>Abstract code manipulation for numerical computation</h4>
<p>Finally, a trend that is picking up in the Python-based scientific
computing is the abstract manipulation of expressions to generate fast
code. This ranges from <a class="reference external" href="http://en.wikipedia.org/wiki/Just-in-time_compilation">JIT (just in time) compilation</a> generating
machine code, to rewriting mathematical expressions. Peter Wang had a
<a class="reference external" href="http://conference.scipy.org/scipy2011/slides/wang_metagraph.pdf">talk</a> in this alley, but the topic was also brough up be Aron Ahmadia.
Of course this is not new: <a class="reference external" href="http://code.google.com/p/numexpr/">numexpr</a> has been using these tricks for
years, and more recently <a class="reference external" href="http://deeplearning.net/software/theano/">Theano</a> has been making good use of GPUs
thanks to them.</p>
<p>Seeing this topic emerges in more and more places fr good reasons: with
faster and more numerous CPU, the number of operations a second is less
the bottleneck, and the order in which they are applied, or the physical
location, is becoming critical.</p>
</div>
</div>
<div class="section" id="my-own-agenda">
<h3>My own agenda</h3>
<div class="section" id="sprinting-on-scikit-learn">
<h4>Sprinting on scikit-learn</h4>
<a class="reference external image-reference" href="http://scikit-learn.org/dev/auto_examples/mixture/plot_gmm.html"><img alt="" src="http://scikit-learn.org/dev/_images/plot_gmm_1.png" /></a>
<p>We had two days of sprints after the conference. A huge number of people
voted for sprint on the <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a> but only two people showed up:
Minwoo Lee and <a class="reference external" href="http://www-etud.iro.umontreal.ca/~wardefar">David Warde-Farley</a>. Thanks heaps to these guys! My
priority for the sprint was to review and merge branches. That worked
beautifully: we merged in the following features:</p>
<ul class="simple">
<li><a class="reference external" href="http://scikit-learn.sourceforge.net/dev/modules/mixture.html#the-dirichlet-process">Dirichlet-Process Gaussian mixture models</a>, by Alex Passos</li>
<li><a class="reference external" href="http://scikit-learn.sourceforge.net/dev/modules/decomposition.html#sparse-principal-components-analysis-sparsepca">Sparse PCA</a> by Vlad Niculae.</li>
<li><a class="reference external" href="http://scikit-learn.sourceforge.net/dev/modules/gaussian_process.html">Speedups in Gaussian processes</a> by Vincent Schut.</li>
<li><a class="reference external" href="http://scikit-learn.sourceforge.net/dev/modules/clustering.html#mini-batch-k-means">Sparse implementation of the mini-batch k-means</a> by Peter
Prettenhofer.</li>
</ul>
<p>In addition, David added dataset downloader for the <a class="reference external" href="http://cs.nyu.edu/~roweis/data/olivettifaces.gif">Olivetti face
datasets</a> which is lightweight, but rich-enough to give very
interesting examples.</p>
</div>
<div class="section" id="my-presentation">
<h4>My presentation</h4>
<p>I gave a talk on my research work, and the software stack that
undermines it: <a class="reference external" href="http://www.slideshare.net/GaelVaroquaux/python-for-brain-mining-neuroscience-with-state-of-the-art-machine-learning-and-data-visualization">Python for brain mining: (neuro)science with state of
the art machine learning and data visualization</a>. I think that it was
well received by the audience. What is really crazy is that I uploaded
the slides on slideshare, and they got a ridiculous amount of viewing. I
suspect that it is because of the title: <em>brain mining</em> does sound
fancy.</p>
</div>
<div class="section" id="mayavi">
<h4>Mayavi</h4>
<p>Because of technical and political reasons, I cannot get <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a>
installed on the computers at work. This, and the fact that many people
ask for help, but little contribute, even in the form of answers on the
mailing list, had been mining me a bit. I got so much great feedback on
Mayavi at the conference that I feel much more motivated to invest
energy on it.</p>
</div>
</div>
</div>
<div class="section" id="the-humain-brain-mapping-conference-in-quebec-city">
<h2>The Humain Brain Mapping conference in Quebec City</h2>
<img alt="" class="align-center" src="http://farm7.static.flickr.com/6018/5968391718_002105ccd1.jpg" style="width: 50%;" />
<p>This blog post is getting too long. It is well beyond my own attention
span. However scipy is not the only conference to which I have been
recently. Two weeks before I was in Quebec, for the <a class="reference external" href="http://www.humanbrainmapping.org/i4a/pages/index.cfm?pageID=3419">Human Brain Mapping
conference</a>. As each year, HBM is a fun ride. It has fantastic parties
in the evenings. But I didn’t stay up too late as, this year was a busy
for me: I was teaching in a educational course, and chairing a
symposium, both on comparing brain functional connectivity across
subjects.</p>
<p>But the really big deal at HBM this year came at the end. As I was
dosing off, vaguely listening to Russ Poldrak’s closing comments, he
brought up on screen a slide entitled <em>the year of Python</em>. This is a
big deal: we’ve been working for years to get Python in the neuroimaging
word, and it is clearly making progress, despite all the roadblocks.</p>
</div>
Euroscipy 2011: early bird deadline soon2011-07-22T00:44:00+02:002011-07-22T00:44:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-07-22:/programming/euroscipy-2011-early-bird-deadline-soon.html<div class="section" id="euroscipy-2011-register-now-for-early-bird-prices">
<h2>Euroscipy 2011: register now for early bird prices</h2>
<p><strong>The deadline for early-bird registration at the Euroscipy conference
is this Sunday</strong>. Beyond this deadline prices will double. <strong>Register
now to get a great deal.</strong></p>
<p>To register, simply go to <a class="reference external" href="http://www.euroscipy.org">www.euroscipy.org</a>, log in using the link on
the top right …</p></div><div class="section" id="euroscipy-2011-register-now-for-early-bird-prices">
<h2>Euroscipy 2011: register now for early bird prices</h2>
<p><strong>The deadline for early-bird registration at the Euroscipy conference
is this Sunday</strong>. Beyond this deadline prices will double. <strong>Register
now to get a great deal.</strong></p>
<p>To register, simply go to <a class="reference external" href="http://www.euroscipy.org">www.euroscipy.org</a>, log in using the link on
the top right, and follow the <em>‘Register now for the conference’</em> link
on the top left.</p>
<p>The conference is a great opportunity to learn the intricacies of
numerical and scientific computing in Python. You can register for the
tutorials in a <a class="reference external" href="http://www.euroscipy.org/track/4010?vid=tracktalkslist">intro track</a>, that will take you from beginner to fully
autonomous user, or for an <a class="reference external" href="http://www.euroscipy.org/track/4011?vid=tracktalkslist">advanced track</a>, to learn from the experts
topics such as image processing, GPU computing, machine learning or
optimization. The tutorials are a fairly unique occasion to improve your
skills, as you will seldom get such a concentration of experts.</p>
</div>
<div class="section" id="some-program-highlights">
<h2>Some program highlights</h2>
<p>After the 2 days of tutorial, the conference itself we host 2 keynotes:
one by <a class="reference external" href="http://mcs.open.ac.uk/mp8/">Marian Petre</a>, of the open university, well-known for her
empirical studies of software development, and another one by <a class="reference external" href="http://fperez.org/">Fernando
Perez</a>, a pioneer in scientific computing in Python and the original
author of IPython.</p>
<p>Glancing at the <a class="reference external" href="http://www.euroscipy.org/track/3992?vid=tracktalkslist">program</a>, we can see how a wide range of topics are
touched:</p>
<ul class="simple">
<li>pure computer-science topics, such as <a class="reference external" href="http://www.euroscipy.org/talk/4186">concurrent programming</a></li>
<li>traditional <em>hard</em> sciences, such as <a class="reference external" href="http://www.euroscipy.org/talk/4201">multi-physics</a></li>
<li>simulation of complex systems, for instance <a class="reference external" href="http://www.euroscipy.org/talk/4219">network modeling in
epidemiology</a></li>
<li>or novel application of quantitative large-data processing, as in
<a class="reference external" href="http://www.euroscipy.org/talk/4182">legal research</a></li>
</ul>
<p>The variety of the topics illustrates what is for me one of the greatest
benefits of the scipy conferences: they form a forum to exchange ideas
and techniques to find new solutions to scientific, numerical and data
analysis problems. Unlike the pure computer science conference, they sit
at the frontier of applications and bleeding edge computer developments,
<strong>because these people really use the tools presented to solve their
problems</strong>.</p>
<p>In addition to this rich program, we will have 2 days of <a class="reference external" href="http://www.euroscipy.org/track/5201">sprints</a>
before the conference as well as 2-day-long satellite conferences on
Python in <a class="reference external" href="http://www.euroscipy.org/card/pyphy2011">Physics</a> and <a class="reference external" href="http://pythonneuro.sciencesconf.org/">NeuroScience</a> after the conference. This is
how what used to be a small conference can now be a full 8-days event if
you order all the extras.</p>
</div>
Hiring a junior engineer on the scikit-learn2011-05-14T19:10:00+02:002011-05-14T19:10:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-05-14:/programming/hiring-a-junior-engineer-on-the-scikit-learn.html<p>The <a class="reference external" href="http://www.scikit-learn.org">scikit-learn</a> is a Python module for machine learning. The
project builds on the scientific and numerical tools of the <a class="reference external" href="http://www.scipy.org">scipy
community</a> to provide state-of-the-art data analysis tools. It is
developed by a community of open source developers to which my research
team (<a class="reference external" href="https://parietal.saclay.inria.fr/">Parietal</a>, <a class="reference external" href="http://www.inria.fr/">INRIA</a>) contributes a lot and is …</p><p>The <a class="reference external" href="http://www.scikit-learn.org">scikit-learn</a> is a Python module for machine learning. The
project builds on the scientific and numerical tools of the <a class="reference external" href="http://www.scipy.org">scipy
community</a> to provide state-of-the-art data analysis tools. It is
developed by a community of open source developers to which my research
team (<a class="reference external" href="https://parietal.saclay.inria.fr/">Parietal</a>, <a class="reference external" href="http://www.inria.fr/">INRIA</a>) contributes a lot and is a <a class="reference external" href="http://github.com/scikit-learn/scikit-learn">striving
project</a>. Its mailing list fosters many discussions on code and machine
learning topics, it has a <a class="reference external" href="http://scikit-learn.sourceforge.net/user_guide.html">a very detailed documentation</a>, and <a class="reference external" href="http://scikit-learn.sourceforge.net/whats_new.html">a tight
release cycle</a>.</p>
<p>Although scikits.learn is mostly developed by volunteers, INRIA has
funded a two year position for a junior engineer —currently <a class="reference external" href="http://fseoane.net/blog/">Fabian
Pedregosa</a>— to help with the core management and integration of the
project. This funding is coming to an end in falls 2011 <a class="reference external" href="#footnote">[*]</a>. The
good news is that we have been allocate new funding to hire an engineer
on the scikit.</p>
<p><strong>We are thus looking to hire a junior engineer for a 2-year position to
work on the scikits.learn at INRIA in Saclay, near Paris</strong>. The position
is only available to candidates that have received a <strong>masters or
equivalent degree at most a year ago</strong> — this is non negotiable: we
cannot hire more senior candidates.</p>
<p>We are looking for a developer with good open-source project management
skills: the successful candidate will review and merge patches, ensure
the quality of the scikit, make releases, coordinate development on the
mailing list and on github. Good knowledge of Python and its scientific
ecosystem is expected. A mathematical or computer-science oriented
mindset is a plus, as the project involves working with machine learning
algorithms.</p>
<p>The candidate should be willing to relocate to work daily in the
<a class="reference external" href="http://www-dsv.cea.fr/en/instituts/institut-d-imagerie-biomedicale-i2bm/services/neurospin-neurospin">Neurospin brain research institute</a> in which the Parietal is located.
Knowledge of French is not required, as the team and the institute are
very international. Non-EU candidates are welcome, but the hiring
process will take longer.</p>
<p>You will be working in a very stimulating environment. You will be
employed by INRIA, the French computer science research institute. As
such, you will benefit from the expertise of the institute’s researchers
and engineers. Team members contribute to various scientific Python
libraries (in addition to scikits.learn, <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a>, <a class="reference external" href="http://nipy.org">nipy</a>, <a class="reference external" href="http://packages.python.org/joblib/">joblib</a>).
In addition, you will be working in a brain research institute, in
collaboration with leading <a class="reference external" href="http://lnao.fr">methods researchers</a> and <a class="reference external" href="http://www.unicog.org/pm/pmwiki.php">neuroscientists</a>
that use machine learning to gain new insights on brain processes.</p>
<p><strong>To apply:</strong> To apply, you need to prepare a CV and a motivation
letter. The deadline for applications is mid June, but we will be
selecting candidates and conducting interviews before. <strong>Don’t send me
CVs</strong>. The formal job description, as well as instructions to apply can
be found on this <a class="reference external" href="http://en.inria.fr/institute/recruitment/offers/young-graduate-engineers/%28view%29/details.html?id=PNGFK026203F3VBQB6G68LOE1&LOV5=4510&ContractType=4545&LG=EN&Resultsperpage=20&nPostingID=5534&nPostingTargetID=10628&option=52&sort=DESC&nDepartmentID=10">page</a>. The page is mostly in French, sorry; use
Google translate if you don’t understand. At the bottom of the page you
will find a link to apply.</p>
<hr class="docutils" />
<p><strong>[*]</strong> Fabian will most probably stay with us to do a PhD on
<a class="reference external" href="https://parietal.saclay.inria.fr/research">analysis of large brain functional imaging datasets</a>.</p>
EuroScipy: the program is filling up, and the submission deadline nearing2011-04-30T17:21:00+02:002011-04-30T17:21:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2011-04-30:/programming/euroscipy-the-program-is-filling-up-and-the-submission-deadline-nearing.html<div class="section" id="submission-deadline-may-8th">
<h2>Submission deadline May 8th</h2>
<p>The deadline for the call for presentation for the EuroScipy conference
is on <strong>May 8th</strong>. There is only a week and a half left.</p>
<p><a class="reference external" href="http://www.euroscipy.org/">EuroScipy</a> will be held in <strong>Paris, August 25-28</strong>. It is the European
meeting for users of Python in scientific and numerical-intensive
applications …</p></div><div class="section" id="submission-deadline-may-8th">
<h2>Submission deadline May 8th</h2>
<p>The deadline for the call for presentation for the EuroScipy conference
is on <strong>May 8th</strong>. There is only a week and a half left.</p>
<p><a class="reference external" href="http://www.euroscipy.org/">EuroScipy</a> will be held in <strong>Paris, August 25-28</strong>. It is the European
meeting for users of Python in scientific and numerical-intensive
applications. It strives to bring together both users and developers of
scientific and numerical tools, as well as academic research and state
of the art industry. The conference will host 2 days of tutorials and 2
days of technical presentations.</p>
<p>Lately, numerical computing in Python has started reaching a much wider
audience than the traditional academic-oriented audience. This is partly
because Python is making its way in major engineering companies, but
also because more and more industries are processing large amounts of
data, and find precious <strong>data analytics tools</strong> in the <a class="reference external" href="http://www.scipy.org">Scipy</a>
community. In this spirit, this year there will be a <a class="reference external" href="http://www.euroscipy.org/talk/4061">tutorial on
machine learning with Python</a>.</p>
</div>
<div class="section" id="poster-session">
<h2>Poster session</h2>
<p>Last year, the organizing committee had to refuse a large fraction of
the proposals, because there were not enough slots available. We had
considered organizing a poster sessions, but the logistics were to
challenging for our little resources. Indeed, EuroSciPy still tries to
be organized as a hackers and coders conference, rather than an
industry-level one. For instance, we keep the prices to a minimum, in
order to make it easy for young people traveling on their own budget to
join us. Getting 200 attendees as we did last year, did strain our small
organization committee.</p>
<p>This year, we had a unexpected backing of the <a class="reference external" href="http://www.phys.ens.fr/">physics department</a> of
the <a class="reference external" href="http://www.ens.fr/?lang=en">ENS</a>. They were extremely enthusiastic about Python, that they now
use for teaching and research. This made me really happy, as this is
where I studied. They proposed help, and in particular help with the
local organization.</p>
<p>Thus I am able to announce that thanks to the physics department of the
ENS, we will be able to host a poster session!</p>
</div>
<div class="section" id="an-exciting-program-shaping-up">
<h2>An exciting program shaping up</h2>
<p>The program is starting to shape up, and it is looking really good, in
my eyes.</p>
<div class="section" id="keynotes">
<h3>Keynotes</h3>
<p>We will be having two keynote speakers, one directly from the SciPy
community, Fernando Perez, and one probably less known to this
community, Marian Petre.</p>
<ul class="simple">
<li><a class="reference external" href="http://mcs.open.ac.uk/mp8/">Marian Petre</a>: Marian is the director of the <a class="reference external" href="http://crc.open.ac.uk/">Center for Research
in Computing</a>, at the <a class="reference external" href="http://www.open.ac.uk/">Open University</a>. She is interested in
empirical studies of software development. I am very excited to hear
a bit more about the often-forgotten human factor that goes behind
every coding job, big or small. In my experience scientific computing
and computational sciences pay a hefty price because they don’t
acknowledge well-enough the gap between good ideas and tractable
code.</li>
<li><a class="reference external" href="http://fperez.org/">Fernando Perez</a>: Fernando is a research scientist in
neuroscience at <a class="reference external" href="http://neuroscience.berkeley.edu/">UC Berkeley</a>. Before that, he was successively a
physicist and a mathematician. He has been an early advocate of the
scientific Python ecosystem, in addition to being the creator of
IPython. His vision has always been oriented toward finding an
computing environment that makes scientific creativity easier.</li>
</ul>
</div>
<div class="section" id="tutorials">
<h3>Tutorials</h3>
<p>The tutorial program is now final, and can be seen on the <a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2011">schedule</a>.
Like last year, we will have two tracks:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.euroscipy.org/track/4010">An introductory track</a>, designed as a two-day course addressing
the different aspects of the Python language and the scientific
computing module to bring up beginners to full speed. At the end of
the two days, attendee should be able to solve simple computational
problems using Python alone.</li>
<li><a class="reference external" href="http://www.euroscipy.org/track/4011">An advanced track</a>, in which experts of various aspects of
scientific and numerical computing in Python share their knowledge in
2-hours long tutorials.</li>
</ul>
</div>
</div>
<div class="section" id="python-in-neuroscience-satellite">
<h2>Python in NeuroScience satellite</h2>
<p>The two days following the conference, their will be <a class="reference external" href="http://pythonneuro.sciencesconf.org/">a satellite
meeting on the use Python in neuroscience</a>. It will be a small and more
focused event, in which neuroscientist will be able to exchange
technical aspects of computation and data management in Python.
Hopefully it will foster interest discussions and collaborations. if you
are interested, you can submit a talk proposal for this satellite
meeting <a class="reference external" href="http://pythonneuro.sciencesconf.org/">here</a>.</p>
<hr class="docutils" />
<img alt="" class="align-center" src="http://farm5.static.flickr.com/4143/4780097256_14c99f3b32.jpg" style="width: 60%;" />
<p><strong>Come and join us at EuroScipy in Paris, Augst 25-28. Paris is a great
city. The SciPy community is a friendly one.</strong></p>
</div>
Scikit-learn sprint on April 1st2011-03-26T13:27:00+01:002011-03-26T13:27:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-03-26:/programming/scikit-learn-sprint-on-april-1st.html<a class="reference external image-reference" href="http://scikit-learn.sourceforge.net/"><img alt="" src="http://scikit-learn.sourceforge.net/stable/_static/scikit-learn-logo-small.png" /></a>
<p>The <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a> team is organizing a sprint on April 1st (that next
Friday). Join us in <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events">Paris, Boston, or on IRC</a>!</p>
<p>With the rise of the data sciences, the scikit-learn, a <strong>BSD-licensed
Python package for machine learning</strong>, is becoming an asset for more and
more endeavors. Machine learning has traditionally …</p><a class="reference external image-reference" href="http://scikit-learn.sourceforge.net/"><img alt="" src="http://scikit-learn.sourceforge.net/stable/_static/scikit-learn-logo-small.png" /></a>
<p>The <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a> team is organizing a sprint on April 1st (that next
Friday). Join us in <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events">Paris, Boston, or on IRC</a>!</p>
<p>With the rise of the data sciences, the scikit-learn, a <strong>BSD-licensed
Python package for machine learning</strong>, is becoming an asset for more and
more endeavors. Machine learning has traditionally been considered as
very technical and inaccessible to the non mathematician. We are aiming
to break this barrier.</p>
<p>The sprint will be focused on pragmatic down-to-earth improvements in
the scikit. Our goal is to make it easy for people to contribute. A list
of tasks and organization details can be found on the <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/wiki/Upcoming-events">sprint planning</a>
wiki page. Amongst other things, we’ll be working on:</p>
<ul class="simple">
<li><strong>integrating new learning algorithms</strong>, in particular merging in the
many excellent pull requests that we have: <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/103">hierarchical
clustering</a>, <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/103">data transforming using linear discriminant
analysis</a>, <a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/107">multinomial naive bayes classifier</a> …</li>
<li><strong>testing and logging framework</strong>,</li>
<li><a class="reference external" href="https://github.com/scikit-learn/scikit-learn/pull/94">**better parallel computing support**</a>,</li>
<li>and many other itches to scratch, as it is a community-driven event.</li>
</ul>
<p>Come and join us. It will be fun, and it’s an occasion to learn new
tricks.</p>
<hr class="docutils" />
<p><a class="reference external image-reference" href="http://farm5.static.flickr.com/4067/4405351641_5675ba000c.jpg"><img alt="image1" src="http://farm5.static.flickr.com/4067/4405351641_5675ba000c.jpg" style="width: 20%;" /></a> <a class="reference external image-reference" href="http://farm6.static.flickr.com/5249/5265835075_ea0b41019c.jpg"><img alt="image2" src="http://farm6.static.flickr.com/5249/5265835075_ea0b41019c.jpg" style="width: 20%;" /></a> <a class="reference external image-reference" href="http://farm5.static.flickr.com/4135/4974339970_566424185f.jpg"><img alt="image3" src="http://farm5.static.flickr.com/4135/4974339970_566424185f.jpg" style="width: 20%;" /></a> <a class="reference external image-reference" href="http://farm6.static.flickr.com/5294/5425114531_6eec316967.jpg"><img alt="image4" src="http://farm6.static.flickr.com/5294/5425114531_6eec316967.jpg" style="width: 20%;" /></a></p>
Windows binaries for the scientific Python ecosystem2011-02-15T09:02:00+01:002011-02-15T09:02:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-02-15:/programming/windows-binaries-for-the-scientific-python-ecosystem.html<p>I just realized yesterday that Christoph Gohlke has <a class="reference external" href="http://www.lfd.uci.edu/~gohlke/pythonlibs/">a repository of
binary installers</a> (<em>.exe</em>) for Windows 32 and 64bit with almost all
the scientific Python packages that you can dream of:</p>
<ul class="simple">
<li><a class="reference external" href="http://numpy.scipy.org">numpy</a>, <a class="reference external" href="http://www.scipy.org/">scipy</a> and <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a>, of course (compiled
with the MKL)</li>
<li><a class="reference external" href="http://cython.org/">cython</a></li>
<li>the <a class="reference external" href="http://enthought.github.com/">ETS</a>, including <a class="reference external" href="http://enthought.github.com/mayavi/mayavi/">Mayavi</a></li>
<li><strong>VTK</strong>, with the Python …</li></ul><p>I just realized yesterday that Christoph Gohlke has <a class="reference external" href="http://www.lfd.uci.edu/~gohlke/pythonlibs/">a repository of
binary installers</a> (<em>.exe</em>) for Windows 32 and 64bit with almost all
the scientific Python packages that you can dream of:</p>
<ul class="simple">
<li><a class="reference external" href="http://numpy.scipy.org">numpy</a>, <a class="reference external" href="http://www.scipy.org/">scipy</a> and <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a>, of course (compiled
with the MKL)</li>
<li><a class="reference external" href="http://cython.org/">cython</a></li>
<li>the <a class="reference external" href="http://enthought.github.com/">ETS</a>, including <a class="reference external" href="http://enthought.github.com/mayavi/mayavi/">Mayavi</a></li>
<li><strong>VTK</strong>, with the Python bindings</li>
<li>a variety of <a class="reference external" href="http://scikits.appspot.com/">scikits</a> (including the <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>,
hurray!)</li>
</ul>
<p>These binaries are incredibly useful, as building all these packages
under Windows does requires some skills, and a compiler. They complement
very well fully-fledge scientific Python distributions such as EPD or
Python(x,y), as they can be installed on top of an existing Python
installation.</p>
<hr class="docutils" />
<p>I should say that I discovered this thanks to a long email discussion in
which Christoph Gohlke and Yakub Nowacki helped me debug a nasty Mayavi
bug on Windows 64bit that I couldn’t reproduce as I don’t have a Windows
64bit available. That was particularly helpful.</p>
Interested in parallel computing and statistics? We are looking for a post-doc2011-01-30T22:30:00+01:002011-01-30T22:30:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-01-30:/programming/interested-in-parallel-computing-and-statistics-we-are-looking-for-a-post-doc.html<p><a class="reference external" href="https://parietal.saclay.inria.fr/">My research group</a> is kick starting a new project, called
<strong>AzureBrain</strong> to do computational analysis of large brain imaging and
genetics population-wise data. One of the goals of the project is to
harness the power of grid computing to do statistical learning on fMRI
data, finding features in an individuals …</p><p><a class="reference external" href="https://parietal.saclay.inria.fr/">My research group</a> is kick starting a new project, called
<strong>AzureBrain</strong> to do computational analysis of large brain imaging and
genetics population-wise data. One of the goals of the project is to
harness the power of grid computing to do statistical learning on fMRI
data, finding features in an individuals brain images that can be
predicted by his genome. The medical applications cover the wide scope
of genetically-related brain pathologies, such as autism.</p>
<p>Want to work in a dynamic and exiting environment, using Python to solve
challenging data analysis? We are looking for a post-doctoral fellow to
hire in spring/beginning of summer. The ideal candidate would have a
strong background in computational statistics or machine learning, as
well as parallel computing, however we will consider any candidate with
good experience in one or the other and a strong desire to learn.</p>
<p>You would be employed by <a class="reference external" href="http://www.inria.fr">INRIA</a>, the lead computing research institute
in France. We are a team of computer scientists specialized in image
processing and statistical data analysis, integrated in one of the top
French brain research centers, <a class="reference external" href="http://www-dsv.cea.fr/en/instituts/institut-d-imagerie-biomedicale-i2bm/services/neurospin-d.-le-bihan">NeuroSpin</a>, south of Paris. We work
mostly in Python. The team includes core contributors to the
<a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn project</a>, for machine learning in Python, and the <a class="reference external" href="http://nipy.sourceforge.net/">nipy
project</a>, for NeuroImaging in Python.</p>
<p>Below follows a summary of <a class="reference external" href="http://parietal.saclay.inria.fr/open-positions/azure-brain-post-doc-proposal">the official job announcement</a>. Please
contact Bertrand Thirion, (first name _dot_ last name _at_ inria
_dot_ fr) if you are interested, referencing the AzureBrain project.</p>
<hr class="docutils" />
<div class="section" id="introduction">
<h2>Introduction</h2>
<p>Imaging genetic studies linking functional MRI data and Single
Nucleotide Polyphormisms (SNPs) data face a dire multiple comparisons
issue. In the genome dimension, genotyping DNA chips allow to record of
several hundred thousands values per subject, while in the imaging
dimension a brain image may contain 100k-1M voxels. Finding the brain
and genome regions that may be involved in this link entails a huge
number of hypotheses, hence a drastic correction of the statistical
significance of pairwise relationships, which in turn reduces crucially
the sensitivity of statistical procedures that aims at detecting the
association. It is therefore desirable to set up as sensitive techniques
as possible to explore where in the brain and where in the genome a
significant link can be detected, while correcting for family-wise
multiple comparisons (controlling for false positive rate). Another
issue is the computational cost of these procedures, that need to be
addressed with adequate algorithmic and computational tools.</p>
</div>
<div class="section" id="objectives">
<h2>Objectives</h2>
<p>In this project, we will consider a unique dataset acquired in the
<a class="reference external" href="http://www.imagen-europe.com">Imagen project</a>, an FP6 project that aims at investigating factors of
addition in a population of adolescents; Imagen’s database contains
multi-modal neuroimaging as well as genetics and psychological data on
about 2000 subjects. This database is hosted and processed at Neurospin
and is available for research purpose. The candidate will be in charge
of:</p>
<ul class="simple">
<li>Setting an analysis pipeline (based on code already available to
analyze neuroimaging/genetics datasets) to extract and pre-process
the relevant data for statistical analysis.</li>
<li>Performing statistical analysis on simulated datasets and sub-parts
of the whole database in order to set all the computational
framework. These procedures will include mass-univariate linear
modeling (with peak- and cluster-level tests), regularized multiple
regression and a permutation-based assessment framework.</li>
<li>Launch data analysis on a large scale grid and cloud environment,
with the help of the Kerdata researchers (see below).</li>
<li>Build the post-analytic framework to ease the interpretation of the
results in both neuroimaging and genetics domains.</li>
</ul>
<p>The analysis framework is based on algorithmic tools developed in
C/Python (numpy, scipy and scikit-learn). The candidate will interact i)
with researchers of the Parietal team for algorithmic aspects, but also
ii) with CEA researchers of Neurospin, who will provide expertise in
genetics domain and iii) with the KerData team (INRIA Rennes) and the
Joint MSR-INRIA Research Center (Microsoft Research), that will provide
help and massive computation facilities. The project has an access to
grid/cloud computing facilities to be used in collaboration with
INRIA/Kerdata and MSR-INRIA partners.</p>
<p>The expected results is the discovery of correlation between brain
activation and genetic information.</p>
</div>
<div class="section" id="required-knowledge-and-background">
<h2>Required knowledge and background</h2>
<p>The candidate should have at least a basic knowledge of standard
statistical concepts. He or she should have a first significant
experience in parallel computation and with python language. It is
important that he or she has some real interest in genetics and/or brain
imaging in order to have strong interactions with specialists of these
domains. He or she will benefit from the algorithmic tools developed at
Parietal and of the database settings and data pre-processing tools
developed by Neurospin researchers.</p>
</div>
EuroSciPy 2011: the dates are out - Aug 25-28, Paris2011-01-16T15:57:00+01:002011-01-16T15:57:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-01-16:/programming/euroscipy-2011-the-dates-are-out-aug-25-28-paris.html<p>We have finally been able to settle on final dates and venue for
<a class="reference external" href="http://www.euroscipy.org/conference/euroscipy_2011">EuroSciPy 2011</a>, the 4th European meeting on Python in Science.</p>
<p>The conference will be held <strong>from Thursday August 25th, to Sunday
August 28th</strong>. The <a class="reference external" href="http://www.ens.fr">ENS</a> will be hosting the conference once again,
right in the center of …</p><p>We have finally been able to settle on final dates and venue for
<a class="reference external" href="http://www.euroscipy.org/conference/euroscipy_2011">EuroSciPy 2011</a>, the 4th European meeting on Python in Science.</p>
<p>The conference will be held <strong>from Thursday August 25th, to Sunday
August 28th</strong>. The <a class="reference external" href="http://www.ens.fr">ENS</a> will be hosting the conference once again,
right in the center of Paris.</p>
Research jobs in France: the black humor of 2010 is the reality of 20112011-01-15T11:41:00+01:002011-01-15T11:41:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-01-15:/science/research-jobs-in-france-the-black-humor-of-2010-is-the-reality-of-2011.html<p>The French basic research landscape is dominated by a few nationwide
institute, similar to the NIST or the NIH in the US. The largest of these
is the <a class="reference external" href="http://www.cnrs.fr/index.php">CNRS</a> (Centre National de la Recherche Scientific). Getting a
tenured job in one of those institutes enables someone to focus on basic …</p><p>The French basic research landscape is dominated by a few nationwide
institute, similar to the NIST or the NIH in the US. The largest of these
is the <a class="reference external" href="http://www.cnrs.fr/index.php">CNRS</a> (Centre National de la Recherche Scientific). Getting a
tenured job in one of those institutes enables someone to focus on basic
research rather than teaching or going in the industry. It has always
been quite challenging to get such position as many people apply for very
few positions, and the choice of the candidates is quite political. Each
year there is a call for applications, through a impressive formal
process that young researchers trying to get jobs in France end up
knowing quite well.</p>
<p>Last year, I was visiting a research lab (<a class="reference external" href="http://www.incm.cnrs-mrs.fr/en_index.php">INCM</a>) and I saw in their
coffee-break room the following poster (below), that I could
clearly recognize as the official call for application for positions at
CNRS.</p>
<p>Now this poster says ‘<strong>The CNRS recruits 3 researchers (m/w) in all
fields of research</strong>‘. Of course it’s a fake poster and black humor: 3
positions nationwide in all fields of research is ridiculously low. It
is however an expression of the nightmare of thousands of young
researchers who are applying each year and keep hearing that the
government will <a class="reference external" href="http://www.latribune.fr/actualites/economie/france/20100415trib000499181/la-fonction-publique-d-etat-perdra-34.000-postes-en-2011-selon-georges-tron.html">slash the number of state employees</a>.</p>
<img alt="" class="align-center" src="attachments/cnrs_recruits.jpg" style="width: 70%;" />
<p>The call for the 2011 applications for research positions at <a class="reference external" href="http://en.inria.fr/">INRIA</a>,
the French national computer science institute, that is another one of
the big research institutions in France, is <a class="reference external" href="http://www.inria.fr/institut/recrutement-metiers/offres/concours-2011-5-postes-de-charge-de-recherche-2e-classe-sont-a-pourvoir/concours-2011">out</a>. The page is entitled
<em>Cinq postes de chargé de recherche 2e classe sont à pourvoir</em> (<strong>5
positions for junior researchers are available</strong>). This is not a joke,
and it is striking to see the similarity between <strong>the dark humor of
2010 and the reality of 2011</strong>. To be fair INRIA is smaller than CNRS,
as it covers only computer science and applications (listed as applied
maths, numerical computing and simulation, algorithm and software
research, networks and distributed systems, and computational modeling
for life sciences). The number of applications is in hundred and not
thousands, but having only 5 jobs available nationwide still feels
really awkward.</p>
<blockquote>
<a class="reference external" href="attachments/cnrs_recruits.pdf">PDF poster</a></blockquote>
<p>A minor detail: I am trying to get a job in computational science
research in France.</p>
Scientific publication for software development2011-01-08T22:40:00+01:002011-01-08T22:40:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2011-01-08:/programming/scientific-publication-for-software-development.html<p>The academic community seems to judge the validity and significance of
any contribution by the number of papers published and the number of
citations they get. To find funding, to get credit, you have to
<strong>publish or perish</strong>. However, the natural output of software
development tends not to be an …</p><p>The academic community seems to judge the validity and significance of
any contribution by the number of papers published and the number of
citations they get. To find funding, to get credit, you have to
<strong>publish or perish</strong>. However, the natural output of software
development tends not to be an article (people who confuse articles and
documentation do a poor job of both, IMHO).</p>
<p>While I believe that this policy is harmful for the quality of research,
I also know that I cannot fight it, and chances are that many other are
in my situation. As such, we need to publish scientific papers about the
scientific softwares that we develop (such as <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a>, or
<a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>, as far as I am concerned). On the other hand, as an
editor of the <a class="reference external" href="http://conference.scipy.org/proceedings.html">Scipy conference proceedings</a>, I have found that the
process of writing a paper on software work and going through peer
review can be greatly beneficial to the software. Indeed, it forces
authors to do a thorough review of the prior work, and to clearly
identify the purpose of the project. Also, such an article can only be
much shorter than a user manual, thus it forces the authors to identify
the key concepts of their software, and explain them clearly. As a
result, it helps finding design and usability flaws and gaining insight
on how the user manual can be structured.</p>
<p>A major challenge to publishing is that most of the highly-ranked
journals tend to disregard software works, unless they are very specific
to a scientific problem, which actually makes them less useful to the
complete ecosystem. Deeply rooted in the minds of the editors and the
reviewers, there tends to be the idea that developing software is easy
compared to doing experiments or proofs. In addition, these top-notch
scientists are not always the most qualified to judge the quality of
software, as they have most often never worked in a major software
project. The good news is that this is slowing changing with the
creation of software tracks in specialized journals, and the development
of new journals focused on scientific software.</p>
<div class="section" id="journals-for-publishing-about-interdisciplinary-scientific-software">
<h2>Journals for publishing about interdisciplinary scientific software</h2>
<p>In my opinion, interdisciplinary scientific software such as <a class="reference external" href="http://numpy.scipy.org/">numpy</a>,
the <a class="reference external" href="http://www.gnu.org/software/gsl/">GSL</a>, <a class="reference external" href="http://www.gnu.org/software/octave/">octave</a>, <a class="reference external" href="http://www.scilab.org/">scilab</a>, <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a>, or <a class="reference external" href="http://www.fenicsproject.org">Fenics</a>, are the
most valuable projects, as they provide foundations to build science in
the open. The challenge that these projects have to face are not only
algorithmic or computational, but also deal with providing good user
interfaces, or developing and catering for very large communities of
users. These problems are considered as <em>solved</em> in a scientific
context, as they have all been solved at least once, often quite
successfully by commercial products such as Matlab. As a result, it is
hard to get some funding for these projects unless there is a political
reason behind the funding, and IMHO politics tend to produce bad
software. Publishing high-profile articles on interdisciplinary
scientific software is thus hard, but critical. For this we need
journals that accept software papers, but are not only read by
researchers in CS or IT departments.</p>
<p>A couple of years ago, some of us made a review of where it was possible
to publish truly wide-scope scientific software, and we found that there
was pretty much no option. It’s crazy to see that things have still not
changed much, and that all lot of major general-purpose widely-used
projects, like the one I cited above, have never been acknowledged by a
publication.</p>
<ul class="simple">
<li><a class="reference external" href="http://cise.aip.org/">Computing in Science and Engineering</a>: a joint publication
between the AIP (American Institute of Physics) and the IEEE, it is a
magazine-style journal and it can be seen in many coffee rooms of
computational-science departments. Thanks to that it gets a lot of
reading, but the articles cannot be too technical (which might be a
good thing) and there is room for only few articles.</li>
<li><a class="reference external" href="http://www.openresearchcomputation.com/">Open Research Computation (ORC)</a>: A newly-created journal, with
a focus on making computational research reproducible. As such, it
favors papers about open source scientific software with good
software-engineering. <strong>Open access</strong>.</li>
</ul>
<p>In addition to these software-friendly journals, some large-scope
journals on computational science sometime accept software papers,
though software production fall out of their scope:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.elsevier.com/locate/jocs/">Journal of Computational Science</a>: a very multidisciplinary
journal.</li>
<li><a class="reference external" href="http://www.siam.org/journals/sisc.php">SIAM Journal on Scientific Computing (SISC)</a>: a journal of the
SIAM (society for industrial and applied mathematics), thus with a
focus on engineering-type applications.</li>
</ul>
</div>
<div class="section" id="journals-for-publishing-domain-specific-scientific-software">
<h2>Journals for publishing domain-specific scientific software</h2>
<p>It is usually easier to publish a domain-specific software contribution,
as you can claim that you have solved a well-identified scientific
roadblock. Until recently, it was hard to get such papers in the best
journals of a community, but things have been changing.</p>
<ul class="simple">
<li><a class="reference external" href="http://www.elsevier.com/locate/cpc">Computer Physics Communications</a>: for algorithms and packages
solving numerical and computational problems related to physics.</li>
<li><a class="reference external" href="http://bioinformatics.oxfordjournals.org/">Bioinformatics</a>: accepts software papers on biology-related
problems.</li>
<li><a class="reference external" href="http://toms.acm.org/">ACM Transactions On Mathematical Software (TOMS)</a>: a journal of
the ACM (Association for Computing Machinery), thus with a focus on
algorithms.</li>
<li><a class="reference external" href="http://www.jstatsoft.org/">Journal of statistical Software</a>: this journal comes from the
community of people who wrote the R language. They know that open
source scientific software is hard and important. <strong>Open access</strong>.</li>
<li><a class="reference external" href="http://jmlr.csail.mit.edu/mloss/">Journal of Machine Learning Research (JMLR), Machine Learning Open
Source (MLOSS) track</a>: reference journal in the machine learning
community, the MLOSS track cares strongly about documentation,
packaging and usability of the software. <strong>Open access</strong>.</li>
<li><a class="reference external" href="http://www.elsevier.com/wps/find/journaldescription.cws_home/398/description#description">Computers & Geoscience</a>: computational geoscience journal that
accepts software papers (thanks Michael Aye for the pointer).</li>
<li><a class="reference external" href="http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291099-0542">Computer Applications in Engineering Education</a>: a journal
about education with computers. AFAIK, no special focus on open
source or software-engineering quality (thanks Doug Holton for the
pointer).</li>
<li><a class="reference external" href="http://www.springer.com/biomed/neuroscience/journal/12021">NeuroInformatics</a> and <a class="reference external" href="http://www.frontiersin.org/neuroinformatics">Frontiers NeuroInformatics</a> (<strong>open
access</strong>): two journals on computer-related issues in neuroscience
that accept software papers. I have the feeling that the latter is a
bit warmer to open source that the former (thanks Andrew Davison for
the pointer).</li>
<li><a class="reference external" href="http://www.elsevier.com/wps/find/journaldescription.cws_home/503304/description#description">Computers and Electronics in Agriculture</a>: for publishing
agriculture-related software (thanks John B. Cole for the pointer).</li>
</ul>
<p>I should stress that, in my opinion, journals such as <a class="reference external" href="http://www.ploscompbiol.org">PLOS
computational biology</a>, or the <a class="reference external" href="http://www.elsevier.com/wps/find/journaldescription.cws_home/622866/description#description">Journal of Computational Physics</a>, or
are not great venues for software papers, as they tend to emphasize what
I would call <em>proof of principle</em>, and not packaged and maintained
software.</p>
<p><strong>I have the feeling that there is need for more communication on
scientific software. The list above is, of course, incomplete. If you
have extra ideas, please do not hesitate to contact me.</strong></p>
<p>As a conclusion, I would like to point out that conferences are also a
good way to advertise scientific software. You may even get approached
by the editor of a journal to open the door for a journal article. Last
year I was at <a class="reference external" href="http://hpfem.org/events/esco-2010/">ESCO</a>, a coupled problems conference, and there was a
track on Python in science. All in all the conference was a huge amount
of fun, and I learned a lot on practical aspects of numerical methods,
given the amount of numerical computing geeks that were around. The same
community is organizing <a class="reference external" href="http://hpfem.org/events/femtec-2011/">FEMTEC</a> in Lake Tahoe (California) this year.
If you are in any field related to FEM or multiphysics, you should
consider it.</p>
<p><strong>Update: added links suggested by Doug Holton, Michael Aye, Andrew
Davison, and John B. Cole</strong></p>
</div>
ICA versus PCA in the scikit-learn: the value of code over pictures2010-11-20T16:12:00+01:002010-11-20T16:12:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-11-20:/programming/ica-versus-pca-in-the-scikit-learn-the-value-of-code-over-pictures.html<p>When I was trying to get an intuitive feeling of the difference between
<strong>Independent Component Analysis</strong> (ICA) and <strong>Principal Component
Analysis</strong> (PCA), I wrote a few Python scripts producing <a class="reference external" href="http://gael-varoquaux.info/scientific_computing/ica_pca/index.html">some
visualizations explaining the difference</a> that have had a bit of
success.</p>
<p>During the last sprint on <a class="reference external" href="http://scikit-learn.org">scikit-learn</a>, a machine learning …</p><p>When I was trying to get an intuitive feeling of the difference between
<strong>Independent Component Analysis</strong> (ICA) and <strong>Principal Component
Analysis</strong> (PCA), I wrote a few Python scripts producing <a class="reference external" href="http://gael-varoquaux.info/scientific_computing/ica_pca/index.html">some
visualizations explaining the difference</a> that have had a bit of
success.</p>
<p>During the last sprint on <a class="reference external" href="http://scikit-learn.org">scikit-learn</a>, a machine learning
toolkit in Python, we cleaned up the ICA code that I had been using, and
we added it to the scikit, along with <a class="reference external" href="http://scikit-learn.org/stable/auto_examples/decomposition/plot_ica_vs_pca.html">an example</a> inspired from this
earlier toy problem.</p>
<a class="reference external image-reference" href="http://scikit-learn.org/stable/auto_examples/decomposition/plot_ica_vs_pca.html"><img alt="" class="align-center" src="http://scikit-learn.org/stable/_images/sphx_glr_plot_ica_vs_pca_001.png" /></a>
<p>While the pictures are not as pretty as the initial ones I had done
(because we wanted to keep the example as simple as possible), I am very
happy that this discussion is know more than a set of static pictures,
but comes with runnable code.</p>
<p>This illustrates very well my feelings on the future of scientific code
and scientific research: paper, books, teaching materials, on numerical
methods or computational science are greatly enhanced when they come
with highly-readable code that illustrates their purpose, because the
reader can start asking questions to the algorithm. Hopefully, <strong>the
documentation of scientific programming toolkits will become the
textbooks of tomorrow</strong>. We still have a lot of work to.</p>
<hr class="docutils" />
<p>It’s funny, I just realized that my vision on software might have been
strongly influenced by the fact that my mother, a high-school math
teacher, spent endless nights when I was a teenager working on
<a class="reference external" href="http://fr.wikipedia.org/wiki/G%C3%A9oplan">Geoplan</a>, a software for teaching geometry by interaction with
figures.</p>
Multitouch with VTK (and MedINRIA and Mayavi)2010-09-18T09:40:00+02:002010-09-18T09:40:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-09-18:/programming/multitouch-with-vtk-and-medinria-and-mayavi.html<p>If the videos on this post are not showing, click through to see
them.</p>
<p>A colleague of mine, <a class="reference external" href="http://sites.google.com/site/pierrefillard/">Pierre Fillard</a>, has just integrated multitouch
in the next generation of the VTK-based medical imaging software
<a class="reference external" href="http://www-sop.inria.fr/asclepios/software/MedINRIA/">MedINRIA</a>. The nice thing is that it works on an Apple laptop out of
the box …</p><p>If the videos on this post are not showing, click through to see
them.</p>
<p>A colleague of mine, <a class="reference external" href="http://sites.google.com/site/pierrefillard/">Pierre Fillard</a>, has just integrated multitouch
in the next generation of the VTK-based medical imaging software
<a class="reference external" href="http://www-sop.inria.fr/asclepios/software/MedINRIA/">MedINRIA</a>. The nice thing is that it works on an Apple laptop out of
the box.</p>
<p>
<object width="640" height="385">
<embed src="http://www.youtube.com/v/UyO4KRnYreU?fs=1&hl=en_US" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="640" height="385">
</embed>
</object>
</p><p>On <a class="reference external" href="https://sites.google.com/site/pierrefillard/coding-blog/multi-touchgesturesinvtk">his blog</a>, he explain how he did this (warning, it involves C++ and
VTK programming). <strong>He also gives the code for this!</strong> Enjoy.</p>
<p>This reminded me of when the <a class="reference external" href="http://www.enthought.com/">Enthought guys</a> had rigged up a large
multitouch screen and wired it in <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a> for 3D plotting, and in
<a class="reference external" href="http://code.enthought.com/projects/chaco/">chaco</a> for 2D plotting, using only a web-cam, a video projector, and
pure Python image-analysis code:</p>
<p>
<object width="480" height="385">
<embed src="http://www.youtube.com/v/bEf3nGjOgpU?fs=1&hl=en_US" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="385">
</embed>
</object>
</p>Machine learning humour2010-09-16T23:11:00+02:002010-09-16T23:11:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-09-16:/science/machine-learning-humour.html<div class="section" id="yes-but-they-overfit">
<h2>Yes, but they overfit</h2>
<p>If you are reading this post through a planet, the movie isn’t showing
up, just <a class="reference external" href="http://gael-varoquaux.info/science/machine-learning-humour.html">click through</a> to understand what the hell this is about.</p>
<p>
<object width="480" height="385">
<embed src="http://www.youtube.com/v/m60lVGz34hU?fs=1&hl=en_US" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="385">
</embed>
</object>
</p></div>
<div class="section" id="some-explanations">
<h2>Some explanations…</h2>
<div class="section" id="machine-learning-geeks-and-beers">
<h3>Machine learning, geeks, and beers</h3>
<p>Sorry for the bad humour. In the previous weeks my social geek life …</p></div></div><div class="section" id="yes-but-they-overfit">
<h2>Yes, but they overfit</h2>
<p>If you are reading this post through a planet, the movie isn’t showing
up, just <a class="reference external" href="http://gael-varoquaux.info/science/machine-learning-humour.html">click through</a> to understand what the hell this is about.</p>
<p>
<object width="480" height="385">
<embed src="http://www.youtube.com/v/m60lVGz34hU?fs=1&hl=en_US" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="385">
</embed>
</object>
</p></div>
<div class="section" id="some-explanations">
<h2>Some explanations…</h2>
<div class="section" id="machine-learning-geeks-and-beers">
<h3>Machine learning, geeks, and beers</h3>
<p>Sorry for the bad humour. In the previous weeks my social geek life had
two strong moments:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.pycon.fr/conference/edition2010">Pycon fr</a>, the French Python conference, and ensuing drinking</li>
</ul>
<img alt="" src="http://farm5.static.flickr.com/4077/4938486734_378f52fd3d.jpg" style="width: 45%;" />
<img alt="" src="http://farm5.static.flickr.com/4114/4938124265_027853c81a.jpg" style="width: 45%;" />
<ul class="simple">
<li><a class="reference external" href="http://fseoane.net/blog/2010/second-scikitslearn-coding-sprint/">The second sprint</a> on the <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit learn</a>, a library for machine
learning in Python.</li>
</ul>
<p>At the first event (or maybe the related drinking) there was a lot of
discussion about NoSQL databases, and I was introduced to <a class="reference external" href="http://www.xtranormal.com/watch/6995033/""">this
fantastic video</a> making fun of MongoDB fanboys. A few days later I was
hacking on the scikit, comparing estimators and discussing hype versus
fact in machine learning algorithms (hint: <a class="reference external" href="http://en.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization">there is no free lunch</a>,
but you may get <a class="reference external" href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.79.2501&rep=rep1&type=pdf">a free brunch</a>). As in brain imaging people seem to
be doing nothing but SVMs over and over while <a class="reference external" href="http://hal.inria.fr/hal-00504095/PDF/icpr_2010_tv.pdf">methods with more
appropriate sparsity clearly perform better</a>, I composed this stupid
video.</p>
</div>
<div class="section" id="anything-to-learn-about-machine-learning-in-there">
<h3>Anything to learn about machine learning in there?</h3>
<p>The short answer is: probably no. This video is humour, and there is
little truth (well, RFE is indeed slow as a dog). However, not every
reader of this blog are machine learning experts, so let me explain the
stakes of the pseudo discussion.</p>
<p><strong>Overfitting</strong>: when you learn a predictive model on a noisy data set,
for instance trying to learn how to predict whether a movie is popular
or not from ratings, if you have a finite amount of data, you should be
careful not to learn by heart every detail of the data. You will learn
noise that, by chance, correlated to what you are trying to predict.
When you try to generalize to new data, these features that you learned
from noise will be detrimental to your prediction performance. For
instance, <a class="reference external" href="http://www.reddit.com/r/Python/comments/cwq37/announcing_python_nltk_demos_natural_language/">the presence of Matt Damon</a> is not the sole predictor of the
quality of movie. This is called overfitting. The goal of
<a class="reference external" href="http://en.wikipedia.org/wiki/Regularization_%28mathematics%29">regularization</a> is to avoid this overfitting.</p>
<p>Both SVM and elasticnet implement regularization, but in different ways.
In the case of brain imaging, as the predictive features (voxels) are
very sparse, but the noise is highly structured, SVM (that do not
operate on voxels directly) are not able to select directly the relevant
voxels and tend to overfit (which can be counter-balanced by univariate
feature selection as in the <a class="reference external" href="http://scikit-learn.org/stable/auto_examples/svm/plot_svm_anova.html">scikit example</a>).</p>
<p><strong>RFE (recursive feature elimination) is slow as dog</strong></p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">scikits.learn</span> <span class="kn">import</span> <span class="n">datasets</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="n">digits</span> <span class="o">=</span> <span class="n">datasets</span><span class="o">.</span><span class="n">load_digits</span><span class="p">()</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="n">X</span> <span class="o">=</span> <span class="n">digits</span><span class="o">.</span><span class="n">data</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="n">y</span> <span class="o">=</span> <span class="n">digits</span><span class="o">.</span><span class="n">target</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">scikits.learn.svm</span> <span class="kn">import</span> <span class="n">LinearSVC</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="n">svc</span> <span class="o">=</span> <span class="n">LinearSVC</span><span class="p">()</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">7</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">scikits.learn.rfe</span> <span class="kn">import</span> <span class="n">RFE</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="o">%</span><span class="n">timeit</span> <span class="n">RFE</span><span class="p">(</span><span class="n">estimator</span><span class="o">=</span><span class="n">svc</span><span class="p">,</span> <span class="n">n_features</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">percentage</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="mi">1</span> <span class="n">loops</span><span class="p">,</span> <span class="n">best</span> <span class="n">of</span> <span class="mi">3</span><span class="p">:</span> <span class="mf">21.5</span> <span class="n">s</span> <span class="n">per</span> <span class="n">loop</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">scikits.learn.glm</span> <span class="kn">import</span> <span class="n">ElasticNet</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="o">%</span><span class="n">timeit</span> <span class="n">ElasticNet</span><span class="p">(</span><span class="n">alpha</span><span class="o">=</span><span class="mf">.1</span><span class="p">,</span> <span class="n">rho</span><span class="o">=</span><span class="mf">0.7</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="mi">10</span> <span class="n">loops</span><span class="p">,</span> <span class="n">best</span> <span class="n">of</span> <span class="mi">3</span><span class="p">:</span> <span class="mf">26.7</span> <span class="n">ms</span> <span class="n">per</span> <span class="n">loop</span>
</pre></div>
<p>Yeah, but it does much more than simply building a predictor, it builds
a ‘heat map’ of which features help predicting (run <a class="reference external" href="http://scikit-learn.sourceforge.net/auto_examples/rfe_digits.html">this scikit-learn
example</a> to get an idea).</p>
<p>I am afraid that all the examples I pointed to require the development
version of the scikit. Sorry, we just finished a sprint, and there will
be a release soon.</p>
</div>
</div>
Scikit Learn coding sprint2010-09-04T17:43:00+02:002010-09-04T17:43:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-09-04:/programming/scikit-learn-coding-sprint.html<p>We have been really crap at communicating the next <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>
coding sprint. It’s next week!</p>
<p>The coding sprint will take place the 8 and 9 September at <a class="reference external" href="http://maps.google.fr/maps/place?oe=utf-8&rls=com.mandriva:en-US:official&client=firefox-a&um=1&ie=UTF-8&q=inria+saclay&fb=1≷=fr&hq=inria&hnear=Saclay&cid=14838681423181723946">INRIA
Saclay</a>, near Paris, in the room K110 (building K).</p>
<p>For those who cannot make it, it will be possible to participate …</p><p>We have been really crap at communicating the next <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>
coding sprint. It’s next week!</p>
<p>The coding sprint will take place the 8 and 9 September at <a class="reference external" href="http://maps.google.fr/maps/place?oe=utf-8&rls=com.mandriva:en-US:official&client=firefox-a&um=1&ie=UTF-8&q=inria+saclay&fb=1≷=fr&hq=inria&hnear=Saclay&cid=14838681423181723946">INRIA
Saclay</a>, near Paris, in the room K110 (building K).</p>
<p>For those who cannot make it, it will be possible to participate using
the IRC chan (#scikit-learn on irc.freenode.net).</p>
<p>We will start at 9am (Paris time), and a sketch of the planning can be
found <a class="reference external" href="http://sourceforge.net/apps/trac/scikit-learn/wiki/SprintPlanning">here</a>. In particular:</p>
<ul class="simple">
<li>More docs! we still need tutorials: features selection, model
selection, cross-validation, etc..</li>
<li>Make the <a class="reference external" href="http://github.com/scikit-learn/scikit-learn/blob/master/scikits/learn/pipeline.py">pipeline object</a> really work + illustration in different
contexts.</li>
<li>Clean up and doc for bayesian approaches.</li>
<li>Implementation of PCA (fit + transform).</li>
<li>FastICA (adapt the <a class="reference external" href="http://github.com/GaelVaroquaux/canica/blob/master/canica/algorithms/fastica.py">CanICA</a> code)</li>
<li>LDA : Covariance estimators (Ledoit-Wolf) and add transform.</li>
<li><a class="reference external" href="http://github.com/scikit-learn/scikit-learn/blob/master/scikits/learn/preprocessing.py">Preprocessing routines</a> (center, standardize) with fit transform.</li>
<li>Anything that you have a particular interest in.</li>
</ul>
<p>Do not hesitate to send on the <a class="reference external" href="https://lists.sourceforge.net/lists/listinfo/scikit-learn-general">mailing list</a> some advices on this
(incomplete…) list, and see you next week!</p>
<hr class="docutils" />
<p><a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a> is a Python module for efficient and easy machine
learning using scipy and numpy.</p>
SVG Word map of countries2010-08-24T10:55:00+02:002010-08-24T10:55:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-08-24:/misc/svg-word-map-of-countries.html<p>To be able to visualize some quantities attached to countries all over
the world, I needed a image with various countries color-coded. The
fantastic <a class="reference external" href="http://matplotlib.sourceforge.net/basemap/doc/html/">matplotlib basemap package</a> was not an option as I really
needed a static image.</p>
<p>So I generated an SVG image with all the countries. It was …</p><p>To be able to visualize some quantities attached to countries all over
the world, I needed a image with various countries color-coded. The
fantastic <a class="reference external" href="http://matplotlib.sourceforge.net/basemap/doc/html/">matplotlib basemap package</a> was not an option as I really
needed a static image.</p>
<p>So I generated an SVG image with all the countries. It was generating by
tracing a bitmap, so it has a lot of imperfections, but being an SVG
with each (major) country as a different object, it can be used to
create a colored-code world map. I am uploading it here under a
public-domain license. Enjoy!</p>
<p><strong>PNG</strong></p>
<img alt="" src="../images/misc/countries.png" style="width: 50%;" />
<p><strong>SVG</strong>: <a class="reference external" href="../images/misc/countries.svg">countries.svg</a></p>
<!-- _ -->
Software design for maintainability2010-08-01T23:47:00+02:002010-08-01T23:47:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-08-01:/programming/software-design-for-maintainability.html<p>I have just spent the best part of my Sunday fixing a bug that turned
out being a <a class="reference external" href="https://svn.enthought.com/enthought/changeset/25699/">seemingly-trivial two-liner</a>. Such unpleasant experiences
are all too frequent, and weight a lot on my view of code design.</p>
<div class="section" id="my-stance-on-code-design">
<h2>My stance on code design</h2>
<img alt="" class="align-right" src="https://gael-varoquaux.info/programming/attachments/software_design_for_maintainability/cool-car-drawing-5.jpg" style="width: 30%;" />
<p>I call <em>code design</em> the process of designing …</p></div><p>I have just spent the best part of my Sunday fixing a bug that turned
out being a <a class="reference external" href="https://svn.enthought.com/enthought/changeset/25699/">seemingly-trivial two-liner</a>. Such unpleasant experiences
are all too frequent, and weight a lot on my view of code design.</p>
<div class="section" id="my-stance-on-code-design">
<h2>My stance on code design</h2>
<img alt="" class="align-right" src="https://gael-varoquaux.info/programming/attachments/software_design_for_maintainability/cool-car-drawing-5.jpg" style="width: 30%;" />
<p>I call <em>code design</em> the process of designing the architecture of a
piece of software: what are the objects it uses? how do they interact?
how is the information passed around?…</p>
<p>My view of code design and software engineering has progressively
evolved to favor <strong>extreme simplicity</strong> over sophistication. I believe
that a good programmer should know <a class="reference external" href="http://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29">design patterns</a>, <a class="reference external" href="http://gael-varoquaux.info/computers/python_advanced/index.html">powerful
language features</a>, <a class="reference external" href="http://scipy2010.blogspot.com/2010/06/tutorials-day-1-advanced-numpy.html">libraries dark corners</a>, and <em>not use them unless
absolutely necessary</em>.</p>
</div>
<div class="section" id="some-rules-of-thumb">
<h2>Some rules of thumb</h2>
<p>Here are some rules that I apply nowadays when writing code that I would
like to last (I am aware that some of them go against well-advertised
best practices).</p>
<ul class="simple">
<li><strong>Keep it as simple a possible, really!</strong> Experimental results have
shown that the tractability of a code base goes down as the square of
the number of interactions, and thus much quicker than the number of
lines in a project. Each time you add a line, think about it: can you
make simpler? If not you’ll have to find resources to maintain your
project as fixing bugs or adding features will grow harder.</li>
<li><strong>Design for the 80% usecases.</strong> In the same vein, a small decrease
in the requirements can make your project much simpler
<a class="reference external" href="http://ieeexplore.ieee.org/Xplore/login.jsp?url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F32%2F35909%2F01702600.pdf%3Farnumber%3D1702600&authDecision=-203">[Woodfield1979]</a>. Corner cases and minor usecases should not make
the whole project complex and hard to maintain. If you can, give up
on what is bringing in complexity. If you cannot, isolate it, and
don’t let it sit at the core of your design.</li>
<li><strong>Don’t design for the future.</strong> Again the same core idea: don’t
start planing for all the usecases, and all the difficulties that you
haven’t encountered, you will most certainly design wrong, and
chances are that you’ll add complexity that you do not use. Design
simple, design cleanly and refactor as you go, based on concrete
problems. This is known as the <a class="reference external" href="http://en.wikipedia.org/wiki/You_aren't_gonna_need_it">“YAGNI principle”</a>.</li>
</ul>
<img alt="" class="align-center" src="https://gael-varoquaux.info/programming/attachments/software_design_for_maintainability/howtobuildmvp.gif" style="width: 60%;" />
<ul class="simple">
<li><strong>Don’t be clever.</strong> Each time you do a clever trick, whoever has to
read and maintain this code will have to understand it (that person
may be you, in a few years). Chances are that they’ll get it wrong
and start by loosing a lot of time.</li>
<li><strong>Repeating yourself may actually be OK.</strong> This is a case of
<em>practicality beats purity</em>. Repeating code is really a bad thing in
software design, because it leads to an increased number of lines to
debug, and tends to hinder reusability. However, adding complexity in
order to save a few lines of duplicated code will cost you more in
the long run.</li>
<li><strong>Use objects sparingly.</strong> Object are great, but are they always
need? An object with a single method <em>eval</em> can probably simply be
implemented by a function. The limitation of objects is that they all
have a different behavior. As a result, the users and maintainers of
your codebase will first have to understand how all your classes
interact before understanding your code. This also means that there
is a lot of benefit in making many different classes that have the
same interface.</li>
<li><strong>Avoid abstractions and levels of indirection.</strong> The more levels of
code piled on top one of the other, the more layers your maintainer
is going to have to inspect to find were the bug might be. An
abstraction hides another object or algorithm. To debug code, chances
are that all the black boxes will first have to be opened.</li>
</ul>
</div>
<div class="section" id="coding-for-others-to-debug">
<h2>Coding for others to debug</h2>
<blockquote class="epigraph">
“Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it.” - Brian W. Kernighan</blockquote>
<img alt="" class="align-right" src="https://gael-varoquaux.info/programming/attachments/software_design_for_maintainability/auto-graveyard-1.jpg" style="width: 40%;" />
<p>You may think that I am overemphasizing simplicity at the cost of
functionality. Well, think about the future of your code. The net is
full of unmaintained and abandoned code. If you want your project to
grow and have a future, you will probably need people to help you. For a
given purpose, the easiest the code is to read and debug, the more
chances you will have to pick momentum.</p>
<hr class="docutils" />
<p>Some external references I like (about software engineering, rather than
debugging):</p>
<ul class="simple">
<li>Edmon Lau: <a class="reference external" href="http://www.theeffectiveengineer.com/blog/hidden-costs-that-engineers-ignore">Hidden costs that engineers ignore</a>
(<strong>Read this</strong>)</li>
<li>Titus Brown: <a class="reference external" href="http://ivory.idyll.org/blog/sep-07/not-sucking-v2">Writing (Python) Code that Doesn’t Suck</a></li>
<li>Peter Norvig: <a class="reference external" href="http://norvig.com/21-days.html">Teach yourself programming in 10 years</a></li>
<li>Paul Stachour and David Collier-Brown: <a class="reference external" href="http://cacm.acm.org/magazines/2009/11/48444-you-dont-know-jack-about-software-maintenance/fulltext">You Don’t Know Jack About
Software Maintenance</a></li>
<li>Greg Wilson: <a class="reference external" href="http://software-carpentry.org/">Software carpentry: a course in software engineering</a></li>
</ul>
</div>
Sprint Scikit learn in Paris2010-07-23T14:31:00+02:002010-07-23T14:31:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-07-23:/programming/sprint-scikit-learn-in-paris.html<p>We are organizing a coding sprint in Paris on <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit learn</a>,
<strong>machine learning in Python</strong>. The goal of this sprint is to set the
API and the general coding guidelines of the scikit to be able to tackle
many different statistical learning problems in a consistent framework.</p>
<p>This is why …</p><p>We are organizing a coding sprint in Paris on <a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit learn</a>,
<strong>machine learning in Python</strong>. The goal of this sprint is to set the
API and the general coding guidelines of the scikit to be able to tackle
many different statistical learning problems in a consistent framework.</p>
<p>This is why we would like to have people with different problems,
applications, and backgrounds to pitch in.</p>
<p>It will be a two-days sprint. Everyone is welcome, so just fill in the
<a class="reference external" href="http://www.doodle.com/4cqxnhuq5rr4qzn5">doodle</a>, so that we can choose the date?</p>
<p>And do not hesitate to suggest some topics that you would like to be
addressed during the sprint, and to discuss them on the <a class="reference external" href="https://lists.sourceforge.net/lists/listinfo/scikit-learn-general">mailing-list</a>!</p>
<p><a class="reference external" href="http://parietal.saclay.inria.fr/Members/vincent-michel">Vincent Michel</a> is organizing the sprint. If you have questions about
the sprint, you are welcomed to contact me, but please do put him in Cc
to.</p>
Simple object signatures2010-07-16T23:31:00+02:002010-07-16T23:31:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-07-16:/programming/simple-object-signatures.html<div class="section" id="a-signature-pattern">
<h2>A <em>signature</em> pattern</h2>
<p>There are many libraries around to specify what I call a <em>‘signature’</em>
for an object, in other words a list of attributes that define its
parameter set. I have heavily used <a class="reference external" href="http://code.enthought.com/projects/traits">Enthought’s Traits library</a> for
this purpose, but the concept is fairly general and can be …</p></div><div class="section" id="a-signature-pattern">
<h2>A <em>signature</em> pattern</h2>
<p>There are many libraries around to specify what I call a <em>‘signature’</em>
for an object, in other words a list of attributes that define its
parameter set. I have heavily used <a class="reference external" href="http://code.enthought.com/projects/traits">Enthought’s Traits library</a> for
this purpose, but the concept is fairly general and can be found <em>eg</em> in
ORMs (Object Relational Mappers) or web frameworks.</p>
<p>Specification of this interface of parameters may be used to answer a
variety of needs:</p>
<ul class="simple">
<li><strong>Typing</strong>: in the case of an ORM, to generate UIs, or for better
error management, it may be desirable to have some control on the
types of certain attributes of an object. In this case, specifying
the signature corresponds to laying out a <strong>data model</strong> for the
object.</li>
<li><strong>Reactive programming</strong>: using properties to react to changes to
attributes, one can fully specify the API of an object in terms of
these attributes. This gives a message-passing like programming style
that can be very well suited to parallel-computing in particular
because it can easily be made thread-safe.</li>
</ul>
</div>
<div class="section" id="signatures-for-statistical-learning-objects">
<h2>Signatures for statistical learning objects</h2>
<p>Recently, I considered the <em>signature</em> pattern in a new context. In the
<a class="reference external" href="http://scikit-learn.sourceforge.net/">scikit-learn</a>, we are interested in statistical learning. This entails
fitting models to data and often tuning parameters to select a model
that fits best (a problem called <em>model selection</em>). Each of our models
is an object that implements a couple of key methods to fit to the data
and to apply to new data (<em>fit</em> and <em>predict</em>).</p>
<p>The approach that we are currently taking for model selection is (more
or less) to generate a list of models with different parameters and fit
and test them on the data.</p>
<p>A very nice feature would be to find out the parameters to vary simply
by inspecting the objects, and such a desire recently got us
<a class="reference external" href="http://sourceforge.net/mailarchive/forum.php?thread_name=201007050958.16199.matthieu.perrot%40cea.fr&forum_name=scikit-learn-general">discussing</a> of defining <em>signatures</em> for our objects. I must confess
that I am a bit weary as this means either depending on a signature
library, or building one. We don’t want to grow our dependencies, and
most signature-definition code that I know involve meta-programming
tricks to avoid code duplication.</p>
</div>
<div class="section" id="solving-the-simple-problem-avoiding-type-checking">
<h2>Solving the simple problem: avoiding type checking</h2>
<p>Today, I had to bite the bullet, because we were in a situation in which
we had to instantiate new models from the existing one during model
selection. For technical reasons, using a <em>copy.copy</em> to create these
new models was not a great idea, and it was better to have the minimal
list of parameters required. Here come signatures again.</p>
<p>After a bit of messing around with the code, I realized that typing
information was useless, and most probably harmful, to our immediate
goals and that I just needed the names of the relevant attributes. I
finally settled down to the following solution (which might still
change):</p>
<ul class="simple">
<li>All parameters need to be specified as keyword arguments of the
<em>__init__</em>. The <em>__init__</em> may not have positional arguments
or ‘*’ arguments. Attributes on the objects have the same names as
the <em>__init__</em> parameters.</li>
<li>A simple base class, with couple of methods relying on a simple use
of the <em>inspect</em> module to find the signature of the <em>__init__</em>.</li>
</ul>
<hr class="docutils" />
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">BaseEstimator</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">_get_param_names</span><span class="p">(</span><span class="bp">cls</span><span class="p">):</span>
<span class="n">args</span><span class="p">,</span> <span class="n">varargs</span><span class="p">,</span> <span class="n">kw</span><span class="p">,</span> <span class="n">default</span> <span class="o">=</span> <span class="n">inspect</span><span class="o">.</span><span class="n">getargspec</span><span class="p">(</span><span class="bp">cls</span><span class="o">.</span><span class="fm">__init__</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">varargs</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">,</span> <span class="p">(</span>
<span class="s1">'scikit learn estimators should always specify their '</span>
<span class="s1">'parameters in the signature of their init (no varargs).'</span>
<span class="p">)</span>
<span class="c1"># Remove 'self'</span>
<span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="k">return</span> <span class="n">args</span>
<span class="k">def</span> <span class="nf">_get_params</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">out</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_param_names</span><span class="p">():</span>
<span class="n">out</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">)</span>
<span class="k">return</span> <span class="n">out</span>
<span class="k">def</span> <span class="nf">_set_params</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">params</span><span class="p">):</span>
<span class="n">valid_params</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_param_names</span><span class="p">()</span>
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">params</span><span class="o">.</span><span class="n">iteritems</span><span class="p">():</span>
<span class="k">assert</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">valid_params</span><span class="p">,</span> <span class="p">(</span><span class="s1">'Invalid parameter </span><span class="si">%s</span><span class="s1"> '</span>
<span class="s1">'for estimator </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span>
<span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="p">))</span>
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
</pre></div>
<p>The full code can be seen <a class="reference external" href="attachments/base_estimator.py">here</a> and adds a bit more features, such as
a clever <em>__repr__</em>.</p>
<p>What I like about this solution is that it (almost) does not use
metaprograming, and avoids code duplication without forcing any specific
pattern on the developer subclassing <em>BaseEstimator</em>.</p>
</div>
<div class="section" id="the-next-step">
<h2>The next step</h2>
<p>This approach solves my immediate problem, but not the bigger one of
finding what values can the different parameters take when varied for
model selection. Of course this second problem is much more complicated,
and maybe it is not worth solving it: the framework could very easily be
bringing in more problems than it solves.</p>
<p>However, it seems that a fairly easy way of specifying possible values
for parameters would be to decorate the <em>__init__</em>, giving the
possible parameters to be tested during the model selection:</p>
<div class="highlight"><pre><span></span><span class="nd">@cv_params</span><span class="p">(</span><span class="n">l1</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">logspace</span><span class="p">(</span><span class="mf">1e-4</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">l1</span><span class="o">=</span><span class="mf">.5</span><span class="p">,</span> <span class="n">fit_intercept</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="c1"># ...</span>
</pre></div>
<p>All the decorator has to do is to store the information in an attribute
attached to the <em>__init__</em> (and probably to check that the
parameters it was given are valid arguments, in order to raise errors
early). Methods on the class can later inspect this information for
model selection, or GUI building (data-model specification will probably
require some typing language, rather than a simple list of possible
parameters).</p>
<p>Once again, here we would be avoiding the difficulty of specifying type
information in a non restrictive way, but avoiding a problem that we
don’t have to solve is probably a good idea.</p>
</div>
Euroscipy 2010: code, science, and a lot of fun2010-07-13T17:31:00+02:002010-07-13T17:31:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-07-13:/programming/euroscipy-2010-code-science-and-a-lot-of-fun.html<p><a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2010">Euroscipy 2010</a>, the third European conference for the use of Python in
science, is just over, and I think it was a great success.</p>
<div class="section" id="euroscipy-in-numbers">
<h2>Euroscipy in numbers</h2>
<p><img alt="image0" src="http://farm5.static.flickr.com/4118/4779625445_0e783484cd_m_d.jpg" /></p>
<p>The attendance this year was huge: there was a grand total of 160 who
came to EuroScipy, with 140 that came only to …</p></div><p><a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2010">Euroscipy 2010</a>, the third European conference for the use of Python in
science, is just over, and I think it was a great success.</p>
<div class="section" id="euroscipy-in-numbers">
<h2>Euroscipy in numbers</h2>
<p><img alt="image0" src="http://farm5.static.flickr.com/4118/4779625445_0e783484cd_m_d.jpg" /></p>
<p>The attendance this year was huge: there was a grand total of 160 who
came to EuroScipy, with 140 that came only to the tutorials, and 130 only
the conference. This up by almost a factor of 3 compared to last year’s
EuroScipy, more than last year’s SciPy conference in Passadena, and
almost as much as this year’s SciPy conference in Austin that hosted 180
person. We had people coming from 16 country, and as far as New Zealand,
the US, or Turkey. Research lab, education, and industry (small to large
companies) were all well represented, with approximately a third of the
delegates coming from the industry. Similarly, many different scientific
field were discussed, ranging from landscape ecology to pure math.</p>
<p>There were 2 tutorial tracks with 10 tutorial slots in each track. We
had 2 keynotes from Hans Petter Langtangen and Konrad Hinsen. With
regards to the contributed talks, the conference this year was highly
selective. We received 52 propositions. We unfortunately could accept
only 30 of them, which corresponds to an acceptance rate of 58%.
Finally, we had 18 <a class="reference external" href="http://www.euroscipy.org/talk/937">lightning talks</a>.</p>
</div>
<div class="section" id="a-warm-and-friendly-atmosphere">
<h2>A warm and friendly atmosphere</h2>
<p><img alt="image1" src="http://farm5.static.flickr.com/4097/4774499149_5dda469dc2_m.jpg" /></p>
<p>As an organizer, I was really pleased to find out how much people were
relaxed and friendly. This certainly facilitates discussions during the
breaks. And the ambiance was undoubtedly warm: 140 people with laptops
in a room without air conditioning in the Paris summer :).</p>
<p>Of course during the evenings, many people met to continue the
passionate discussions in restaurants and bars.</p>
</div>
<div class="section" id="trends-i-noticed">
<h2>Trends I noticed</h2>
<p>What one remembers from a conference is obviously biased by personal
interests. With that disclaimer, here are the recurrent and important
topics that I noticed, both in the talks, but also in the coffee break
discussions:</p>
<ul class="simple">
<li><strong>Parallel computing</strong>, in particular making it easy to do parallel
computing. <a class="reference external" href="http://www.euroscipy.org/talk/2011">Konrad’s keynote</a> had many interesting directions to
explore. (talks: <a class="reference external" href="http://www.euroscipy.org/talk/2009">Playdoh</a>, <a class="reference external" href="http://www.euroscipy.org/talk/1686">DANA</a>).</li>
<li><strong>Code generation</strong>. In the various conferences I have been to
recently, I heard much talking about symbolic manipulation of
numerical problems to generate optimal computing kernels (talks:
<a class="reference external" href="http://www.euroscipy.org/talk/1657">Efficient computation tutorial</a>, <a class="reference external" href="http://www.euroscipy.org/talk/1666">Theano</a>, <a class="reference external" href="http://www.euroscipy.org/talk/2045">Algorithmic
Differentiation</a>.</li>
<li><strong>Data management</strong>, with problems such as provenance tracking for
reproducibility (talks: <a class="reference external" href="http://www.euroscipy.org/talk/1960">Sumatra</a>, <a class="reference external" href="http://www.euroscipy.org/talk/880">Knowledge management
tutorial</a>).</li>
</ul>
<p>Finally installation problems of scientific tools were the subject of
many discussions, as each year. One thing that I did notice, is that
people stopped simply blaming each others and acknowledged that nobody
knew how to fix the problem. Somebody even pointed out that installing
any major scientific code was not a piece of cake. Hans Petter and
others said that they had solved the problem by relying on a virtual
machine and Ubuntu.</p>
<p>Konrad has also <a class="reference external" href="http://khinsen.wordpress.com/2010/07/12/euroscipy-2010/">blogged</a>, giving his own view of the conference.</p>
<p><img alt="image2" src="http://farm5.static.flickr.com/4097/4778812305_9217c5d3c2_m.jpg" /></p>
</div>
<div class="section" id="thanks">
<h2>Thanks</h2>
<p>The conference could happen only because of the help of many people.
First we need to thank our sponsors: <a class="reference external" href="http://www.enthought.com">Enthought</a>, <a class="reference external" href="http://www.python-academy.com/">Python Academy</a>,
<a class="reference external" href="http://www.pytables.org">Pytables</a>, and especially our host <a class="reference external" href="http://www.ens.fr">Ecole Normale Supérieure</a>, which
not only provided us with the rooms, but also made sure that everything
was going well with the sound system, the projection, or the access to
the building. With regards to organization and planing, Nicolas and I
received a lot of help from <a class="reference external" href="http://www.saint-gobain-recherche.com/svi/en/emmanuelle_gouillart.html">Emmanuelle Gouillart</a>.</p>
</div>
Making posters for scientific conferences2010-07-12T00:00:00+02:002010-07-12T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-07-12:/science/making-posters-for-scientific-conferences.html<p class="first last">Some advices and examples on making posters for scientific conference.</p>
<p>This page gives some advices and examples on making posters for
scientific conference.</p>
<p>Here are some posters I made (one in 2007, the other in 2011). They don’t
follow all the advice on this page, but should.</p>
<p><a class="reference external image-reference" href="attachments/poster_YAO.pdf"><img alt="poster1" src="attachments/poster_YAO.jpg" style="width: 33%;" /></a> <a class="reference external image-reference" href="attachments/poster_hbm2011.pdf"><img alt="poster2" src="attachments/poster_hbm2011.png" style="width: 33%;" /></a></p>
<div class="topic">
<p class="topic-title">LaTeX sources</p>
<p>This poster is written in LaTeX. You can download the whole source of
the posters for <a class="reference external" href="attachments/poster.zip">the first poster (left)</a>,
and <a class="reference external" href="attachments/poster_hbm2011.zip">the second one (right)</a>. These
are some of my personnal projects, not meant for sharing. As a result
they have a fair amount of hacking. I have been asked for source code
more than once, so I put it on the web. I do not however have time to
provide <strong>any</strong> support for it (I am already to busy supporting other
things. Any mail asking for help on these files will unanswered. Sorry.</p>
</div>
<p>Here is another example, a bit more visually appealing, as it is intended
for a less technical audience.</p>
<a class="reference external image-reference" href="attachments/poster_ICE.pdf"><img alt="" class="align-center" src="attachments/poster_ICE.jpg" /></a>
<p>One more about my work: this one was made to convey a strong message and
simplified the content a lot to get the message accross. I am not too sure
it worked, but I still find the poster pretty.</p>
<a class="reference external image-reference" href="attachments/poster_ICOLS07.pdf"><img alt="" class="align-center" src="attachments/poster_ICOLS07.jpg" /></a>
<p>And finally two made by Emmanuelle with really nice colours.</p>
<a class="reference external image-reference" href="attachments/poster_Emmanuelle.pdf"><img alt="" src="attachments/poster_Emmanuelle.jpg" /></a>
<a class="reference external image-reference" href="attachments/poster_blue.pdf"><img alt="" src="attachments/poster_blue.jpg" /></a>
<div class="section" id="advice-on-poster-presentation">
<h2>Advice on poster presentation</h2>
<p>See also <a class="reference external" href="http://www.ncsu.edu/project/posters">http://www.ncsu.edu/project/posters</a></p>
<div class="section" id="fonts">
<h3>Fonts</h3>
<p>Sans-serif fonts look really nice, but are less readable in
paragraphs. Use them for titles and headers. Use serif fonts for
paragraphs. Stick to a simple font family like times. Use bold fonts
when writing with a light colour on a dark background.</p>
</div>
<div class="section" id="colours">
<h3>Colours</h3>
<p>Stick to a rather little numbers of colours, but well chosen.
Put a very light colour behind your text blocks. If ink is not too
expensive, I would use a dark background, and have light text blocks on
it. Have well separated areas of your posters (like the background, and
the text blocks), and have the background, or other decorative elements,
have little contrast: they should not stand out too much (mine stood out
too much in my poster, its because the print-out didn’t look like want
was on the screen).</p>
</div>
<div class="section" id="page-layout">
<h3>Page layout</h3>
<p>Break symmetry and order. A well aligned poster is boring to the
eye, and does not catch attention from afar. People read your poster by
first scanning through it and stopping at a few key points (usually
first at the upper left, then the upper left, then down right, and down
left), then they might read it more thoroughly after their first scan.
You want to define visually these key points, make them appealing, and
put key ideas there.</p>
<p>Long lines are difficult to read. Pick up a book, a flyer, anything made
by a professional publisher, it will never have long lines. A good rule
of thumb is that if a text block has lines longer than 80 characters, it
needs breaking down in several columns.</p>
</div>
</div>
<div class="section" id="which-software-to-use">
<h2>Which software to use</h2>
<p>Many people use PowerPoint to make their posters. It is easy to use, but
it is not dedicated to making posters, and it does horrible pdfs.</p>
<p>If you want to pay a lot there is Quark Xpress that is very good for that
kind of things. Adobe PageMaker is also a very good software. <a class="reference external" href="http://www.xara.com/">Xara</a> is a cheap and good design program, and a free
version will soon be available for linux.</p>
<p>I use LaTeX. Just because I love the way it positions characters. But I
admit it is a bit brutal. What I would advice you to use is <a class="reference external" href="http://www.scribus.net">scribus</a> it is dedicate to making posters and is free
and open source. I sometimes use LaTeX to create the text boxes, and
scribus to lay them around. I wrote a <a class="reference external" href="LaTeX-scribus.html">page</a>
describing how I do it.</p>
<!-- See also :
http://theoval.cmp.uea.ac.uk/~nlct/jpgfdraw/manual/postertutorial.html -->
<p>One last remark: use vector graphics (eps, ps, pdf, svg), not bitmaps,
they scale up really badly.
Try to get a vector logo of your institution. Usually asking the PR
people is the only thing it take to get one. Of course if you are using
powerpoint chances are that you wont be able to insert it in your poster.</p>
</div>
A simple LaTeX example2010-06-01T00:00:00+02:002010-06-01T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-06-01:/science/a-simple-latex-example.html<p class="first last">A simple LaTeX document, to use as a skeletton</p>
<p>Here is a very simple example of a laTeX document that uses good package
to have a simple but nice layout:</p>
<ul class="simple">
<li><a class="reference external" href="attachments/simple.tex">The LaTeX source</a></li>
<li><a class="reference external" href="attachments/simple.pdf">The pdf document</a></li>
</ul>
<div class="topic">
<p class="topic-title"><strong>Some advice</strong></p>
<ul class="simple">
<li>Use <a class="reference external" href="http://www.texniccenter.org/">texniccenter</a> if you don’t have a
favorite editor.</li>
<li>Read the <a class="reference external" href="http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf">not so short introduction to latex</a></li>
</ul>
</div>
Personal views on scientific computing2010-05-20T00:00:00+02:002010-05-20T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-05-20:/programming/view_on_scientific_computing.html<p>My contributions to the scientific computing software ecosystem are
motivated by my vision on computational science.</p>
<p>Scientific research relies more and more on computing. However, most of
the researchers are not software engineers, and as computing is becoming
ubiquitous, the limiting factor becomes more and more the <strong>human
factor</strong> <a class="reference external" href="http://software-carpentry.org/articles/amsci-swc-2006.pdf">[G …</a></p><p>My contributions to the scientific computing software ecosystem are
motivated by my vision on computational science.</p>
<p>Scientific research relies more and more on computing. However, most of
the researchers are not software engineers, and as computing is becoming
ubiquitous, the limiting factor becomes more and more the <strong>human
factor</strong> <a class="reference external" href="http://software-carpentry.org/articles/amsci-swc-2006.pdf">[G. Wilson, 2006]</a> <a class="reference external" href="http://download.on9pc.com/ebook/programing/Teach%20Yourself%20Programming%20in%20Ten%20Years.pdf">[P.
Norvig, 2009]</a>.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">To address the needs of computing accross scientific fields, I believe
that we need a <strong>general-purpose</strong>, <strong>high-level</strong>, <strong>interactive</strong>, and
<strong>highly-readable</strong> language and set of tools for scientific computing.</p>
</div>
<ul class="simple">
<li>C does not answer my needs: does a molecular biologist know about
pointers? Should she?</li>
<li>Matlab does not answer my needs either: scientific work with computers
is not only about numerical computation. Have you tried writing an
experiment-control software with Matlab? How about file management?
Inserting the algorithms in a web server.</li>
</ul>
<p>We need better teaching material, that sit at interfaces between software
engineer, and general science. Most top notch tools and libraries are
full of domain-specific jargon and conventions.</p>
<p>For reproducible science, we need the code to be readable and to reflect
the corresponding scientific operation. We need it to be unit-tested to
ensure its correctness.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">We need to consider scientific libraries as end-result of our
research with the same importance than articles <a class="reference external" href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.6201">[J. Buckheit and D.
Donoho. 1995]</a>.
They need to convey a scientific message, to be <strong>understandable</strong> and
<strong>refutable</strong>. New results should be <strong>reproducible</strong> via published code
<a class="reference external" href="http://www.computer.org/portal/web/csdl/doi/10.1109/MCSE.2009.14">[CISE Jan. 2009]</a>. As
for established algorithms, scientific libraries with their
<strong>documentation</strong> and <strong>examples</strong> should be the textbooks of tomorrow.</p>
</div>
<hr class="docutils" />
<ul class="simple">
<li><strong>Scientific software should be as reusable as possible</strong>, to enable the
advancement of Science via software, year after year. This means that
we need to build general-purpose libraries.</li>
<li><strong>Code quality and documentation are crucial</strong>, as human factors are
often the limitation. As a corollary, scientific code should be
unit-tested to ensure correctness.</li>
<li><strong>Core scientific software should be open source</strong>, as scientific work
cannot build on black boxes</li>
<li><strong>Algorithms should be written as simply as possible</strong>. A high level of
sophistication in software engineering should not be a requirement to
all scientists</li>
<li><strong>Prefer high-level languages</strong>. The code should be written at the right
level of abstraction.</li>
<li><strong>We need to build common and shared tools</strong>. Scientific software
shouldn’t be ‘owned’ by a lab.</li>
<li><strong>The source code should a deliverable of the research</strong>. As a result, it
should read clearly and be understandable to all.</li>
<li><strong>Documentation and examples are the textbooks of tomorrow</strong>.</li>
<li><strong>Publications should be reproducible</strong>. Ideally they should become an
example of the library. This should be mitigated by the fact that code
maintainance is costly, and achieving good code takes more work that
publishing. Focus should be on publications that will give rise to reference
results.</li>
<li><strong>Academia need to value sotware maintainance</strong>. It is hard and costly,
but it determines our future.</li>
<li><strong>Tools that develop the environment, rather than a specific algorithm or
scientific field are crucial</strong> (one example is IPython).</li>
</ul>
<!-- Cite V Stodden -->
<hr class="docutils" />
<p>Further reading:</p>
<blockquote>
<ul class="simple">
<li>Open source Machine Learning software <a class="reference external" href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.77.5605&rep=rep1&type=pdf">[S. Sonnenburg et al. 2007]</a></li>
<li>Open source mathematical sofware <a class="reference external" href="http://www.ams.org/notices/200710/tx071001279p.pdf">[D. Joyner and W. Stein 2007]</a></li>
<li>Licensing, intellectual property in scientific work
<a class="reference external" href="http://jolt.unc.edu/sites/default/files/7_nc_jl_tech_321.pdf">[A. Gonzalez 2006]</a></li>
<li>Scientific software development best practices
<a class="reference external" href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.0020087">[S. Baxter et al. 2006]</a></li>
</ul>
</blockquote>
EuroScipy abstract submission deadline extended2010-05-15T23:36:00+02:002010-05-15T23:36:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-05-15:/programming/euroscipy-abstract-submission-deadline-extended.html<p>Given that we have been able to turn on registration only very late, the
<a class="reference external" href="http://www.euroscipy.org">EuroScipy</a> conference committee is extending the deadline for abstract
submission for the 2010 EuroScipy conference.</p>
<p>On Thursday May 20th, at midnight Samoa time, we will turn off the
abstract submission on the conference site. Up to …</p><p>Given that we have been able to turn on registration only very late, the
<a class="reference external" href="http://www.euroscipy.org">EuroScipy</a> conference committee is extending the deadline for abstract
submission for the 2010 EuroScipy conference.</p>
<p>On Thursday May 20th, at midnight Samoa time, we will turn off the
abstract submission on the conference site. Up to then, you can modify
the already-submitted abstract, or submit new abstracts.</p>
<p>We are very much looking forward to your submissions to the conference.</p>
<div class="line-block">
<div class="line">Gaël Varoquaux</div>
<div class="line">Nicolas Chauvat</div>
</div>
<hr class="docutils" />
<div class="line-block">
<div class="line">EuroScipy 2010 is the annual European conference for scientists using Python. It will be held July 8-11 2010, in ENS, Paris, France.</div>
<div class="line"><strong>Links: `Conference website`_, `Call for papers`_, `Practical information`_</strong></div>
</div>
EuroScipy is finally open for registration2010-05-13T13:23:00+02:002010-05-13T13:23:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-05-13:/programming/euroscipy-is-finally-open-for-registration.html<a class="reference external image-reference" href="attachments/poster_euroscipy_2010.pdf"><img alt="" src="attachments/poster_euroscipy_2010.jpg" /></a>
<div class="section" id="the-registration-for-euroscipy-is-finally-open">
<h2>The registration for <a class="reference external" href="http://www.euroscipy.org//conference/euroscipy2010">EuroScipy</a> is finally open.</h2>
<p>To register, go to the <a class="reference external" href="http://www.euroscipy.org//conference/euroscipy2010">website</a>, create an account, and you will see a
<em>‘register to the conference’</em> button on the left. Follow it to a page
which presents a <em>‘shoping cart’</em>. Simply submitting this information
registers you to the conference, and on …</p></div><a class="reference external image-reference" href="attachments/poster_euroscipy_2010.pdf"><img alt="" src="attachments/poster_euroscipy_2010.jpg" /></a>
<div class="section" id="the-registration-for-euroscipy-is-finally-open">
<h2>The registration for <a class="reference external" href="http://www.euroscipy.org//conference/euroscipy2010">EuroScipy</a> is finally open.</h2>
<p>To register, go to the <a class="reference external" href="http://www.euroscipy.org//conference/euroscipy2010">website</a>, create an account, and you will see a
<em>‘register to the conference’</em> button on the left. Follow it to a page
which presents a <em>‘shoping cart’</em>. Simply submitting this information
registers you to the conference, and on the left of the website, the
button will now display <em>‘You are registered for the conference’</em>.</p>
<p>The registration fee is 50 euros for the conference, and 50 euros for
the tutorial. Right now there is no payment system: you will be
contacted later (in a week) with instructions for paying.</p>
<p>We apologize for such a late set up. We do realize this has come as an
inconvenience to people.</p>
<p><strong>Do not wait to register: the number of people we can host is
limited.</strong></p>
</div>
<div class="section" id="an-exciting-program">
<h2>An exciting program</h2>
<div class="section" id="tutorials-from-beginners-to-experts">
<h3>Tutorials: from beginners to experts</h3>
<p>We have two tutorial tracks:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.euroscipy.org/track/871">**Introductory tutorial**</a>: to get you to speed on scientific
programming with Python.</li>
<li><a class="reference external" href="http://www.euroscipy.org/track/872">**Advanced tutorial**</a>: experts sharing their knowledge on specific
techniques and libraries.</li>
</ul>
</div>
<div class="section" id="scientific-track-doing-new-science-in-python">
<h3>Scientific track: doing new science in Python</h3>
<p>Although the abstract submission is not yet over, I can say that we are
going to have a rich set of talks, looking at the current submissions.
In addition to the contributed talks, we have:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2010">**Keynote speakers**</a>: Hans Petter Langtangen and Konrard Hinsen,
two major player of scientific computing in Python.</li>
<li><a class="reference external" href="http://www.euroscipy.org/talk/937">**Lightning talks**</a>: one hour will be open for people to come up
and present in a flash an interesting project.</li>
</ul>
</div>
<div class="section" id="publishing-papers">
<h3>Publishing papers</h3>
<p>We are talking with the editors of a major scientific computing journal,
and the odds are quite high that we will be able to publish a special
issue on scientific computing in Python based on the proceedings of the
conference. The papers will undergo peer-review independently from the
conference, to ensure high quality of the final publication.</p>
</div>
</div>
<div class="section" id="call-for-papers">
<h2>Call for papers</h2>
<p>Abstract submission is still open, though not for long. We are
soliciting contributions on scientific libraries and tools developed
with Python and on scientific or engineering achievements using Python.
These include applications, teaching, future development directions, and
current research. See the <a class="reference external" href="http://www.euroscipy.org/card/euroscipy2010_call_for_papers">call for papers</a>.</p>
<p><strong>I am very much looking forward to passionate discussions about
Python in science in Paris</strong></p>
</div>
Status of the EuroScipy registration2010-05-02T22:57:00+02:002010-05-02T22:57:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-05-02:/programming/status-of-the-euroscipy-registration.html<p>It is still not possible to register for the <a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2010">Euroscipy conference</a>: we
are having difficulties with payment for the registration, and we are
still not sure that we will be able to actually charge money!</p>
<p>This might not be a bad news, because it might mean that the conference
will …</p><p>It is still not possible to register for the <a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2010">Euroscipy conference</a>: we
are having difficulties with payment for the registration, and we are
still not sure that we will be able to actually charge money!</p>
<p>This might not be a bad news, because it might mean that the conference
will be completely free. This would mean that we would be able to
provide lunch which is a pity as there is nothing like eating with a
bunch of passionate experts to learn new tricks, but it would not hamper
the conference in any other way, as the rooms are already booked and
various little expenses covered.</p>
<p>If we manage to sort out payments in the next weeks, the fee should be
50 euros for the 2 days of tutorial, and between 50 and 100 euros for
the full conference, depending on exactly what catering we offer.</p>
<p>Anyhow, we should open the registration real-soon, with or without
payment. We will need to have some formal registration, as the number of
people that can fit in the rooms will be limited.</p>
<p>All in all, with or without registration fees, it should be possible to
make it to Euroscipy keeping expenses low: we have indicated a few cheap
accommodation on the <a class="reference external" href="http://www.euroscipy.org/card/euroscipy2010_practical_information">practical details page</a>, and it is easy to get
good food for a good price in the area.</p>
<hr class="docutils" />
<p>I am very excited about this conference. We have two keynotes that I am
really looking forward to hearing, and I can say that we have been
getting pretty good submissions for presentations. Also, changes are
that we should be able to publish proceedings in a peer-reviewed
journal, although I can’t say more about that right now.</p>
<p>Also, even if you are not interested in scientific research done using
Python, the tutorials are a unique opportunity: we are having top-notch
experts presenting with two tracks, <a class="reference external" href="http://www.euroscipy.org/track/871">one</a> to get beginners up to speed
and efficient in a couple of days, and the <a class="reference external" href="http://www.euroscipy.org/track/872">other</a> for exploring
advanced subjects. I know the speakers, and I can tell you that I won’t
be talking in the corridor, but sitting with my laptop and listening to
them. People pay large chunks of money for such training, usually.</p>
Mayavi: Representing an additional scalar on surfaces2010-04-05T00:30:00+02:002010-04-05T00:30:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2010-04-05:/programming/mayavi-representing-an-additional-scalar-on-surfaces.html<p>We have been getting a few questions on the <a class="reference external" href="https://mail.enthought.com/mailman/listinfo/enthought-dev">enthought-dev</a>
mailing-list on how to represent an additional information on a surface
with <a class="reference external" href="http://code.enthought.com/projects/mayavi">Mayavi</a>, using color not given eg by the elevation. A <a class="reference external" href="http://dpinte.wordpress.com/2010/03/30/4d-surface-plots-in-mayavi/">recent
post</a> on his blog by Didrik Pinte shows the problem quite well:</p>
<a class="reference external image-reference" href="http://dpinte.wordpress.com/2010/03/30/4d-surface-plots-in-mayavi/"><img alt="" src="http://dpinte.files.wordpress.com/2010/03/option_valuation_3d.png" /></a>
<p>This problem can be seen …</p><p>We have been getting a few questions on the <a class="reference external" href="https://mail.enthought.com/mailman/listinfo/enthought-dev">enthought-dev</a>
mailing-list on how to represent an additional information on a surface
with <a class="reference external" href="http://code.enthought.com/projects/mayavi">Mayavi</a>, using color not given eg by the elevation. A <a class="reference external" href="http://dpinte.wordpress.com/2010/03/30/4d-surface-plots-in-mayavi/">recent
post</a> on his blog by Didrik Pinte shows the problem quite well:</p>
<a class="reference external image-reference" href="http://dpinte.wordpress.com/2010/03/30/4d-surface-plots-in-mayavi/"><img alt="" src="http://dpinte.files.wordpress.com/2010/03/option_valuation_3d.png" /></a>
<p>This problem can be seen as taking a standard <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_helper_functions.html#enthought.mayavi.mlab.surf">surf</a> plot:</p>
<a class="reference external image-reference" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_helper_functions.html#enthought.mayavi.mlab.surf"><img alt="" src="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/_images/enthought_mayavi_mlab_surf.jpg" /></a>
<p>but coloring it with a different scalar than the elevation.</p>
<p>I would like to present two ways of solving this problem. First a very
simple way specific to the exact problem, second a more complicated but
quite generic approach.</p>
<div class="section" id="representing-surfaces-more-complex-than-an-elevation-map">
<h2>Representing surfaces more complex than an elevation map</h2>
<p>The first option is simply to use the <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html#d-data">tools</a> that Mayavi’s <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab</a>
interface provide to represent surfaces that are not the particular case
of an elevation plot. In our case, it is very easy to use the <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_helper_functions.html#enthought.mayavi.mlab.mesh">mesh
function</a> which can take the x, y, z positions of a grid giving the
surface, but also an additional scalar value at these position:</p>
<div class="highlight"><pre><span></span><span class="c1"># Create some data</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mgrid</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">10</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="mi">0</span><span class="p">:</span><span class="mi">10</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">]</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arctan</span><span class="p">(</span><span class="n">x</span><span class="o">/</span><span class="n">y</span><span class="p">)</span>
<span class="c1"># Visualize it</span>
<span class="kn">from</span> <span class="nn">enthought.mayavi</span> <span class="kn">import</span> <span class="n">mlab</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">mesh</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="mf">.05</span><span class="o">*</span><span class="n">z</span><span class="p">,</span> <span class="n">scalars</span><span class="o">=</span><span class="n">w</span><span class="p">)</span>
<span class="c1"># Finally, add a few decorations.</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">axes</span><span class="p">()</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">outline</span><span class="p">()</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">view</span><span class="p">(</span><span class="o">-</span><span class="mi">177</span><span class="p">,</span> <span class="mi">82</span><span class="p">,</span> <span class="mi">32</span><span class="p">)</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
<img alt="" src="attachments/mesh_example.png" />
<p>As you can see, this solution is really simple, and solves the problem.</p>
</div>
<div class="section" id="a-generic-way-of-representing-several-scalar-attributes-with-one-visualization">
<h2>A generic way of representing several scalar attributes with one visualization</h2>
<p>If we think of the visualization problem as a way of representing two
scalar values, ‘z’ and ‘w’, and a function of two others, ‘x’ and ‘y’,
the above solution is not really satisfactory: the surf function really
turns the scalar value ‘z’ in elevation (using a WarpScalar filter). We
would like to be able to add an addition scalar value ‘w’ and turn it
into color, just like ‘z’ is turned into elevation. The pipeline that is
created by the surf function is the following:</p>
<img alt="" src="attachments/surf_pipeline.png" />
<p>The first element of the pipeline after the scene is the data source
created for us by the surf function: it is a 2D array that contains the
‘z’ value as a scalar value. The ‘WarpScalar’ filter is applied, and
transform that value into elevation. After that, a ‘PolyDataNormals’
filter is used to calculate normals, so as to have a smooth rendering,
and finally, a ‘Surface’ module is applied to display the resulting
elevation map as a surface, with a color reflecting the scalar value.</p>
<p>The way we can operate on two scalar values and turn them into elevation
and color successively is to embed these two scalar values on the
dataset, ‘z’ and ‘w’, and use a ‘SetActiveAttribute’ to control on which
one the ‘Surface’ module is applied. This approach is much more powerful,
because we can tweak the pipeline ourselves, and use any filter to
replace the WarpScalar, and display the ‘z’ information (more on that
below).</p>
<p>Here is how to do achieve a visualization with a similar look as above,
but with two scalar values transformed successively in elevation and
color:</p>
<div class="highlight"><pre><span></span><span class="c1">###############################################################</span>
<span class="c1"># Create some data</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mgrid</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">10</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="mi">0</span><span class="p">:</span><span class="mi">10</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">]</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span>
<span class="n">w</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arctan</span><span class="p">(</span><span class="n">x</span><span class="o">/</span><span class="n">y</span><span class="p">)</span>
<span class="c1">###############################################################</span>
<span class="c1"># Visualize the data</span>
<span class="kn">from</span> <span class="nn">enthought.mayavi</span> <span class="kn">import</span> <span class="n">mlab</span>
<span class="c1"># Create the data source</span>
<span class="n">src</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">pipeline</span><span class="o">.</span><span class="n">array2d_source</span><span class="p">(</span><span class="n">z</span><span class="p">)</span>
<span class="c1"># Add the additional scalar information 'w', this is where we need to be a bit careful,</span>
<span class="c1"># see</span>
<span class="c1"># http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/example_atomic_orbital.html</span>
<span class="c1"># and</span>
<span class="c1"># http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/data.html</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">src</span><span class="o">.</span><span class="n">mlab_source</span><span class="o">.</span><span class="n">dataset</span>
<span class="n">array_id</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">point_data</span><span class="o">.</span><span class="n">add_array</span><span class="p">(</span><span class="n">w</span><span class="o">.</span><span class="n">T</span><span class="o">.</span><span class="n">ravel</span><span class="p">())</span>
<span class="n">dataset</span><span class="o">.</span><span class="n">point_data</span><span class="o">.</span><span class="n">get_array</span><span class="p">(</span><span class="n">array_id</span><span class="p">)</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="s1">'color'</span>
<span class="n">dataset</span><span class="o">.</span><span class="n">point_data</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>
<span class="c1"># Here, we build the very exact pipeline of surf, but add a</span>
<span class="c1"># set_active_attribute filter to switch the color, this is code very</span>
<span class="c1"># similar to the code introduced in:</span>
<span class="c1"># http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html#assembling-pipelines-with-mlab</span>
<span class="n">warp</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">pipeline</span><span class="o">.</span><span class="n">warp_scalar</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="n">warp_scale</span><span class="o">=</span><span class="mf">.5</span><span class="p">)</span>
<span class="n">normals</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">pipeline</span><span class="o">.</span><span class="n">poly_data_normals</span><span class="p">(</span><span class="n">warp</span><span class="p">)</span>
<span class="n">active_attr</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">pipeline</span><span class="o">.</span><span class="n">set_active_attribute</span><span class="p">(</span><span class="n">normals</span><span class="p">,</span>
<span class="n">point_scalars</span><span class="o">=</span><span class="s1">'color'</span><span class="p">)</span>
<span class="n">surf</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">pipeline</span><span class="o">.</span><span class="n">surface</span><span class="p">(</span><span class="n">active_attr</span><span class="p">)</span>
<span class="c1"># Finally, add a few decorations.</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">axes</span><span class="p">()</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">outline</span><span class="p">()</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">view</span><span class="p">(</span><span class="o">-</span><span class="mi">177</span><span class="p">,</span> <span class="mi">82</span><span class="p">)</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
<p>The pipeline that is created is the following:</p>
<img alt="" src="attachments/complex_pipeline.png" />
<p>In the first part of the pipeline, the ‘WarpScalar’ filter is applied to
the ‘z’ scalar value, whereas, due to the ‘SetActiveAttribute’ filter,
the ‘Surface’ module uses the ‘w’ scalar value to display the color.</p>
<p>This pattern is very powerful, and can be used with other sets of
filters or modules. The example of this pattern that we use in the
Mayavi documentation is the following:</p>
<a class="reference external image-reference" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/example_atomic_orbital.html"><img alt="" src="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/_images/example_atomic_orbital.jpg" /></a>
<p>We use a ‘Contour’ filter to contour on the amplitude of a complex a
field defined in the volume, and then switch to the phase to display the
color. See the <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/example_atomic_orbital.html">atomic orbital example</a> in the Mayavi documentation for
more details.</p>
</div>
Book review: Matplotlib for Python Developpers2010-03-26T10:49:00+01:002010-03-26T10:49:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-03-26:/programming/book-review-matplotlib-for-python-developpers.html<p><em>Packt publishing</em> sent me a copy of Sandro Tosi’s book <a class="reference external" href="http://www.packtpub.com/matplotlib-python-development/book">Matplotlib for
Python Developpers</a> a while ago. Unfortunately, it arrived after I had
left for the Christmas break, and I couldn’t find time to review it for
a while (I am terribly bad at time-management, and I do …</p><p><em>Packt publishing</em> sent me a copy of Sandro Tosi’s book <a class="reference external" href="http://www.packtpub.com/matplotlib-python-development/book">Matplotlib for
Python Developpers</a> a while ago. Unfortunately, it arrived after I had
left for the Christmas break, and I couldn’t find time to review it for
a while (I am terribly bad at time-management, and I do too many things,
as I result I am always overworked). 3 months later, I have finally
found time to read it and post a review.</p>
<div class="section" id="content">
<h2>Content</h2>
<p>The book introduces <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> which is, for those who don’t know, a
truly fantastic library for scientific plotting in Python. Matplotlib is
great because it is really easy to pick up, and can be used to produce
very high-quality plots.</p>
<p>The book starts by progressively introducing the simple, imperative API
for matplotlib, with a focus on getting the user immediately plotting
data. It then moves on to a review of the functionality for plotting in
matplotlib and the object-oriented usage of matplotlib. Finally, Sandro
shows us how to embedded matplotb in various environment such as GUI
toolkits or web development tools.</p>
<p>The last part of the book is, in my opinion the most original and
precious, as these subjects are less well-known and documented in
classical references accessible to people with a scientific computing
background.</p>
</div>
<div class="section" id="target-audience">
<h2>Target audience</h2>
<p>The book can pretty much be picked by a scientific Python beginner. It
does require some knowledge of the Python language, but if the reader
has programmed in another language, I don’t see this as a big problem.
In this regard, the book is especially interesting, as it can lead a
scientist from newbie to writing simple end-user programs. There is a
clear need for more of these documents currently.</p>
<p>The book will also be useful for the experienced Python developers
looking to pick up quickly matplotlib.</p>
</div>
<div class="section" id="personal-comments-on-the-book">
<h2>Personal comments on the book</h2>
<p>In my experience, exposing a tool such as matplotlib is a challenge:
everybody has different plotting needs and there is an infinity of
variation in ways that you can use a powerful library like matplotlib.
Thus, Sandro’s exposition of matlplotlib will not suffice: people should
absolutely read more, and I can’t stress too much that the matplotlib
documentation is excellent, and people should read more of it.</p>
<p>In general, I found that the books reads fairly well. Off course, I am
not the best critic in term of ease of read, as I know matplotlib very
well. I do find that the book lacks a <em>personal touch</em> such as
interesting examples, or profound insights on specific problems. There
is nothing that got me excited in the book (again, maybe it’s because I
know what’s in the book quite well).</p>
<p>Once again, in my eyes, the biggest contribution of this book is to put
together an introduction to matplotlib, and examples of application
building using matplotlib. I would especially recommend the book for
people wanting to build simple data visualization GUIs.</p>
<hr class="docutils" />
<p>Finally, with regards to interactive data visualization, in my
experience, scientific programmers achieve better productivity when
avoiding to work at the widget level and using an abstraction library. I
strongly recommend looking at <a class="reference external" href="http://code.enthought.com/projects/traits/docs/html/">TraitsUI</a> for this purpose. you can find
a tutorial <a class="reference external" href="http://gael-varoquaux.info/computers/traits_tutorial/index.html">here</a> (disclaimer: I wrote that tutorial).</p>
<p>Also, if you are going to write a data visualization program that is
interactive in the sens that it enables the user to interact with the
data, using <a class="reference external" href="http://code.enthought.com/chaco/">Chaco</a> instead of matplotlib may make your life easier.
Chaco is not as well polished and documented as matplotlib, and I would
never use it for a quick scripting work, but it has a strong focus on
data interaction, and as such makes it really easy to build very
responsive user interfaces, because it is very fast and has a clear
object-oriented API.</p>
</div>
New Mayavi release2010-03-14T12:58:00+01:002010-03-14T12:58:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-03-14:/programming/new-mayavi-release.html<p>A week ago, the Peter Wang released a new version of the <a class="reference external" href="http://code.enthought.com/">Enthought Tool
Suite (ETS)</a>. With it came a new version of <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi2</a>.</p>
<p>Prabhu and I have been horribly busy we real life, and I had the bad
feeling that we were not giving enough love to Mayavi. I …</p><p>A week ago, the Peter Wang released a new version of the <a class="reference external" href="http://code.enthought.com/">Enthought Tool
Suite (ETS)</a>. With it came a new version of <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi2</a>.</p>
<p>Prabhu and I have been horribly busy we real life, and I had the bad
feeling that we were not giving enough love to Mayavi. I was surprised
when I put together the list of features and bugs fixes that went in
Mayavi for the last two releases. The full list can be found <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/changes.html">in the
documentation</a>.</p>
<div class="section" id="contributors">
<h2>Contributors</h2>
<p>We are not being terribly good at tracking external ideas and patches,
so I hope that I haven’t forgotten anybody, but I am very happy to say
that Prabhu and I have received a fair amount of help from non core
contributors:</p>
<ul class="simple">
<li>Chris Colbert</li>
<li>Darren Dale</li>
<li>Dave Martin</li>
<li>Dave Peterson</li>
<li>Emmanuelle Gouillart</li>
<li>Erik Tollerud</li>
<li>Evan Patterson</li>
<li>Gary Ruben</li>
<li>Kyle Mandli</li>
<li>Michele Mattioni</li>
<li>Ondrej Certik</li>
<li>Ram Rachum</li>
<li>Robert Kern</li>
<li>Scott Warts</li>
<li>Suyog Jain</li>
</ul>
<p>On top of these people, I wish to thank the people making sure that the
Mayavi packages are available in the different Linux distributions:
Varun Hiremath, Lev Givon, Andrea Colangelo, Rakesh Pandit, as well as
Pierre Raybault for integrating in <a class="reference external" href="http://pythonxy.com">Pythonxy</a>.</p>
</div>
<div class="section" id="important-features-added-in-3-3-0">
<h2>Important features added in 3.3.0</h2>
<p>3.3.0 was released last fall. We had not compiled the list of changes at
the time, I am giving it here:</p>
<ul class="simple">
<li>An <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/examples.html">example gallery</a> in the documentation.</li>
<li>A <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_figure.html#sync-camera">sync_camera</a> helper function to synchronize camera between two
scenes.</li>
<li>A <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_other_functions.html#text3d">text3d</a> module, for position text in 3D that is scaled and hidden
like a data object.</li>
<li>A <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_figure.html#close">close</a> function to close scenes, similar to that in pylab or
matlab.</li>
<li>A new filter to crop datasets: <em>DataSet Clipper</em>. This filter is
terribly useful.</li>
<li>All the <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab_pipeline_reference.html">mlab.pipeline</a> functions now take a <em>figure=</em> keyword
argument. This is very useful when coding with several figures
embedded in GUIs, as in a GUI you can’t rely on a context. This is
illustrated in this <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/example_multiple_mlab_scene_models.html">example</a>.</li>
</ul>
</div>
<div class="section" id="important-features-added-in-3-3-1">
<h2>Important features added in 3.3.1</h2>
<p>In latest release the following important features were added:</p>
<ul class="simple">
<li><a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_figure.html#savefig">mlab.savefig</a> can now reliably save images of a size larger than
the window.</li>
<li>The interactive VTK documentation browser is now available in the
GUI.</li>
<li>New functions added to <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab</a> to control position of the camera:
<a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#move">move</a>, <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#yaw">yaw</a>, and <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#pitch">pitch</a>. These complement the existing <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#view">view</a>
and <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#roll">roll</a>.</li>
<li>Make the lines smoother when using <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_helper_functions.html#enthought.mayavi.mlab.plot3d">mlab.plot3d</a> (use a VTK Stripper
filter)</li>
<li>Add a <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_figure.html#enthought.mayavi.mlab.screenshot">screenshot</a> function to mlab for easy screen capture as a
numpy array. This is very useful when creating figures that combine
3D using Mayavi and 2D using pylab. I use it all the time.</li>
<li>Add a <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_pipeline_data.html#probe-data">probe_data</a> function to return the data values of Mayavi
objects at given locations as numpy arrays. This is very useful to
combine numerics with Mayavi.</li>
<li>Add a auto mode to <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/auto/mlab_camera.html#view">mlab.view</a> to compute position and distance
based on the objects on the image.</li>
<li>Add a helper function to easily interact with the data: a callback
can easily be registered to picking data with the mouse. <a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/trunk/examples/mayavi/data_interaction/">Two
examples</a> illustrate this new functionality. This is a major step
forward in making life easier for people using Mayavi to build custom
interfaces.</li>
</ul>
</div>
Using Python, Scipy, ETS, … to implement art2010-02-14T14:14:00+01:002010-02-14T14:14:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-02-14:/programming/using-python-scipy-ets-to-implement-art.html<p>The <a class="reference external" href="http://sites.google.com/site/aikonproject/">Aikon project</a> has just been slashdotted.</p>
<p>The project is about implementing a robotic artist, with a special
artistic touch:</p>
<img alt="" src="http://lh5.ggpht.com/_MJv2VVPLQaA/SUTeNkUi3zI/AAAAAAAAADA/aJkTS88XqGo/s512/0046.jpg" />
<p>The Co-principal investigator, Patrick Tresset, gave a talk at the
French Pycon this year and I was simply flabbergasted by the project. It
is amazing to mix together art and …</p><p>The <a class="reference external" href="http://sites.google.com/site/aikonproject/">Aikon project</a> has just been slashdotted.</p>
<p>The project is about implementing a robotic artist, with a special
artistic touch:</p>
<img alt="" src="http://lh5.ggpht.com/_MJv2VVPLQaA/SUTeNkUi3zI/AAAAAAAAADA/aJkTS88XqGo/s512/0046.jpg" />
<p>The Co-principal investigator, Patrick Tresset, gave a talk at the
French Pycon this year and I was simply flabbergasted by the project. It
is amazing to mix together art and technology in such a way, you should
really have a look at the <a class="reference external" href="http://sites.google.com/site/aikonproject/">videos of the robotic arm making sketches of
people.</a></p>
<p>But I was even more startled when I discovered that the project was
using <a class="reference external" href="http://www.scipy.org">scipy</a> and all my beloved stack for scientific computing in
Python, including the <a class="reference external" href="http://code.enthought.com/">Enthought Tool Suite</a>: <a class="reference external" href="http://sites.google.com/site/aikonproject/isle">check it out</a>. I really
want scientific computing software to be tools opening new ideas and new
research. This research goes beyond my dreams.</p>
<p>
<object width="425" height="344">
<embed src="http://www.youtube.com/v/AOtQAhblRps&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344">
</embed>
</object>
</p>EuroScipy 2010, Paris July 8-11. Save the date!2010-02-14T00:02:00+01:002010-02-14T00:02:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-02-14:/programming/euroscipy-2010-paris-july-8-11-save-the-date.html<p><strong>EuroScipy 2010</strong>, the 3rd European meeting on Python in Science, will
be held July 8-11 in the center of Paris, at the <a class="reference external" href="http://www.ens.fr/?lang=en">Ecole Normale
Supérieure</a>.</p>
<p>We have made good progress in the organization, and we already have <a class="reference external" href="http://www.euroscipy.org/conference/867?vid=primary">an
exciting program</a> although paper submission is not yet even open.</p>
<div class="section" id="tutorial-tracks">
<h2>Tutorial tracks …</h2></div><p><strong>EuroScipy 2010</strong>, the 3rd European meeting on Python in Science, will
be held July 8-11 in the center of Paris, at the <a class="reference external" href="http://www.ens.fr/?lang=en">Ecole Normale
Supérieure</a>.</p>
<p>We have made good progress in the organization, and we already have <a class="reference external" href="http://www.euroscipy.org/conference/867?vid=primary">an
exciting program</a> although paper submission is not yet even open.</p>
<div class="section" id="tutorial-tracks">
<h2>Tutorial tracks</h2>
<p>There will be two tutorials tracks:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.euroscipy.org/track/871">An introductory track</a>, to bring attendees up to speed with Python
in science. Even if you are a complete beginner, after these two
days, you should be able to be efficient using Python for scientific
purposes.</li>
<li><a class="reference external" href="http://www.euroscipy.org/track/872">An advanced tutorial track</a>, covering in-depth specific tools and
projects, aimed at experienced users and presented by leading experts
of the topic.</li>
</ul>
<p>We will soon be requesting feedback from you to help us choose between
the different thrilling tutorial propositions that we have for these
tracks. More on that later…</p>
</div>
<div class="section" id="keynote-speakers">
<h2>Keynote speakers</h2>
<div class="section" id="hans-petter-langtangen">
<h3>Hans Petter Langtangen</h3>
<ul class="simple">
<li><a class="reference external" href="http://simula.no/">Simula laboratory</a>, Oslo, director of scientific computing and
bio-medical research</li>
<li>Author of the famous book <a class="reference external" href="http://www.springer.com/mathematics/numerical+and+computational+mathematics/book/978-3-540-73915-9">Python scripting for computational
science</a></li>
</ul>
</div>
<div class="section" id="konrad-hinsen">
<h3>Konrad Hinsen</h3>
<ul class="simple">
<li><a class="reference external" href="http://www.synchrotron-soleil.fr/">Synchrotron SOLEIL</a> and <a class="reference external" href="http://dirac.cnrs-orleans.fr/plone">Centre de Biophysique Moléculaire</a>
(Orléans)</li>
<li>One of the fathers of numeric, and developer of <a class="reference external" href="http://dirac.cnrs-orleans.fr/plone/software/scientificpython/">Scientific Python.</a></li>
</ul>
</div>
</div>
<div class="section" id="help-us-spread-the-word">
<h2>Help us spread the word</h2>
<p>The poster of the conference can be downloaded:</p>
<a class="reference external image-reference" href="attachments/poster_euroscipy_2010.pdf"><img alt="" src="attachments/poster_euroscipy_2010.jpg" /></a>
<p>Help us spread the word: print it and post it at your workplace!</p>
</div>
<div class="section" id="the-exciting-city-of-paris">
<h2>The exciting city of Paris</h2>
<p>The conference will take place in the center of Paris, in the very
lively “quartier latin”, in the prestigious and historical ‘Ecole
Normale Supérieure’. In the morning, on your way to ENS, drop by a café
for a French croissant, served by a French waiter with a typical French
accent in English. In the evenings, walk one block to enjoy the night
life “rue Mouffetard”, or venture further to stroll on the river banks
of the Seine, along which people dance to street music.</p>
<img alt="" src="attachments/banner_euroscipy.jpg" />
</div>
PCA and ICA: Identifying combinations of variables2010-02-05T00:00:00+01:002010-02-05T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2010-02-05:/science/ica_vs_pca.html<div class="topic">
<p class="topic-title"><strong>Dimension reduction and interpretability</strong></p>
<p>Suppose you have statistical data that too many dimensions, in other
words too many variables of the same random process, that has been
observed many times. You want to find out, from all these variables (or all
these dimensions when speaking in terms of multivariate data …</p></div><div class="topic">
<p class="topic-title"><strong>Dimension reduction and interpretability</strong></p>
<p>Suppose you have statistical data that too many dimensions, in other
words too many variables of the same random process, that has been
observed many times. You want to find out, from all these variables (or all
these dimensions when speaking in terms of multivariate data),
what are the relevant combinations, or directions.</p>
</div>
<div class="section" id="dimension-reduction-with-pca">
<h2>Dimension reduction with PCA</h2>
<p>If we have three-dimensional data, for instance simultaneous measurements
made by three thermometers positioned at different locations in a room.
The data forms a cluster of points in a 3D space:</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/science/attachments/ica_pca/3d_data.jpg" style="width: 50%;" />
<p>If the temperature in that room is conditioned by only two parameters,
the setting of a heater and the outside temperature, we probably have
too much data: the three sets of measurements can be expressed as a
linear combination of two fluctuating variable, and an additional much
smaller noise parameter. In other words, the data mostly lies in a 2D
plane embedded in the 3D measurement space.</p>
<p>We can use PCA (Principal Component Analysis) to find this plane: PCA
will give us the orthogonal basis in which the covariance matrix of our
data is diagonal. The vectors of this basis point in successive
orthogonal directions in which the data variance is maximum. In the case
of data mainly residing on a 2D plane, the variance is much greater along
the two first vectors, which define our plane of interest, than along the
third one:</p>
<div class="figure align-center">
<img alt="" src="https://gael-varoquaux.info/science/attachments/ica_pca/3d_data_pca_axis.jpg" style="width: 50%;" />
<p class="caption">The covariance eigenvectors identified by PCA are shown in red. The
plane defined by the 2 largest eigenvectors is shown in light red.</p>
</div>
<p>If we look at the data in the plane identified by PCA, it is clear that
it was mostly 2D:</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/science/attachments/ica_pca/3d_data_pca.jpg" style="width: 50%;" />
</div>
<div class="section" id="understanding-pca-with-a-gaussian-model">
<h2>Understanding PCA with a Gaussian model</h2>
<p>Let <cite>x</cite> and <cite>y</cite> be two normal-distributed variables, describing the
processes we are observing:</p>
<div class="formula">
<i>x</i> = <span class="scriptfont">N</span>(0, 1)
</div>
<p>and</p>
<div class="formula">
<i>y</i> = <span class="scriptfont">N</span>(0, 1)
</div>
<p>Let <cite>a</cite> and <cite>b</cite> be two observation variables, linear combinations of <cite>x</cite>
and <cite>y</cite>:</p>
<div class="formula">
<i>a</i> = <i>x</i> + <i>y</i>
</div>
<p>and</p>
<div class="formula">
<i>b</i> = 2 <i>y</i>
</div>
<p>PCA is performed by applying an SVD (singular value decomposition) on the
observed data matrix:</p>
<div class="formula">
<i>Y</i> = [<i>a</i><sub>1</sub><i>a</i><sub>2</sub><i>a</i><sub>3</sub>...; <i>b</i><sub>1</sub><i>b</i><sub>2</sub><i>b</i><sub>3</sub>...]
</div>
<p>This is equivalent to find the eigenvalues and eigenvectors of
<span class="formula"><i>Y</i><sup> <i>T</i></sup><i>Y</i></span>, the correlation matrix of the observed data. The
multidimensional (or multivariate, in statistical jargon) probability
density function of Y is written:</p>
<div class="formula">
<i>p</i>(<i>Y</i>) ∼ <i>exp</i>( − <i>r</i><sup> <i>T</i></sup><i>M</i> <i>r</i>)
</div>
<p>where <cite>r</cite> is the position is the <cite>(a,b)</cite> observation space, and <cite>M</cite> the
correlation matrix. Diagonalizing the matrix <cite>M</cite> corresponds to finding
a rotation matrix <cite>U</cite> such that:</p>
<div class="formula">
<i>p</i>(<i>Y</i>) ∼ <i>exp</i>( − <i>r</i><sup> <i>T</i></sup><i>U</i><sup> <i>T</i></sup><i>S</i> <i>U</i> <i>r</i>)
</div>
<p>With <cite>S</cite> a diagonal matrix. In other words, <cite>U</cite> is a rotation of the
observation space to change to a basis where the probability density
function is written:</p>
<div class="formula">
<i>p</i>(<i>Y</i>) ∼ <i>exp</i>( − <span class="limits"><sup class="limit"> </sup><span class="limit"><span class="bigoperator">∑</span></span><sub class="limit"><i>i</i></sub></span> <i>σ</i><sub><i>i</i></sub> <i>r</i><span class="scripts"><sup class="script">2</sup><sub class="script"><i>i</i></sub></span>) = <span class="limits"><sup class="limit"> </sup><span class="limit"><span class="bigoperator">∏</span></span><sub class="limit"><i>i</i></sub></span> <i>exp</i>( − <i>σ</i><sub><i>i</i></sub> <i>r</i><span class="scripts"><sup class="script">2</sup><sub class="script"><i>i</i></sub></span>)
</div>
<p>In this new basis, <cite>Y</cite> can thus be interpreted as a sum of independent
normal processes of different variance.</p>
<p>We can thus picture the PCA as a way of finding independent normal
processes. The different steps of the argument exposed above can be
pictured in the following figure:</p>
<div class="figure align-center">
<img alt="" src="https://gael-varoquaux.info/science/attachments/ica_pca/pca_on_gaussian_data.png" style="width: 80%;" />
<p class="caption">First we represent samples drawn from
<cite>x</cite> and <cite>y</cite> in their original space, the basis of the independent
variables. Then we represent the (<cite>a</cite>, <cite>b</cite>) samples, and we apply PCA on
these samples, to estimate the eigenvectors of the covariance matrix.
Then we represent the data projected in the basis estimated by PCA. One
important detail to note, is that after PCA, the data is most often
rescaled: each direction is divided by the corresponding sample standard
deviation identified by PCA. After this operation, all directions of
space play the same role, the data is spheric, or “white”.</p>
</div>
<p>PCA was able to identify the original independent variables <cite>x</cite> and <cite>y</cite>
in the <cite>a</cite> and <cite>b</cite> samples only because they were mixed with different
variance. For a isotropic Gaussian model, any basis can describe the data
in terms of independent normal process.</p>
</div>
<div class="section" id="pca-on-non-normal-data">
<h2>PCA on non normal data</h2>
<p>More generally, the PCA algorithm can be understood as an algorithm
finding the direction of space with the highest sample variance, and
moving on to the orthogonal subspace of this direction to find the next
highest variance, and iteratively discovering an ordered orthogonal basis
of highest variance. This is well adapted to normal processes, as their
covariance is indeed diagonal in an orthogonal basis. In addition, the
resulting vectors come with a “PCA score”, ie the variance of the data
projected along the direction they define. Thus when using PCA for
dimension reduction, we can choose the subspace defined by the first <cite>n</cite>
PCA vectors, on the basis that they explain a given percentage of the
variance, and that the subspace they define is the subspace of dimension
<cite>n</cite> that explains the largest possible fraction of the total variance.</p>
<p>However, on strongly non-Gaussian processes, the variance may not be the
quantity of interest.</p>
<p>Let us consider the same model as above, with two independent variables
<cite>x</cite> and <cite>y</cite> thought with strongly non-Gaussian distributions. Here we
use a mixture of a narrow Gaussian, and wide one, to populate the tails:</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/science/attachments/ica_pca/non_gaussian_pdf.png" style="width: 40%;" />
<p>We can apply the same operations on these random variables: change of
basis to an observation basis made of <cite>a</cite> and <cite>b</cite>, and PCA on the
resulting sample:</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/science/attachments/ica_pca/pca_on_non_gaussian_data.png" style="width: 80%;" />
<p>We can see that the PCA did not properly identify the original
independent variables. The variance criteria is not good-enough when the
principle axis of the observed distribution are not orthogonal, as the
highest variance can be found in a direction mixing the two process.
Indeed the largest PCA direction is found slightly off axis. In addition
the second direction can only be found orthogonal to the first one, as
this is a restriction of PCA.</p>
<p>On the other side, the data after PCA is much more spheric than the
original data. No strong anisotropy is found in the central part of the
sample cloud, which contributes most to the variance.</p>
</div>
<div class="section" id="ica-independent-non-gaussian-variables">
<h2>ICA: independent, non-Gaussian variables</h2>
<p>For strongly non-Gaussian processes, the above example shows that
separating independent process should be done by looking at fine details
of the distribution, such as the tails. Indeed, after PCA, the Gaussian
part of the processes have been separated by their variance, and the
resulting, rescaled, samples cannot be decomposed in independent process
in a Gaussian model, as they all have the same variance, and would
already be considered independent under a Gaussian hypothesis.</p>
<p>A popular class of algorithms to separate independent sources, called ICA
(independent component analysis) makes the simplification that finding
independent sources out of such data can be reduced to finding maximally
non-Gaussian. Indeed, the central-limit theorem tells us that the sum of
non-Gaussian processes lead to Gaussian process. Conversely, with equal
variance multivariate samples, the more non-Gaussian a signal extracted
from the data, the less independent -and non-Gaussian- variables it
contains.</p>
<p>A good discussion of these arguments can be found in following paper:
<a class="reference external" href="http://www.cis.hut.fi/aapo/papers/IJCNN99_tutorialweb/IJCNN99_tutorial3.html">http://www.cis.hut.fi/aapo/papers/IJCNN99_tutorialweb/IJCNN99_tutorial3.html</a></p>
<p>ICA is thus an optimization algorithm that from the data extracts the
direction with the least-Gaussian PDF, removes the data explained by this
variable from the signal, and iterates.</p>
<p>Applying ICA to the previous model yields the following:</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/science/attachments/ica_pca/ica_on_non_gaussian_data.png" style="width: 80%;" />
<p>We can see that ICA has well identified the original independent data
variables. Its use of the tails of the distribution was paramount for
this task. In addition, ICA relaxes the constraint that all identified
directions must be perpendicular. This flexibility was also important to
match our data.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This discussion can now be seen as an <a class="reference external" href="http://scikit-learn.org/stable/auto_examples/decomposition/plot_ica_vs_pca.html">example of the scikit-learn</a>.
Thus you can replicate the figure using the code in the scikit.</p>
</div>
<!-- vim:set spell:
vim:set autoindent: -->
</div>
The SciPy 2009 proceedings are online2009-12-20T18:49:00+01:002009-12-20T18:49:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-12-20:/programming/the-scipy-2009-proceedings-are-online.html<p>We are finally announcing the online edition of SciPy proceedings:</p>
<p><a class="reference external" href="http://conference.scipy.org/proceedings/SciPy2009/">http://conference.scipy.org/proceedings/SciPy2009/</a></p>
<p>This year, we tried to raise the bar in terms of article quality. This
involved having a more strict review process, and we must thank a lot
all the reviewers. I have the feeling …</p><p>We are finally announcing the online edition of SciPy proceedings:</p>
<p><a class="reference external" href="http://conference.scipy.org/proceedings/SciPy2009/">http://conference.scipy.org/proceedings/SciPy2009/</a></p>
<p>This year, we tried to raise the bar in terms of article quality. This
involved having a more strict review process, and we must thank a lot
all the reviewers. I have the feeling it did improve the quality of the
final papers. Actually, I must say that there are some really nice
papers in the proceedings. I am not going to list them here, you can
have a glance at the contents, but they range from fairly technical
papers on tools development that are more in the software engineering
and computer science fields, to application papers demonstrating how the
tools can be used.</p>
<p>I must apologize for the time it took to publish the proceedings. All
this was actually a lot of work, and it has taken me a lot of energy. I
hope that you will it was worth it.</p>
Announcing EuroScipy 20102009-12-14T01:01:00+01:002009-12-14T01:01:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-12-14:/programming/announcing-euroscipy-2010.html<div class="section" id="the-3rd-european-meeting-on-python-in-science">
<h2>The 3rd European meeting on Python in Science</h2>
<p><strong>Paris, Ecole Normale Supérieure, July 8-11 2010</strong></p>
<p>We are happy to announce the 3rd EuroScipy meeting, in Paris, July 2010.</p>
<div class="line-block">
<div class="line">The EuroSciPy meeting is a cross-disciplinary gathering focused on the</div>
<div class="line">use and development of the Python language in scientific research. This</div>
<div class="line">event …</div></div></div><div class="section" id="the-3rd-european-meeting-on-python-in-science">
<h2>The 3rd European meeting on Python in Science</h2>
<p><strong>Paris, Ecole Normale Supérieure, July 8-11 2010</strong></p>
<p>We are happy to announce the 3rd EuroScipy meeting, in Paris, July 2010.</p>
<div class="line-block">
<div class="line">The EuroSciPy meeting is a cross-disciplinary gathering focused on the</div>
<div class="line">use and development of the Python language in scientific research. This</div>
<div class="line">event strives to bring together both users and developers of</div>
<div class="line">scientific tools, as well as academic research and state of the art</div>
<div class="line">industry.</div>
</div>
<div class="section" id="important-dates">
<h3>Important dates</h3>
<div class="line-block">
<div class="line"><strong>Registration opens</strong>: Sunday March 29</div>
<div class="line-block">
<div class="line"><strong>Paper submission deadline</strong>: Sunday May 9</div>
<div class="line"><strong>Program announced</strong>: Sunday May 22</div>
<div class="line"><strong>Tutorials tracks</strong>: Thursday July 8 - Friday July 9</div>
<div class="line"><strong>Conference track</strong>: Saturday July 10 - Sunday July 11</div>
</div>
</div>
</div>
<div class="section" id="tutorial">
<h3>Tutorial</h3>
<p>There will be two tutorial tracks at the conference, an introductory one,
to bring up to speed with the Python language as a scientific tool, and
an advanced track, during which experts of the field will lecture on
specific advanced topics such as advanced use of numpy, scientific
visualization, software engineering…</p>
</div>
<div class="section" id="main-conference-topics">
<h3>Main conference topics</h3>
<p>We will be soliciting talks on the follow topics:</p>
<ol class="arabic simple">
<li>Presentations of scientific tools and libraries using the Python
language, including but not limited to:<ul>
<li>Vector and array manipulation</li>
<li>Parallel computing</li>
<li>Scientific visualization</li>
<li>Scientific data flow and persistence</li>
<li>Algorithms implemented or exposed in Python</li>
<li>Web applications and portals for science and engineering</li>
</ul>
</li>
<li>Reports on the use of Python in scientific achievements or ongoing
projects.</li>
<li>General-purpose Python tools that can be of special interest to the
scientific community.</li>
</ol>
<p>Keynote Speaker: Hans Petter Langtangen</p>
<p>We are excited to welcome Hans Petter Langtangen as our keynote speaker.</p>
<ul class="simple">
<li>Director of scientific computing and bio-medical research at Simula
labs, Oslo</li>
<li>Author of the famous book Python scripting for computational science
<a class="reference external" href="http://www.springer.com/math/cse/book/978-3-540-73915-9">http://www.springer.com/math/cse/book/978-3-540-73915-9</a></li>
</ul>
<div class="section" id="the-organizers">
<h4>The organizers:</h4>
<div class="line-block">
<div class="line">Gaël Varoquaux (INRIA Saclay, Parietal), conference co-chair</div>
<div class="line-block">
<div class="line">Nicolas Chauvat (Logilab), conference co-chair</div>
</div>
</div>
</div>
<div class="section" id="program-committee">
<h4>Program committee</h4>
<div class="line-block">
<div class="line">Romain Brette (ENS Paris, DEC)</div>
<div class="line">Mike Müller (Python Academy)</div>
<div class="line">Christophe Pradal (CIRAD/INRIA, DigiPlantes team)</div>
<div class="line">Pierre Raybault (CEA, DAM)</div>
<div class="line">Jarrod Millman (UC Berkeley, Helen Wills NeuroScience institute)</div>
</div>
</div>
</div>
</div>
General relativity, quantum physics, freely-falling planes and Bayesian statistics2009-12-08T22:20:00+01:002009-12-08T22:20:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-12-08:/science/general-relativity-quantum-physics-freely-falling-planes-and-bayesian-statistics.html<p>We’re famous: the <a class="reference external" href="http://gael-varoquaux.info/science/acceleration-estimation-in-atom-interferometric-tests-of-the-einstein-equivalence-principle.html">work</a> that concluded my PhD is now picked up by the
press <a class="reference external" href="http://www.physorg.com/news179481148.html">http://www.physorg.com/news179481148.html</a></p>
<p>I hadn’t realized before reading this journalist’s version of the story,
but we have all the proper buzz words:</p>
<ul class="simple">
<li>general relativity</li>
<li>quantum physics</li>
<li>freely-falling planes</li>
<li>Bayesian …</li></ul><p>We’re famous: the <a class="reference external" href="http://gael-varoquaux.info/science/acceleration-estimation-in-atom-interferometric-tests-of-the-einstein-equivalence-principle.html">work</a> that concluded my PhD is now picked up by the
press <a class="reference external" href="http://www.physorg.com/news179481148.html">http://www.physorg.com/news179481148.html</a></p>
<p>I hadn’t realized before reading this journalist’s version of the story,
but we have all the proper buzz words:</p>
<ul class="simple">
<li>general relativity</li>
<li>quantum physics</li>
<li>freely-falling planes</li>
<li>Bayesian statistics.</li>
</ul>
<p>This kind of stuff makes great headlines, but the way we are judged on
this “success” is actually harmful (I believe), as there is so much
interesting research that lies away of the trendy words and that needs
to be done.</p>
Decoration in Python done right: Decorating and pickling2009-11-13T00:14:00+01:002009-11-13T00:14:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-11-13:/programming/decoration-in-python-done-right-decorating-and-pickling.html<p>Decoration is a fantastic pattern in Python that allows for very
light-weight metaprograming with functions rather than objects (see
<a class="reference external" href="http://www.ibm.com/developerworks/linux/library/l-cpdecor.html">this article</a> for an in-depth discussion). However, when decorating,
it is very easy to break another great feature of the language: its
reflectivity and its ability to do static representations of …</p><p>Decoration is a fantastic pattern in Python that allows for very
light-weight metaprograming with functions rather than objects (see
<a class="reference external" href="http://www.ibm.com/developerworks/linux/library/l-cpdecor.html">this article</a> for an in-depth discussion). However, when decorating,
it is very easy to break another great feature of the language: its
reflectivity and its ability to do static representations of its
internal objects: pickling.</p>
<p>In this blog post, I’d like to rewrite a post I made on the IPython
mailing list a month ago, summing up the few things to have in mind when
decorating a function.</p>
<div class="section" id="a-pattern-to-avoid">
<h2>A pattern to avoid?</h2>
<p>I have recently been revisiting my decoration code, to fight a common
mistake I had been doing, and it was partly due to the heavy use of a
simplified pattern for decorating:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">with_print</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" Decorate a function to print its arguments.</span>
<span class="sd"> """</span>
<span class="k">def</span> <span class="nf">my_func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">my_func</span>
<span class="nd">@with_print</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'f called'</span>
</pre></div>
<p>The nice thing about this pattern is that is it quite easy to type, and
to read.</p>
</div>
<div class="section" id="why-it-is-harmful">
<h2>Why it is harmful</h2>
<p>The decorated function is actually the function ‘my_func’, with a
reference to the original function ‘func’, a part of the scope of the
decorator ‘with_print’, and thus in the closure of the with_print
function.</p>
<p>The problem is that we have a closure here. Thus we have variables that
are hard to get to (the undecorated function), and the decorated
function is not picklable (which is more and more important to me, e.g.
for parallel computing).</p>
</div>
<div class="section" id="some-solutions">
<h2>Some solutions</h2>
<div class="section" id="avoiding-the-closure">
<h3>Avoiding the closure</h3>
<p>Use objects as a scope, rather than a closure:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">WithPrint</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">func</span> <span class="o">=</span> <span class="n">func</span>
<span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
</pre></div>
<p>This solution is not enough: the following code won’t pickle:</p>
<div class="highlight"><pre><span></span><span class="nd">@WithPrint</span>
<span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'g called'</span>
</pre></div>
<p>The reason this won’t pickle is that we have a name collision: the code
above expands to:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'g called'</span>
<span class="n">g</span> <span class="o">=</span> <span class="n">WithPrint</span><span class="p">(</span><span class="n">g</span><span class="p">)</span>
</pre></div>
<p>and trying to pickle raises the following PicklingError:</p>
<div class="highlight"><pre><span></span><span class="n">Can</span><span class="s1">'t pickle <function g at 0x6ed2a8>: it'</span><span class="n">s</span> <span class="ow">not</span> <span class="n">the</span> <span class="n">same</span> <span class="nb">object</span> <span class="k">as</span> <span class="n">__main__</span><span class="o">.</span><span class="n">g</span>
</pre></div>
<p>If we do:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'g called'</span>
<span class="n">h</span> <span class="o">=</span> <span class="n">WithPrint</span><span class="p">(</span><span class="n">g</span><span class="p">)</span>
</pre></div>
<p>we can pickle h, hurray!</p>
</div>
<div class="section" id="using-functools-wraps">
<h3>Using functools.wraps</h3>
<p>However, Python comes with the answer in the standard libary:
functools.wraps does the name unmangling.</p>
<p>Thus the following code produces a pickleable f:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">wraps</span>
<span class="k">def</span> <span class="nf">with_print</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" Decorate a function to print its arguments.</span>
<span class="sd"> """</span>
<span class="nd">@wraps</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">my_func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">my_func</span>
<span class="nd">@with_print</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="nb">print</span> <span class="s1">'f called'</span>
</pre></div>
<div class="line-block">
<div class="line">The pickling works simply because using functools.wraps resets the</div>
<div class="line">.func_name attribute of f to have a well-defined import path. Thus</div>
<div class="line">pickling works, simply by storing the import path, as all pickling of</div>
<div class="line">functions.</div>
</div>
<p>Notice that there is only a one-line difference with the original code!</p>
<p>I actually tend to use a combination of both solution (an object, using
functools.wraps), to keep a reference on the undecorated functions.</p>
<p><strong>Note</strong>: Demo code of this blog post can be found <a class="reference external" href="attachments/pickling_tests_py">here</a>.</p>
</div>
</div>
<div class="section" id="take-home-messages-for-pickling">
<h2>Take home messages for pickling</h2>
<ul class="simple">
<li>Decorators can be more clever than you think, and might not return
objects as simple as you think</li>
<li>Think about pickling, or you’ll get bitten at some point (for
instance when doing parallel computing).</li>
</ul>
<p>and most important:</p>
<ul class="simple">
<li>Use functools.wraps</li>
</ul>
</div>
<div class="section" id="a-remark-about-object-oriented-programming">
<h2>A remark about object-oriented programming</h2>
<p>To jump on the band-wagon behind <a class="reference external" href="http://blog.enthought.com/?p=223">Travis</a>, I believe that this
discussion teaches us a bit about object-oriented programming. When
decorating, we are really taking a callable object, and redefining how
the call is handled. If we do this the naive way, we loose
introspection (there is no way to access the original callable from
Python), and as a result pickling, and many of the nice feature going
with reflexivity in Python. This is because we trapped information in a
scope that is not accessible by normal Python code (without playing at
the frame level). If on the other hand, we accept that what we have
behind all this are nested scope with a control of lookups, and we
create a full-blown object, we have the benefits of the black box, and
the benefits of reflexivity.</p>
<p>But this is not the point I want to make. The point I want to make is
that, by decorating, we are piggy-backing on an absolutely universal
object/interface: the callable. Everybody knows what a callable is, and
knows how to employ it. From a pure object-oriented point of view,
decorating is simply some kind of proxy design pattern. But, to stress
Travis’s point, introducing new objects that have their own behavior
puts cognitive load on the programmer. The real value of decoration is
that it is object-oriented programming without adding any new or
surprising interface. You don’t really have to care what is going on,
you can still use the resulting ‘proxied’ function as the original
function: a simple function.</p>
</div>
Writing parallel code in a readable way2009-11-09T00:10:00+01:002009-11-09T00:10:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-11-09:/programming/writing-parallel-code-in-a-readable-way.html<p>Although I often have embarrasingly parallel problems (data parallel),
and I have an 8-CPU box at work, I used to frown on writing parallel
computing code when doing exploratory coding. We now have fantastic
parallel computing facilities in Python (amongst other,
<a class="reference external" href="http://docs.python.org/library/multiprocessing.html">multiprocessing</a>, <a class="reference external" href="http://ipython.scipy.org/doc/rel-0.9.1/html/parallel/index.html">IPython</a>, and <a class="reference external" href="http://www.parallelpython.com/">parallel Python</a>). However, in my
opinion …</p><p>Although I often have embarrasingly parallel problems (data parallel),
and I have an 8-CPU box at work, I used to frown on writing parallel
computing code when doing exploratory coding. We now have fantastic
parallel computing facilities in Python (amongst other,
<a class="reference external" href="http://docs.python.org/library/multiprocessing.html">multiprocessing</a>, <a class="reference external" href="http://ipython.scipy.org/doc/rel-0.9.1/html/parallel/index.html">IPython</a>, and <a class="reference external" href="http://www.parallelpython.com/">parallel Python</a>). However, in my
opinion, there are two reasons to hesitate to use them, especially when
the code is very imature (which is almost always my case, in research
settings):</p>
<ol class="arabic simple">
<li>It makes the code look less like the ideas it is trying to express.
Peter Norvig made <a class="reference external" href="http://www.archive.org/details/scipy09_day1_03-Peter_Norvig">a pretty convincing case</a> for scientific code
reading like math at <a class="reference external" href="http://conference.scipy.org/">SciPy2009</a>.</li>
<li>Because parallel computing is out of process, in Python, it is simply
harder to debug (though I hear that the IPython guys are on that).</li>
</ol>
<p>I have progressively developed a tiny tool to adress both problems, at
least for my embarrasingly-parallel problems. I address the second
problem by having a trivial switch to run my code without importing any
fancy parallel computing tools. And I address the first problem using
syntactic sugar to be able to type in map/reduce code that actually
looks like standard procedural code:</p>
<div class="highlight"><pre><span></span><span class="n">results</span> <span class="o">=</span> <span class="n">Parallel</span><span class="p">(</span><span class="n">n_jobs</span><span class="o">=</span><span class="mi">2</span><span class="p">)(</span>
<span class="n">delayed</span><span class="p">(</span><span class="n">my_calculation</span><span class="p">)(</span><span class="n">data1</span><span class="p">,</span> <span class="n">data2</span><span class="p">,</span>
<span class="n">parameter1</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">parameter2</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="k">for</span> <span class="n">data1</span> <span class="ow">in</span> <span class="n">store1</span> <span class="k">for</span> <span class="n">data2</span> <span class="ow">in</span> <span class="n">store2</span><span class="p">)</span>
</pre></div>
<p>There are several tricks here:</p>
<ol class="arabic simple">
<li>I use a ‘<em>delayed</em>‘ decorator that creates the argument list and
keyword argument dictionary for me so that I can type something that
really looks like a function call. Also, the decorator checks to see
if the function and the arguments can be pickled, because if not the
parallel computing libraries will raise errors, sometimes with
hard-to-understand messages.</li>
<li>I use list comprehension to create the list to apply the map/reduce
onto. List comprehension is really readable, and very powerful.</li>
<li>The ‘<em>Parallel</em>‘ object hides all the cleverness. If the
‘<em>n_jobs</em>‘ parameter is set to 1, it does not call any parallel
computing library. If it is set to -1, all the CPUs are used. The
object instantiates the parallel computing context and also destroys
it. While this is inefficient, it is great for catching errors early.
And finally, while I have implemented this only for the
multiprocessing module, any fork/join-based parallel computing
library could be encapsulated the same way, thus providing a uniform
API to do multi-node parallel computing or single-computer shared
memory (as multi-processing uses the Unix fork call, and all modern
Unices implement copy on write of memory pages, you get some shared
memory for free without worrying about race conditions).</li>
</ol>
<div class="topic">
<p class="topic-title"><strong>Update</strong></p>
<p>This pattern has actually evolved in the <a class="reference external" href="https://pythonhosted.org/joblib/">joblib project</a> ,
which provides a lot of cleverness under the hood.</p>
</div>
Acceleration estimation in atom-interferometric tests of the Einstein equivalence principle2009-11-07T15:24:00+01:002009-11-07T15:24:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-11-07:/science/acceleration-estimation-in-atom-interferometric-tests-of-the-einstein-equivalence-principle.html<p>Hurray! The pivot article that marks my transition from physics to
statistic modeling is finally out:</p>
<blockquote>
<a class="reference external" href="http://www.iop.org/EJ/article/1367-2630/11/11/113010/njp9_11_113010.pdf">How to estimate the differential acceleration in a two-species atom interferometer to test the equivalence principle</a>
<em>G Varoquaux, R A Nyman, R Geiger, P Cheinet, A Landragin and P Bouyer</em></blockquote>
<p>To put things in …</p><p>Hurray! The pivot article that marks my transition from physics to
statistic modeling is finally out:</p>
<blockquote>
<a class="reference external" href="http://www.iop.org/EJ/article/1367-2630/11/11/113010/njp9_11_113010.pdf">How to estimate the differential acceleration in a two-species atom interferometer to test the equivalence principle</a>
<em>G Varoquaux, R A Nyman, R Geiger, P Cheinet, A Landragin and P Bouyer</em></blockquote>
<p>To put things in context, at the end of my PhD, we had been building an
atom interferometer to test the Einstein equivalence principle and my
reflections on the limits of atom interferometry shifted from worrying
about the underlying physics, to worrying about the estimation: the
inverse problem of going from the experimental signal, to the underlying
quantities that we are measuring, confounded by all the horrible
experimental noise.</p>
<div class="section" id="atoms-light-gravity-fields-and-free-fall-planes">
<h2>Atoms, light, gravity fields and free-fall planes</h2>
<p>The problem is: we want to do high precision metrologic tests in a
free-falling plane. We use interferometry to measure gravity fields. But
rather than doing interferometry with light, we use atoms, that are much
more coupled to gravity. When probing gravity fields with light, the
trick is to use huge highly-sensitive interferometers. For instance the
<a class="reference external" href="http://www.ligo.caltech.edu/">ligo</a> and <a class="reference external" href="http://www.virgo.infn.it/">virgo</a> projects are kilometer-long light interferometers
listening for gravitational waves, and the <a class="reference external" href="http://www.ringlaser.org.nz/content/facilities.php">giant ring lasers</a> can test
for tiny modifications in the Earth rotation and gravity field.
Gravimetric coupling with matter waves and light waves describes the
<a class="reference external" href="http://www.turpion.org/php/paper.phtml?journal_id=pu&paper_id=6425">very exact same underlying physics</a>. However, matter waves, atoms in
the case of PhD, fall in gravity fields. While this is the expression of
the very exact phenomena we are trying to measure, it also means that to
build a very large atom interferometer, you have to let the atoms fall
for a large distance. And I can attest that even laboratory-sized
versions of atom-interferometric experiments are fairly nasty to
run:</p>
<img alt="" class="align-center" src="attachments/P1010619.jpg" style="width: 55%;" />
<p>This is why we simply decided to build an experiment in a
<a class="reference external" href="http://arxiv.org/pdf/0705.2922">freely-falling plane</a>: let’s fall with the atoms for 6
kilometers (30 seconds).</p>
<img alt="" src="http://gael-varoquaux.info/physics/ICELog/07/0328/DSCF0662.jpg" style="width: 40%;" />
<img alt="" src="http://gael-varoquaux.info/physics/ICELog/07/0327/100_6838.jpg" style="width: 40%;" />
</div>
<div class="section" id="measuring-free-fall-while-in-free-fall">
<h2>Measuring free fall, while in free fall?</h2>
<img alt="" class="align-right" src="attachments/coyote.png" />
<p>Of course, the plane is not really in free fall. The pilots try as hard
as possible to compensate for drag and atmospheric turbulence but there
is a limit to what they can achieve with an Airbus. The atoms are a
vacuum apparatus, so they are indeed in free fall (before they crash in
the side of the apparatus). However, making sens of measure of fall-free
made relative to an unstable, and unpredictable platform is not trivial.
This is where the statistical modeling kicked in. After reading a bit
about noise in interferometers, I realized that we had a well-known
problem in statistics: estimation of hidden variables from noisy
observations. I learned about <a class="reference external" href="http://www.google.fr/url?sa=t&source=web&ct=res&cd=1&ved=0CAcQFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FRecursive_Bayesian_estimation&ei=S331StLdCof34Ab117i4BA&usg=AFQjCNFeQT7-ruBii_IfqL5C7smW9jBL3Q&sig2=fYSw1ieKbBFPLqnoBEsdEQ">recursive Bayesian estimation</a>, coded a
proof-of-principle algorithm for our problem (in Python, of course), and
was sold. The rest of the story is about noise simulations, and trying
to convince a metrology community that you could perform good
measurements in a noisy environment.</p>
<p>It took us a lot of time (2 years) to write an article that was
acceptable to the target scientific community, while keeping the core
estimation and statistics message. Publishing new ideas is hard, because
you are not answering questions that people already have in mind. This
is why the fact that <a class="reference external" href="http://www.iop.org/EJ/abstract/1367-2630/11/11/113010">this article</a> is out is a huge deal for me. It
marks a turning point in my reflection: I switched from worrying only
about forward models, with which try to describe as well as possible the
system at hand, to inverse problems, in which you worry about estimating
the parameters from the data.</p>
<p>I was startled to see that people are ready to spend a huge amount of
money and efforts in improving complicated experiments involving quantum
physics and very sophisticated technology, but can be weary of
processing the output signal to increase statistical power. Scientific
communities have their own goals that they pitch (e.g. reducing the
phase noise in lasers) and there can be huge divides between different
scientific interests. Realizing this played an important role in <a class="reference external" href="http://gael-varoquaux.info/personnal/update-on-my-life.html">my
career shift</a>. I wanted to know more about the power of statistical
modeling and machine learning applied to real-life system. I decided
that to learn more, I had to work with people that had a different
culture from mine. It’s been a huge amount of fun so far… More about
that later.</p>
</div>
EuroScipy 2010 in Paris2009-10-27T23:22:00+01:002009-10-27T23:22:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-10-27:/programming/euroscipy-2010-in-paris.html<p>Next year’s EuroScipy will be in Paris, as Nicolas Chauvat and myself
announced in Leipzig this summer. We are still busy organizing, but we
have pretty much settled down for a dates: <strong>July 8th- July 11th</strong>. So
mark those dates, and get ready to come to Paris for a …</p><p>Next year’s EuroScipy will be in Paris, as Nicolas Chauvat and myself
announced in Leipzig this summer. We are still busy organizing, but we
have pretty much settled down for a dates: <strong>July 8th- July 11th</strong>. So
mark those dates, and get ready to come to Paris for a fantastic event
where Science meets computing thanks to Python.</p>
<p>On the Thursday and Friday, we will have 2 days of optional tutorials;
introductory ones to get up to speed with Python, and advanced ones,
where experts explain the tools they know best. On the Saturday and
Sunday, the main conference will be held, and if it is anywhere like
last year’s, we will be hearing thrilling discussions with topics
ranging from the latest libraries for better scientific computing to how
Python was used in top-notch scientific achievements.</p>
Useful trick for functions and tests using np.random2009-08-29T16:00:00+02:002009-08-29T16:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-08-29:/programming/useful-trick-for-functions-and-tests-using-nprandom.html<p class="first last">How to test functions that use the numpy random number generator</p>
<p>Recently, listening to Robert Kern taught a new useful trick to use when
you write functions that use the numpy random number generator:</p>
<p>As always, when using random number generation in code, the problem is
to get ‘repeatable results’. Of course, you want only repeatable
statistics, and with statistics, the problem is to define what
repeatable is. Anyhow, for various reasons, it is useful to be able to
reproduce exactly runs, for instance when testing, fine tuning, or
debugging. That is why you would like to be able to control the seed of
your random number generation. Robert Kern showed us (at the SciPy
conference tutorial) a way to control the pseudo random number generator
(PRNG) in a function, without affecting the rest of the code. This does
not involve setting the seed of the global PRNG, as this is evil,
because it has global effects. The idea is to pass in to your functions
a PRNG instance (by default the global one):</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="n">prng</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="p">):</span>
<span class="nb">print</span> <span class="n">pnrg</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
</pre></div>
<p>if you want to use your function with a controlled PRNG, you can
instantiate one with a specific seed:</p>
<div class="highlight"><pre><span></span><span class="n">prng</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">RandomState</span><span class="p">(</span><span class="n">seed</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
</pre></div>
<p>and pass it to your function.</p>
SciPy 2009 is over!2009-08-29T12:21:00+02:002009-08-29T12:21:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-08-29:/programming/scipy-2009-is-over.html<p>The week is over, and I am finally catching up with things, back here in
France.</p>
<p>The SciPy conference was exciting and fun as usual. It was great to meet
old friends and put faces on names on the mailing list.</p>
<p>The turn out was very good: we had 150 …</p><p>The week is over, and I am finally catching up with things, back here in
France.</p>
<p>The SciPy conference was exciting and fun as usual. It was great to meet
old friends and put faces on names on the mailing list.</p>
<p>The turn out was very good: we had 150 people total. This is more than
last year (125), which shows that there is high interest, given that
most institutions have travel restrictions due to this year’s low
budget.</p>
<p>The year, the conference was very international. I was really happy
that, partly thanks to the <a class="reference external" href="http://conference.scipy.org/student-funding">PSF contribution</a>, we had the visit of
young contributors coming from far away, such as David Cournapeau
(Japan), Dag Sverre Seljebotn (Norway), Pauli Virtanen (Finland), and
Stéfan van der Walt (South Africa). For me, living in France, it was
also great to have people from major European institutions, such as the
<a class="reference external" href="http://conference.scipy.org/abstract?id=7">ESRF</a> (European Synchrotron Radiation Facility), the <a class="reference external" href="http://conference.scipy.org/abstract?id=14">Fraunhofer
institute</a>, the <a class="reference external" href="http://conference.scipy.org/abstract?id=14">Max Planck Institute</a>. These people not only
committed important projects to the scientific Python tools, but made
the effort of coming all the way to California to talk about it, which
is non negligible given the cost of the trip all the way from Europe. To
me this is important because it means that we are getting more
interaction wordwide, and thus the tools are more likely to converge to
something of generaluse. Also, for the first time ever, one of my bosses
came to the conference. It is fantastic to be working with great
scientists who actually understand that technology is important to do
good science, and that programming is actually hard, and a matter of
interest per se.</p>
<p>On the other hand, I was disappointed that we had no presentations from
the industry. There were a lot of industry people in the audience, and
it is always fun to here what they use Python for.</p>
<p>I really enjoyed the keynote by Peter Norvig because Peter talked about
the importance of having a clear language to expose and formulate
scientific ideas. This is something that is very dear to me, and I must
say that the code snippets he presented were crystal clear, and
involving non-trivial maths explained in a way that made them look
simple, similar to his famous blog post on <a class="reference external" href="http://norvig.com/spell-correct.html">spell checking</a>. It was
really inspiring for me, and driving me into trying to write even
cleared and simpler code.</p>
<p>The technical keynote by Jon Guyer also hit a soft spot for me, not only
because the physics presented was very beautiful, but also because my
partner is doing research in similar fields (with Python, of course),
and Jon made an excellent argument for using Python, which is not always
easy when you are discussing heavily computational problems.</p>
<p>For my personal work, SciPy was very exciting, because I had so many
discussions with different people on how we could share efforts, by
tweaking a data structure in an existing package, or simply having a
look at a package I wasn’t aware of. The machine learning BoF was
extremely enthusiastic, and I am really looking forward to October, when
we will be able to start working on that. If only half of the things was
talked about ever get done, I will be thrilled.</p>
<p>I should point out that, thanks to hard work by Jeff Teeters and Kilian
Koepsell from Berkeley, the videos of every talk are on the <a class="reference external" href="http://www.archive.org/search.php?query=">web</a> for
the first time.</p>
<p>Also, we have a nice <a class="reference external" href="http://www.flickr.com/photos/irees/sets/72157622006161097/">photo gallery</a> with a group picture.</p>
<p>We have so many people to thank. I think special thanks go to Leah
Jones, at Enthought, and Julie Ponce, at CACR, Caltech. They made sure
that the organization committee didn’t forget anything important and did
a lot of the grunt work. Thanks also to Enthought and CACR, and many of
their staff, for the support in the organization. PSF founded students,
and that is a big deal. We should thank all the tutorial presenters, it
takes a lot of work to put together the material. We were very grateful
to the <a class="reference external" href="http://conference.scipy.org/organizers">program committee</a> for reviewing the papers. Also thanks to all
the speakers, and to all the attendees. The SciPy conference is a bit
special to me, because it is very laid back, and I can trust that it
will be great almost by self-organization, as you put together nice and
clever people, and they find ways of discussing of interested things
with enthusiasm.</p>
<p><strong>Update:</strong> That blog post feels way too ‘political’. I dislike sale
pitches, and it does feel like one. But, how to sum up some important
contributions and thanks the people who helped out? I made a point of
always wearing a T-shirt of the conference, rather than a shirt, but I
guess that there is a point at which trying to dodge etiquette with a
T-shirt and a pony tail is just another cliché[*] and formalism.</p>
<p>[*] <em>cliché</em> is a French word for cliché.</p>
Mayavi: 2 videos of tutorial-like presentation2009-07-16T23:35:00+02:002009-07-16T23:35:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-07-16:/programming/mayavi-2-videos-of-tutorial-like-presentation.html<p>I gave a presentation on Mayavi in the Python for science seminar
organised by Fernando Perez at Berkeley. I was loudmouth and obnoxious
as usual, and unfortunately for me, I was recorded.</p>
<p>More seriously, Jeff Teeters has filmed the presentation and recorded
the sound was a microphone I was wearing …</p><p>I gave a presentation on Mayavi in the Python for science seminar
organised by Fernando Perez at Berkeley. I was loudmouth and obnoxious
as usual, and unfortunately for me, I was recorded.</p>
<p>More seriously, Jeff Teeters has filmed the presentation and recorded
the sound was a microphone I was wearing. I find that he did a really
excellent job. Getting a good recording is <strong>hard</strong>, and he really got
good audio and good framing. I am amazed and I don’t know how to thank
him enough.</p>
<p><a class="reference external" href="http://www.archive.org/details/ucb_py4science_2009_07_14_Gael_Varoquaux">http://www.archive.org/details/ucb_py4science_2009_07_14_Gael_Varoquaux</a></p>
<p>Also, Stefan van der Waalt gave a talk in Souft Africa on Mayavi that
was recorded. Another very useful resource for learning Mayavi:</p>
<p><a class="reference external" href="http://www.archive.org/details/ctpug-2008-09-mayavi">http://www.archive.org/details/ctpug-2008-09-mayavi</a></p>
Announcing the SciPy conference schedule2009-07-16T03:02:00+02:002009-07-16T03:02:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-07-16:/programming/announcing-the-scipy-conference-schedule.html<p>The SciPy conference committee is pleased to announce the schedule of
the conference:</p>
<p><a class="reference external" href="http://conference.scipy.org/schedule">http://conference.scipy.org/schedule</a></p>
<p>This year’s program is very rich. In order to limit the number of
interesting talks that we had to turn down, we decided to reduce the
length of talks. Although this …</p><p>The SciPy conference committee is pleased to announce the schedule of
the conference:</p>
<p><a class="reference external" href="http://conference.scipy.org/schedule">http://conference.scipy.org/schedule</a></p>
<p>This year’s program is very rich. In order to limit the number of
interesting talks that we had to turn down, we decided to reduce the
length of talks. Although this results in many short talks, we hope that
it will foster discussions, and give new ideas. Many subjects are
covered, both varying technical subject in the scientific computing
spectrum, and covering a lot of different research areas.</p>
<p>I would personally like to thank the members of the program committee,
who spent time reviewing the proposed abstracts and giving the chairs
feedback.</p>
<p>Fernando Perez and the tutorial presenters are hard at work finishing
planning all the details of the two-day tutorial session that will
precede the conference. An introduction tutorial track and an advanced
tutorial track, both covering various aspect of scientific computing in
Python, presented by experts in the field, should help many people
getting up to speed on the amazing technology driving this community.</p>
<p><strong>The SciPy 2009 program committee</strong></p>
<ul class="simple">
<li><strong>Co-Chair</strong> Gaël Varoquaux, Applied Mathematics and Neuroscience,
Neurospin, CEA - INRIA Saclay (France)</li>
<li><strong>Co-Chair</strong> Stéfan van der Walt, Applied Mathematics, University of
Stellenbosch (South Africa)</li>
<li>Michael Aivazis, Center for Advanced Computing Research, California
Institute of Technology (USA)</li>
<li>Brian Granger, Physics Department, California Polytechnic State
University, San Luis Obispo (USA)</li>
<li>Aric Hagberg, Theoretical Division, Los Alamos National Laboratory
(USA)</li>
<li>Konrad Hinsen, Centre de Biophysique Moléculaire, CNRS Orléans
(France)</li>
<li>Randall LeVeque, Mathematics, University of Washington, Seattle (USA)</li>
<li>Travis Oliphant, Enthought (USA)</li>
<li>Prabhu Ramachandran, Department of Aerospace Engineering, IIT Bombay
(India)</li>
<li>Raphael Ritz, International Neuroinformatics Coordinating Facility
(Sweden)</li>
<li>William Stein, Mathematics, University of Washington, Seattle (USA)</li>
</ul>
<p><strong>Conference Chair</strong>: Jarrod Millman, Neuroscience Institute, UC
Berkeley (USA)</p>
My article on scientific computing with Python2009-07-13T03:23:00+02:002009-07-13T03:23:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-07-13:/programming/my-article-on-scientific-computing-with-python.html<p>I have never sold the rights to the article I published in LinuxMagazine
France on scientific computing with Python. So I am uploading it to the
net, under a CC-by-SA license : <a class="reference external" href="http://hal.inria.fr/hal-00776672/">http://hal.inria.fr/hal-00776672/</a></p>
<p>It is in French, so it restricts the audience.</p>
Tutorial on scientific use of Python2009-07-08T19:38:00+02:002009-07-08T19:38:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-07-08:/programming/tutorial-on-scientific-use-of-python.html<p>The notes of the tutorial I gave on scientific use of Python at PyconFR
are online. They are in French, but I am giving the link here, just in
case it is needed:</p>
<p><a class="reference external" href="http://dl.afpy.org/pycon-fr-09/python_scientifique/index.html">http://dl.afpy.org/pycon-fr-09/python_scientifique/index.html</a></p>
Object-oriented design: framework objects versus data containers2009-07-01T05:13:00+02:002009-07-01T05:13:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-07-01:/programming/object-oriented-design-framework-objects-versus-data-containers.html<p>I find that in object oriented design, there are two kinds of objects:</p>
<ul class="simple">
<li>A first kind is the object encoding logics. This is an object for
which clever and complex design will hold together the logics of a
state-full application. It can often be part of a forest of objects …</li></ul><p>I find that in object oriented design, there are two kinds of objects:</p>
<ul class="simple">
<li>A first kind is the object encoding logics. This is an object for
which clever and complex design will hold together the logics of a
state-full application. It can often be part of a forest of objects
that are linked together via design patterns. The interfaces of these
objects are driven by their active role in the application. These
objects are prominently present in interactive application and
interactive application. They are mostly particular to an application
or a framework, and are mostly implementation-defined.</li>
<li>The second type of object is a data container. It strives to expose a
data model that can be of use in various situations, as it is the
link between different parts of the code that do not talk to each
other apart through data. It is responsible for loose coupling
(something that is very important to achieve a maintainable code
base) by having a light and shallow interface. It must be
interfaced-designed, rather than implementation-designed. One should
very easily get a grasp, an almost physical feeling, for the object
by simple interaction with it. I have what I call the ‘explaining
test’ for these objects: can I explain fully and completely to
somebody what the object does, and any possible caveat, without being
sidetracked into special discussions? If not, back to the drawing
board: the object will not gain acceptance. In my experience, only
the objects of the second kind can easily be shared between different
projects.</li>
</ul>
SciPy abstract submission deadline extended2009-06-27T08:14:00+02:002009-06-27T08:14:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-06-27:/programming/scipy-abstract-submission-deadline-extended.html<p>Greetings,</p>
<div class="line-block">
<div class="line">The conference committee is extending the deadline for abstract</div>
<div class="line-block">
<div class="line">submission for the Scipy conference 2009 one week.</div>
</div>
</div>
<div class="line-block">
<div class="line">On Friday July 3th, at midnight Pacific, we will turn off the abstract</div>
<div class="line-block">
<div class="line">submission on the conference site. Up to then, you can modify the</div>
<div class="line">already-submitted abstract, or submit new abstracts.</div>
</div>
</div>
<p><strong>The …</strong></p><p>Greetings,</p>
<div class="line-block">
<div class="line">The conference committee is extending the deadline for abstract</div>
<div class="line-block">
<div class="line">submission for the Scipy conference 2009 one week.</div>
</div>
</div>
<div class="line-block">
<div class="line">On Friday July 3th, at midnight Pacific, we will turn off the abstract</div>
<div class="line-block">
<div class="line">submission on the conference site. Up to then, you can modify the</div>
<div class="line">already-submitted abstract, or submit new abstracts.</div>
</div>
</div>
<p><strong>The SciPy 2009 executive committee</strong></p>
<ul class="simple">
<li>Jarrod Millman, UC Berkeley, USA (Conference Chair)</li>
<li>Gaël Varoquaux, INRIA Saclay, France (Program Co-Chair)</li>
<li>Stéfan van der Walt, University of Stellenbosch, South Africa
(Program Co-Chair)</li>
<li>Fernando Pérez, UC Berkeley, USA (Tutorial Chair)</li>
</ul>
SciPy 2009 conference opened up for registration2009-06-19T14:53:00+02:002009-06-19T14:53:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-06-19:/programming/scipy-2009-conference-opened-up-for-registration.html<p>We are finally opening the registration for the SciPy 2009 conference.
It took us time, but the reason is that we made careful budget
estimations to bring the registration cost down.</p>
<p>We are very happy to announce that this year registration to the
conference will be only $150, tutorial $100 …</p><p>We are finally opening the registration for the SciPy 2009 conference.
It took us time, but the reason is that we made careful budget
estimations to bring the registration cost down.</p>
<p>We are very happy to announce that this year registration to the
conference will be only $150, tutorial $100, and students get half
price! We made this effort because we hope it will open up the
conference to more people, especially students that often have to
finance such trip with little budget. As a consequence, however,
catering at noon is not included.</p>
<p>This does not mean that we are getting a reduced conference. Quite on
the contrary, this year we have two keynote speakers. And what speakers:
Peter Norvig and Jon Guyer! Peter Norvig is the director of research at
Google and Jon Guyer is a research scientist at NIST, in the
Thermodynamics and Kinetics Group, where he leads a fiPy, a finite
element project in Python.</p>
<div class="section" id="the-scipy-2009-conference">
<h2>The SciPy 2009 Conference</h2>
<p>SciPy 2009, the <a class="reference external" href="http://conference.scipy.org/">8th Python in Science conference</a>, will be held from
August 18-23, 2009 at Caltech in Pasadena, CA, USA.</p>
<p>Each year SciPy attracts leading figures in research and scientific
software development with Python from a wide range of scientific and
engineering disciplines. The focus of the conference is both on
scientific libraries and tools developed with Python and on scientific
or engineering achievements using Python.</p>
</div>
<div class="section" id="call-for-papers">
<h2>Call for Papers</h2>
<p>We welcome contributions from the industry as well as the academic
world. Indeed, industrial research and development as well academic
research face the challenge of mastering IT tools for exploration,
modeling and analysis.</p>
<p>We look forward to hearing your recent breakthroughs using Python!
Please read the <a class="reference external" href="http://conference.scipy.org/call_for_papers">full call for papers</a>.</p>
</div>
<div class="section" id="important-dates">
<h2>Important Dates</h2>
<ul class="simple">
<li>Friday, June 26: Abstracts Due</li>
<li>Saturday, July 4: Announce accepted talks, post schedule</li>
<li>Friday, July 10: Early Registration ends</li>
<li>Tuesday-Wednesday, August 18-19: Tutorials</li>
<li>Thursday-Friday, August 20-21: Conference</li>
<li>Saturday-Sunday, August 22-23: Sprints</li>
<li>Friday, September 4: Papers for proceedings due</li>
</ul>
<div class="section" id="the-scipy-2009-executive-committee">
<h3>The SciPy 2009 executive committee</h3>
<ul class="simple">
<li>Jarrod Millman, UC Berkeley, USA (Conference Chair)</li>
<li>Gaël Varoquaux, INRIA Saclay, France (Program Co-Chair)</li>
<li>Stéfan van der Walt, University of Stellenbosch, South Africa
(Program Co-Chair)</li>
<li>Fernando Pérez, UC Berkeley, USA (Tutorial Chair)</li>
</ul>
<p><strong>Update</strong>: I correct the typo in the original blog post: the sprints
are free, the tutorial are $100.</p>
</div>
</div>
Fuzzy on OOP and the French2009-06-14T10:38:00+02:002009-06-14T10:38:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-06-14:/programming/fuzzy-on-oop-and-the-french.html<p><a class="reference external" href="http://www.voidspace.org.uk/python/weblog/arch_d7_2009_06_13.shtml#e1097">Fantastic</a>:</p>
<blockquote class="epigraph">
Haha - I shake my fuzzywuzzy beard at you in bewilderment. Do you people
dislike OOP, the class statement is mere boilerplate to you, I mumble
incoherent French obscenities in your general direction. (Did you know
the French acronym for object-oriented programming is POO?).</blockquote>
Job offering for junior Python developer2009-06-07T19:53:00+02:002009-06-07T19:53:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-06-07:/programming/job-offering-for-junior-python-developer.html<p>Our lab is seeking to hire an engineer to work on porting our machine
learning code to the scikit learn, adding tests and documentation and
packaging it.</p>
<p>We are looking for someone motivated by quality in software and open
source. No prior scientific computing experience is required. You will
be …</p><p>Our lab is seeking to hire an engineer to work on porting our machine
learning code to the scikit learn, adding tests and documentation and
packaging it.</p>
<p>We are looking for someone motivated by quality in software and open
source. No prior scientific computing experience is required. You will
be working in a highly stimulating research environment (<a class="reference external" href="http://www-dsv.cea.fr/neurospin/">Neurospin</a>),
near Paris and employed by the French research institute in computer
science and applied math (<a class="reference external" href="http://www.inria.fr/saclay/home/view?set_language=en">INRIA</a>), a prestigious institution.</p>
<p><a class="reference external" href="http://www-dsv.cea.fr/neurospin/">Neurospin</a> is a research institute dedicated to the understanding of
the brain. You will be working with computer-assisted neurology
laboratory, the image-analysis and branch of Neurospin, in the small
‘Parietal’ INRIA team embedded in NeuroSpin and dedicated to statistical
modeling.</p>
<p>Over the years, the lab has developed a set of tools for machine
learning and statistical analysis in Python (with some C). There are
some tools for this purpose available in the open-source world
(BSD-licensed) in the scikit learn. We want to extract the good and
unique parts of our internal library, and release it in the open source
world through the scikit learn. Our code is fully Python code, using
scipy and matlab, with some bindings to R. As we want the code to be
BSD-licensed, we will remove the bindings with R, and replace when
possible. The job does not involve developing new algorithms, but
testing, improving, and documenting the existing one. There is a big
quality assurance work to be done. The code needs to be put to the right
coding standards; APIs should be cleaned; tests added. Dead code should
be delete. There is some optimization work to be done. Also, if there is
any duplicated funcitonnality with the scikit learn, you should analyse
both code and determine which one to code. The job also involves working
with the community, documentating the code, and releasing the project,
including binary packages. And finally, all the original authors of the
algorithms, and experts in the field, are in the lab. So you will be
able to learn from them and pester them if there is a problem with the
code.</p>
<p>In one word, this is about transforming an internal project, into a
leading open source project that will rock and live on!</p>
<p>The job description is available <a class="reference external" href="http://www.inria.fr/travailler/mrted/fr/jd/details.html?id=PGTFK026203F3VBQB6G68LONZ&LOV5=4510&ContractType=4545&LG=FR&Resultsperpage=20&nPostingID=3487&nPostingTargetID=7675&option=52&sort=DESC&nDepartmentID=10">here</a>.</p>
<p>There are to caveats: first it is a 2 year position. Second, you need to
have graduated recently (how recently I don’t know exactly, but I will
inquire).</p>
<p>If you are interested, or just want to ask questions, don’t hesitate to
send me an e-mail, I am _really_ looking forward to collaborate with
someone motivated on this project.</p>
<p><strong>UPDATE:</strong> I have more details on the restrictions of the job offering:
you need to have graduated in 2008 or 2009. This is a very hard
restriction, and I am recieving many excellent CVs that I even consider
because of this restriction. I am sorry, I cannot do anything about
it<strong>.</strong></p>
Pycon FR: presentations and tutorials2009-05-16T16:25:00+02:002009-05-16T16:25:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-05-16:/programming/pycon-fr-presentations-and-tutorials.html<p>May 30th and 31st the French Python conference, <a class="reference external" href="http://fr.pycon.org">Pycon FR</a>, will be
held at ‘la citée des sciences’, la Villette, in Paris.</p>
<p>The first day, I will be giving a one-hour-long tutorial (in French) on
numpy, scipy, and all the Python for Science jazz. On the following day,
I will …</p><p>May 30th and 31st the French Python conference, <a class="reference external" href="http://fr.pycon.org">Pycon FR</a>, will be
held at ‘la citée des sciences’, la Villette, in Paris.</p>
<p>The first day, I will be giving a one-hour-long tutorial (in French) on
numpy, scipy, and all the Python for Science jazz. On the following day,
I will be giving a half-hour-long talk to ilustrate the use of Python in
my current work: statistical analysis and modelling of brain activity.</p>
<p>I’ll be giving my tutorial in one room, while David Larlet (the famous
Biologeek) will be giving one on Django in another room. Tough
competition :-P .</p>
<p>The <a class="reference external" href="http://fr.pycon.org/sessions">program</a> of the conference is very eclectic, ranging from general
programming talks, to GUIs or web development. While this might deter
the pure scientific computing folks, I strongly encourage you to attend.
Indeed, a lot of the development, packaging, quality assurance, …
problems encountered in scientific computing are universal in computing.</p>
<p>You might think that you are only interested in writing algorithms,or
processing data, but this code will have to live on. My experience is
that it is terribly hard to have code in a lab that can be somewhat
shared and live on when people move away to another lab, or stop having
time to maintain the code. Talks like</p>
<ul class="simple">
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/correction_d'un_bug_et_naissance_d'une_nouvelle_fonctionnalité_dans_cpython">Correction d’un bug et naissance d’une nouvelle fonctionnalité dans
CPython</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/contrôle_de_versions_de_source%3A_pourquoi%3F_comment%3F">Contrôle de versions de source: pourquoi? comment?</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/de_la_qualité_dans_un_projet_en_python">De la qualité dans un projet en Python</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/amusez-vous_à_tester_avec_les_objets_farceurs_(mock)">Amusez-vous à tester avec les objets farceurs (mock)</a></li>
</ul>
<p>can probably be of some use.</p>
<p>Also, don’t underestimate the fact that some other communities might
have solved some of the issues you struggle with. When dealing with
real-world problems, and not only developing algorithms on a few set of
test data, a large fraction of the code lines and related to IO,
interfaces, data massaging… Two years ago, I remember that I was not
terribly interested in the web-development talks. I tried to be
open-minded and listen to them, but… Now I have done a bit of web
development myself, and I have played with some of the famous ‘web
frameworks’. I can tell you, there are some really interesting concepts
there. The web guys have managed to extract a set of patterns from the
problems they face and provide excellent abstracts to data handling and
display. Can we learn from them? I am especially interested in getting
more insight from things like ORMs (object relational mappers), and
understanding better the web frameworks:</p>
<ul class="simple">
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/django-roa_pour_une_architecture_orient%C3%A9e_ressources">Django-ROA pour une architecture orientée ressources</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/django-roa_pour_une_architecture_orient%C3%A9e_ressources">PyQt4: Un exemple de sur-mesure en Model/View/Delegate</a> (this is
not about web, but MVC/MVD pattern has been used in web a lot and is
universal and very important, IMHO).</li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/oubliez_le_sql_avec_sqlalchemy">Oubliez le sql avec SQLAlchemy</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/developpement_d'applications_maintenables_avec_django">Developpement d’applications maintenables avec Django</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/turbogears_2%2C_présentation_et_introduction_(tutoriel)">Turbogears 2, présentation et introduction (tutoriel)</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/programmer_couchdb_avec_couchdbkit">Programmer CouchDB avec couchdbkit</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/réflexion_sur_l'utilisation_de_python_pour_le_développement_d'une_plateforme_web_d'annotation_génomique">Réflexion sur l’utilisation de python pour le développement d’une
plateforme web d’annotation</a><a class="reference external" href="http://fr.pycon.org/sessions/seances/réflexion_sur_l'utilisation_de_python_pour_le_développement_d'une_plateforme_web_d'annotation_génomique">génomique</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/django_par_la_pratique%2C_premiers_pas">Oubliez le sql avec
SQLAlchemy</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/django_par_la_pratique%2C_premiers_pas">Django par la pratique</a></li>
<li><a class="reference external" href="http://fr.pycon.org/sessions/seances/python_et_les_bases_de_données_non_sql.">Python et les bases de données non sql.</a></li>
</ul>
<p>And finally, one more reason to come: it is so nice to actually get to
meet in real life people, and have a chat.</p>
<p>So, see you there, for those who live in France.</p>
Minimum spanning tree2009-05-10T23:52:00+02:002009-05-10T23:52:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-05-10:/programming/minimum-spanning-tree.html<p>Gary Ruben came up with the excellent idea of visualizing the minimum
spanning tree of a Delaunay tesselation in addition to Delaunay
tessalation itself. After he sent me his code, I spent some times
playing with it, because I found out that, with the right choice of
visualization parameter, it …</p><p>Gary Ruben came up with the excellent idea of visualizing the minimum
spanning tree of a Delaunay tesselation in addition to Delaunay
tessalation itself. After he sent me his code, I spent some times
playing with it, because I found out that, with the right choice of
visualization parameter, it gave me a nice understanding of what a
minimum spanning tree was: a tree structure of minimal total length
connecting all the vertices of the graphs, and embedded in the graph. On
the visualization, the Delaunay graph is displayed in grey, and the
minimum spanning tree in thick and colors.</p>
<img alt="" src="attachments/mst.jpg" />
<p>The minimum spanning tree is calculated using Prim’s algorithms, on the
fullly-connected distance-weighted graph of all points. One can clearly
see that is it embedded in the Delaunay graph. In fact I have tested
that calculating a minimum spanning tree on the Delaunay graph, or on
the complete graph, gave the same result.</p>
<p>The code to create this picture can be found <a class="reference external" href="attachments/mst_py">here</a>.</p>
Extracting the data from the Delaunay triangulation2009-05-01T16:42:00+02:002009-05-01T16:42:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-05-01:/programming/extracting-the-data-from-the-delaunay-triangulation.html<p>Gary Ruben just asked me if it was possible to retrieve the
triangulation information from my previous Delaunay example. Actually
the reason I came up with this example is that Emanuelle Gouillart, my
partner[*], needed to do Delaunay triangulation on some data. She was
kind enough to extract that code …</p><p>Gary Ruben just asked me if it was possible to retrieve the
triangulation information from my previous Delaunay example. Actually
the reason I came up with this example is that Emanuelle Gouillart, my
partner[*], needed to do Delaunay triangulation on some data. She was
kind enough to extract that code from her code base. <a class="reference external" href="attachments/extract_delaunay_edges_py">Here</a> it is.</p>
<p>[*] The various languages do not seem to have evolved quickly enough to
cope with the fact that people can now have a stable long-term
relationship with someone you are not married to. What word should I be
using here: ‘girlfriend’, ‘partner’… ?</p>
Mayavi image of the … month2009-04-27T22:42:00+02:002009-04-27T22:42:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-04-27:/programming/mayavi-image-of-the-month.html<p>Tonight I sat down and played a bit with VTK’s Delaunay tessalation
filter. I wanted to inspect the local structure of a graph created by
Delaunay tessalation of random points. To see better the structure, I
selected a slab of the resulting unstructured grid. I think the image is …</p><p>Tonight I sat down and played a bit with VTK’s Delaunay tessalation
filter. I wanted to inspect the local structure of a graph created by
Delaunay tessalation of random points. To see better the structure, I
selected a slab of the resulting unstructured grid. I think the image is
not only instructive to explain what a Delaunay tessalation is, it also
looks pretty cool. Here is the image and the Mayavi <a class="reference external" href="attachments/delaunay_py">script</a> that
creates it.</p>
<img alt="" src="attachments/delaunay_mayavi.jpg" />
Long sys.path and consequences, one more reason not to use easy_install2009-04-09T08:43:00+02:002009-04-09T08:43:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2009-04-09:/programming/long-syspath-and-consequences-one-more-reason-not-to-use-easy_install.html<p>For those who don’t know, sys.path is the path that the Python
interpreter traverse at each module import to look for the module file
imported.</p>
<p>This blog post is about the consequences of having a long sys.path. I’ll
try and make it short, but I would …</p><p>For those who don’t know, sys.path is the path that the Python
interpreter traverse at each module import to look for the module file
imported.</p>
<p>This blog post is about the consequences of having a long sys.path. I’ll
try and make it short, but I would have a lot to say. I am just reacting
on <a class="reference external" href="http://artificialcode.blogspot.com/2009/04/short-circuiting-python-module-lookup.html">Noah Gift’s post on performance improvement</a>, not making a full
essay on why overloading sys.path is considered harmful.</p>
<p>When using easy_install (or setuptools), each new project is installed
in a different directory, and the directory is added at runtime to the
sys.path (the addition at runtime confuses many users who are not aware
of it). As a result, you quickly end up with more than 40 directory on
your sys.path. These directories are ‘stat-ed’ one after the other on
each module import. Thus if you have a long sys.path, there are a large
amount of system calls to read directories. To check this out, simply
try:</p>
<div class="highlight"><pre><span></span>strace<span class="w"> </span>python<span class="w"> </span>-c<span class="w"> </span><span class="s2">"import foobar"</span><span class="w"> </span><span class="m">2</span>><span class="p">&</span><span class="m">1</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>less
</pre></div>
<p>You can see the amount of noise created by a simple (failing) import
statement. On a system with high latency (such as an NFS, as we use at
work), this is very costly.</p>
<p>Noah joyfully reports performance improvements by hijacking the Python
import mechanism. I claim that part of what Noah has done is not really
hijacking the import mechanism, it is undoing the hijacking performed by
setuptools.</p>
<p>I know I am being rude, but many people raised this point before, and it
is not getting any traction from the setuptools maintainer. I claim that
you should not be using setuptools or easy_install if you want
performance or control. I claim that you should not be using setuptools
unless you understand well what you are doing (which defeats the name
easy_install).</p>
<p>The way I install packages when I want good control via easy_install is
in a virtual environment to discovered the dependencies, and then:</p>
<div class="highlight"><pre><span></span>easy_install<span class="w"> </span>-Zeab<span class="w"> </span>.<span class="w"> </span>package_name
</pre></div>
<p>to download the package for each required package, and</p>
<div class="highlight"><pre><span></span>python<span class="w"> </span>setup.py<span class="w"> </span>install<span class="w"> </span>--single-version-externally-managed<span class="w"> </span>--record<span class="w"> </span>./foobar
</pre></div>
<p>if the package itself is using setuptools.</p>
<p>As you can see, setuptools make it really hard to do a clean install.
Its a design choice :(.</p>
<p>Another alternative is to use <a class="reference external" href="http://pypi.python.org/pypi/pip">pip</a> which I strongly encourage.</p>
Mayavi documentation: in multiple small pages, or a few long ones2009-03-15T00:58:00+01:002009-03-15T00:58:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-03-15:/programming/mayavi-documentation-in-multiple-small-pages-or-a-few-long-ones.html<p>Prabhu and I can’t decide: what is best for the documentation, have more
pages, and thus have them be small, or have longer pages, but have less.
Two specific examples:</p>
<p><a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/examples.html">http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/examples.html</a></p>
<p><a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">http://code.enthought.com/projects/mayavi/docs …</a></p><p>Prabhu and I can’t decide: what is best for the documentation, have more
pages, and thus have them be small, or have longer pages, but have less.
Two specific examples:</p>
<p><a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/examples.html">http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/examples.html</a></p>
<p><a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html</a></p>
<p>Right now, these pages are split in smaller pages. Should all these
smaller pages be folded back in one long page? It would be a long page,
but all the information would be there.</p>
<p>Neither Prabhu, nor I, want to decide solely on our personnal
preference. We would to do what suits users most. This I why we want
FEEDBACK :). Could you please give feedback by mail, or in a comment on
this blog. Thank you!</p>
Mayavi on the web2009-03-07T13:06:00+01:002009-03-07T13:06:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-03-07:/programming/mayavi-on-the-web.html<p><a class="reference external" href="http://ondrejcertik.blogspot.com/">Ondrej Certik</a> has installed a <a class="reference external" href="http://www.sagemath.org/">sage</a> notebook on a server opened to
the net, with <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a> installed on it. The result is that you have a
command line interface on the web, in which you can enter Mayavi
commands, and see the result. You have to be very careful to …</p><p><a class="reference external" href="http://ondrejcertik.blogspot.com/">Ondrej Certik</a> has installed a <a class="reference external" href="http://www.sagemath.org/">sage</a> notebook on a server opened to
the net, with <a class="reference external" href="http://code.enthought.com/projects/mayavi/">Mayavi</a> installed on it. The result is that you have a
command line interface on the web, in which you can enter Mayavi
commands, and see the result. You have to be very careful to switch
Mayavi in offscreen mode as soon as you load it. To see the result of a
plot, just save it in a file. The sage notebook will display the image.</p>
<a class="reference external image-reference" href="http://nb.hpfem.org/home/pub/16/"><img alt="" src="attachments/mayavi2_in_sage1.png" /></a>
<p>I have always had in mind the use of Mayavi as a backend for a
scientific web application, for instance for a neuromaging database, but
what is really stuning with this implementation, is the way you interact
with it: full-blown Python comand line.</p>
Mayavi UI issue2009-02-18T09:25:00+01:002009-02-18T09:25:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-02-18:/programming/mayavi-ui-issue.html<p>I have been wanting to change slightly the design of a Mayavi dialog for
a while. Here is the issue: when you create a visualization, eg throught
the command line in <a class="reference external" href="http://ipython.scipy.org/">IPython</a>, whith <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab</a>, you get a nice and small
window with only your visualization, and a toolbar. If you …</p><p>I have been wanting to change slightly the design of a Mayavi dialog for
a while. Here is the issue: when you create a visualization, eg throught
the command line in <a class="reference external" href="http://ipython.scipy.org/">IPython</a>, whith <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab</a>, you get a nice and small
window with only your visualization, and a toolbar. If you want to
change the properties of the objects on the visualization, or add some
more, you need to click on a button on the toolbar, which displays a
dialog, from which you can open more dialogs to edit the objects:</p>
<a class="reference external image-reference" href="attachments/mayavi_old_ui.png"><img alt="" class="align-center" src="attachments/mayavi_old_ui.png" /></a>
<p>I am thinking to changing this to a single dialog:</p>
<a class="reference external image-reference" href="attachments/mayavi_new_ui.png"><img alt="" class="align-center" src="attachments/mayavi_new_ui.png" /></a>
<p>The single-object-editing dialogs could still be opened by
double-clicking on the pipeline.</p>
<p>I am not going to discuss why I believe the new version would be better
than the old one, because I do not want to bias people. However, I would
prefer not making the decision to change based only on my feelings. So I
ask everybody, users of Mayavi or not: what do you think is better? And
why? I will probably leave an option to have the old behavior, anyhow,
but the default is very important.</p>
Frank Willison on the French2009-01-29T17:20:00+01:002009-01-29T17:20:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-01-29:/misc/frank-willison-on-the-french.html<p>I couldn’t help myself, this Franck-Willison quote is brilliant:</p>
<blockquote class="epigraph">
“And the Europeans accept chaos and know how to live with it. I came
away from France believing in the strong faith of the French people,
not because of their magnificent cathedrals, but because I saw them
drive around L …</blockquote><p>I couldn’t help myself, this Franck-Willison quote is brilliant:</p>
<blockquote class="epigraph">
“And the Europeans accept chaos and know how to live with it. I came
away from France believing in the strong faith of the French people,
not because of their magnificent cathedrals, but because I saw them
drive around L’Arc du Triomphe. As they careened around this monument
in vehicles that were little more than motorized Altoid containers,
they looked like a network run by Linux servers: little packets
hurtling within inches of each other, traveling together for a time,
separated by other packets coming from other sources, reassembling as
they sped out of the circle and onto the Champs Élysées. It’s no
surprise to me that much of the interest in IPv6 is coming from
France–if they don’t understand delay-sensitive traffic in France,
they understand it nowhere.”</blockquote>
Error in my article2009-01-27T22:00:00+01:002009-01-27T22:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-01-27:/programming/error-in-my-article.html<p>There is an error in a code example in my article that just came out in
Linux Magazine France. I am so ashamed. I did test the code, but I
didn’t have automated tests, so I broke it when tweaking it :(. I think
the lesson is that you need …</p><p>There is an error in a code example in my article that just came out in
Linux Magazine France. I am so ashamed. I did test the code, but I
didn’t have automated tests, so I broke it when tweaking it :(. I think
the lesson is that you need to do more than doc-testing articles (it was
doc-tested).</p>
<p>The code example is about calculating the Mandebrot set. The idea is
that you take a grid of the numbers c complex plane, and iterate on it
the function f = lambda z: z**2 + c. You look at the divergence of
this iteration, and plotting a mesure of the divergence gives you a nice
figure. The code I wrote was:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="n">ogrid</span><span class="p">,</span> <span class="n">zeros</span><span class="p">,</span> <span class="nb">abs</span><span class="p">,</span> <span class="n">isnan</span><span class="p">,</span> <span class="n">ones</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">x</span> <span class="o">+</span> <span class="n">y</span> <span class="o">*</span> <span class="mi">1</span><span class="n">j</span>
<span class="n">threshold_time</span> <span class="o">=</span> <span class="n">zeros</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">))</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">zeros</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">))</span>
<span class="n">mask</span> <span class="o">=</span> <span class="n">ones</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">),</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">bool</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">50</span><span class="p">):</span>
<span class="n">z</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span> <span class="o">=</span> <span class="n">z</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">c</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span>
<span class="n">mask</span> <span class="o">=</span> <span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">z</span><span class="p">)</span> <span class="o">></span> <span class="mi">10</span><span class="p">)</span>
<span class="n">threshold_time</span> <span class="o">+=</span> <span class="n">mask</span>
</pre></div>
<p>The error is subtle. First there is the not so subtle mask error: I am
masking the points that diverge, and iterate them even further. This is
exactly the opposite that I meant to do. Then there is the more subtle
bug: the line “z[mask] = z[mask]**2 + c[mask]” is an in-place
assignment. As a result the dtype of z is not modified: z is not
magically cast in a complex. Thus the imaginary information coming from
c is lost. And that information is crucial to Mandelbrot.</p>
<p>The right code is:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="n">ogrid</span><span class="p">,</span> <span class="n">zeros</span><span class="p">,</span> <span class="nb">abs</span><span class="p">,</span> <span class="n">isnan</span><span class="p">,</span> <span class="n">ones</span><span class="p">,</span> <span class="nb">complex</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">x</span> <span class="o">+</span> <span class="n">y</span> <span class="o">*</span> <span class="mi">1</span><span class="n">j</span>
<span class="n">threshold_time</span> <span class="o">=</span> <span class="n">zeros</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">))</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">zeros</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">),</span> <span class="nb">complex</span><span class="p">)</span>
<span class="n">mask</span> <span class="o">=</span> <span class="n">ones</span><span class="p">((</span><span class="mi">500</span><span class="p">,</span> <span class="mi">500</span><span class="p">),</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">bool</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">50</span><span class="p">):</span>
<span class="n">z</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span> <span class="o">=</span> <span class="n">z</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">c</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span>
<span class="n">mask</span> <span class="o">=</span> <span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">z</span><span class="p">)</span> <span class="o"><</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">threshold_time</span> <span class="o">+=</span> <span class="n">mask</span>
</pre></div>
<p>Plot the threshold_time array with pylab.imshow (from the matlplotlib
project) to get a nice figure.</p>
Mayavi image of the fortnight2009-01-25T19:21:00+01:002009-01-25T19:21:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-01-25:/programming/mayavi-image-of-the-fortnight.html<p>It’s been two weeks since I posted a ‘Mayavi image of the week’. Prabhu
has made a really cool example of integrating trajectories in a 3D
vector field, using, of course, the <a class="reference external" href="http://en.wikipedia.org/wiki/Lorenz_system">Lorenz equation</a> for the 3D field.
With nice colors, it makes a new fantastic image:</p>
<img alt="" src="https://gael-varoquaux.info/programming/attachments/lorentz_mayavi.png" />
<p>The green …</p><p>It’s been two weeks since I posted a ‘Mayavi image of the week’. Prabhu
has made a really cool example of integrating trajectories in a 3D
vector field, using, of course, the <a class="reference external" href="http://en.wikipedia.org/wiki/Lorenz_system">Lorenz equation</a> for the 3D field.
With nice colors, it makes a new fantastic image:</p>
<img alt="" src="https://gael-varoquaux.info/programming/attachments/lorentz_mayavi.png" />
<p>The green surface represented is an isosurface of the z component of the
vector field: on this surface, the z-component changes sign. This can be
seen from the trajectories, as they start going up once they pass the
surface. The script generating this image is checked in as an example:
<a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/trunk/examples/mayavi/lorenz.py">https://svn.enthought.com/enthought/browser/Mayavi/trunk/examples/mayavi/lorenz.py</a></p>
LinuxMag special edition on Python2009-01-24T12:42:00+01:002009-01-24T12:42:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-01-24:/programming/linuxmag-special-edition-on-python.html<p>The French LinuxMag just published a special edition on Python, in which
I authored a 12-page article on scientific computing. The edition is in
French, so if you don’t speak French, it is of limited interested.</p>
<hr class="docutils" />
<p>Ce dossier hors-série est une excellente ressource pour découvrir
Python, entre autre par …</p><p>The French LinuxMag just published a special edition on Python, in which
I authored a 12-page article on scientific computing. The edition is in
French, so if you don’t speak French, it is of limited interested.</p>
<hr class="docutils" />
<p>Ce dossier hors-série est une excellente ressource pour découvrir
Python, entre autre par ce qu’il présente Python sous beaucoup d’aspects
différents, et permet donc de découvrir quelles sont les outils avancés
disponibles pour s’attaquer à une tâche particulière.</p>
<p>Lien vers la présentation du magazine, ainsi que où l’acheter:
<a class="reference external" href="http://www.gnulinuxmag.com/index.php/2009/01/23/gnulinux-magazine-hs-n%C2%B040-janvierfevrier-2009-chez-votre-marchand-de-journaux">http://www.gnulinuxmag.com/index.php/2009/01/23/gnulinux-magazine-hs-n%C2%B040-janvierfevrier-2009-chez-votre-marchand-de-journaux</a></p>
<p>Feuilletez en ligne: <a class="reference external" href="http://ed-diamond.com/feuille_lmhs40/index.html">http://ed-diamond.com/feuille_lmhs40/index.html</a></p>
<p>Il ne coûte que 6 euros 50, et est disponible dans tous les kiosques de
France (sauf au Monoprix à coté de chez moi :<). Ce n’est pas bien cher
pour une soixantaine de pages d’informations spécialisés. Achetez le,
même si vous connaissez bien Python, et n’apprendrez rien de nouveau.
Vous le laisserez traîner au boulot, pour faire de la propangande
passive :).</p>
<p>Cela fait plus d’un ans que les auteurs travaillent sur leurs articles.
Je ne sais pas si cela dénote un grand perfectionnisme, ou une grande
inefficacité :). En tout cas, un grand merci à Philippe Biondi qui a été
le maître d’oeuvre du project, et qui l’a tiré en avant.</p>
<div class="topic">
<p class="topic-title"><strong>Le PDF de l’article</strong></p>
<p>J’ai mis le PDF de l’article <a class="reference external" href="./my-article-on-scientific-computing-with-python.html">en ligne</a></p>
</div>
Mayavi image of the week2009-01-13T00:38:00+01:002009-01-13T00:38:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2009-01-13:/programming/mayavi-image-of-the-week.html<p>The title of this post is a lure: there won’t be a Mayavi image each
week, because I would run out quickly. But it sounded cool.</p>
<p>Anyway, here is an image of a graph, visualized with Mayavi. The graph
is actually a protein structure, downloaded from the PDB. The …</p><p>The title of this post is a lure: there won’t be a Mayavi image each
week, because I would run out quickly. But it sounded cool.</p>
<p>Anyway, here is an image of a graph, visualized with Mayavi. The graph
is actually a protein structure, downloaded from the PDB. The Python
script producing this visualization is checked in as a Mayavi example:
<a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/trunk/examples/mayavi/protein.py">https://svn.enthought.com/enthought/browser/Mayavi/trunk/examples/mayavi/protein.py</a></p>
<p>The part of the code to read the PDB file is actualy way longer than the
visualization part.</p>
<p>I hope this script inspires people trying to visualize graphs. The
combination of the GaussianSplatter filter and the volume rendering to
create a halo renders really well, IMHO.</p>
<img alt="" src="https://gael-varoquaux.info/programming/attachments/protein_mayavi.png" />
GPS coordinates for the world’s major cities2008-12-31T00:53:00+01:002008-12-31T00:53:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-12-31:/misc/gps-coordinates-for-the-worlds-major-cities.html<p>This may sound stupid, but I have failed to find on the web a
simply-accessible text file giving the GPS coordinates for the world’s
major cities. The web site <a class="reference external" href="http://www.world-gazetteer.com">World-gazetteer</a> is helpful, but doesn’t
really give what I want. So I wrote a small Python web-scraping script
to …</p><p>This may sound stupid, but I have failed to find on the web a
simply-accessible text file giving the GPS coordinates for the world’s
major cities. The web site <a class="reference external" href="http://www.world-gazetteer.com">World-gazetteer</a> is helpful, but doesn’t
really give what I want. So I wrote a small Python web-scraping script
to get the data I needed. BeautifulSoup is really cool, but somehow I
feel I could be spending my time in a better way. And on a side note, I
still struggle to get good conversion from unicode to ascii (I needed
ascii, as this will be fed to software that don’t understand unicode).</p>
<p>I am attaching both the data file produced, and the Python script, in
case someone finds them useful.</p>
<ul class="simple">
<li><a class="reference external" href="../images/misc/cities.txt">GPS coordinate list</a></li>
<li><a class="reference external" href="../images/misc/cities.py">Web-scraping Python script</a></li>
</ul>
<p><strong>Update</strong>: About 10 minutes after posting, I did a better google
search, and found a page that might have been easier to scrape:
<a class="reference external" href="http://www.bcca.org/misc/qiblih/latlong_oc.html">http://www.bcca.org/misc/qiblih/latlong_oc.html</a></p>
Tracking objects in scientific code2008-12-23T01:26:00+01:002008-12-23T01:26:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-12-23:/programming/tracking-objects-in-scientific-code.html<p>When I started working in my new field (data analysis of functional
brain images), I was surprised to find in our data-analysis scripts what
I thought was a very particular <a class="reference external" href="http://en.wikipedia.org/wiki/Code_smell">code smell</a>: the numerical code is
always doing a lot of filename and path manipulation, loading and saving
data even …</p><p>When I started working in my new field (data analysis of functional
brain images), I was surprised to find in our data-analysis scripts what
I thought was a very particular <a class="reference external" href="http://en.wikipedia.org/wiki/Code_smell">code smell</a>: the numerical code is
always doing a lot of filename and path manipulation, loading and saving
data even in the core routines. I couldn’t picture what seemed wrong
with this, but I was uncomfortable with it.</p>
<div class="section" id="the-good">
<h2>The good</h2>
<div class="section" id="memory-management">
<h3>Memory management</h3>
<p>In the data-processing work I am currently doing, we deal with large
objects, mostly huge numpy arrays, though sometimes some domain-specific
data containers creep in. As a result, simple calculations take time (an
SVD is 10 minutes), and I am always fighting with memory.</p>
<p>Saving to disk is a handy way of freeing memory. Moreover, with
memmapping, reading only the relevant parts of pre-calculated arrays
becomes very cheap.</p>
</div>
<div class="section" id="crash-resistance">
<h3>Crash-resistance</h3>
<p>When the simplest operation takes ten minutes, you want to save
intermediate steps, to be able to resume calculations, or to inspect why
the code crashed. And who knows, you might need this intermediate step.</p>
</div>
</div>
<div class="section" id="the-bad">
<h2>The bad</h2>
<p>The immediate apparent problem is that your code becomes riddled with
path-management code. We often joke that once we have figured out the
algorithm, the longest surviving piece of code is the path-related junk.</p>
<p>But, I believe this is only the tip of the iceberg, and that this code
smell hints to deeper problems.</p>
</div>
<div class="section" id="the-ugly">
<h2>The ugly</h2>
<div class="section" id="loss-of-scoping">
<h3>Loss of scoping</h3>
<p>When I started working on these problems, I was startled to encounter
basic domain-specific algorithmic functions taking input and output data
filenames. It took me a while to realize that the huge problem with this
is that I loose scoping, or in other words naming locality. Let us
pretend that I have a function ‘foo’ that does basic numerics on large
numpy arrays, but to save memory it takes as a signature the name of the
file where the input array is stored, and the name of the filename where
the output array should be stored. So I have some code that looks like
this:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">process_sessions</span><span class="p">(</span><span class="n">session_files</span><span class="p">):</span>
<span class="k">for</span> <span class="n">session_file</span> <span class="ow">in</span> <span class="n">session_files</span><span class="p">:</span>
<span class="n">foo</span><span class="p">(</span><span class="n">session_file</span><span class="p">,</span> <span class="n">session_file</span> <span class="o">+</span> <span class="s1">'.out'</span><span class="p">)</span>
</pre></div>
<p>Saving to files in the loop is a huge gain of memory;</p>
<p>Now I decide I want to add a parameter to foo, and vary this parameter,
with, eg:</p>
<div class="highlight"><pre><span></span><span class="k">for</span> <span class="n">param</span> <span class="ow">in</span> <span class="n">params</span><span class="p">:</span>
<span class="n">process_sessions</span><span class="p">(</span><span class="n">session_files</span><span class="p">,</span> <span class="n">param</span><span class="p">)</span>
</pre></div>
<p>My code is hard to refactor, because I need to introduce modifications
deep in all subroutines to make sure they do not save their outputs in
the same file.</p>
<p>Suppose session_files are actually extracted from an upstream dataset,
and now I want to apply my algorithm on a set of these upstream
datasets, and in parallel. Once again I need to generate a score of new
filenames and keep track of them. I can use temporary files, but I need
to keep hold of this information too, and I loose most of my
crash-resistance.</p>
<p>When you think it over, the way programming languages solve this problem
elegantly, is by the rules connecting names to objects, and in
particular scoping: a name corresponds to an object in a given function.
Using files is equivalent to using globals, and we have to cook up our
own scoping rules (which results in a lot of path-massaging code).</p>
</div>
<div class="section" id="no-history-tracking">
<h3>No history tracking</h3>
<p>When I find a file on the disk, I do not really know how it has been
generated. As a results, the crash-resistance is compromised. Moreover,
when tweaking algorithms, we often try to rerun only the necessary parts
of the algorithms, relying on the precomputed parts saved to the disk.
We comment out code, or exercise different code paths. As a result we
often end up in situations where the whole code does not actually run.
And once again refactoring is hard, because we have not expressed the
dependency relations between our intermediate results.</p>
</div>
</div>
<div class="section" id="doing-better">
<h2>Doing better?</h2>
<p>Once again, today I was refactoring my algorithm, or my “pipeline” as we
call it. And once again, I felt the failure to have the proper tools,
the proper abstractions, words, to express the problem in the code.
Manipulating files directly seems wrong, for the reason expressed above.
But can we do better?</p>
<p>The problem, I believe, is that we need a lightweight persistence
framework adapted to scientific purposes. I remember telling Travis
Vaught a few weeks before beginning my new job that scientists had no
problem with their persistence. Well, I was so wrong.</p>
<p>By a persistence framework, I do not mean a persistence mechanism, like
numpy.save, or hdf5, or a database. I am interested in the objects with
which we represent it in the code. How do we solve the scoping problem?
And the history problem? Can we implement a “trajectory tracking”, to
reuse the <a class="reference external" href="http://article.gmane.org/gmane.comp.python.french/5423">words of Alexandre Fayolle</a>, for our data containers?</p>
<p>I am thinking about a small set of well-thought abstractions, a bit like
the use of ORM (object relational mappers) in web application, that
would take care of the mapping from in-memory objects to objects on the
disk for us.</p>
<p>I am starting to have some ideas. I am thinking in terms of context
objects, with getattr tricks to do the mapping to a database doing the
bookkeeping and the trajectory tracking, and doing the impedance
matching with objects stored as numpy “.npy” files, hdf5 files, nifti
files, or whatever you want. The added value of a database would be that
it would give some robust locking, and possible network abstraction, to
allow for crash-safety, and parallel or distributed computing.</p>
<p>This may sound overkill, or overcomplicated. I’ve tried simple things.
They all failed.</p>
<p>This is a problem that matters a lot to me. I feel I am loosing a lot of
time on this. However I feel that the effort to do something good is
quite important. I am also afraid of polluting my numerical code with
unnecessary abstractions. The main problem is that attempting to solve
this problem would require a significant investment in time, and I don’t
really see where I can find this time.</p>
<p>Have people encountered similar problems? Do you have any suggestions,
any trick to share?</p>
<p>I’d be very happy to read any comments that can move forward my
thinking, even if it is about pointing out problems and not solutions. I
still think I haven’t identified the problems well.</p>
<p><strong>Update</strong>: I have just realized that I will be almost without internet
access for the next week, starting from pretty much now. Looks like it
was a bad moment to start a thrilling discussion. I guess I got carried
away by the discontent of a day doing some bad refactoring. I really
look forward to catching up when I come back. Please forgive me for the
bad timing.</p>
<div class="topic">
<p class="topic-title"><strong>Update</strong></p>
<p>Patterns that derived from this line of thoughts are now implemented
in the <a class="reference external" href="https://pythonhosted.org/joblib/">joblib</a> library.</p>
</div>
</div>
What’s new in Mayavi 3.1.0?2008-12-11T00:56:00+01:002008-12-11T00:56:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-12-11:/programming/whats-new-in-mayavi-310.html<p>Mayavi 3.1.0 has just been released, and I think it is a fantastic
version. We are starting to be able to focus on the details and the
focus. In addition, we are getting user feedback, which helps identify
the pain points.</p>
<p><strong>Automatic scripting</strong></p>
<p>This is a huge deal …</p><p>Mayavi 3.1.0 has just been released, and I think it is a fantastic
version. We are starting to be able to focus on the details and the
focus. In addition, we are getting user feedback, which helps identify
the pain points.</p>
<p><strong>Automatic scripting</strong></p>
<p>This is a huge deal! You now have a <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/automatic_scripting.html%20">record button</a> on the pipeline
view. In record mode, the modifications that you make to the objects
properties are recorded as valid Python lines: Mayavi tells you what are
the line of code to modify those properties or create new objects. I use
this a lot: I first build a skeletton of a visualization using <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab</a>
but when it comes to tuning parameters, I do it interactively, and
record.</p>
<p><strong>Much more testing</strong></p>
<p>We added a huge amount of testing (many thanks to Suyog who contribed
quite few). From an user’s point of view this has two consequences. First
the code is more robust (for instance the mlab commands are more flexible
on the shape of the arguments passed in). Second the rendering part of
the Mayavi engine is well-separated from the algorithmes, which means
that the VTK algorithms <a class="reference external" href="https://mail.enthought.com/pipermail/enthought-dev/2008-December/018935.html">can now be used</a> easily to manipulate numpy
arrays through Mayavi.</p>
<p><strong>Two new mlab functions: barchart and triangular_mesh</strong></p>
<p>Mlab has <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html#d-data">two new functions</a>: one to create nice bar chart, for 2D
histograms displayed in 3D, and one to build meshes defined from their
triangle.</p>
<p><strong>Control of the pipeline through mlab is easier and more robust</strong></p>
<p>As the mlab.pipeline is getting more usage, it is being ironed out. For
instance applying a module to a source object (may it be a Mayavi
source, or a vtk dataset) adds it automatically to the figure if it is
not already in it. Also, when adding an additional module on an existing
source, a new module manager (object controlling the colormap) is added
automatically if the colormaps or extents differ. Many modules take
keyword arguments to make common operations easier.</p>
<p><strong>IPython in Mayavi</strong></p>
<p>If you have a recent version of IPython installed (> 0.9), Mayavi will
use an IPython widget, instead of the vanilla pyshell.</p>
<p><strong>mlab.view has now a sensible behavior</strong></p>
<p>The mlab.view no longer gives a bad roll angle to the camera. This makes
it much easier to do animations during which the camera moves.</p>
<p><strong>Axes and outline extents</strong></p>
<p>mlab.axes and mlab.outline now adjust by default to the extents of the
object they are applied on. This removes a bad surprise for people having
tuned the scale of their visualization.</p>
<p><strong>enthought.tvtk.tools.visual in Mayavi</strong></p>
<p>enthought.tvtk.tools.visual can now be used inside Mayavi, to provide a
<a class="reference external" href="https://mail.enthought.com/pipermail/enthought-dev/2008-October/018402.html">visual-like</a> API in mayavi.</p>
<p><strong>Documentation has recieved some love</strong></p>
<p>Documentation has been added and completed, with a focus on making it
easier for the beginner to discover the features of Mayavi. We try more
and more to walk the user through complete usecases of Mayavi, in a
task-oriented documentation, such as in the introductory examples, or in
<a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html#case-studies-of-some-visualizations">case-studies</a>.</p>
<p><strong>Two new sources</strong></p>
<p>There are two new sources that do not require data. The first creates
objects, such as an arrow, a cube, or a view of the earth, to be viewed
with a ‘surface’ module. The second creates image data, such as a disk,
or a 2d gaussian, or (my favorite) the Mandelbrot set. This can be
viewed with an ImageActor, or (even better) with a WarpScalar filter and
a Surface. These sources have been contributed by Suyog.</p>
<div class="section" id="a-word-of-thanks">
<h2>A word of thanks</h2>
<p>I am sure I am going to forget some people here, but I’d like to thank a
lot those who have been helping us with getting Mayavi2 going. First of
all, Dave Peterson, who is doing the release management for ETS. This is
a lot of work, and we would never have frequent releases without him.
I’d also like to thank Suyog Jain, who contributed some code. This is
fantastic, and I am sure we are going to have more people contributing
improvements. Finally, I’d like to thank Pierre Raybault, of
Python(x,y), and Varun Hiremath, of Debian. Packaging is very important
to our users, and it is not a trivial piece of work… Hum, I almost
forgot Chris Casey. Chris has been updating the docs on the net and
making sure the docs build well. This is also very important, as the web
page is a major means of communication with our users.</p>
</div>
120 pages!2008-12-09T01:00:00+01:002008-12-09T01:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-12-09:/programming/120-pages.html<p>The mayavi manual in SVN has now 120 pages when compiled to pdf. I know
that this is a stupid metric, and that the quality is more important
then the number of pages, but it does give me a warm and fuzzy feeling.</p>
<p>More seriously, next release of Mayavi (coming …</p><p>The mayavi manual in SVN has now 120 pages when compiled to pdf. I know
that this is a stupid metric, and that the quality is more important
then the number of pages, but it does give me a warm and fuzzy feeling.</p>
<p>More seriously, next release of Mayavi (coming soon, we promise) is
going to have a lot of added documentation for the casual users. In
particular the <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">mlab section</a> has been expended a lot and is starting
to hint at Mayavi’s full power.</p>
<p>Thanks to Chris Casey, who is making sure that the docs land on the net
as soon as they are written.</p>
Using Mayavi to explore a potential field2008-11-22T15:22:00+01:002008-11-22T15:22:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-11-22:/programming/using-mayavi-to-explore-a-potential-field.html<p>As promised, here is the sequel to the tutorial I posted yesterday on
using Mayavi with scipy to understand the trajectories of a particle in
a potential. (chances are you are reading this before my previous post.
I suggest you first jump to my previous post, and then come back …</p><p>As promised, here is the sequel to the tutorial I posted yesterday on
using Mayavi with scipy to understand the trajectories of a particle in
a potential. (chances are you are reading this before my previous post.
I suggest you first jump to my previous post, and then come back here).</p>
<p>This tutorial shows you how to use the powerfull VTK and Mayavi feature
to explore the trajectories in the same potential. However, the tools we
are using do not given us as much control on the dynamics of the system,
so this time we do not add damping or oscillation of the potential. At
the end of the day, the resulting visualization is however much more
interactive. Once again, I would like as much feedback as possible, as
this is intended for the Mayavi User Guide.</p>
<hr class="docutils" />
<p>In this example, we create a vector field from the gradient of a scalar
field and explore it interactively. This example shows you how to do
some operations similar to the previous example, but interactively,
using the filters and module. This approach requires a better knowledge
of Mayavi and the VTK filters, but the big gain is that the resulting
visualization can be explored interactively.</p>
<p>First, let us create the same scalar field as the previous example:. We
open Mayavi and enter the following code in the Python shell:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">enthought.mayavi</span> <span class="kn">import</span> <span class="n">mlab</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="k">def</span> <span class="nf">V</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" A 3D sinusoidal lattice with a parabolic confinement. """</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">x</span><span class="p">)</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">y</span><span class="p">)</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">z</span><span class="p">)</span> <span class="o">+</span> <span class="mi">2</span><span class="o">*</span><span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">z</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span>
<span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mgrid</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">]</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">contour3d</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="n">V</span><span class="p">)</span>
</pre></div>
<p>As in the previous example, we can change the color map and the values
chosen in the isosurfaces.</p>
<p>We want to take the gradient of the scalar field, to create a vector
field. To do this we are going to use the CellDerivatives filter, that
takes derivatives of the data located in the cells (that is, between the
points, see <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/data.html">*Creating data for Mayavi*</a>). For this, we first need to
interpolate the data from the points where it is located to the cells,
using a PointToCellData filter. We can then apply our CellDerivatives
filter, and then a CellToPointData filter to get point data back.
(remark: if you are not using the latest Mayavi from SVN - 3.1.0 - you
need to enable the ‘pass data’ option in the two CellToPointData and
PointToCellData filters).</p>
<p>To visualize the vector field, we can use a VectorCutPlane module. The
resulting vectors are too large, and we can go to the Glyph tab, (and
the Glyph tab in this tab), to reduce the scale factor to 0.2. The
vector field is still too dense, therefore we go to the Masking tab to
enable masking, mask with an on ratio of 6 (one arrow out of 6 is
masked) and turn off the random mode.</p>
<a class="reference external image-reference" href="attachments/vector_cut_plane2.jpg"><img alt="" src="attachments/vector_cut_plane2.jpg" /></a>
<p>To have nice colors, we also changed the color map of the vector field
by going to the Colors and legend node just above the VectorCutPlane,
and choosing a look up table <strong>in the VectorLUT</strong> tab, as there can be
different color maps for vector data and scalar data.</p>
<p>Unlike the previous example, we can play with all the parameters in the
dialog box, like masking, or select color_by_scalar in the Glyph tab,
to display the value of the potential. We can also move the cut plane
used to display the vectors by dragging it.</p>
<p>Now that we have a 3D vector field, we can also use Mayavi to integrate
the trajectory of a particle in it. For this we can use the streamline
module. It displays trajectories starting from the vertices of a seed
surface. We choose (in the Seed tab) a Point Widget as a seed. We can
then move the seed point by dragging it along in the 3D scene. This
allows us to explore the trajectories in the potential created by the
initial scalar field. In our case, all the trajectories end up in a
local potential minimum, and moving the seed point along lets us see in
which minimum each point will fall into, in other world the basin of
attraction of each local minimum.</p>
<a class="reference external image-reference" href="attachments/streamline.jpg"><img alt="" src="attachments/streamline.jpg" /></a>
Using Mayavi with Scipy: a tutorial2008-11-22T00:19:00+01:002008-11-22T00:19:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-11-22:/programming/using-mayavi-with-scipy-a-tutorial.html<p>Many years ago, I was working with a bright undergrad on the
trajectories of a atoms in a complex light field created by the
intersection of two laser beams. She had developped a code in C, and I
was starting to discover Python, so we had binded in t in …</p><p>Many years ago, I was working with a bright undergrad on the
trajectories of a atoms in a complex light field created by the
intersection of two laser beams. She had developped a code in C, and I
was starting to discover Python, so we had binded in t in Python. We
where using the Python binding, together with ipython and matplotlib to
explore and debug the code. However, our problem was readlly
fundementally 3D, and I din’t find the status of the 3D plotting tools
in Python satisfying.</p>
<p>That usecase was very much on my mind while working on Mayavi, as I have
always believed that Mayavi and ipython could make a fantastic steering
and debugging tool for 3D Physics code. I think Mayavi is starting to be
pretty mature for this set of problems and as I am improvong the docs, I
decided to write a tutorial example on this specific problem. I am
posting it here as a preview. This is going to go in the docs, so
please, if you have any comments that might improve it, fire away.</p>
<hr class="docutils" />
<p>This tutorial example shows you how how you can use Mayavi interactively
to visualize <a class="reference external" href="http://www.scipy.org/">numpy</a> arrays while doing numerical work with <a class="reference external" href="http://www.scipy.org/">scipy</a>.
It assumes that you are familiar with numerical Python tools, and shows
you how to use Mayavi in combination with these tools.</p>
<p>Let us study the trajectories of a particle in a potential. This is a
very common problem in physics and engineering, and visualization of the
potential and the trajectories is key to developing an understanding of
the problem.</p>
<p>The potential we are interested is a periodic lattice, immersed in a
parabolic confinement. We will shake this potential and see how the
particle jumps from a hole of the lattice to another. The parabolic
confinement is there to limit the excursions of the particle:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="k">def</span> <span class="nf">V</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" A 3D sinusoidal lattice with a parabolic confinement. """</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">x</span><span class="p">)</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">y</span><span class="p">)</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="n">z</span><span class="p">)</span> <span class="o">+</span> <span class="mi">2</span><span class="o">*</span><span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">z</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span>
</pre></div>
<p>Now that we have defined the potential, we would like to see what it
looks like in 3D. To do this we can create a 3D grid of points, and
sample it on these points:</p>
<div class="highlight"><pre><span></span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mgrid</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">100</span><span class="n">j</span><span class="p">]</span>
<span class="n">V</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">)</span>
</pre></div>
<p>We are going to use the mlab module (see <a class="reference external" href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html">*Simple Scripting with
mlab*</a>) to interactively visualize this volumetric data. For this it is
best to type the commands in an interactive Python shell, either using
the built-in shell of the Mayavi2 application, on in ipython -wthread.
Let us visualize the 3D isosurfaces of the potential:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">enthought.mayavi</span> <span class="kn">import</span> <span class="n">mlab</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">contour3d</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="n">V</span><span class="p">)</span>
</pre></div>
<p>We can interact with the visualization created by the above command by
rotating the view, but to get a good understanding of the structure of
the potential, it is useful to vary the iso-surfaces. We can do this by
double-clicking on the IsoSurface in the Mayavi pipeline tree (if you
are running from ipython, you need to click on the Mayavi icon on the
scene to pop up the pipeline). This opens a dialog which lets us select
the values of the contours used. A good view of the potential can be
achieved by turning off auto contours and choosing -0.5 as a first
contour value (eg by entering it in the text box on the right, and
pressing tab). A second contour can be added by clicking on the blue
arrow and selecting “Add after”. Using a value of 15 gives a nice
result.</p>
<p>We can now click on the Colors and legends on the pipeline and change
the colors used, by selecting a different LUT (Look Up Table). Let us
select ‘Paired’ as it separates well levels.</p>
<a class="reference external image-reference" href="attachments/potential_ipython.jpg"><img alt="" src="attachments/potential_ipython.jpg" /></a>
<p>To get a better view of the potential, we would like to display more
contours, but the problem with this approach is that closed contours
hide their interior. On solution is to use a cut plane. Right-click on
the IsoSurface node and add a ScalarCutPlane through the “Add module”
sub menu. You can move the cut plane by clicking on it and dragging.</p>
<p>To make the link between our numpy arrays and the visualization, we can
use the same menu to add a Axes and an Outline. Finally, let us add a
colorbar. We can do this by typing:</p>
<div class="highlight"><pre><span></span><span class="n">mlab</span><span class="o">.</span><span class="n">colorbar</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Potential'</span><span class="p">,</span> <span class="n">orientation</span><span class="o">=</span><span class="s1">'vertical'</span><span class="p">)</span>
</pre></div>
<p>Or using the options in the LUT dialog visited earlier.</p>
<a class="reference external image-reference" href="attachments/potential.jpg"><img alt="" src="attachments/potential.jpg" /></a>
<p>We want to study the motion of a particle in this potential. For this we
need to derive the corresponding force, given by the gradient of the
potential. We create a gradient function:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">gradient</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="n">d</span><span class="o">=</span><span class="mf">0.01</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" Return the gradient of f in (x, y, z). """</span>
<span class="n">fx</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="o">+</span><span class="n">d</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">)</span>
<span class="n">fx_</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="o">-</span><span class="n">d</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">)</span>
<span class="n">fy</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">+</span><span class="n">d</span><span class="p">,</span> <span class="n">z</span><span class="p">)</span>
<span class="n">fy_</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">-</span><span class="n">d</span><span class="p">,</span> <span class="n">z</span><span class="p">)</span>
<span class="n">fz</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="o">+</span><span class="n">d</span><span class="p">)</span>
<span class="n">fz_</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="o">-</span><span class="n">d</span><span class="p">)</span>
<span class="k">return</span> <span class="p">(</span><span class="n">fx</span><span class="o">-</span><span class="n">fx_</span><span class="p">)</span><span class="o">/</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">d</span><span class="p">),</span> <span class="p">(</span><span class="n">fy</span><span class="o">-</span><span class="n">fy_</span><span class="p">)</span><span class="o">/</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">d</span><span class="p">),</span> <span class="p">(</span><span class="n">fz</span><span class="o">-</span><span class="n">fz_</span><span class="p">)</span><span class="o">/</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">d</span><span class="p">)</span>
</pre></div>
<p>To check that our gradient function works well, let us visualize the
vector field it creates. To avoid displaying too many vectors, we will
evaluate the gradient only along a cut for X=50, and every three points
on our grid:</p>
<div class="highlight"><pre><span></span><span class="n">Vx</span><span class="p">,</span> <span class="n">Vy</span><span class="p">,</span> <span class="n">Vz</span> <span class="o">=</span> <span class="n">gradient</span><span class="p">(</span><span class="n">V</span><span class="p">,</span> <span class="n">X</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">],</span> <span class="n">Y</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">],</span> <span class="n">Z</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">])</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">quiver3d</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">],</span> <span class="n">Y</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">],</span> <span class="n">Z</span><span class="p">[</span><span class="mi">50</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">,</span> <span class="p">::</span><span class="mi">3</span><span class="p">],</span>
<span class="n">Vx</span><span class="p">,</span> <span class="n">Vy</span><span class="p">,</span> <span class="n">Vz</span><span class="p">,</span> <span class="n">scale_factor</span><span class="o">=-</span><span class="mf">0.2</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
<a class="reference external image-reference" href="attachments/gradient.jpg"><img alt="" src="attachments/gradient.jpg" /></a>
<p>Now we can use scipy to integrate the trajectories. We first have to
define a dynamical flow, the function that returns the derivative of the
different parameters as a function of these parameters and of time. The
flow is used by every <a class="reference external" href="http://en.wikipedia.org/wiki/Ordinary_differential_equation">ODE</a> (ordinary differential equation) solver, it
give the dynamic of the system. The dynamics we are interested in is
made of the force deriving from the potential, that we shake with time
in the three direction, as well as a damping force. The damping
coefficient and the amount and frequency of shaking have been tuned to
give an interesting dynamic.</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">flow</span><span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="n">t</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" The dynamical flow of the system """</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="n">vx</span><span class="p">,</span> <span class="n">vy</span><span class="p">,</span> <span class="n">vz</span> <span class="o">=</span> <span class="n">r</span>
<span class="n">fx</span><span class="p">,</span> <span class="n">fy</span><span class="p">,</span> <span class="n">fz</span> <span class="o">=</span> <span class="n">gradient</span><span class="p">(</span><span class="n">V</span><span class="p">,</span> <span class="n">x</span><span class="o">-</span><span class="mf">.2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="mi">6</span><span class="o">*</span><span class="n">t</span><span class="p">),</span> <span class="n">y</span><span class="o">-</span><span class="mf">.2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="mi">6</span><span class="o">*</span><span class="n">t</span><span class="o">+</span><span class="mi">1</span><span class="p">),</span> <span class="n">z</span><span class="o">-</span><span class="mf">.2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="mi">6</span><span class="o">*</span><span class="n">t</span><span class="o">+</span><span class="mi">2</span><span class="p">))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">((</span><span class="n">vx</span><span class="p">,</span> <span class="n">vy</span><span class="p">,</span> <span class="n">vz</span><span class="p">,</span> <span class="o">-</span><span class="n">fx</span> <span class="o">-</span> <span class="mf">0.3</span><span class="o">*</span><span class="n">vx</span><span class="p">,</span> <span class="o">-</span><span class="n">fy</span> <span class="o">-</span> <span class="mf">0.3</span><span class="o">*</span><span class="n">vy</span><span class="p">,</span> <span class="o">-</span><span class="n">fz</span> <span class="o">-</span> <span class="mf">0.3</span><span class="o">*</span><span class="n">vz</span><span class="p">))</span>
</pre></div>
<p>Now we can integrate the trajectory:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">scipy.integrate</span> <span class="kn">import</span> <span class="n">odeint</span>
<span class="c1"># Initial conditions</span>
<span class="n">R0</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="c1"># Times at which we want the integrator to return the positions:</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">500</span><span class="p">)</span>
<span class="n">R</span> <span class="o">=</span> <span class="n">odeint</span><span class="p">(</span><span class="n">flow</span><span class="p">,</span> <span class="n">R0</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span>
</pre></div>
<p>And we can now plot the trajectories, after removing the cut plane and
the vector field by right-clicking on the corresponding pipeline node
and selecting delete. We also turn the first color bar off in the
corresponding Colors and legends node. We plot the trajectories with an
extra scalar information attached to it, to display the time via the
colormap:</p>
<div class="highlight"><pre><span></span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="n">vx</span><span class="p">,</span> <span class="n">vy</span><span class="p">,</span> <span class="n">vz</span> <span class="o">=</span> <span class="n">R</span><span class="o">.</span><span class="n">T</span>
<span class="n">trajectory</span> <span class="o">=</span> <span class="n">mlab</span><span class="o">.</span><span class="n">plot3d</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">colormap</span><span class="o">=</span><span class="s1">'hot'</span><span class="p">,</span>
<span class="n">tube_radius</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
<span class="n">mlab</span><span class="o">.</span><span class="n">colorbar</span><span class="p">(</span><span class="n">trajectory</span><span class="p">,</span> <span class="n">title</span><span class="o">=</span><span class="s1">'Time'</span><span class="p">,</span> <span class="n">orientation</span><span class="o">=</span><span class="s1">'vertical'</span><span class="p">)</span>
</pre></div>
<a class="reference external image-reference" href="attachments/trajectories.jpg"><img alt="" src="attachments/trajectories.jpg" /></a>
<p>If I have time, I’ll show later how some of the operations we have done
with numpy can be done with VTK and Mayavi. This will give us control of
these operation via widgets and thus more interativity.</p>
Numpy documentation editor2008-10-27T00:50:00+01:002008-10-27T00:50:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-10-27:/programming/numpy-documentation-editor.html<p>Pauli Virtanen and myself have finally finished transfering the numpy
documentation editor to <a class="reference external" href="http://docs.scipy.org">http://docs.scipy.org</a>. The documentation editor
is a project that has been mainly championed by Pauli. It allows you to
edit in a wiki-like fashion the documentation for numpy, including the
docstring. The changes are reviewed …</p><p>Pauli Virtanen and myself have finally finished transfering the numpy
documentation editor to <a class="reference external" href="http://docs.scipy.org">http://docs.scipy.org</a>. The documentation editor
is a project that has been mainly championed by Pauli. It allows you to
edit in a wiki-like fashion the documentation for numpy, including the
docstring. The changes are reviewed by editors, and eventually merged in
the numpy svn. As a result, they are shipped with numpy and end up on
everybody’s install of numpy.</p>
<p>the documentation editor has been deployed during the summer on my
girlfriend’s hosted server, but we where afraid it wouldn’t scale there
(and beside using my girlfriend’s server was not ideal). The
contributions made throught the web portal have already helped improve
the numpy documentation tremendously. It is a pleasure to look at the
docstring of a function and find it actually helpful. Now that it is
hosted on the main scipy servers, we are no longer afraid of making as
much publicity as possible around it. So please, go straight to
<a class="reference external" href="http://docs.scipy.org">http://docs.scipy.org</a> and start improving the docs. More seriously, when
you think a feature is poorly documented, when you have faught for a few
hours to understand how a function works, improve the docs, it is very
easy, and if everybody does this, you’ll save time too.</p>
<p>In the long run we would like to get scipy itself under the same
mechanism, and I would love to open the service to other major Python
scientific computing librairies that form the scipy ecosystem.</p>
What’s wrong with young academic careers in France2008-10-13T22:36:00+02:002008-10-13T22:36:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-10-13:/science/whats-wrong-with-young-academic-careers-in-france.html<p><a class="reference external" href="http://cournape.wordpress.com/">David</a> just blogged a link to an <a class="reference external" href="http://insidehighered.com/views/2008/09/15/altbach">article</a> about careers in higher
education. I thought the paragraph on the French system was so much to
the point that I would like to quote it entirely here:</p>
<blockquote>
In France, the access to a first permanent position as <em>maître de
conférences</em> occurs …</blockquote><p><a class="reference external" href="http://cournape.wordpress.com/">David</a> just blogged a link to an <a class="reference external" href="http://insidehighered.com/views/2008/09/15/altbach">article</a> about careers in higher
education. I thought the paragraph on the French system was so much to
the point that I would like to quote it entirely here:</p>
<blockquote>
In France, the access to a first permanent position as <em>maître de
conférences</em> occurs rather early compared with other countries (on
average prior to the age of 33 years) and opens the path to 35 to 40
years of an academic career. These recruitments happen after a
period of high uncertainty as in almost all disciplines the ratio of
“open positions per doctors” has worsened, while the doctoral degree
is still not recognized as a qualification by businesses or the
public sector. Recruiting a new <em>maître de conférences</em> thus
constitutes a high-stakes decision. But currently university
departments have about two months to examine the candidates, select
some of them, hold a 20- to 30-minute interview with those on the
short list, and rank the best ones. Despite the highly selective
process that the first candidate on the list successfully passes,
this new colleague is rarely considered as a chance on which to
build by the recruiting university. Not only is the salary based on
a national bureaucratic scale below the average GDP per capita for
France, but new academics are frequently not offered a personal
office and may be asked to teach the classes colleagues do not want
to offer or to accept administrative duties. The difficult road
toward the doctorate leads to a rather disappointing and frequently
non-well-remunerated situation, thus undermining the attractiveness
of the career.</blockquote>
<p>I don’t regret doing a PhD, but I think the current situation needs to
be stressed, especially to future PhD students: high risk, little gain
career. You better really love what you’ll be doing. And keep in mind an
exit door.</p>
My travels this summer2008-10-05T23:24:00+02:002008-10-05T23:24:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-10-05:/programming/my-travels-this-summer.html<p>This summer has been hectic (life is hectic anyhow!). As I was switching
fields from physics to neuro-imaging, I took the chance to travel to the
US and to spend the summer doing Python-related stuff.</p>
<div class="section" id="austin-enthought">
<h2>Austin - Enthought</h2>
<p>I spent most of my time this summer at <a class="reference external" href="http://www.enthought.com">Enthought</a>, in Austin, Texas …</p></div><p>This summer has been hectic (life is hectic anyhow!). As I was switching
fields from physics to neuro-imaging, I took the chance to travel to the
US and to spend the summer doing Python-related stuff.</p>
<div class="section" id="austin-enthought">
<h2>Austin - Enthought</h2>
<p>I spent most of my time this summer at <a class="reference external" href="http://www.enthought.com">Enthought</a>, in Austin, Texas.
Actually it was the Enthought guys who really made my fantastic summer
possible by paying me a good salary and thus indirectly funding my
travels. Thanks Enthought, you guys rock.</p>
<a class="reference external image-reference" href="attachments/enthought-longhorn1.png"><img alt="" src="attachments/enthought-longhorn1.png" /></a>
<p>Austin is a nice city. It’s got a very nice night life (thought the
tequilla doesn’t match Jarrod’s expectations, and they don’t always
accept South Africans in bars because they look like kids and “any
college kid could fake this ID”).</p>
<p>I don’t know if it is particular to the Enthought guys, or it is just
Texans in general, but the hospitality was simply incredible. I spent
the summer at Eric Jones’ place (I had a separate little cabin for me,
fantastic). I enjoyed a lot interacting with Eric’s family: Courtney and
the kids, Zach and Liz. It was fun to be at a place with young children.
By the way, they go around in scipy T-shirts and love when daddy’s
friends (aka geeks) come around at the house. I bet they know the scipy
community better than I do.</p>
<p>Of course, there is a catch: Eric lives out in the boonies (just in case
you don’t speak Texan, this means “out in the bush”, ie in a remote
location). It might even have been a trap: he lives in a <a class="reference external" href="http://en.wikipedia.org/wiki/Dry_counties">dry county</a>.
Moreover, I woke up one morning to find this in my living room:</p>
<a class="reference external image-reference" href="attachments/scorpion-300x240.jpg"><img alt="" src="attachments/scorpion-300x240.jpg" /></a>
<p>It’s not deadly, I have been told, it just hurts a lot. On top of that,
the kids brought back a dead black widow one day. Now it may seem to
some that I am making a stupid fuss about nothing, but the scariest
animals we have where I live are probably cats.</p>
<p>The Enthought office is a very pleasant place to work. It large with a
lot of space for everybody. It is full of very nice people (I knew that
before coming, but it was nice to have a confirmation). I had the nicest
office I have ever had so far (they seem to be improving each time I
change jobs, that a good sign). I’ll talk about what I did there in
another post (I actually started a screen cast about this, but the state
of video-editing software under Linux is abymissimal, and I could never
edit the sound track).</p>
<a class="reference external image-reference" href="attachments/p1010588-300x187.jpg"><img alt="" src="attachments/p1010588-300x187.jpg" /></a>
</div>
<div class="section" id="visits-to-california">
<h2>Visits to California</h2>
<p>I made two trips to California. First I went to UCLA for a summer school
on mathematical method in neuro-imaging. It was the occasion for me to
discover the field. One thing that stroke me was the importance of
software in the field, and how little people are organized to limit
duplication of efforts. On the other hand, we had a very nice
presentation about a beautiful software engineering effort (<a class="reference external" href="http://www.slicer.org">slicer</a>)
trying to build a platform to unite tools by Steve Pieper. Fernando and
I where gritting our teethes during the talk, wondering what the license
of the tools would be, and it turned out that Steve ended his talk with
a discussion about why it seemed to him that the BSD was most suited. We
were delighted.</p>
<p>My second visit was for the <a class="reference external" href="http://conference.scipy.org">SciPy conference</a> which was great fun. Fun
to meet new people in real life, fun to meet old friends. I had the
feeling the talks where excellent and I learned a lot of things. After
the conference I went to Berkeley with the <a class="reference external" href="http://projects.scipy.org/neuroimaging/ni/wiki">nipy</a> team. Nipy stands for
NeuroImaging in Python. It is a project led by a crack team at Berkeley
with Chris Burns, Jarrod Millman, Fernando Perez, Tom Waite and now
Matthew Brett. The <a class="reference external" href="http://lnao.fr">team</a> I am starting my new job with, in France,
hopes to be able to integrate their software with nipy.</p>
<p>Berkeley is a nice place. I stayed at Fernando’s and Jarrod’s and it is
always a pleasure to hang out with these guys. As far as work goes, we
tried to do some 3D visualization of neuro-imaging data with Mayavi. We
got some things done and the week ended in a party at Chris’s place
where we greated Jarrod with a mac-book air displaying a really cool
view of a brain. However I have the feeling I stayed just long-enough to
understand the problems, and not to solve them. Damn, software is hard.
Hopefuly my work will allow me to move further in this direction.</p>
</div>
<div class="section" id="back-in-france-and-off-to-prague">
<h2>Back in France, and off to Prague</h2>
<p>Well, after all these travels, I got back to France. Off course the
first thing I did with my girlfriend, Emmanuelle, was to shoot out of
the country. I need to get away of a computer, sometimes. We went to
Prague. It is a very beautyful city, it has good beers, and it is the
home town of Ondrej Certik, so that made three good reasons to go there.
We had a great week end strolling through the city, in the old streets
or in the castles (photos <a class="reference external" href="http://gael-varoquaux.info/journal/prague/index.html">here</a>). Most people there speak English, but
I was lucky-enough to order fried cheese (doesn’t that sound nice?) to a
lady not speaking English, so we fell back on Russian as a common
language, and it is always fun to me to speak Russian (now that’s a long
sentence).</p>
<p>We had a bunch of beers with Ondrej. We spent some times talking the
world into a better place. We discussed licensing, and agreed that
<a class="reference external" href="http://cython.org">Cython</a> was sooooo cool, and talked about all things under the
(scientific Python) sun.</p>
<p>In the past year I have the feeling I have been all over the place. I
changed jobs three times, moved houses once in France and twice abroad,
and visited a lot of countries for the fun of it. In the near future I
am planning to settle down in France to get some work done.</p>
</div>
<div class="section" id="we-need-help">
<h2>We need help</h2>
<p>Speaking of work, I am starting a new career in a new academic field.
This is going to require a lot of focus from me, and it will not leave
too much time for open source work in the near future. We need help!
There are many ways to help and not all of them involve coding. I think
I spend not more than 50% of my open-source-devoted time on coding. We
need better docs. We need more marketing (we are really bad at this: we
have fantastic tools, but it is hard to see them). We need people to
help each-others on the mailing lists. We need packaging. All this is
paramount and takes a lot of time… And we need coding.</p>
<a class="reference external image-reference" href="attachments/s_p10105761-225x300.jpg"><img alt="" src="attachments/s_p10105761-225x300.jpg" /></a>
<a class="reference external image-reference" href="attachments/s_p10105732-225x300.jpg"><img alt="" src="attachments/s_p10105732-225x300.jpg" /></a>
<a class="reference external image-reference" href="attachments/s_p10105831.jpg"><img alt="" src="attachments/s_p10105831.jpg" /></a>
</div>
SciPy Conference proceedings2008-09-22T14:54:00+02:002008-09-22T14:54:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-09-22:/programming/scipy-conference-proceedings.html<p>The SciPy conference proceedings are finally available online:
<a class="reference external" href="http://conference.scipy.org/proceedings/SciPy2008">http://conference.scipy.org/proceedings/SciPy2008</a> .</p>
<p>I hope you enjoy them. I find it great to have this set of excellent
articles talking about works done with, or for, Python in science. For
me, it is a reference to remember what was …</p><p>The SciPy conference proceedings are finally available online:
<a class="reference external" href="http://conference.scipy.org/proceedings/SciPy2008">http://conference.scipy.org/proceedings/SciPy2008</a> .</p>
<p>I hope you enjoy them. I find it great to have this set of excellent
articles talking about works done with, or for, Python in science. For
me, it is a reference to remember what was said at the conference. I
hope it can also be interesting for people who were not present at the
conference.</p>
<p>I apologize for being so slow at publishing them. In addition to the
round trip between authors and editors taking a while, I have been
travelling back home and spent way too much time last week finishing off
administrative duties in the US.</p>
Rendering static pages with Turbogears2008-09-07T07:12:00+02:002008-09-07T07:12:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-09-07:/programming/rendering-static-pages-with-turbogears.html<div class="section" id="turbogears-hack">
<h2>Turbogears hack</h2>
<p>Suppose you have a dynamic website using turbogears, and you want to
publish part of the content of this dynamic site to a static website,
for instance to garanty its perenity. Well turbogears makes it really
hard for you to do this. On the mailing lists they pretty …</p></div><div class="section" id="turbogears-hack">
<h2>Turbogears hack</h2>
<p>Suppose you have a dynamic website using turbogears, and you want to
publish part of the content of this dynamic site to a static website,
for instance to garanty its perenity. Well turbogears makes it really
hard for you to do this. On the mailing lists they pretty much advise
you to create a webserver and crawl it. Ugly. So here is the code
required to render the kid templates that you have been using with
turbogears to an html string (consider this as a brain dump, so that
Google picks this up, hopefuly to help somebody not to loose a
day like I did):</p>
<div class="highlight"><pre><span></span><span class="c1"># First set up the environment you need for your webapp:</span>
<span class="kn">import</span> <span class="nn">turbogears</span>
<span class="n">turbogears</span><span class="o">.</span><span class="n">update_config</span><span class="p">(</span><span class="n">configfile</span><span class="o">=</span><span class="s2">"dev.cfg"</span><span class="p">,</span>
<span class="n">modulename</span><span class="o">=</span><span class="s2">"sanum.config"</span><span class="p">)</span>
<span class="kn">from</span> <span class="nn">itertools</span> <span class="kn">import</span> <span class="n">izip</span>
<span class="kn">import</span> <span class="nn">turbogears.view</span>
<span class="n">turbogears</span><span class="o">.</span><span class="n">view</span><span class="o">.</span><span class="n">load_engines</span><span class="p">()</span>
<span class="kn">import</span> <span class="nn">turbogears.util</span> <span class="k">as</span> <span class="nn">tg_util</span>
<span class="kn">from</span> <span class="nn">turbogears.widgets</span> <span class="kn">import</span> <span class="n">js_location</span>
<span class="n">engine</span> <span class="o">=</span> <span class="n">turbogears</span><span class="o">.</span><span class="n">view</span><span class="o">.</span><span class="n">engines</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'kid'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">render_static</span><span class="p">(</span><span class="n">data_dict</span><span class="p">,</span> <span class="n">template</span><span class="p">):</span>
<span class="w"> </span><span class="sd">""" Render a given template + its data dictionnary to a static html.</span>
<span class="sd"> """</span>
<span class="n">data_dict</span><span class="p">[</span><span class="s1">'tg_css'</span><span class="p">]</span> <span class="o">=</span> <span class="n">tg_util</span><span class="o">.</span><span class="n">setlike</span><span class="p">()</span>
<span class="n">data_dict</span><span class="p">[</span><span class="s1">'tg_flash'</span><span class="p">]</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">js</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">izip</span><span class="p">(</span><span class="n">js_location</span><span class="p">,</span> <span class="nb">iter</span><span class="p">(</span><span class="n">tg_util</span><span class="o">.</span><span class="n">setlike</span><span class="p">,</span> <span class="kc">None</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">l</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="n">js_location</span><span class="p">):</span>
<span class="n">data_dict</span><span class="p">[</span><span class="s2">"tg_js_</span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="nb">str</span><span class="p">(</span><span class="n">l</span><span class="p">)]</span> <span class="o">=</span> <span class="n">js</span><span class="p">[</span><span class="n">l</span><span class="p">]</span>
<span class="k">return</span> <span class="n">engine</span><span class="o">.</span><span class="n">render</span><span class="p">(</span><span class="n">data_dict</span><span class="p">,</span> <span class="n">template</span><span class="o">=</span><span class="n">template</span><span class="p">)</span>
</pre></div>
<p>You can call this function with data_dict being a dictionary as
returned by your controller methods, and template the path to your
template, as in the expose decorator.</p>
</div>
pyreport: literate programming in python2008-07-23T00:00:00+02:002008-07-23T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-07-23:/programming/pyreport-literate-programming-in-python.html<div class="sidebar">
<p class="first sidebar-title">Downloads :</p>
<p>pyreport is hosted on
<a class="reference external" href="https://pypi.python.org/pypi/pyreport">pypi</a></p>
<p><strong>Project page :</strong></p>
<p class="last"><a class="reference external" href="http://github.com/joblib/pyreport">Source code on github</a></p>
</div>
<p><cite>pyreport</cite> is a program that runs a python script and captures its
output, compiling it to a pretty report in a pdf or an html file. It can
display the output embedded in the code that produced it …</p><div class="sidebar">
<p class="first sidebar-title">Downloads :</p>
<p>pyreport is hosted on
<a class="reference external" href="https://pypi.python.org/pypi/pyreport">pypi</a></p>
<p><strong>Project page :</strong></p>
<p class="last"><a class="reference external" href="http://github.com/joblib/pyreport">Source code on github</a></p>
</div>
<p><cite>pyreport</cite> is a program that runs a python script and captures its
output, compiling it to a pretty report in a pdf or an html file. It can
display the output embedded in the code that produced it and can process
special comments (literate comments) according to markup languages (
<a class="reference external" href="http://docutils.sourceforge.net/rst.html">rst</a> or <a class="reference external" href="http://www.latex-project.org/">LaTeX</a> ) to compile a very readable document.</p>
<p>This allows for extensive literate progamming <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> in python, for generating
reports out of calculations written in python, and for making nice
tutorials.</p>
<p><strong>License</strong> pyreport is free software released under a BSD-like
license. You can chek out the latest code, submit bugs, ask questions…
on the <a class="reference external" href="http://github.com/joblib/pyreport">github project page</a>.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p><strong>Pyreport is unmaintained</strong></p>
<p class="last">Due to lack of time, pyreport is unmaintained and looking for a
contributor. Please do not contact me to ask questions, unless it is
to take over development.</p>
</div>
<div class="contents topic" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#requirements" id="toc-entry-1">Requirements</a></li>
<li><a class="reference internal" href="#installing" id="toc-entry-2">Installing</a></li>
<li><a class="reference internal" href="#examples-and-use-cases" id="toc-entry-3">Examples and use cases</a><ul>
<li><a class="reference internal" href="#initial-goals" id="toc-entry-4">Initial goals</a></li>
<li><a class="reference internal" href="#other-possible-uses" id="toc-entry-5">Other possible uses</a></li>
<li><a class="reference internal" href="#examples" id="toc-entry-6">Examples</a></li>
<li><a class="reference internal" href="#limitations" id="toc-entry-7">Limitations</a></li>
</ul>
</li>
<li><a class="reference internal" href="#command-line-switches" id="toc-entry-8">Command line switches</a></li>
</ul>
</div>
<div class="section" id="requirements">
<h2><a class="toc-backref" href="#toc-entry-1">Requirements</a></h2>
<p><cite>pyreport</cite> is Python, and needs the Python interpreter to run. It should
work on any operating system where Python and the other requirements are
available.</p>
<ul>
<li><p class="first"><strong>External programs</strong>:</p>
<p>Under windows make sure they are in the path, elsewhere <cite>pyreport</cite>
cannot find them. All these programs are available with the <a class="reference external" href="http://www.miktex.org/">MikTeX</a> LaTeX distribution.</p>
<ul class="simple">
<li><cite>pyreport</cite> does not need any external programs to generate html
files.</li>
<li>LaTeX : currently <cite>pyreport</cite> calls LaTeX to generate pdf files.
Hopefully one day this will be optional and <cite>pyreport</cite> will use
reportlabs to output pdf files.</li>
<li>epstopdf or ps2pdf if you want to use pylab to insert graphs in your
documents. Once <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> has a pdf backend this will not be
needed.</li>
</ul>
</li>
<li><p class="first"><strong>Python packages</strong>:</p>
<ul class="simple">
<li>docutils only</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="installing">
<h2><a class="toc-backref" href="#toc-entry-2">Installing</a></h2>
<p>The easiest way to install pyreport is using pip:</p>
<pre class="literal-block">
pip install pyreport
</pre>
<p>It is recommended to install to your user drive:</p>
<pre class="literal-block">
pip install --user pyreport
</pre>
</div>
<div class="section" id="examples-and-use-cases">
<h2><a class="toc-backref" href="#toc-entry-3">Examples and use cases</a></h2>
<div class="section" id="initial-goals">
<h3><a class="toc-backref" href="#toc-entry-4">Initial goals</a></h3>
<p>I use python to write small scripts that can do, for instance, numerical
calculations, or simple operations <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a> and I want to have nice
print-outs of these scripts to study off-screen or to hand out to
colleagues. Having the code with its relevant output is great for code
reviewing. This also allows something similar to mathematica’s notebook
in python without having to use a special IDE.</p>
<blockquote>
<ul>
<li><p class="first">First I want to be able to have a print-out of these calculations
where I can see the code ran, and the results produced:</p>
<a class="reference external image-reference" href="attachments/pyreport/simple.png"><img alt="The ouput of a session." class="align-center" src="attachments/pyreport/simple.png" /></a>
</li>
</ul>
</blockquote>
<hr class="docutils" />
<blockquote>
<ul>
<li><p class="first">Second I would like all the plots produced by <a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> to be
captured and displayed too.</p>
<a class="reference external image-reference" href="attachments/pyreport/pylab.png"><img alt="The "show()" command gets overloaded." class="align-center" src="attachments/pyreport/pylab.png" /></a>
</li>
</ul>
</blockquote>
<hr class="docutils" />
<blockquote>
<ul>
<li><p class="first">Last I would be able to comment these reports, give them titles,
sections, … This can be done via “literate programming”: comment
lines begin with a special sets of characters are interpreted as
<a class="reference external" href="http://docutils.sourceforge.net/rst.html">rst</a> or <a class="reference external" href="http://www.latex-project.org/">LaTeX</a>. I also want these files to still be standard
python files, and to be able to run them with the python interpreter.</p>
<pre class="literal-block">
#! Just a title
#!---------------
a = "a"
#! *ooo oo
b = "b"
#$ This is \LaTeX : $c = 2\cdot(a+b)$
c = 2*(a+b)
print c
</pre>
<a class="reference external image-reference" href="attachments/pyreport/latex.png"><img alt="A python report with a title and a LaTeX formula." class="align-center" src="attachments/pyreport/latex.png" /></a>
</li>
</ul>
</blockquote>
</div>
<div class="section" id="other-possible-uses">
<h3><a class="toc-backref" href="#toc-entry-5">Other possible uses</a></h3>
<blockquote>
<ul class="simple">
<li>Hidding the source code allows to generate nice reports from
calculation scripts without worrying about writing document generating
code in the script itself. The use of the literal comments and <cite>print</cite>
statements allow your report to be well structured and
self-explaining. The major advantage of having the text of the report
in the source of the calulation is that the report always discribes
the calculation that was actually ran, and not a previous one, with
incorrect constants, for instance.</li>
<li>With a moin-moin syntax and a pdf output this can be a very useful
tool for writing tutorials and putting them on line, with a pdf
version.</li>
</ul>
</blockquote>
</div>
<div class="section" id="examples">
<h3><a class="toc-backref" href="#toc-entry-6">Examples</a></h3>
<p>Here are two examples showing what you can do with <cite>pyreport</cite> :</p>
<blockquote>
<ul>
<li><p class="first">A calculation of the bifurcation diagram of the logistic mapping</p>
<p><a class="reference external" href="attachments/pyreport/example2.python">The code</a> , the <a class="reference external" href="attachments/pyreport/example2.pdf">pdf
generated</a> , and the <a class="reference external" href="attachments/pyreport/example2.html">html file
generated</a></p>
</li>
<li><p class="first">An exploration of the Julia sets, this example uses a LaTeX equation
(LaTeX embedding does not work with html output, so far):</p>
<p><a class="reference external" href="attachments/pyreport/example.python">The code</a> , the <a class="reference external" href="attachments/pyreport/example.pdf">pdf
generated</a> , and the <a class="reference external" href="attachments/pyreport/example.html">html file
generated</a></p>
</li>
</ul>
</blockquote>
</div>
<div class="section" id="limitations">
<h3><a class="toc-backref" href="#toc-entry-7">Limitations</a></h3>
<p>The “sys” module is imported in your code, whether you want it or not. As
a general rule, beware of the namespace when running your scripts with
<cite>pyreport</cite>, <cite>pyreport</cite> injects a few variables in your namespace.</p>
<p><cite>from __future__ import foobar</cite> in a script does not work. This a a big
caveat ! This is simply not possible in python, as the <cite>from __future__</cite>
imports have to be the first statement of a script.</p>
<p>As a consequence I have made the decision to always import <cite>division</cite>, so
that 2/3 = 0.6666.</p>
<p>Some stange bugs can occur depending on the backend you use for
matplotlib. WXAgg has played me a few tricks.</p>
</div>
</div>
<div class="section" id="command-line-switches">
<h2><a class="toc-backref" href="#toc-entry-8">Command line switches</a></h2>
<p>This is not as useful as a well written documentation, but it is better
than nothing:</p>
<pre class="literal-block">
usage: pyreport [options] pythonfile
Processes a python script and pretty prints the results using LateX. If
the script uses "show()" commands (from pylab) they are caught by
pyreport and the resulting graphs are inserted in the output pdf.
Comments lines starting with "#!" are interprated as rst lines
and pretty printed accordingly in the pdf.
By Gael Varoquaux
options:
--version show program's version number and exit
-h, --help show this help message and exit
-o FILE, --outfile=FILE
write report to FILE
-x, --noexecute do not run the code, just extract the literate
comments
-n, --nocode do not display the source code
-d, --double compile to two columns per page (only for pdf or tex
output)
-t TYPE, --type=TYPE output to TYPE, TYPE can be ps, dvi, trac, eps, tex,
html, pdf, rst, moin
-f TYPE, --figuretype=TYPE
output figure type TYPE (TYPE can be of pdf, jpg,
eps, png, ps depending on report output type)
-c CHAR, --commentchar=CHAR
literate comments start with "#CHAR"
-l, --latexliterals allow LaTeX literal comment lines starting with "#$"
-e, --latexescapes allow LaTeX math mode escape in code wih dollar signs
-p, --nopyreport disallow the use of #pyreport lines in the processed
file to specify options
-q, --quiet don't print status messages to stderr
-v, --verbose print all the message, including tex messages
-s, --silent Suppress the display of warning and errors in the
report
--noecho Turns off the echoing of the output of the script on
the standard out
-a ARGS, --arguments=ARGS
pass the arguments "ARGS" to the script
</pre>
<hr class="docutils" />
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td><a class="reference external" href="http://en.wikipedia.org/wiki/Literate_programming">literate programming</a> is a programming style
that embeds the documentation of a program in its source code. The
documentation is generated in the same time that the program is
built. I am using the term in a very loose way, as <cite>pyreport</cite> is
capable of literate programming, but also of much more, as it embeds
the documentation, but also weaves the output of the script in the
documentation.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td><a class="reference external" href="http://scipy.org/">scipy</a> provides a very powerful math package to python, and
<a class="reference external" href="http://matplotlib.sourceforge.net/">matplotlib</a> is a great plotting interface.</td></tr>
</tbody>
</table>
</div>
Scipy2008 Early-bird registration deadline ends today2008-07-11T09:15:00+02:002008-07-11T09:15:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-07-11:/programming/scipy2008-early-bird-registration-deadline-ends-today.html<p>I have been planning to make a more interesting post highlighting the
large trends of the SciPy2008 conference, but it is 3AM local time, and
I am still hacking on Mayavi, so I think I’ll keep it short.</p>
<p>As far a the <a class="reference external" href="http://conference.scipy.org/conference">conference program</a> goes, we can see a …</p><p>I have been planning to make a more interesting post highlighting the
large trends of the SciPy2008 conference, but it is 3AM local time, and
I am still hacking on Mayavi, so I think I’ll keep it short.</p>
<p>As far a the <a class="reference external" href="http://conference.scipy.org/conference">conference program</a> goes, we can see a few major themes
emerging. There will be talks about the use of Python for scientific
works, but also talks about the growing stack of Python scientific
tools. Interesting trends are the non-purely-numerical tools: symbolic
and graph theory, and the race towards more optimisation through
compilation from Python code. In addition this year we see a major
effort on documentation. I think this is the sign of a numerical stack
that is maturing.</p>
<p>As for <a class="reference external" href="http://conference.scipy.org/tutorials">the tutorials</a>, I am personnally very interested in the
advanced track tutorial. The newest and coolest technologies, like
Cython, are also not the one I know best, and we have the chance to be
able to listen to their authors presenting them.</p>
<p><strong>The early bird registration deadline is ending tomorrow</strong>, as I point
out in my title. If you miss this deadline, the conference fees will be
higher, and the reason is simply that late registration makes
organisation harder and more expensive. I would be happier if everybody
registered before this deadline and paid less. I am not too sure what
the accountant would say.</p>
Student sponsorship for the SciPy08 conference2008-06-27T05:26:00+02:002008-06-27T05:26:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-06-27:/programming/student-sponsorship-for-the-scipy08-conference.html<p>I am delighted to announce that the Python Software Foundation has
answered our call and is providing sponsoring to the <a class="reference external" href="http://conference.scipy.org">SciPy08
conference</a>.</p>
<p><strong>We will use this money to sponsor the registration fees and travel for
up to 10 college or graduate students to attend the conference</strong>. The
PSF did not …</p><p>I am delighted to announce that the Python Software Foundation has
answered our call and is providing sponsoring to the <a class="reference external" href="http://conference.scipy.org">SciPy08
conference</a>.</p>
<p><strong>We will use this money to sponsor the registration fees and travel for
up to 10 college or graduate students to attend the conference</strong>. The
PSF did not provide all the founds required for all 10 students and once
again Enthought Inc. (<a class="reference external" href="http://www.enthought.com">http://www.enthought.com</a>) is stepping up to fill
in[1].</p>
<p><strong>To apply, please send a short description of what you are studying and
why you’d like to attend to info@enthought.com</strong>. Please include
telephone contact information.</p>
<p>From my perspective, this is excellent news. First of all this means
that the SciPy community is working a bit more closely with the PSF and
the broader Python community. But the is also very dear to me as last
year I was sponsored as a student to come to the SciPy conference. I got
to meet fantastic people and discover thrilling new developments. This
was the beginning of a move away of my core physics activity to a more
software-related work, and the realization that yes, I could do it, I
could maybe contribute something useful to the community (well, I’ll let
you judge that). Thanks a lot to Travis Vaught from Enthought for
bringing this project to a success.</p>
<p>[1] I feel like we (the SciPy community) are like an aging teenager,
wanting a lot of independence, but still living a lot out of parent’s
money (Enthought). And I feel the first concerned, as I am spending the
summer working at Enthought to get a chance to work on interesting
SciPy-related projects.</p>
<p>Disclaimer: The second part of this post reflects my own opinions, and
not those of my employer (obviously) or the SciPy08 organising comittee.</p>
Alex Martelli giving the SciPy2008 Keynote2008-06-14T19:08:00+02:002008-06-14T19:08:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-06-14:/programming/alex-martelli-giving-the-scipy2008-keynote.html<p>On behalf of the <a class="reference external" href="http://conference.scipy.org">SciPy2008 conference</a> organizing committee, I am
happy to announce that the Keynote at the conference will be given by
Alex Martelli.</p>
<p>It is a pleasure for us to receive Alex. He currently works as “Uber
Tech Leader” at Google and is the author of two of …</p><p>On behalf of the <a class="reference external" href="http://conference.scipy.org">SciPy2008 conference</a> organizing committee, I am
happy to announce that the Keynote at the conference will be given by
Alex Martelli.</p>
<p>It is a pleasure for us to receive Alex. He currently works as “Uber
Tech Leader” at Google and is the author of two of the Python classics:
“Python in a nutshell” and the “Python CookBook”. Alex graduated in
electronic engineering from the university of Bologna and worked in chip
design first for Texas Instrument, and later for IBM Research. During
the 8 years he spent at IBM, he gradually shifted from hardware design
to software development while winning three Outstanding Technical
Achievement Awards. Then he joined think3 inc., and Italian CAD company,
as Senior Software Consultant where he developed libraries, network
protocols, GUI engines, event frameworks, and web access frontends.
After 12 years at think3, he worked for 3 years as a freelance
consultant, mostly doing Python development, before joining Google.</p>
<p>Alex won the 2002 Activators’ Choice Award, and the 2006 Frank Willison
award for outstanding contributions to the Python community.</p>
<p>Alex has also taught courses on programming, development methods,
object-oriented design, and numerical computing, at Ferrara University
(Italy) and other venues. Alex’s proudest achievement is the articles
that appeared in Bridge World (January/February 2000), which were hailed
as giant steps towards solving issues that had haunted contract-bridge
game theoreticians for decades.</p>
<p>This biography was loosely adapted from Alex’s autobiography
(<a class="reference external" href="http://www.aleax.it/bio.txt">http://www.aleax.it/bio.txt</a>), more information can be found on his
website <a class="reference external" href="http://www.aleax.it">http://www.aleax.it</a> .</p>
Arrived in Texas2008-06-13T15:10:00+02:002008-06-13T15:10:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-06-13:/programming/arrived-in-texas.html<p>I just arrived in Austin, Texas. I need to settle down a bit more, blog
about my fantastic holidays, but I wanted to give an update of where I
was.</p>
<p>The hospitality here has been fantastic so far. I am sitting in a confy
chair, sipping a fresh orange juice …</p><p>I just arrived in Austin, Texas. I need to settle down a bit more, blog
about my fantastic holidays, but I wanted to give an update of where I
was.</p>
<p>The hospitality here has been fantastic so far. I am sitting in a confy
chair, sipping a fresh orange juice, after having spend a night in a
very cosy bungalow and waking up to see two fawns in the garden. I would
say that the Enthought guys really know how to treat there hosts. It
seems to be something the Python scientific community is really good at,
judging from my different experiences (Fernando, and JB Poline).</p>
Asie centrale: 17 mai - 8 juin 2008: Kyrgyzstan2008-06-02T00:00:00+02:002008-06-02T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-06-02:/personnal/asie_centrale_kyrgyzstan.html<p>Deuxième partie du voyage: au Kirghizstan.</p>
<p>Photos et panoramas : Gaël | Récit, mise en page et retouches photos :
Emmanuelle</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last"><strong>Première partie du voyage</strong>: <a class="reference external" href="asie_centrale_uzbekistan.html">la première partie du récit, sur la page de
l’Ouzbekhistan.</a></p>
</div>
<a class="reference external image-reference" href="map.jpg"><img alt="Carte voyage" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/map.jpg" style="width: 100%;" /></a>
<div class="contents topic" id="table-des-matieres">
<p class="topic-title">Table des matières</p>
<ul class="simple">
<li><a class="reference internal" href="#vendredi-23-mai-tashkent-bishkek" id="toc-entry-1">Vendredi 23 mai : Tashkent -> Bishkek</a></li>
<li><a class="reference internal" href="#samedi-24-mai-bishkek-karakol" id="toc-entry-2">Samedi 24 mai : Bishkek -> Karakol</a></li>
<li><a class="reference internal" href="#dimanche-25-mai-vallee-de-la-karaka" id="toc-entry-3">Dimanche 25 mai …</a></li></ul></div><p>Deuxième partie du voyage: au Kirghizstan.</p>
<p>Photos et panoramas : Gaël | Récit, mise en page et retouches photos :
Emmanuelle</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last"><strong>Première partie du voyage</strong>: <a class="reference external" href="asie_centrale_uzbekistan.html">la première partie du récit, sur la page de
l’Ouzbekhistan.</a></p>
</div>
<a class="reference external image-reference" href="map.jpg"><img alt="Carte voyage" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/map.jpg" style="width: 100%;" /></a>
<div class="contents topic" id="table-des-matieres">
<p class="topic-title">Table des matières</p>
<ul class="simple">
<li><a class="reference internal" href="#vendredi-23-mai-tashkent-bishkek" id="toc-entry-1">Vendredi 23 mai : Tashkent -> Bishkek</a></li>
<li><a class="reference internal" href="#samedi-24-mai-bishkek-karakol" id="toc-entry-2">Samedi 24 mai : Bishkek -> Karakol</a></li>
<li><a class="reference internal" href="#dimanche-25-mai-vallee-de-la-karaka" id="toc-entry-3">Dimanche 25 mai : vallée de la Karaka</a></li>
<li><a class="reference internal" href="#lundi-26-mai-aller-retour-au-lac-ala-kul" id="toc-entry-4">Lundi 26 mai : aller-retour au lac Ala-Kul</a></li>
<li><a class="reference internal" href="#mardi-27-mai-passage-du-col-tileti" id="toc-entry-5">Mardi 27 mai : passage du col Tileti</a></li>
<li><a class="reference internal" href="#mercredi-28-mai-sortie-dans-la-vallee-de-djeti-oguz" id="toc-entry-6">Mercredi 28 mai : sortie dans la vallée de Djeti-Oguz</a></li>
<li><a class="reference internal" href="#jeudi-29-mai-karakol-bar-skoon" id="toc-entry-7">Jeudi 29 mai : Karakol -> Bar-Skoon</a></li>
<li><a class="reference internal" href="#vendredi-30-mai-vers-la-jokuu" id="toc-entry-8">Vendredi 30 mai : vers la Jokuu</a></li>
<li><a class="reference internal" href="#samedi-31-mai-vallee-de-la-joku" id="toc-entry-9">Samedi 31 mai : vallée de la Joku</a></li>
<li><a class="reference internal" href="#dimanche-1er-juin-sortie-de-la-vallee-de-la-joku" id="toc-entry-10">Dimanche 1er juin : sortie de la vallée de la Joku</a></li>
<li><a class="reference internal" href="#lundi-2-juin-kizil-suu-bakonbaiev" id="toc-entry-11">Lundi 2 juin : Kizil-Suu -> Bakonbaiev</a></li>
<li><a class="reference internal" href="#mardi-3-juin-bakonbaiev-kotchkor" id="toc-entry-12">Mardi 3 juin : Bakonbaiev -> Kotchkor -> ??</a></li>
<li><a class="reference internal" href="#mercredi-4-juin-remontee-dans-la-vallee" id="toc-entry-13">Mercredi 4 juin : remontée dans la vallée</a></li>
<li><a class="reference internal" href="#jeudi-5-juin-passage-du-col" id="toc-entry-14">Jeudi 5 juin : passage du col</a></li>
<li><a class="reference internal" href="#vendredi-6-juin-parc-ala-archa" id="toc-entry-15">Vendredi 6 juin : parc Ala-Archa</a></li>
<li><a class="reference internal" href="#samedi-7-juin-retour-a-bishkek" id="toc-entry-16">Samedi 7 juin : retour à Bishkek</a></li>
<li><a class="reference internal" href="#dimanche-8-juin-retour-a-paris" id="toc-entry-17">Dimanche 8 juin : retour à Paris</a></li>
</ul>
</div>
<hr class="docutils" />
<div id="kyrgyzstan" class="align-center"></div>
<script type="text/javascript"
src="http://comfort-works.com/news/wp-content/plugins/javascript-flickr-badge/javascript-flickr-badge.min.js"></script>
<script type="text/javascript">
jsFlickrBadge(document.getElementById('kyrgyzstan'), {
// your Flickr ID (find it here)
flickrId: '66885349@N03',
// feed type. user, group, contacts, etc.
feed: 'user',
// optional comma-delimited tags to filter by, only used with 'user' feed
tags: 'kyrgyzstan',
// number of rows to display
rows: 2,
// number of columns to display
columns: 8,
// size of each thumbnail (any bigger than 75 will cause pixelization)
size: 75,
// animation to use.
// one of: vscroll, random, vscroll, shuffle, zoom, scroll, flipX, flipY
animation: 'vscroll',
// seconds each animation takes
animationSpeed: 1,
// seconds between each animation
animationPause: 20
});
</script><div class="section" id="vendredi-23-mai-tashkent-bishkek">
<h2><a class="toc-backref" href="#toc-entry-1">Vendredi 23 mai : Tashkent -> Bishkek</a></h2>
<p>Aujourd’hui, nous quittons l’Ouzbékistan pour le Kirghistan. Nous avons
un vol à 8h30 pour Bishkek, la capitale kirhize. Nous partons de l’hôtel
à 6h, il fait déjà jour. Nous n’avons aucun problème à trouver juste
devant la gare un taxi qui nous emmène à l’aéroport. Le passage de la
douane s’effectue sans aucune problème, contrairement à ce qu’on nous
avait raconté : le policier regarde rapidement la feuille où nous avons
marqué la quantité de devises que nous transportons, et nous laisse
passer sans rien nous demander. C’est un soulagement ! Nous volons avec
la compagnie Uzbekistan Airlines, et notre avion n’est pas le Tupolev
annoncé sur le billet, mais un Airbus. L’avion est majoritairement
occupé par un groupe d’Allemands du troisième âge, qui nous demandent si
ça ne nous fait pas peur de voyager tous seuls. Nein, keine Angst !</p>
<p>Arrivés à l’aéroport, nous faisons faire notre visa au service consulaire
: le Kirghistan permet aux ressortissants d’un certain nombre de pays
occidentaux, dont la France, de faire leur visa automatiquement en
arrivant à l’aéroport. Il suffit de donner un photo, et de payer 35
dollars pour le visa : c’est extrêmement pratique. Nous changeons des
dollars en soms kirghizes, puis on prend la marshrutka qui va de
l’aéroport de Manas à Bishkek, après avoir réussi à traverser le
bataillon de taxis.</p>
<p>A Bishkek nous nous rendons à la guesthouse “Ultimate Adventure”, qui est
le camp de base d’une agence de voyage française. Son équipe nous a donné
beaucoup de renseignements sur la Kirghizie pendant que nous étions
encore en France. Nous devons marcher un petit moment pour rejoindre la
guesthouse, un peu excentrée. Les gens dans la rue sont très gentils et
nous aident à trouver le chemin. Nous trouvons enfin la guesthouse. Smaïl
nous y accueille : c’est un Algérien qui parle 9 langues et a pas mal
roulé sa bosse avant de se fixer en Kirghizie il y a une dizaine
d’années. Il dirige l’équipe kirghize de l’agence avec beaucoup de bonne
humeur et d’efficacité. Nous discutons un peu dans le jardin pendant
qu’il nourrit son bébé, la petite Kahina qui est la vedette de la maison
!</p>
<p>Puis nous partons faire les courses pour nos randonnées : nous achetons
des cartes dans un petit magasin à l’étage d’un immeuble sur l’avenue
Kiev, à peine indiqué. Ce sont des 1:200000, donc pas très précises,
mais c’est tout ce qu’il y a. Pour les cartes plus précises, Gaël avait
fait un gros travail à Paris pour aller chercher des cartes de l’armée
soviétique sur Internet, puis les faire imprimer sur du papier spécial.
Ensuite nous galérons longtemps pour essayer de trouver du gaz : notre
réchaud fonctionne au gaz, mais c’est très dur d’en trouver, ici la
plupart des gens utilisent du pétrole. Sur l’avenue Moskva, nous
trouvons un magasin de sport qui a un rayon randonnée, mais pas de gaz.
Ils nous conseillent un magasin de chasse/pêche juste à côté, mais qui
est fermé. Nous remettons donc la quête de gaz à plus tard, en espérant
que nous pourrons en acheter à la guesthouse. Nous faisons ensuite les
courses alimentaires au supermarché Beta Store, qui ressemble en tout
point à un supermarché parisien. Ca aurait été beaucoup moins cher de
faire nos courses au bazaar, mais nous n’avons pas beaucoup de temps et
le Beta Store est juste sur notre chemin. Nous y trouvons quand même
certains produits qu’il n’y a pas dans les bazaars, comme du muesli ou de
l’instant pudding.</p>
<p>Nous rentrons à la guesthouse, où est arrivé Sacha, le chef des guides
d’Ultimate Adventure. Nous lui posons plein de questions sur les
randonnées, la quantité de neige, les passages de col, les transports, le
logement, etc. Il connaît super bien les montagnes qu’il sillonne sans
cesse avec les groupes, ou pour découvrir de nouveaux endroits pour
randonner. Il nous conseille un programme constitué de plein de petits
sauts de puce : cela aurait l’avantage de nous faire découvrir plein de
jolis endroits sans trop nous fatiguer, mais nous souhaitons faire des
randos un peu plus longues, sans devoir reprendre un véhicule tous les
deux jours. En tout cas, Sacha est une véritable mine de renseignements,
dont certains nous seront bien utiles par la suite. Il est tard
quand nous sortons du bureau du guide, nous sortons manger rapidement un
morceau au café du coin. On a juste le temps de commander manti, et
laghman (une soupe épicées de nouilles au boeuf, super bon), puis la
lumière s’éteint et c’est une soirée karaoké qui commence. Il n’y a pas
beaucoup de monde, mais presque tout le monde danse et a l’air de bien
s’amuser, pendant qu’une fille très concentrée fait une reprise réussie
de <em>Smooth operator</em>. Il est tard et nous avons encore nos sacs à
préparer, les cartes à regarder, donc nous partons. Mais nous serions
bien restés un peu plus danser avec les Bishkekois !</p>
</div>
<div class="section" id="samedi-24-mai-bishkek-karakol">
<h2><a class="toc-backref" href="#toc-entry-2">Samedi 24 mai : Bishkek -> Karakol</a></h2>
<p>Aujourd’hui nous partons à Karakol, une ville dans le sud est du pays, au
pied des montagnes du Tien Shan. Pour les touristes, Karakol est un camp
de base pour de nombreuses randonnées.</p>
<p>Après un somptueux petit déjeuner à la guesthouse, nous commençons par
aller retirer de l’argent à l’un des rares distributeurs de billets de la
ville, situé dans une banque au croisement de l’avenue Manas et de la
perspective Chuy. Puis nous cherchons à nous rendre à la gare routière
pour trouver une marshrutka pour Karakol. Le tram ne vient pas, on est un
peu embêtés, du coup on finit par héler un taxi. En arrivant à la gare
routière on se rend compte qu’elle était en fait beaucoup plus près que
ce qu’on croyait, en qu’on aurait pu y aller en 20 minutes du marche. Au
moment où il s’arrête, le chauffeur du taxi ne peut s’empêcher de sourire
car il a repéré la nuée de chauffeurs de taxi et marshrutka qui va
s’abattre sur nous. En effet, ça ne manque pas : on nous propose en vrac
plein de destinations, Almaty, Osh et aussi Karakol, pour des prix plus
ou moins intéressants. Un gamin plus malin que les autres remporte le
marché avec un bon prix (250 soms par personne). Il se dépêche alors de
nous emmener au fond du parking où est garée la marshrutka en question,
c’est à peine s’il ne nous prend par la main pour nous y emmener !</p>
<p>Nous montons dans la marshrutka, déjà pleine aux 2/3 : il faut attendre
qu’elle soit bien remplie pour partir. Encore une fois, nous serons les
seuls touristes. Nous partons assez vite. Notre chauffeur est un vrai
champion et semble avoir décidé de battre le record de vitesse sur le
trajet Bishkek-Karakol. Il dépasse tout le temps, souvent de manière
assez agressive ! Sur la grand route il y a une voie de chaque côté, plus
une voie centrale pour doubler où les voitures évoluent dans un
chassé-croisé assez impressionnant. Il y a beaucoup de policiers au bord
de la route avec un radar, et notre chauffeur se fait arrêter trois fois.
A chaque fois il sort discuter avec les flics, on suppose qu’il s’en
sort bien car il n’a pas l’air trop préoccupé. Nous quittons la région de
Bishkek pour entrer dans la vallée de la Karaka, une puissante rivière
qui a créé des canyons encaissés de pierre rouge sur ses rives. Le
chauffeur est toujours aussi pressé et double beaucoup, parfois sans
visibilité. Une fois il évite de justesse un camion qui le klaxonne
furieusement. On arrive au lac Issy Kul et les gens commencent à
descendre tandis que d’autres montent. Des familles se disent au revoir
sur le bord de la route.</p>
<p>Le lac Issy Kul est tout à fait charmant, bordé de montagnes enneigées
qui descendent jusqu’à ses rives. Sur la côte nord que nous longeons,
plusieurs stations balnéaires ont connu leur heure de gloire pendant la
période soviétique, et d’énormes paquebots de bétons rappellent ça et là
leur splendeur passée : les touristes préfèrent maintenant aller dans les
montagnes au sud du lac.</p>
<p>La marshrutka nous dépose dans le bazaar de Karakol. La ville nous semble
d’emblée chaude, sale et poussiéreuse. Pas une seule rue goudronnée, des
nids de poule qui labourent toutes les artères et ralentissent
considérablement le trafic… Ca nous fait un peu bizarre. Il règne à
Karakol une impression de bout de monde, de <em>frontier town</em>, comme si les
gens qui s’y trouvent s’y étaient arrêtés car ils ne pouvaient pas aller
plus loin… Nous nous rendons à un hôtel pour backpackers, le Yak Tur.
Nous toquons à un grand portail, pas de réponse, puis nous voyons
apparaître le museau d’un chien sous le portail : il doit donc y avoir
du monde ! En effet, on nous ouvre pendant qu’un joli chien se glisse
sous la porte. Les prix sont très honnêtes (300 soms ou 450 avec le petit
déj) mais c’est bien plus cher que ce que dit le Lonely Planet. L’hôtel
est un ancien hôtel particulier à la décoration abracadabrante : notre
chambre fait bien 30 m^2, et contient entre autres un piano, une table
d’ébène, deux immenses miroirs, deux lits divans d’une autre époque…
les murs et le sol sont recouverts de tapis et de tentures rouge bordeau.
C’est amusant. Par contre, la propreté de la salle de bain commune laisse
un peu à désirer. A un moment, la lumière s’éteint, il n’y a plus
d’électricité. Nous nous disons que nos hôtes bricolent, et nous sortons
faire un tour.</p>
<p>En nous baladant dans la ville nous nous rendons compte qu’il s’agit
d’une coupure d’électricité générale. On nous explique que le
gouvernement a vendu trop d’électricité en hiver (aux Kazakhs notamment,
d’où une certaine amertume envers ce peuple plus riche) et que pour
faire des économies, l’électricité est coupée chaque jour entre 16h et
20h. Ceci renforce évidemment l’impression de ville fantôme que nous
avons eue en arrivant : les gens traînent mélancoliquement devant leurs
magasins en attendant le retour de la lumière à l’intérieur… Nous
arrivons trop tard à l’office du tourisme où nous aurions aimé obtenir
quelques renseignements, mais sur la place du bazaar, nous nous
arrangeons avec un taxi 4x4 pour qu’il nous emmène le lendemain dans les
montagnes de Jeti Oguz. Nous avons du mal à trouver un endroit pour
manger : tout a l’air fermé. On finit par trouver un café où nous
mangeons des salades et des manti dans le noir. La serveuse est aussi
jolie que désagréable. Les cheveux courts, mince, impeccable dans son
ensemble veste pantalon à l’occidentale, elle arpente nerveusement le
café, claquant ses talons sur le sol. La coupure d’électricité a l’air de
lui porter sur les nerfs… L’électricité revient à 20h pile, et le café
semble tout de suite plus gai. La télévision est tout de suite allumée
sur une chaîne de musique. En sortant du café, un jeune mendiant – il
n’a pas dix ans – nous tourne autour en nous demandant de l’argent. Nous
ne savons que faire, car tous les guides recommandent de ne pas donner
d’argent aux enfants seuls, exploités par leurs parents. Et c’est vrai
que si les gens sont généreux avec les vieux ou les femmes qui demandent
l’aumône, les gens dans la rue crient au gamin de partir. Mais le pauvre
gosse a l’air assez mal en point, sale et très maigre… J’ai encore mon
portefeuille à la main après avoir réglé l’addition, et il me tourne
autour en ne le quittant pas des yeux, j’ai peur qu’il me l’arrache…
Il finit par se décourager et partir, mais on se sent vraiment bêtes et
impuissants…</p>
</div>
<div class="section" id="dimanche-25-mai-vallee-de-la-karaka">
<h2><a class="toc-backref" href="#toc-entry-3">Dimanche 25 mai : vallée de la Karaka</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435634555/in/set-72157648345589451"><img alt="La vallée" class="align-center" src="https://farm3.staticflickr.com/2950/15435634555_66f8c1f89a_b_d.jpg" style="width: 100%;" /></a>
<p>Notre premier jour de rando ! Nous commençons par prendre des forces au
petit déjeuner, puisque nous avons choisi l’option chambre+petit déjeuner
à la suite des recommandations insistantes de notre hôte, très fier de
sa cuisine. On nous sert en effet – entre autres – un plat roboratif à
base d’oeufs, herbes et poissons, très bon et original. Nous discutons
ensuite avec Valentin, un vieux russe malin qui organise des balades en
montagne pour les touristes de l’hôtel. Il nous dit qu’il y a trop neige
sur la rando que nous avions programmée, et nous conseille de remonter
plutôt la vallée de la Karaka, puis d’aller vers les sources chaudes
d’Alty Arachan (où il a des yourtes pour les touristes). Nous décidons
donc d’aller dans la vallée de la Karaka et d’aviser là-bas en fonction
de l’état de la neige.</p>
<p>A la station de taxis nous proposons donc à notre chauffeur de changer de
destination, mais il estime que sa Lada Niva 4x4 n’est pas nécessaire
pour ce coin et nous confie plutôt à un de ses collègues, un taxi
“classique”, pas 4x4. Comme d’habitude, un attroupement de taxis se
penche sur notre carte (les gens connaissent bien le coin, mais ne savent
souvent pas bien lire une carte, et ça prend un certain temps), puis nous
faisons affaire. Nous sortons lentement de Karakol, chaque rue devant
être négociée doucement pour ne pas abîmer la voiture dans les
innombrables nids de poule, puis nous suivons une petite route qui
remonte la vallée de la Karaka jusqu’au parc national. Après avoir payé
l’entrée du parc, la voiture peut même nous déposer encore un peu plus
loin, près d’un groupe de maisons. C’est la rando qui commence…</p>
<p>Pendant que nous rangeons un peu nos sacs, des hommes sortent deux petits
veaux du coffre d’une voiture. Ils les ont probablement achetés au bazaar
aux bestiaux de Karakol. Les deux bestioles ont eu beaucoup d’émotions
et sont complètement tétanisées ! Pour remonter le vallée nous devons
choisir quel côté de la rivière emprunter : un pont permet de choisir un
côté ou l’autre pour partir. La carte n’est pas très claire non plus à
cet endroit (nous utilisons une carte 100 000è pas très précise). Nous
partons à droite (rive ouest), ce que semble indiquer la carte, mais le
chemin devient assez vite impraticable, avec notamment des rivières à
traverser à gué ou des hautes herbes boueuses à traverser : pour un parc
national, on s’attend à un chemin meilleur. On rebrousse donc chemin pour
passer sur la rive est, où la route est en effet plus large et
meilleure. On croise de nombreuses familles venues pique-niquer, qui nous
hèlent joyeusement. De manière générale, les Kirghizes adorent leurs
montagnes et aiment aussi beaucoup faire la fête : ils profitent de
chaque occasion pour aller y pique-niquer ! Ils étalent plein de tapis
colorés sur l’herbe, sortent les samovars, les bières… et passent des
après-midi à manger et discuter pendant que des ribambelles d’enfants
courent autour ! Nous marchons dans une belle vallée boisée aux pentes
escarpées, le temps est changeant mais plutôt beau. Le torrent que nous
remontons est large et impétueux, ils est impensable de le traverser sans
pont à cet endroit. Un large pont nous permet plus tard de passer sur la
rive ouest pour suivre le chemin.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15412584426/in/set-72157648345589451"><img alt="La vallée" class="align-right" src="https://farm3.staticflickr.com/2944/15412584426_b2c1968334_z_d.jpg" style="width: 40%;" /></a>
<p>Nous marchons 5h avant d’arriver à un grand plateau où la vallée
s’élargit et la rivière sinue joliment. Nous avançons un peu jusqu’à
arriver à un pont qui permet de partir vers le lac Ala Kul, vers l’est.
L’endroit est idéalement adapté pour poser le camp, et nous y laissons
les sacs pour poursuivre un peu l’exploration du plateau en suivant le
chemin principal. Arrivés à un bras de la rivière, nous croisons trois
cavaliers nomades : leurs chevaux traversent sans problèmes le gué, ils
ont fière allure ! Nous mettons nos guêtres pour traverser la rivière en
courant, ce qui permet d’éviter – presque – de se mouiller les
chaussures. Plus loin, il y a les traces d’un campement très organisé,
avec foyer, toilettes, pierres pour marquer le seuil des tentes, …,
sûrement mis en place pour les groupes en trekking organisé. Un cheval
blessé à sa patte arrière gauche semble attendre mélancoliquement qu’on
le soigne, c’est un spectacle assez navrant. Nous pensons qu’il a dû
venir ici car il y a souvent des hommes, malheureusement nous serions
bien incapables de soigner un cheval…</p>
<p>Nous retournons au camp où nous montons la tente, que nous n’avions pas
montée depuis tout juste trois ans ! (la thèse…). Comme nous avons
beaucoup de nourriture nous faisons un énorme repas de pâtes. Notre camp
est idéalement situé, tout près de la rivière et bien à l’abri sous de
grands pins. Nous assistons un peu surpris au passage d’une grosse moto
tout terrain qui est venue jusqu’ici… Une petite gerboise nous regarde
manger ; elle amuse beaucoup Gaël en déchiquetant méthodiquement les
feuilles qu’elle mange.</p>
<p>C’était une très belle journée de rando, avec des conditions presque
idéales : temps frais, terrain facile… Mais c’est bien pour commencer,
car les sacs sont lourds (entre 18 et 22 kg je dirais).</p>
<a class="reference external image-reference" href="cavaliers.jpg"><img alt="Des cavaliers" class="align-center" src="https://farm3.staticflickr.com/2941/15248916470_f5d3268515_b_d.jpg" style="width: 90%;" /></a>
<div class="line-block">
<div class="line"><br /></div>
</div>
</div>
<div class="section" id="lundi-26-mai-aller-retour-au-lac-ala-kul">
<h2><a class="toc-backref" href="#toc-entry-4">Lundi 26 mai : aller-retour au lac Ala-Kul</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15249079507/in/set-72157648345589451"><img alt="Lac Ala-Kul" class="align-center" src="https://farm4.staticflickr.com/3930/15249079507_7ee6522e64_b_d.jpg" style="width: 100%;" /></a>
<p>Aujourd’hui on laisse la tente au camp pour faire une balade à la
journée, ça fera moins à porter. Avant de partir nous montons le sac de
Gaël en hauteur sous la branche d’un arbre, comme nous l’avions fait pour
la nuit, pour éviter que les ours (ou les rongeurs) ne s’y attaquent.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248878699/in/set-72157648345589451"><img alt="" class="align-right" src="https://farm3.staticflickr.com/2941/15248878699_d33bd27300_z_d.jpg" style="width: 40%;" /></a>
<p>Nous partons au lac Ala Kul, un lac glaciaire à 3500 m d’altitude (ça nous fera
donc 1000m de dénivelé). Le chemin traverse la rivière, puis monte sec dans une
petite forêt de sapins et de bouleaux. Il est très bien tracé, et poursuit dans
une succession de montées raides et de plateaux. Des marmottes ou des lièvres
pas très farouches gambadent près de nous. Il fait beau, avec un léger voile
nuageux. Nous arrivons au lieu-dit de Sirota, un camp de base pour continuer
ensuite sur Ala Kul. Il y a de nombreuses traces de camp, même une petite
baraque. Nous atteignons la limite de la végétation peu après Sirota, et nous
continuons à monter dans des pierriers. Gaël porte gentiment le sac pour nous
deux. On devine le lac derrière un piton rocheux : encore un effort ! Le col
(au sens mathématique) passerait par une cascade de glace, donc le chemin passe
plutôt par un long pierrier qui vient rejoindre un épaulement au-dessus du lac.
Nous avons une surprise à la vue du lac : il est gelé, et recouvert par
endroits d’une épaisse couche de neige ! Ca contraste pas mal avec les photos
de carte postale que nous avons vues du lac, dont les eaux sont habituellement
d’un beau bleu turquoise ! Par rapport à ces cartes postales, le paysage gagne
en majesté ce qu’il perd en charme : au creux d’un cirque de montagnes acérées,
le lac déroule sa longue silhouette sinueuse de glace. C’est un paysage
austère, de roche et de glace. Nous pique-niquons rapidement, sans nous
attarder car il fait un peu froid. Nous constatons que si les faces nord sont
bien enneigées, les faces sud sont complètement sèches jusqu’à 4000 m : c’est
bon à savoir pour la suite.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435648145/in/set-72157648345589451"><img alt="" class="align-right" src="https://farm6.staticflickr.com/5597/15435648145_5699db15f2_z_d.jpg" style="width: 30%;" /></a>
<p>La descente est à peine plus rapide car la progression est difficile dans
les pierriers. C’est d’ailleurs un peu fatigant pour les articulations !
Le temps se couvre de plus en plus. On reprend un peu de pique-nique à
Sirota, à couvert du vent sous les pins. Le fromage légèrement spongieux
acheté à Bishkek est absolument délicieux. J’avance lentement, assez
fatiguée. Quelques gouttes de pluies commencent à tomber.</p>
<p>Arrivés au camp, nous constatons que quelqu’un a monté sa tente à côté de
nous : c’est un jeune Anglais au look roots, fort mal équipé (jeans,
mauvaise tente…), qui mange un grand plat de nouilles allongé dans
l’herbe. Il est 16h30 (on a marché 8h !), je me lave rapidement avec
l’eau du torrent (bien froide, mais ça fait du bien !), avant de faire
une lessive avec Gaël. Un troupeau de chevaux descend la vallée et
traversent la rivière, pour rester tout près de notre camp (ils ne
peuvent de toute façon pas aller plus loin, car la vallée est fermée. Ces
vallées très étroites par endroits sont très pratiques pour les éleveurs
qui peuvent laisser leurs troupeaux pendant des semaines sans
surveillance, en mettant juste une barrière à un endroit). Il commence à
pleuvoir pas mal. Nous mangeons une grande gamelle de semoule bien
bourrative.</p>
</div>
<div class="section" id="mardi-27-mai-passage-du-col-tileti">
<h2><a class="toc-backref" href="#toc-entry-5">Mardi 27 mai : passage du col Tileti</a></h2>
<p>Il a plu une bonne partie de la nuit, mais nous avons une très bonne
tente et j’ai très bien dormi. Au réveil tout est trempé, y compris la
tente ou le sac de Gaël qui a passé la nuit sous une branche. De longues
langues de brume s’élèvent paresseusement dans la vallée : ce n’est pas
forcément idéal pour passer un col à 3800m comme nous l’envisageons… Du
coup on hésite beaucoup pour savoir quoi faire, et on est très lents. Les
chevaux ont dû passer une très mauvaise nuit sous la pluie et commencent
à s’étendre doucement au soleil pour faire la sieste.</p>
<p>Le temps de faire sécher la tente, il est 9h quand on part (on se
réveille à 6h en général). On continue dans la vallée, très plate et
large à cet endroit. Il fait un temps superbe et tout brille au soleil
après la pluie. Nous croisons quelques chevaux, et aussi des vaches.
Arrivés à l’endroit où la vallée se sépare en deux, nous devons faire un
choix. Soit nous partons à gauche (sud-est) vers une balade à la journée
en aller-retour vers un glacier, soit on part à droite pour essayer de
passer le col Tiletti, mais il est bien tard… Après un goûter censé
porter conseil, on décide de tenter le col parce que c’est plus rigolo
d’essayer de traverser, quitte à devoir camper un peu plus haut.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15249092007/in/set-72157648345589451"><img alt="Montée au col" class="align-right" src="https://farm4.staticflickr.com/3932/15249092007_ede98e18ee_z_d.jpg" style="width: 40%;" /></a>
<p>On a un peu de mal à trouver le chemin qui monte au col à partir de la
vallée. Il faut en fait traverser d’abord un ruisseau (peut-être sec en
plein été), puis deux rivières plus importantes, la dernière a un pont
composé de plusieurs morceaux de troncs. Il y a ensuite un chemin qui
part droit dans la pente sur la droite, c’est celui-là qu’il faut
prendre, et pas les bouts de chemin qu’on peut rencontrer plus tôt et qui
ne mènent nulle part. Le chemin est extrêmement raide (on fait 500m de
dénivelé en 30 min !) mais très bien tracé donc on avance bien, même si
c’est vraiment fatigant. Après cette montée abrupte, on arrive à un grand
plateau qui monte en pente douce dans la direction du col. Une petite
rivière coule doucement au milieu du plateau, la progression est
maintenant très agréable. C’est le moment de pique-niquer. Il y a
beaucoup de vent donc la limite des arbres est basse, mais il y a
beaucoup de buissons. Dans le fond on aperçoit les montagnes qu’il va
falloir franchir, avec de la neige. Nous avançons plusieurs heures dans
ce plateau ensoleillé, seul avec les marmottes. On finit par apercevoir
le col qui fait un S, dans le fond sur la gauche. La progression se fait
maintenant dans des pierriers et des langues de neige, c’est assez
pénible. A la suite d’une glissade j’ai couche d’un bon cm d’épaisseur de
boue neigeuse sur mon pantalon… La fin de la montée au col est plus
raide. Il y a beaucoup de neige au centre du couloir, Gaël nous fait
passer par des pierriers sur le côté. Ca roule un peu sous les pieds et
c’est raide, mais ça vaut vraiment mieux que d’enfoncer dans la neige
molle. Il y a quand même quelques langues de neige à traverser : Gaël
fait la trace. Il avance plus vite que moi et monte poser son sac au col,
puis redescend prendre mon sac pour la fin de la montée. C’est vraiment
gentil car du coup c’est beaucoup plus facile pour moi !</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15412571856/in/set-72157648345589451"><img alt="Montée au col" class="align-right" src="https://farm6.staticflickr.com/5599/15412571856_80124f1c5f_z_d.jpg" style="width: 50%;" /></a>
<p>Au col, il fait froid, mais le paysage est très beau. On a une vue
plongeante sur la vallée de l’autre côté, et la chaîne de montagnes au
fond, un peu perdue dans les nuages. Mais la descente est encore beaucoup
plus difficile que la montée en raison de la neige où on enfonce
beaucoup, souvent jusqu’à mi-cuisse. Quand on peut éviter la neige on
avance sur des pierriers d’ardoise très glissants, ou dans des éboulis de
gros rochers. C’est épuisant, et je n’en peux plus, avec la neige qui me
refroidit beaucoup : j’ai tout le temps l’onglée. Nous mettons presque
deux heures à traverser ce faux plat… Quelques cairns sur le côté nous
donnent espoir de se diriger vers le bon endroit. On arrive enfin sur un
terrain plus agréable, avec même un chemin tracé ! Il faut remonter un
peu sur le côté droit pour traverser une rivière, et passer une dernière
langue de neige au somment d’un promontoire rocheux. Sous nos pieds
s’ouvre le bout de la vallée, qu’il nous faut atteindre pour camper à une
altitude raisonnable (ce sera à 3000m tout de même !). Il reste 1h30 ou
2h de jour. On descend super vite, j’ai retrouvé beaucoup de forces sur
ce terrain facile. La lumière du soir donne de super couleurs bleutées à
la vallée. Les montagnes environnantes sont impressionnantes, il y a un
glacier suspendu presque vertical sur une paroi. Arrivés dans une zone
plus plate, il y a plein de petits ruisseaux à traverser. On pose le camp
dès qu’on trouve un terrain à peu près plat, même si le coin est peu
abrité. On bat notre record de temps de montage de tente ! Pendant qu’on
se prépare un énorme plat de nouilles instantanées, un groupe de chevaux
nous rejoignent silencieusement dans le crépuscule.</p>
<p>Quelle journée ! On se couche vite fait après avoir fini de manger.</p>
</div>
<div class="section" id="mercredi-28-mai-sortie-dans-la-vallee-de-djeti-oguz">
<h2><a class="toc-backref" href="#toc-entry-6">Mercredi 28 mai : sortie dans la vallée de Djeti-Oguz</a></h2>
<p>Après les émotions d’hier, on s’octroie une petite grasse mat’ et on ne
se lève qu’à 7h30. On se lève tout doucement, en profitant du beau temps,
des marmottes. L’eau de la rivière a un très bon goût ici.</p>
<p>On part sans se presser, il nous faut descendre toute la vallée en
longeant la rivière qui naît un peu plus haut. Sur la carte, le chemin
est indiqué de l’autre côté de la rivière que celui où nous sommes. On
passe pas mal de temps à essayer de chercher un gué au fur et à mesure
qu’on avance, mais la rivière est profonde et rapide et on n’ose pas
traverser. On continue en espérant trouver un pont plus loin. On
rencontre en effet un premier pont, constitué de 2 rondins étroits, à
l’air pas très solide. On préfère continuer sur la même rive, même si le
terrain devient un peu marécageux. On fait bien, car on arrive vite à un
deuxième pont, beaucoup plus solide avec 4 larges rondins. C’est bon, on
est passés du bon côté !</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248944750/in/set-72157648345589451"><img alt="Petit Kyrgyz cavalier" class="align-right" src="https://farm3.staticflickr.com/2946/15248944750_3e37645ea6_z_d.jpg" style="width: 30%;" /></a>
<p>Un peu plus tard on croise un campement de nomades, et un troupeau de
chevaux qu’on traverse en essayant de ne pas effrayer les poulains. Trois
gamins dévalent de la montagne en nous faisant de grands signes. On
répond à leurs salutations puis on continue, mais on est vite rejoints
par les trois petits cavaliers qui sont montés sur leurs chevaux. Ils ont
entre 9 et 11 ans et nous proposent de monter en croupe pour un bout de
chemin. On hésite un tout petit peu : je n’ai jamais fait de cheval !
Mais ce serait dommage de manquer cette expérience. Les chevaux sont
grands et j’ai un peu de mal à monter dessus : je monte d’abord sans mon
sac, puis Gaël me le passe. C’est plus dur pour lui de se débrouiller
avec son sac… Mon petit guide monte admirablement bien, et conduit
doucement son cheval qui a le pas très sûr sur ce chemin accidenté (on
traverse quelques gués, et ça descend assez raide). Du coup je n’ai pas
peur du tout, et j’apprécie beaucoup. On parle un peu, mais la
conversation est assez vite limitée par mon russe ! Enfin, heureusement
qu’il y a les signes… On fait un bon bout de chemin comme ça. Nous
sommes accompagnés par quelques poulains du troupeau, que le petit garçon
caresse de son pied nu quand ils passent à côté. L’un des poulains se
met à trotter, du coup notre cheval se met aussi à trotter : ça secoue !
On arrive à un endroit plus plat où la rivière que nous suivons rejoint
une plus grosse rivière. Il y a un large pont, et une route de 4x4
commence derrière. A l’arrivée au pont, Gaël et moi descendons de cheval
et faisons une distribution de chocolat. On demande à l’aîné comment on
peut les remercier. Avec un sourire malin, il nous demande si on a de
l’argent. On leur donne 50 soms (1 euro) chacun. On a calculé que c’était
sûrement un bon prix, mais c’était très chouette et ça nous a bien
avancé.</p>
<p>Nous continuons sur la route de 4x4, où nous rejoignons la “civilisation”
: à nouveau pas mal de familles en pique-nique. Les gens n’ont en général
pas de 4x4, mais des vieilles guimbardes qui ont vécu. Certaines
souffrent d’ailleurs : une vieille voiture cabossée est arrêtée au
milieu de la route pour laisser le radiateur refroidir. De jeunes hommes
s’affairent sur le moteur, pendant que leur patron est assis sur le siège
du passager. Il nous dit bonjour de manière très aimable, nous demande
comment on va, puis me demande de me pencher comme pour me dire un
secret… et me vole un court baiser. Tout le monde rigole, ça a l’air de
le rendre content… pourquoi pas ! La vallée devient de plus en plus
peuplée, avec plein de yourtes que les gens montent pour la saison
touristique qui commencera bientôt. On croise différents troupeaux sur la
route, que leurs bergers emmènent dans les montagnes pour l’été :
moutons, vaches, chevaux (par ordre croissant d’intelligence des
bestiaux).</p>
<p>Il se met tout d’un coup à bien pleuvoir. A un moment nous sommes
rattrapés par mon petit cavalier nomade, qui galope à toute vitesse sur
son cheval. Tous les gens qui le voient passer envient son allure à
cheval : c’est le roi de la vallée ! On le reverra repasser plus tard :
il est probablement allé faire quelques courses avec nos soms. Le soleil
revient un peu plus tard, on arrive au “sanatorium” de Jeti Oguz, une
station thermale célèbre pour ses formations rocheuses d’un rouge
profond. Les falaises rouges que nous apercevons ressemblent un peu à
certains paysages de l’Utah. Nous arrivons au village. Il y a quelques
maisons et un magasin où nous achetons une bière. Le bus scolaire qui
passe va jusqu’à Karakol et accepte de nous y emmener pour 350 soms (on
aurait sans doute pu mieux négocier…).</p>
<p>C’est la fin de cette belle rando. Arrivés au bazaar, on se rend à
nouveau au Yak Tour. Le beau chien est toujours frétillant à notre
arrivée, par contre l’hôte, Sergei, a l’air assez abattu. A la suite
d’une coupure, il n’y a plus d’eau courante dans le bloc depuis deux
jours. Ca veut dire qu’on ne peut entre autres pas prendre de douche.
C’est un coup dur pour lui. On décide d’aller ailleurs, car on a vraiment
besoin d’une douche !</p>
<p>L’hébergement pas cher est limité à Karakol. En plus du Yak Tour il y a
le Turkestan Yourt Camp un peu plus loin. On trouve l’endroit sans
problèmes, heureusement car il est tard. Le gardien nous ouvre et nous
mène à une yourte bien accueillante. Pour régler les détails du paiement,
il va chercher les responsables, une bande de trois russes complètement
bourrés car ils fêtent aujourd’hui l’ouverture de la saison (il n’est que
18h…). Nous avons de la chance car la veille, le camp aurait été fermé.
L’un des trois rigolos, un grand roux qui bégaye, se reprend un peu et
note nos noms et encaisse l’argent, mais un autre est beaucoup plus
lyrique et essaye de nous vanter ses talents de guide de haute montagne,
tout en s’excusant tous les trois mots de son mauvais anglais… ou
d’être aussi bourré ! Il transpire beaucoup, a l’oeil vitreux et le
sourire fixe : il est déjà dans un bel état ! Les gars sont sympas et
assez comiques, mais on décline leur offre de se joindre à leur fête,
pour éviter de devoir trop boire.</p>
<p>Nous allons dîner dans un café en plein air, où il n’y a pas
d’électricité car c’est l’heure de la coupure, mais un générateur fait
quand même tourner la sono. On se régale de shaslyks, manti et salade, et
on siffle une bière et 1L de jus d’orange. Puis nous prenons la douche
chaude qui nous faisais tant envie, et on fait une lessive. La nuit
tombe, et le gardien nous montre une petite chouette qui est tombée du
nid, sans se faire mal, mais qui n’arrive pas à y remonter. Gaël et lui
la mettent dans un pneu tout en haut d’un camion, en espérant que les
chats ne pourront pas l’atteindre. Je m’endors à moitié dans la yourte en
attendant Gaël quand quelque chose me saute sur les pieds. Je crois
d’abords que c’est Gaël qui me fait une blague avant de réaliser qu’il
s’agit d’un gros chat. Je lui fais faire un joli vol plané et il s’enfuit
sans demander son reste.</p>
</div>
<div class="section" id="jeudi-29-mai-karakol-bar-skoon">
<h2><a class="toc-backref" href="#toc-entry-7">Jeudi 29 mai : Karakol -> Bar-Skoon</a></h2>
<p>Aujourd’hui nous repartons de Karakol pour aller vers notre deuxième
rando. Nous voulons nous faire déposer dans les montagnes, sur la route
de Kumtor, une mine d’or franco-canadienne, et de là rejoindre la vallée
de la Juku qui redescend jusqu’aux rives d’Issy-Kul. Mais pour ça, il
faut arriver à la route de Kumtor…</p>
<p>On se réveille à 6h et on regarde un peu les cartes avant d’aller au
bazaar. On y arrive vers 7h30, c’est clairement trop tôt, les marchands
commencent tout juste à s’installer. Quelques stands servent le petit
déjeuner. On essaye de repérer les endroits où on peut acheter les choses
utiles pour la rando : pâtes instantanées, pain, fromage, chocolat,
fruits
secs. On achète aussi 1L de jus de cerise qu’on boit goulûment dans les
allées du bazaar. On prend aussi un bol de nouilles piquantes comme petit
déjeuner. On trouve aussi de la crème solaire de fort indice à un prix
dérisoire. Je suis super contente, car le tube qu’on avait était un peu
sous-dimensionné, et j’ai quelques coups de soleil. On fait l’ensemble de
nos courses alimentaires pour un tout petit prix, c’est vraiment bon
marché de se fournir dans les bazaars !</p>
<p>On passe reprendre nos sacs au Yak Tour puis on part prendre une
marshrutka pour Bar Skoon. La station “longue distance” n’est pas au
centre, et les taxis nous disent à tort qu’il n’y a pas de marshrutka
pour Bar Skoon. Evidemment, on sait qu’il y en a. L’un entre eux nous
indique la direction de la gare routière. C’est reparti pour un nouveau
voyage en marshrutka ! La route au sud du lac Issy Kul est assez mauvaise
par rapport à celle du nord, ça secoue pas mal ! Nous sommes déposés à un
arrêt à un bon kilomètre du bled de Barskoon, qui n’est pas directement
sur la route. Alors que nous marchons vers Barskoon, une voiture
s’arrête et nous propose de nous emmener jusqu’au bled. Dans ce break
rouge qui a vécu, il y a le conducteur, sa femme, son frère et son gamin,
et un certain nombre de courses, mais visiblement encore plein de place
pour nos deux sacs et nous. Nous leur demandons comment se rendre vers
Kumtor, et, très gentils et serviables, ils nous proposent de nous
déposer un peu en dehors du bled en nous disant que des taxis passent par
là. On se fait donc déposer au croisement de deux routes, au milieu de
collines désertiques qui surplombent Bar Skoon. On fait des photos de
groupe avec les téléphones portables des deux frères, nous leur donnons
un petit peu d’argent pour l’essence, on échange plein de grands
sourires, et ils repartent vers Barskoon.</p>
<p>On se met donc sur le bord de la route et on commence à faire du stop.
C’est le début d’une longue attente… Il fait très chaud et le soleil
tape dur. Les collines autour de nous sont sèches et pelées, sauf dans
le creux de la vallée où une petite oasis se forme autour d’une rivière
rapide. Mais il n’y a pas beaucoup de circulation : quelques voitures
fatiguées qui s’arrêtent, mais ne vont pas très loin, dans les fermes des
environs, et les véhicules rutilants de la mine qui roulent à tombeau
ouvert et semblent avoir reçu des consignes de ne pas s’arrêter. Ca nous
décourage un peu… on se dit que nos chances d’être pris en stop sont
assez minces.</p>
<p>Puis arrive un groupe de trois cavaliers, accompagnés d’un jeune homme
qui pousse devant lui un poulain, une vache et son veau. Ils vont
presque aussi loin que nous, à trente kilomètres, et tentent de nous
convaincre de faire le voyage avec eux, à pied et à cheval. On refuse
poliment : nous ne sommes pas bons cavaliers et ça nous fatiguerait trop,
et quitte à marcher, on préfère le faire dans les montagnes. Mais ils
insistent et nous disent qu’il y a une voiture à la prochaine ferme, qui
pourrait nous emmener (on est bien sûr prêts à payer). Du coup on se
retrouve à les accompagner un peu sur la route, mais ils sont vraiment
bizarres, voire un peu louches. Ils ont un comportement lunatique, voire
violent, maltraitent leurs animaux un instant, et l’instant d’après
éclatent de rire et prennent Gaël par le bras en l’appelant “brat”
(frère). Ils nous posent plusieurs fois de suite les mêmes questions
sans avoir l’air de s’en rendre compte, font tomber le contenu de leurs
sacs par terre.. bref ils ont l’air franchement débiles. Ils proposent
plusieurs fois de prendre nos sacs sur les chevaux pour nous décharger,
mais en fait on ne leur fait vraiment pas confiance… donc on refuse
catégoriquement. Un premier incident : la vache s’enroule les pieds dans
la corde qui relie ses pattes arrières au cou de veau et se met au
courir, étranglant à moitié le petit veau. Le plus jeune rattrape la
vache et entreprend de désemmêler les pattes de la vache, tout ça en
continuant à courir à côté d’elle et sans sembler se soucier de ses
violentes ruades… Ca nous semble super dangereux. Puis le summum :
alors qu’on traverse un pont au-dessus de la rivière, nos Don Quichotte
descendent vers la rivière pour faire boire leur bêtes. Mais alors qu’un
des chevaux se penche pour boire, son cavalier laisse échapper le
contenu du sac plastique qu’il tient à la main… contenu qui file bien
entendu avec le courant. Panique générale : ils se mettent tous à courir,
on nous donne rapidement les rênes de deux des chevaux… évidemment ils
ne retrouvent rien, et sont très embêtés car il y avait certains de leurs
papiers dans ce petit sac en plastique. Nous sommes à la fois atterrés et
énervés devant tant de négligence, et très désolés pour eux. On décide
une bonne fois pour toutes qu’ils sont trop pas doués, et de revenir en
arrière. Pendant qu’on tente de leur remettre les rênes dans les mains,
ils nous montrent des plaies à vif sur le dos d’une jument et expliquent
que ce sont des loups qui l’ont mordue cet hiver. Notre premier réflexe
serait d’attribuer ces plaies à leurs mauvais traitements – ils montent
les chevaux à cru pour commencer, et les font avancer à la baguette –,
mais ça assombrit encore le tableau : qu’est-ce qu’on fait dans cet
endroit pourri ? Quand ils ont enfin compris qu’on ne les suivait pas,
ils se mettent à nous demander de la vodka, ou de l’argent,
mi-suppliants, mi-menaçants. Bref, c’est avec une impression bien pénible
qu’on les quitte d’un coup, un peu déboussolés, presque en courant, nous
retournant de temps en temps pour être sûr qu’ils ne retournent pas vers
nous avec des intentions plus ou moins bonnes…</p>
<p>Bon, on en a assez pour aujourd’hui : on décide de retourner vers Bar
Skoon pour y passer la nuit. Il est seize heures. Il nous faut marcher
une bonne heure pour revenir au village. Comme on est un peu bêtes, on ne
sort le Lonely Planet qu’à l’entrée du village pour s’apercevoir qu’il
n’y a pas d’hôtel à Barskoon, alors que le village de Tamga, quelques
kilomètres plus loin, est plus touristique. Et nous aurions pu aller à
Tamga à partir de l’endroit où nous faisions du stop (enfin, ça aurait
été plus long que d’aller à Barskoon)… Donc c’est la looze. On commence
à se demander où on dormira cette nuit. De vive vue, le village de
Barskoon n’est pas plus pittoresque ou touristique que ce qu’annonce le
guide : un bout de route goudronnée fait un grand S… et c’est la fin du
village, il n’y a que de toutes petite rue qui partent de la route vers
les champs. Un espoir tout de même : le guide annonce qu’il y a dans le
village un Sheperd’s way trekking, c’est-à-dire un camp de base pour des
randonnées à cheval. Pour trouver l’adresse on se renseigne dans une
épicerie, où un jeune homme nous donne des indications très précises (et
nous n’aurions jamais trouvé sans indications !). En sortant on se fait
accoster par deux gars un peu bizarres qui bégayent en riant de façon
mécanique, et nous demandent de l’argent pour boire. On ne comprend pas :
est-ce l’alcool qui fait des ravages dans le coin ? Ca fait un peu trop
de gens pas très nets rencontrés en quelques heures…</p>
<p>Apercevoir le portail pimpant du Sheperd’s way trekking est un énorme
soulagement, surtout quand un chien se précipite vers nous en remuant la
queue : nous n’étions pas sûrs de trouver du monde à cet saison. Une
jeune femme blonde, Vicky, vient à notre rencontre. Elle est anglaise,
pas besoin de parler russe ! Avec son compagnon Rash, et un ami russe,
ils font des travaux dans leur propriété pendant la saison creuse. Ils
ont toute la place pour nous loger : nous pouvons choisir une chambre
avec balcon et vue sur les montagnes, et des tapis sur les mur, dans le
joli chalet où ils accueillent les touristes. Ouf, ça fait du bien après
nos petites mésaventures de l’après-midi ! En plus Vicky nous dit assez
vite que Rash peut nous conduire à l’endroit où on veut le lendemain,
donc on est complètement soulagés, et on peut pleinement profiter de
l’instant. Surtout quand elle sort sa confiture de cassis et son pain
maisons, avec un peu de thé vert, la vie est vraiment belle ! On discute
un peu autour du thé : c’est une passionnée de chevaux, une grande fille
blonde très sympathique qui aime les “horses with personality”, ceux qui
n’ont pas forcément un caractère facile. Elle avait commencé une
formation de vétérinaire en Angleterre, mais est partie au Kirghistan
s’occuper de chevaux et n’est jamais rentrée… Du coup elle monte un
projet pour faire venir des vétos britanniques pour former les
vétérinaires kirghizes.</p>
<p>On la laisse un peu tranquille pendant qu’elle prépare le repas. Les
toilettes sont au fond du jardin, et pour y arriver je dois négocier avec
le vieux chien de la maison qui est très câlin : il se met pile devant
moi et gémit lamentablement en se frottant contre moi, jusqu’à ce que je
le caresse. Au bout d’une minute de caresses je décide qu’il a eu sa
dose, et il me laisse avancer d’un mètre… avant de se remettre devant
moi et de gémir de plus belle ! On recommence ainsi, mètre par mètre…</p>
<p>Nous mangeons avec nos hôtes, un succulent plat de pommes de terre au
chou rouge accompagné d’une salade de concombres. Et on boit du thé, bien
sûr (personne ne boit d’eau à table, la boisson la plus courante est de
loin le thé). Après le repas nous restons encore discuter un peu avec
Vicky : ses deux compagnons sont plutôt taciturnes, et ça doit la
changer de parler avec nous ! Elle nous raconte les différentes
randonnées qu’ils font avec Sheperd’s way. Les traversées d’un mois
jusqu’à la frontière chinoise nous font rêver… Puis on va se coucher :
Gaël est un peu malade et fiévreux, il est temps de se reposer.</p>
</div>
<div class="section" id="vendredi-30-mai-vers-la-jokuu">
<h2><a class="toc-backref" href="#toc-entry-8">Vendredi 30 mai : vers la Jokuu</a></h2>
<p>Rash a donc accepté de nous conduire au pied de notre randonnée sur la
route de Kumtor, au prix standard de 15 soms le kilomètre. Il a une
lada niva, ce qui est un peu la Rolls Royce de la région (malgré toutes
les blagues qu’on peut faire sur les Ladas en France, ici c’est vraiment
la voiture idéale : c’est le véhicule tout terrain qu’on peut faire
réparer partout).</p>
<p>Avant de partir, Vicky essaye de réparer une couture du sac de Gaël qui
est en train de se défaire, ce qui l’inquiète beaucoup. Bon, on va voir
comment ça tient…</p>
<p>On commence par aller chercher de l’essence, dans une arrière-cour à
laquelle on accède par une allée poussiéreuse jonchée d’ornières. Le
“camion scolaire” est justement en train de prendre de l’essence : les
gamins sont entassés dans sa benne et gesticulent dans un beau tapage.</p>
<p>La route de la mine est extrêmement pittoresque, une tranchée entre des
montagnes grandioses. Rash n’est pas très bavard mais ce n’est pas grave,
on admire le paysage. Avec ses cheveux noirs longs dans le cou, ses
traits puissants et sa bouche fine où flotte toujours un léger sourire,
il
ressemble à un indien Navajo. Il connaît très bien la rando que nous
voulons faire, et nous donne quelques conseils pour trouver le chemin en
nous déposant.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248885509/in/set-72157648345589451"><img alt="Djuku pass" class="align-right" src="https://farm6.staticflickr.com/5597/15248885509_560e257f77_z_d.jpg" style="width: 40%;" /></a>
<p>Pour arriver à la vallée de la Juku, il faut d’abord passer un col à 3800
m : nous ne savons pas encore si nous y arriverons aujourd’hui ou demain.
Le début monte sec, on remonte une vallée aride et caillouteuse. Les pans
de montagne qui tombent à pic ont des couleurs magnifiques : roches
rouges, pierriers presque violets, vert des herbes rases, le tout sous un
ciel d’un bleu implacable. Il y a beaucoup de chevaux et de vaches. Nous
pique-niquons en regardant le manège de deux chevaux rivaux : le plus
jeune semble avoir irrité le plus vieux qui le poursuit pour se battre,
mais il est maintenu à distance par les ruades du jeune. Ils virevoltent
dans la montagne jusqu’à être lassés de ce curieux ballet.</p>
<p>Le chemin est très bien tracé et il n’y a pas de neige de notre côté : on
décide de passer le col aujourd’hui. Plusieurs cols ont l’air possibles
pour traverser la barrière de montagnes, mais avec la carte on repère
assez vite le bon. La fin monte en pente douce : c’est un vrai plaisir de
voir le paysage apparaître lentement de l’autre côté. Au col le sol est
un peu spongieux, sillonné d’une multitude de petits ruisseaux. Quelques
plaques de neige, mais rien de bien méchant. Curieusement, la rivière qui
descend du col est trouble, on prend donc de l’eau un peu plus loin à un
ruisseau venant d’un glacier. Comme il se fait tard, on pose le camp près
d’une énorme pierre qui nous abritera du vent. Le ruisseau voisin nous
sert de frigo pour faire prendre l’instant pudding au chocolat qui
améliore notre ordinaire ! Ce soir nous dormons à 3400m d’altitude, mais
la nuit n’est pas trop fraîche (avec une bonne tente et des vêtements
chauds…).</p>
</div>
<div class="section" id="samedi-31-mai-vallee-de-la-joku">
<h2><a class="toc-backref" href="#toc-entry-9">Samedi 31 mai : vallée de la Joku</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435331302/in/set-72157648345589451"><img alt="Djuku" class="align-right" src="https://farm6.staticflickr.com/5599/15435331302_5577d5d165_z_d.jpg" style="width: 40%;" /></a>
<p>Je me réveille avec de la fièvre, mal à la gorge et aux articulations,
comme Gaël il y a deux jours. Bon, on va y aller doucement. Pour
rejoindre la longue vallée de la Joku, on commence par perdre pas mal
d’altitude pour arriver à un Y où la rivière naît : c’est le début de la
vallée. Il y a un nombre incroyable de marmottes. Seuls leurs sifflements
rompent parfois le grand calme de la vallée. Quelques chevaux broutent
paisiblement dans un pli de la vallée. Le temps est magnifique, c’est une
superbe vallée sauvage qui s’offre à nous.</p>
<p>On fait beaucoup de pauses pour que je ne me fatigue pas trop. Je fais
même une courte sieste à un moment ! On décide de faire une journée pas
trop longue. De toute façon, il se met à pleuvoir brusquement, ce qui
calme nos ardeurs… La montagne devant nous prend des allures
fantômatiques derrière le rideau de pluie qui ne révèle que ses pics les
plus acérés. Des bancs de brume commencent à lécher les sommets, puis
descendent vite. On cherche un abri dans un bouquet d’arbres près de la
rivière, et on trouve un endroit presque idéal, dans une petite élévation
au-dessus de deux ruisseaux, au coeur des arbres pour nous protéger du
vent vigoureux. Je me couche pour récupérer un peu, Gaël part se balader.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435686105/in/set-72157648345589451"><img alt="yourte" class="align-left" src="https://farm4.staticflickr.com/3930/15435686105_1f1a1e185d_z_d.jpg" style="width: 40%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15412619256/in/set-72157648345589451"><img alt="yourte" class="align-right" src="https://farm4.staticflickr.com/3932/15412619256_d6f127ecff_z_d.jpg" style="width: 40%;" /></a>
<div style="clear: both;"></div><p>Je me réveille quand il rentre un peu plus tard : il a rencontré une
famille de nomades qui ont leur yourte juste à côté, et nous invitent à
leur rendre visite. Le père passe très droit sur son grand cheval blanc,
avec ses deux jeunes enfants en croupe, très mignons. Nous les rejoignons
à la yourte. Assis sur un tronc, le père fume tranquillement une pipe
près de l’enclos des vaches, un court moment de détente avant d’aller
rassembler les vaches pour la traite. Sa petite fille nous regarde avec
des grands yeux timides, sans vraiment oser s’approcher. Son frère aîné
n’a pas l’air impressionné par contre ! Le père parle très bien russe, et
nous explique comment il s’occupe de deux petits agneaux noirs qui
viennent de naître. Les deux sont attachés près de nous : si l’un deux se
blottit tranquillement au chaud dans un tas de tissus bien chauds – les
oreilles dépassent à peine ! –, l’autre tire sur sa corde et bêle
furieusement, cabré autant qu’il peut sur ses toutes petites pattes.
L’homme les libère, ils bondissent vers le troupeau dans un concert de
bêlements pour aller têter longuement leur mère. Pendant qu’il part
rassembler ses vaches, il nous invite à aller prendre le thé dans la
yourte. Son épouse, une jolie jeune femme au visage très doux, nous offre
du yaourt délicieux, et un excellent pain nan que nous trempons dans une
sorte de crème fraîche, entre la crème et le beurre : tout cela est
simple mais succulent. Ca a l’air d’être leur repas habituel. Nous avons
apporté des raisins secs et une tablette de chocolat que les gamins
dévorent, mais ce n’est pas grand chose. Notre hôtesse sourit timidement,
nous discutons un peu. Nous finissons par comprendre que nous l’empêchons
d’aller traire les vaches, mais qu’elle n’ose rien dire par politesse.
Nous partons donc en la remerciant. Il se met à pleuvoir violemment peu
après : comme nous avons déjà pas mal mangé dans la yourte, Gaël cuisine
juste un peu de semoule (pendant que je suis à l’abri dans la tente !),
que nous mangeons dans la tente.</p>
</div>
<div class="section" id="dimanche-1er-juin-sortie-de-la-vallee-de-la-joku">
<h2><a class="toc-backref" href="#toc-entry-10">Dimanche 1er juin : sortie de la vallée de la Joku</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248924029/in/set-72157648345589451"><img alt="Une famille Kyrgyz" class="align-left" src="https://farm4.staticflickr.com/3931/15248924029_7bbc09d6b8_z_d.jpg" style="width: 40%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15432489241/in/set-72157648345589451"><img alt="Un bébé Kyrgyz" class="align-right" src="https://farm3.staticflickr.com/2945/15432489241_7ede645f38_z_d.jpg" style="width: 40%;" /></a>
<div style="clear: both;"></div><p>Ce matin ma fièvre est passée et je vais beaucoup mieux. Malgré la pluie
de la nuit, il fait maintenant très beau. Nous décidons de faire une
rando sur la journée pour aller voir des lacs en altitude, pendant que
la tente séchera au camp. On s’avance donc un peu dans la vallée, puis on
commence à monter dans une vallée perpendiculaire, mais on se rend compte
au bout d’un certain temps qu’on s’est trompés de position sur la carte,
et qu’on s’est avancés pour rien : c’est en fait directement près du camp
qu’il aurait fallu monter. Un peu dépités, on décide de rentrer au camp
et de repartir avec la tente. Sur le chemin du retour, on rencontre deux
hommes qui travaillent dans la forêt : ils refont un pont sur la rivière.
Ils nous invitent à venir prendre le thé dans leur yourte voisine à
l’occasion de leur casse-croûte de midi. Ce campement est plus riche que
celui de nos hôtes de la veille : ces nomades ont à la fois une grande
tente, et une très grande yourte bien meublée, avec de beaux tissus à
l’intérieur. En plus des deux hommes, nous voyons deux femmes et quatre
enfants, dont un bébé qui dort sur une pile de petits tapis. Pour le
déjeuner, il y a un pain nan énorme, de la confiture de
cassis délicieuse, de la crème, et du tchai bien chaud qui est un délice
pour ma gorge un peu enflammée. La chaleur légèrement humide qui règne
dans la yourte est bien agréable ! Nous mangeons aussi un plat non
identifié à la consistance légèrement grumeleuse, inhabituel mais bon.
Nous discutons un peu avec nos hôtes : ce sont de riches éleveurs qui
possèdent encore une autre yourte, et une maison en ville. C’est un grand
moment quand Gaël essaye d’expliquer en russe en quoi consiste mon
travail et ce que peut bien faire un ingénieur de recherche…</p>
<p>Nous repartons au camp pour démonter la tente, et nous repartons dans la
vallée. Le terrain est facile, on avance bien. Mais le ciel se couvre
beaucoup. On croise un petit père sur son cheval, emmitouflé dans un
grand duffle coat. Il nous demande de la vodka d’un air rigolard. Mais
nous n’en avons pas ! Il se met tout à coup à pleuvoir des trombes : on
prend rapidement congé et on marche vite ! Heureusement, ça ne dure pas
trop longtemps. Gaël a un peu mal au dos, donc on fait de fréquentes
pauses pique-nique dans l’après-midi. La vallée s’ouvre de plus en plus,
nous rejoignons des zones plus habitées par les nomades. Il commence à
être temps de songer à camper. Près d’une yourte, deux gros chiens se
mettent à courir très vite dans notre direction, on décampe sans
s’affoler, mais quand même bien vite. Pas très agréable…</p>
<p>Nous apercevons un endroit qui serait bien pour camper, près d’une petite
rivière qui descend de la montagne, mais l’endroit est déjà occupé par
une fête de joyeux kirghizes. Tant pis, on continue, mais on est hélés
par une femme qui nous convainc de prendre au moins le thé. Il y a là
une dizaine d’adultes et une nombreuse marmaille. On nous explique que
c’est la fête des enfants – la version non officielle qu’on ne tarde pas
à nous donner est qu’il s’agit de la fête du plaisir à faire des enfants
! Tous sont vendeurs ou artisans au bazaar de Kizil Suu. Ils ont tué un
mouton dont la viande a longuement mijoté avec des pommes de terre. En
attendant ce plat, on nous sert du bon thé chaud avec du lait, et des
salades. Les gens sont extrêmement gentils, rient beaucoup et nous
parlent comme si on se connaissait depuis toujours. Ils nous servent
généreusement en mouton et patates, ça change de nos éternelles pâtes !
Les adolescentes du groupe nous font signer des autographes, on procède
donc à une séance d’écriture en cyrillique. Un tout petit garçon fait
bien rire tout le monde en essayant mes grosses chaussures de marche. Les
commentaires sur le fait que je n’aie pas d’enfant à mon âge sont un peu
énervants, même s’ils ne pensent pas à mal, c’est culturel, par contre
ils sont très impressionnés que je soie ingénieur !</p>
<p>La dame qui nous a convaincu de venir manger avec eux – je n’ai
malheureusement pas compris ou retenu son prénom – est un vrai
personnage : c’est la doyenne du groupe, qu’elle préside naturellement
grâce à un charisme certain. Elle porte un grand chapeau, éclate de rire
tout le temps… Elle insiste pour que nous ne campions pas cette nuit,
mais que nous allions plutôt dormir chez elle quand la joyeuse compagnie
repartira. On hésite un peu, alors elle explique à Gaël avec de grands
gestes qu’une jeune femme doit dormir dans un lit bien confortable, et
pas dans une tente où on se cogne partout ! Hilarité des autres, bien
sûr… On accepte, du coup on peut profiter du reste du repas sans devoir
repartir et chercher un camp. Tout le monde est de plus en plus gai au
fur et à mesure que la bière et le kvas circulent. Un rigolo avec un
béret crânement enfoncé sur le crâne nous explique qu’il est appelé “dva
zhina”, “deux femmes” en russe, en prenant dans ses bras sa femme et la
soeur de celle-ci. Les autres se moquent gentiment du vantard, connu pour
ses fanfaronnades. Pour la fin du repas on passe à la vodka, servie dans
de petits verres. Le chauffeur de la marshrutka qui a amené tout le
monde, lui, ne boit pas. En discutant avec lui, on est impressionné par
sa
connaissance de l’histoire de France. Dans le mouton, tout est bon : le
repas se finit par une distribution d’abats et de soupe faite avec la
carcasse. J’ai un peu de mal avec les abats, Gaël aussi : je mange
certaines parties mais pour d’autres, je n’y arrive vraiment pas !</p>
<p>Il fait nuit quand on repart. La marshrutka n’est pas vraiment adaptée à
la route très accidentée mais ce n’est pas grave, ça passe quand même. Le
chauffeur négocie les cahots un par un, avec “dva zhina” qui sort de
temps en temps pour le guider. La marshrutka est très pleine, même si
d’expérience nous savons qu’elle pourrait l’être encore un petit peu plus
– si, si. Gaël et moi sommes assis, avec sur les genoux des petits
garçons qui essayent de se disputer un peu pour s’occuper, mais
s’endorment très vite ! Les gens commencent à chanter. La chef en
particulier chante très bien, d’une voix bien timbrée et légèrement
plaintive, et les autres reprennent, les uns sur un ton comique, les
autres avec émotion. On arrive dans le village de Kizil Suu. Une épicerie
est encore ouverte sur la grand route, Gaël descend avec quelques uns des
hommes et leur achète du schnaps, de la bière et du saucisson. On arrive
à l’appartement de la dame, dans un immeuble soviétique à deux étages. On
prend le thé dans le salon, la dame commence à avoir l’air fatiguée.
Devant la télé, un de ses fils, un bel adolescent boudeur, fait très
attention à ne pas avoir l’air de s’intéresser à ce qui se passe autour
de lui. Nous prenons quelques gâteaux avec le thé et malgré leur aspect
inoffensif, je me rends compte assez vite que je suis allergique à l’un
d’eux. La dame nous prépare gentiment un beau lit avec plein de tapis, on
va être très bien ! Pendant que Gaël fait un brin de toilette, je
m’endors très vite. Je suis réveillée par quelqu’un qui me parle, en
russe évidemment. Je murmure quelques mots en espérant que la personne va
réaliser que je suis très endormie, mais non, rien n’y fait. OK, une
minute, je me réveille complètement, je mets mes lunettes : que se
passe-t-il ? Un homme jeune a l’air tout a fait exalté se penche vers
moi, il a manifestement un peu bu. Mais il a l’air très gentil, et me
serre la main avec effusion. Je ne comprends que peu de choses dans son
flot de paroles. Il disparaît un instant : vais-je pouvoir me rendormir ?
Non, il ressort de la chambre de sa mère en tenant tout un tas de
tableaux qu’il dispose par terre devant moi, assise sur mon lit
improvisé. Les tableaux représentent des montagnes où courent de grands
chevaux, des paysages somptueux au clair de lune (un peu kitsch), ou des
motifs rupestres peints en relief sur les toiles. Certains des tableaux
me plaisent bien. Sa mère engueule le peintre, lui explique que je suis
fatiguée et qu’il faut me laisser dormir. Gaël revient, et commence à
discuter : mon russe avait atteint ses limites depuis longtemps ! L’homme
est un peintre professionnel qui souhaiterait être exposé à l’Ouest
(zapat, zapat, comme il répète tout le temps). Il voudrait notre avis
pour savoir si sa peinture peut plaire en Europe, et comment faire pour
attirer l’attention des galeries. Pas facile… on convient de discuter
plus longuement le lendemain car là on tombe de sommeil ! On s’endort,
mais je me réveille plus tard avec un fort mal de ventre, conséquence de
mon allergie au biscuit. Je prends un antihistaminique, mais il n’y a pas
grand chose d’autre à faire que d’attendre que ça passe. Je change de lit
pour aller me serrer contre Gaël qui me rassure et me réchauffe, et je ne
tarde pas à me réendormir.</p>
</div>
<div class="section" id="lundi-2-juin-kizil-suu-bakonbaiev">
<h2><a class="toc-backref" href="#toc-entry-11">Lundi 2 juin : Kizil-Suu -> Bakonbaiev</a></h2>
<p>Je me réveille complètement dans le coton, c’est l’allergie qui m’a
épuisée. On s’attendait à devoir se lever tôt pour que la dame puisse
aller au bazaar, mais en fait il ne se passe rien avant 9h au bazaar. Les
horaires des commerces sont très réduits ici, ce qui est plutôt un signe
de mauvaise santé économique. Nous prenons donc le petit déjeuner
tranquillement avec la dame et ses deux plus jeunes fils (le peintre dort
encore) : je mange beaucoup de nan et de confiture de cassis pour
reprendre des forces. On mange aussi des nouilles grillées pimentées avec
des restes du mouton d’hier, c’est très bon. Et bien sûr, on boit plein
de tchai !</p>
<p>On ne repart pas tout de suite car Gaël veut faire réparer la couture de
son sac. Le matamore “dva zhina” d’hier soir est cordonnier au bazaar,
on a convenu de lui apporter le sac ce matin. Mais on a encore le temps
pour aller au bazaar. Le fils adolescent part pour l’école avec un bel
uniforme. Le fils peintre se lève, on parle de peinture et du “milieu” de
l’art à Paris — et de la difficulté d’y rentrer. En plus on n’y connaît
quand même pas grand chose donc c’est difficile de lui donner des
conseils… Nous partons au bazaar avec la mère et le fils peintre. Les
commerçants commencent à installer leurs étals, il fait déjà très chaud à
9h. Mais le cordonnier n’est pas encore arrivé. Du coup le fils peintre
nous emmène nous promener un peu dans le village. On fait une séance
photo souvenir avec lui, et comme il y a un magasin qui peut imprimer des
photos, on donne quelques photos à imprimer. Nous proposons d’offrir les
photos mais il refuse. La modernité du magasin contraste d’ailleurs
curieusement avec d’autres aspects de la vie quotidienne : il n’y a par
exemple pas l’eau courante dans l’appartement de la famille. Puis nous
entrons dans un bar où le peintre se commande… un grand schnaps, qu’il
vide d’un trait (à 9 h du matin !). Nous déclinons sa proposition de
prendre aussi du schnaps, et notre désapprobation doit se lire sur notre
visage car il a quand même l’air un peu gêné… Comme il n’a pas d’argent
on lui paye son schnaps pour partir : il est clairement dépendant à
l’alcool et on devine que sa femme et sa mère doivent contrôler l’argent
qu’il a sur lui pour éviter qu’il abuse. Du coup on n’est pas très
contents de s’être fait avoir comme ça… Nous retournons sur le bazaar
où le cordonnier est arrivé. Il nous propose d’acheter une bouteille de
bière pour rafraîchir les commerçants, et qu’il passe bientôt dans
l’appartement de la dame réparer le sac de Gaël. Marché conclu : on
achète la bière et on rentre à l’appartement. On voit le fils donner à sa
mère le ticket de réservation des photos pour qu’elle paye plus tard…
On se retrouve à nouveau dans le petit salon où nous avons dormi. Le
peintre a l’alcool triste. Il sort son album photo pour nous montrer des
clichés de sa jeunesse (il n’est pas vieux non plus, entre trente-cinq et
quarante ans, mais il a l’air tellement fatigué…). Nous découvrons un
jeune officier au corps vigoureux et au visage inspiré, photographié dans
des exercices de kung-fu ou en permission avec des amis. Un portrait en
noir en blanc retient particulièrement mon attention : une masse
mouvementée de cheveux noirs, des traits extrêmement fins, c’est un très
bel homme qui regarde fixement la caméra. Je lève les yeux : il a bien
changé… ses traits se sont ramollis – l’alcool n’y est sûrement pas
pour rien –, et surtout le regard a bien perdu de sa flamme. Il a laissé
tomber d’un coup la carrière militaire pour se consacrer à la peinture.
Mais la vie d’artiste n’est pas facile au Kirghistan : s’il est reconnu
dans son pays, le marché de l’art est presque inexistant et il en réduit
à peindre des toiles plus “commerciales” qui plaisent aux touristes de
passage. Il parle très vite, je comprends très peu, parfois il se tourne
vers moi et me parle très sérieusement plusieurs minutes de suite. Je me
sens très faible à cause de l’allergie, j’ai la tête qui tourne, je
voudrais qu’il parle moins vite… Il décide que Gaël ressemble à Jésus
avec ses cheveux et sa barbe, et ne l’appelle plus que Iesus Christos.
Par moments il a l’air de se sentir vraiment mal, nous dit que la vie ici
est une prison : il est au bord de la dépression, et l’alcool est un
refuge empoisonné… La mère revient du bazaar, a l’air fâchée que le
cordonnier ne soit pas encore là. Elle est très gentille et prévenante,
elle nous cuisine des oeufs pour midi, et je mange plein de tartines de
confiture. Puis je m’endors, épuisée.</p>
<p>Le cordonnier “dva zhina” vient enfin, et fait une réparation bien
solide au sac de Gaël : il connaît visiblement bien son boulot. Il faut
qu’on se dépêche de partir pour aller prendre la seule marshrutka de
l’après-midi en direction de l’ouest. Gaël donne 500 soms à la dame pour
la remercier de son hébergement : elle commence par refuser, nous disant
qu’elle nous a accueillis juste parce qu’on est sympas, mais Gaël lui
dit que ça pourra aider à financer une autre fête avec les rigolos du
bazaar. Pendant ce temps je discute avec le peintre en essayant de
l’occuper pour qu’il ne voie pas que sa mère a de l’argent : vu comme il
est en manque d’alcool, il doit tout le temps lui demander de l’argent…
Il nous accompagne pour nous montrer l’arrêt de la marshrutka, et réussit
à nous soutirer 20 soms avant de partir. On a un peu honte de céder, sans
trop savoir quoi faire. L’accueil de la dame était absolument génial,
rencontrer des gens comme ça est une très belle expérience, par contre le
fils était un peu fatigant à la fin. Mais on ne peut pas trop lui en
vouloir : ravagé par l’ennui et l’alcool, il n’en restait pas moins
gentil et intéressant.</p>
<p>On the road again! La marshrutka peut nous emmener jusqu’à Bakonbaiev,
une autre petite ville, où nous descendons dans l’hôtel de la ville, tenu
par un employé de la gare routière et sa femme (il faut donc demander à
la gare pour trouver l’hôtel !). Vassilia, notre hôtesse, nous donne une
jolie petite chambre toute propre, c’est agréable de se reposer un peu.
Mais il n’y a pas l’eau courante ici non plus, seulement une pompe dans
le jardin. Il faut maintenant qu’on décide où on fait notre prochaine et
dernière rando. Il y a plusieurs options, la plus tranquille consisterait
à aller au lac Song Kul, un lac d’altitude sur un haut plateau à 3000m où
se trouvent beaucoup de nomades. Mais Gaël trouve que les paysages
ressemblent plus à des collines qu’à de vraies montagnes, du coup ça ne
le tente pas du tout. On sort les cartes pour réfléchir à une autre
destination, mais on est fatigués et on a beaucoup de mal à prendre des
décisions. Une tablette de chocolat est engloutie d’un coup, ça nous
redonne un peu d’énergie. Parmi les destinations possibles, l’une d’entre
elles est particulièrement tentante. Il s’agit de se faire déposer au
beau milieu de nulle part, un peu au sud du parc national d’Ala Tau, et
de traverser le parc national pour se retrouver près de Bishkek. Rentrer
à Bishkek en traversant les montagnes, c’est tentant ! Le problème est
qu’à part un gros trait rouge indiquant des itinéraires de randonnée sur
la carte à plus grande échelle (qui couvre toute l’Asie centrale !), on
n’a pas plus de précision sur la faisabilité de la rando. Les cartes plus
précises qu’on a montrent qu’il y a une difficulté principale : un col à
3900 m avec une approche très escarpée. On ne peut atteindre ce col
qu’après un jour de marche et demi, si ça ne passe pas on aura tout
juste le temps de revenir en arrière et de négocier un transport jusqu’à
Bishkek. C’est un peu risqué, mais on décide de tenter le coup. Ranimés
par cette décision, nous sortons manger. Bakonbaiev est comme Kizil Suu,
un petit village où il ne se passe pas grand chose. Les quelques
commerces sont regroupés sur la grand route. Le bazaar est fini et il n’y
a presque personne dans les rues, ça donne une impression bizarre. Il y a
deux cafés d’ouvert, nous en choisissons un où nous mangeons du lagman
et des mantis, le tout arrosé de bière bien sûr. Un groupe de
plusieurs familles arrive et commande à manger en sortant plusieurs
bouteilles de schnaps de leurs sacs. La serveuse a froid, elle frissonne
devant la fenêtre comme si elle attendait quelque chose, mais reste en
T-shirt dans se couvrir.</p>
<p>En rentrant à l’hôtel nous croisons un homme légèrement bourré qui est
tout content car sa femme a eu un enfant, une petite fille – et il a
dignement fêté ça au schnaps. Il est accompagné de deux gamins, dont un
petit garçon sur une bicyclette rouge aussi haute que lui ! Il nous
propose de venir manger avec sa famille pour fêter cette naissance. C’est
très gentil, mais nous avons eu suffisamment d’émotions pour la journée !
L’homme est visiblement déçu qu’on ne puisse pas lui donner les photos
qu’on prend de lui et ses enfants : un polaroid doit faire beaucoup
d’heureux dans ce pays !</p>
</div>
<div class="section" id="mardi-3-juin-bakonbaiev-kotchkor">
<h2><a class="toc-backref" href="#toc-entry-12">Mardi 3 juin : Bakonbaiev -> Kotchkor -> ??</a></h2>
<p>Ce matin on reprend la marshrutka pour Baliki, et on peut même s’asseoir
! Notre voyage jusqu’au pied de la rando se fera en au moins trois
étapes, on croise les doigts pour que tout s’enchaîne bien et que le
voyage ne nous prenne pas trop de temps (sinon on devra marcher plus
vite pour rentrer à Bishkek !). Arrivés à Baliki on a de la chance de
trouver une marshrutka qui part bientôt pour Naryn et passe donc par
Kotchkor où on veut aller. Heureusement on ne se laisse pas arrêter pas
les dénégations des chauffeurs de taxi qui affirment mordicus qu’il n’y a
pas de marshrutka pour Kotchkor… ils perdent un peu contenance quand
même quand on arrive juste à côté de la marshrutka en question… Le
chauffeur balance nos sacs dans le coffre, et une femme imposante sort de
la marshrutka en râlant très fort car il a mis nos sacs au-dessus de ses
sacs de pain qui vont être écrasés. Le chauffeur arrange ça, on se dit
qu’elle n’a pas l’air très commode… Mais quand la marshrutka démarre on
est assis à côté d’elle, et elle très sympa et discute pas mal avec nous.
Elle a appris pas mal de choses sur la France à l’école, nous parle de la
Sorbonne ou de la tour Eiffel…</p>
<p>La route jusqu’à Kotchkor est fort pittoresque, on traverse des montagnes
très rouges et arides. Nous arrivons à Kotchkor à midi, jusqu’ici on est
allés plus vite que nos plans les plus optimistes. On déjeune dans la
petit bazaar dans une petite échoppe où on prend du plov et du tchai. On
compète aussi nos provisions avec un très beau nan et une livre de
cerises qui nous font très envie.</p>
<p>Il faut maintenant trouver un taxi qui accepte de nous conduire à
l’endroit bizarre où on veut aller. On va s’asseoir stratégiquement pas
loin des taxis pour étudier nos cartes, en attendant qu’ils nous tombent
dessus, ce qui ne tarde guère ! La course qu’on voudrait est difficile,
elle prend une route avec un col très haut. Certains chauffeurs renoncent
tout de suite, d’autres discutent entre eux si c’est possible. Nous on
regarde, assez amusés. Je vais acheter une glace. Les chauffeurs étalent
la carte sur le chariot de la petite vendeuse qui finit par s’énerver et
les chasser. Un des chauffeurs sait bien lire les cartes (c’est loin
d’être le cas de tous). Il propose de nous conduire jusqu’à la fin de la
route praticable en véhicule normal (non 4x4) mais pas jusqu’au bout.
Certains suggèrent qu’on fasse le reste en louant des chevaux aux gens
sur place. Si on avait le temps, ce serait l’option la plus pratique,
mais là on ne peut pas. On suggère qu’il faut une 4x4. Le chauffeur le
plus dégourdi par avec son taxi à la recherche d’une 4x4. Il revient en
4x4 avec le propriétaire de la voiture, un homme solide à l’air sérieux.
A nouveau, on sort la carte et c’est l’attroupement autour de nous. Le
chauffeur de la 4x4 nous propose une variante pour l’itinéraire : plutôt
que de prendre le chemin le plus court, qui est une très mauvaise route,
on peut prendre une route en meilleur état, puis revenir en arrière sur
la mauvaise route à un croisement. C’est plus long, mais sûrement plus
raisonnable aussi : ça évite notamment de passer par le col en altitude.
On se met d’accord sur le prix : 15 som au kilomètre. Mais alors qu’on va
partir, ils nous disent que c’est en comptant les kilomètres aller+retour
: dans ce cas-là, on n’a pas assez d’argent. Ca nous semble d’ailleurs
curieux car d’habitude les chauffeur ne comptent que les kilomètre de
l’aller. On leur dit donc qu’il nous reste seulement 150 dollars, et
qu’on ne peut leur donner que ça (il nous reste à peine plus en effet
!). Heureusement ça leur va. On commence par changer 50 dollars à la
banque pour prendre de l’essence. Jusqu’ici le chauffeur qui a cherché la
4x4 était resté avec nous, mais il reste à Kotchkor : il aura sûrement
une petite commission sur la course.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248977130/in/set-72157648345589451"><img alt="Djuku" class="align-center" src="https://farm4.staticflickr.com/3936/15248977130_29457fac4b_b_d.jpg" style="width: 100%;" /></a>
<p>C’est parti pour 250 km, on pense en avoir au moins pour quatre heures.
La voiture est tout à fait adaptée au trajet, une 4x4 bien haute de
fabrication chinoise. On roule pendant une heure et demi sur une route en
terre, sans traverser le moindre village. Au Sud se trouvent les
montagnes qui encerclent Song Kul : c’est très beau, même Gaël le
reconnaît : si le relief n’est pas très alpin, les vallonnements se
succèdent dans un ensemble très esthétique, et les couleurs des
montagnes sont très belles, avec des nuances de verts, bleus et bruns
très variées. On commence à croiser quelques yourtes, puis on traverse
quelques tout petits villages. En dehors des murs des villages se trouve
toujours un petit cimetière. Ces cimetières sont très beaux et émouvants,
une collection de petits mausolées en terre cuite qui ressemble à une
ville en miniature. Le paysage change au fur et à mesure qu’on avance,
devient plus aride, les pentes herbeuses deviennent des pierriers sur les
flancs des montagnes. On rejoint le canyon tracé par une rivière
tumultueuse. La roche est d’un rouge profond, la rivière blanche à force
de remous (celle-là, on ne la traverse pas à pied !). On arrive au
croisement où la bonne route que nous avons suivie jusqu’ici continue, et
où il faut bifurquer pour prendre une route moins carrossable. Notre
chauffeur sait très bien où aller, il connaît bien le coin. Nous comptons
les rivières sur la carte pour être sûrs de l’endroit où s’arrêter. La
route passe bien avec le 4x4. Il y a quelques yourtes et même des maisons
dans la vallée où nous nous engageons, et des troupeaux de chevaux. Le
pont qui était notre repère sur la carte est bien là où on l’attendait,
on s’arrête là ! On paye le chauffeur puis on lui dit au revoir : encore
quelqu’un qu’on a été heureux de croiser, à la fois compétent et
sympathique.</p>
<p>On s’écarte un peu de la petite route pour camper. Les montagnes au sud
sont superbes dans la lumière rasante qui vient lécher les glaciers, et
projeter des ombres veloutées au creux des vallonnements.Il y a plein de
petits insectes dans les hautes herbes, du coup on avance vite. On fait
un ou deux kilomètres puis on pose le camp, tout heureux d’être arrivés
aussi vite au pied des montagnes. Aujourd’hui on a le droit à une entrée,
une sauce au vin qui est engloutie avec du nan, puis à des pâtes
instantanées qui disparaissent aussi vite. La vallée est très jolie, il y
a plein d’oiseaux jaunes et rouges avec des aigrettes colorées et une
grande queue, nous n’en avions jamais vu de tels auparavant.</p>
</div>
<div class="section" id="mercredi-4-juin-remontee-dans-la-vallee">
<h2><a class="toc-backref" href="#toc-entry-13">Mercredi 4 juin : remontée dans la vallée</a></h2>
<p>Nous avons posé notre camp du côté gauche (ouest) de la rivière, en
suivant le chemin indiqué par la carte. D’ailleurs le chemin est très bon
et large, ce qui est bon signe. Cependant, on voit que le chemin
traverse la rivière cinq kilomètres plus loin, et ça nous ennuie un peu
de ne pas savoir s’il y a un pont ou non : il a pas mal plu ces derniers
temps et la rivière est rapide et profonde, si on doit passer à gué ça ne
va pas être facile. Comme il y avait un pont à l’endroit où le chauffeur
nous a déposé hier, on décide de rebrousser chemin et de partir de
l’autre côté de la rivière.</p>
<p>C’est la solution la plus prudente, mais ce n’est pas très agréable car
le chemin est beaucoup moins bon du côté est, ou plutôt, il n’y a
pas vraiment de chemin, juste des traces d’animaux. On avance dans des
grandes herbes où on enfonce beaucoup sans voir où on met les pieds, on
passe notre temps à descendre, monter puis redescendre pour trouver le
meilleur passage… Du coup on avance lentement, ce qui est un peu
embêtant car aujourd’hui il faut qu’on s’approche le plus près possible
du col. On continue notre avancée dans la brousse, sans machette hélas.
Il fait un temps magnifique. Les collines autour de nous n’ont rien
d’extraordinaire mais offrent un décor tout à fait plaisant. Il y a très
peu d’animaux, contrairement aux autres endroits où nous avons marché,
juste un troupeau de chevaux qui vient boire à la rivière, mais reste
près de la vallée habitée. On se met à faire des blagues sur les loups
pour expliquer ce manque d’animaux…</p>
<p>Soudain on aperçoit le pont sur la rivière, un peu plus loin d’ailleurs que
l’endroit où la carte fait traverser le chemin. Bon, on a perdu du temps, mais
on n’avait pas vraiment le choix. C’est bon à savoir pour les prochains
randonneurs : il y a bien un pont… La bonne nouvelle est qu’on récupère un
chemin meilleur. C’est même presque une route, avec des pavés en pierre sur une
bonne largeur, ça fait penser à une voie romaine. Mais la route ne doit pas
être très utilisée, et dans les creux elle disparaît sous les herbes hautes.
Mais on réussit toujours à la retrouver grâce aux talents de pisteur de Gaël.
Une fois qu’on a pris un peu de hauteur il y a moins de grandes herbes et il
n’y a plus de difficultés pour suivre la route. Elle continue à être aussi
large et bien pavée, on se demande vraiment de quelle époque elle date. Et on
se dit que si une telle route existe, ça doit bien passer au col… Le chemin
monte, mais doucement. On fait un premier pique-nique, abrités dans une
ondulation de la pente. Au deuxième Y de la vallée on part à gauche dans les
montagnes, en suivant toujours l’autoroute comme on l’appelle maintenant. On
trouve une hache plantée dans la terre du chemin, sûrement là depuis des
années, mais on oublie de la prendre : tant pis, on ne jouera pas avec les
loups… La vallée où nous montons maintenant s’ouvre sur un grand cirque
montagneux. L’herbe se raréfie, les pentes sont rocheuses avec pas mal de
neige. Tout au fond on devine l’endroit où doit être le col, mais il n’est pas
possible de le repérer précisément. On cherche un camp alors qu’il reste encore
un peu d’herbe : ce n’est pas très agréable de camper dans un pierrier ! La
vallée est étroite et en pente, on trouve un terrain à peu près plat entre deux
branches de la rivière. Gaël va faire une petite reconnaissance en direction du
col, et je m’offre le luxe d’une lessive et de me laver rapidement dans la
rivière. Pour dîner nous avons une sauce à l’oignon, un lyophilisé de pasta
milanese et un instant pudding que nous avons fait prendre dans la neige : un
vrai festin ! Mais on en a besoin, demain c’est le col…</p>
</div>
<div class="section" id="jeudi-5-juin-passage-du-col">
<h2><a class="toc-backref" href="#toc-entry-14">Jeudi 5 juin : passage du col</a></h2>
<p>C’est le grand jour et les loups ne nous ont pas mangés. On part vite
pour en découdre avec le col. Le temps est encore superbe : une bonne
météo pour un passage de col ! On peut encore suivre la grande route qui
disparaît parfois sous de grandes plaques de neige. Il faut alors
deviner par où passent ses lacets pour la retrouver, jeu auquel Gaël
excelle. Le terrain devient alpin, des pentes de pierres et de grandes
étendues de neige. Un peu au-dessous de nous se trouve un lac gelé
d’un turquoise magnifique. Le paysage est très beau, mais aussi un peu
hostile : de la pierre et de la glace à perte de vue. Nous progressons
maintenant sur une grande pente de neige dure. Il faut faire très
attention à ne pas glisser, ce n’est pas sûr qu’on pourrait s’arrêter
avant le lac. Par contre on n’enfonce pas dans la neige, heureusement. On
avance ainsi dans une alternance de pierriers et de neige : ça avance
bien. On arrive à un petit replat entouré de parois rocheuses bien
verticales : par où va-t-on passer ? On finit par deviner que la route
doit suivre la base d’un de ces murs rocheux avant de grimper à un
endroit un peu moins haut que le reste, donc on y va. Ca correspond en
gros à la carte (nous n’avons pas de carte très précise, seulement une
1/200000). La montée est rude dans le pierriers et la neige, ça monte
très sec, ça glisse aussi et je m’essouffle. Un petit peu avant la fin de
la montée Gaël monte rapidement au col et redescend me prendre mon sac,
et c’est beaucoup plus facile pour finir ! Il me dit galamment qu’il aime
bien faire deux fois les cols.</p>
<p>La montée n’a pas été très agréable, mais au col il ne fait pas trop
froid et on peut même pique-niquer. Un poteau nous indique qu’on est bien
au bon endroit, et il y a même une pelle pour dégager la corniche de
neige qui empêche de passer de l’autre côté (nous n’en aurons pas besoin,
il y a déjà un trou dans la corniche). Le paysage qui s’offre à nous est
un très large cirque glaciaire, c’est très beau et beaucoup plus serein
que l’endroit d’où on vient. On devine le début de la gorge qui va nous
ramener vers Bishkek. Le début de la descente est un vrai plaisir : la
pente est recouverte de neige, et s’il y a quelques passages un peu
raides au début, après on peut courir ou faire des glissades contrôlées
dans une neige juste assez molle pour qu’on y enfonce les talons, mais
pas plus. On se croirait aux sports d’hiver : on descend ainsi une longue
pente à toute vitesse, c’est assez génial.</p>
<p>Nous arrivons à une combe barrée par un lac gelé : ça se complique un peu pour
avancer. Intimidés par les pierriers qui semblent raide sur le côté gauche, on
choisit de passer par les pentes de neige sur le côté droit. Vu de plus près
c’était clairement la mauvaise option, les pentes sont très raides, à certains
endroits on manque glisser sur la neige très dure, le plus souvent on enfonce
jusqu’à mi-cuisse et c’est très dur d’avancer. En plus on se refroidit vite à
force de barboter dans la neige : heureusement j’ai gardé mon coupe-vent et mes
gants. Je manque glisser une ou deux fois : au-dessus du lac gelé, ce n’est pas
très rassurant. Ouf, on arrive à dépasser le lac. Après il y a encore une
grande pente neigeuse à traverser pour rejoindre les pierriers. Ca glisse
terriblement et j’ai beaucoup de mal à avancer. Après une glissade de plusieurs
mètres je m’aperçois que je contrôle pas mal ma trajectoire avec les pieds, et
je descends toute la pente en une grande glissade sur mon sac… Gaël décide
alors de descendre en courant et en enfonçant les talons, ce que je n’arrivais
pas à faire. On a descendu cette pente plus vite que prévu… Plus loin il y a
encore des pierriers et de petites bouts de neige mais l’essentiel des
difficultés est passé : il y a même un peu d’herbe ! Heureusement car on
commençait à en avoir un peu assez de la neige…</p>
<p>Nous arrivons au début de la vallée d’Ala Archa. Il y a une route de
l’autre côté (est) de la vallée, mais juste à côté de nous il y a aussi
une route qui ressemble fort à notre grand route de l’autre côté des
montagnes. On décide donc de la suivre. A un moment il n’y a plus de
route, car toute une pente s’est effondrée. Il faut désescalader une
petite pente neigeuse et aller reprendre la route dans le lit d’un
ruisseau. Ca fait une difficulté de plus, et on est un peu fatigués… La
randonnées est tout à fait faisable, mais on a compris aujourd’hui
pourquoi les groupes ne passent pas par cet endroit, c’est quand même dur
! Un vieux chasse-neige abandonné dans un amoncellement de pierres donne
une image de fin du monde.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435341332/in/set-72157648345589451"><img alt="" class="align-right" src="https://farm6.staticflickr.com/5598/15435341332_aeb02c5c21_z_d.jpg" style="width: 40%;" /></a>
<p>Dans le fond de la vallée nous apercevons le bassin verdoyant d’une rivière
sinueuse… et deux tentes, juste là où on voudrait camper. On voit d’ailleurs
une personne se diriger vers nous. A son équipement (gore-tex marmott, bâtons
de marche) on le pense américain, mais non, Ivan est russe. C’est un alpiniste
professionnel (sponsorisé par Marmott, d’où l’équipement) qui est venu se
balader quelques jours dans la montagne avec sa famille et des amis. Il les a
laissés au camp et gambade dans la montagne à la recherche de belles photos. Il
est très sympa, avec un énorme sourire, et nous invite à venir prendre le tchai
dans leur tente.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248931769/in/set-72157648345589451"><img alt="" class="align-right" src="https://farm6.staticflickr.com/5599/15248931769_1fae509778_z_d.jpg" style="width: 40%;" /></a>
<p>Nous arrivons au camp, situé près d’une ancienne station météo russe,
dont les instruments sont restés plantés comme des dards sur le sol de la
vallée. A cet endroit le terrain est très plat et la rivière fait de
jolis méandres : c’est un paysage très doux qui contraste avec les
montagnes de glace et de pierre que nous quittons.</p>
<p>Les Russes sont sympas et rigolos. Ils nous font bien rire avec leurs
coussins qu’ils tiennent autour de la taille avec un élastique pour
s’asseoir confortablement dès qu’ils le veulent. Ils insistent pour nous
nourrir, et leur provisions sont appétissantes : oignons cru et saucisson
fumé excellent, fruits secs et bonbons pour l’apéritif, puis une soupe à
l’avoine et au porc très bonne. Je vais me coucher tout de suite après le
repas car je suis crevée : la journée a été rude et je recommence à avoir
de la fièvre.</p>
</div>
<div class="section" id="vendredi-6-juin-parc-ala-archa">
<h2><a class="toc-backref" href="#toc-entry-15">Vendredi 6 juin : parc Ala-Archa</a></h2>
<p>Il a plu toute la nuit et on ne se presse pas de partir. Les Russes
partent faire une balade à la journée. On décide d’aller faire une petite
balade dans une petite vallée pour débuter la journée, mais mal nous en
prend : on part assez loin avant de se rendre compte que le temps se
dégrade vraiment. Nous voyons un rideau de pluie arriver bien vite, et le
temps de se décider à revenir en arrière on est déjà complètement
trempés. La pluie est très violente, presque de la grêle par moments. On
rentre à la tente presque en courant. Une fois arrivés on s’engouffre
dans la tente, on se change un peu et on attend que ça passe.
Heureusement la pluie ne dure pas trop longtemps et on peut sortir
sécher. Après avoir englouti une tablette de chocolat pour se donner du
courage, on part pour du bon. On refait exactement le même chemin que le
matin, mais sous une lumière différente puisqu’il fait maintenant assez
beau. Le passage d’une rivière se révèle assez délicat : il n’y a pas de
pont et la rivière est gonflée par les orages successifs. On essaye de
passer à un endroit où la rivière se divise en plusieurs branches : la
première branche se traverse assez bien mais après on se retrouve devant
un torrent qui nous embête bien : l’eau est soit profonde soit rapide (et
souvent les deux), avec beaucoup de courant. On finit par remarquer une
planche qui traîne dans l’eau d’une rive à l’autre, mais n’est pas très
engageante pour autant car beaucoup d’eau passe au-dessus et ça doit
être à peu près impossible d’avancer dessus avec le courant dans les
chevilles, sans parler des risques de glisser. Il y a quand même trois ou
quatre mètres à traverser. Gaël a la bonne idée de sortir la planche de
l’eau pour la poser sur des pierres : maintenant elle sort de l’eau. Mais
elle reste bien étroite et glissante. Je tente le début sans mon sac : en
avançant toujours le même pied, ça va à peu près. Arrivée au milieu je
n’ai aucune envie de me retourner ou de marcher, donc je vais jusqu’au
bout, puis je reviens. Donc c’est possible de passer… Je recommence
avec mon sac, forcément moins agile avec le poids en plus, puis c’est le
tour de Gaël : ouf, on est passés ! On remarque que les passages de
rivière étaient mieux aménagés dans les endroits où vivaient les nomades
que dans ce parc national.</p>
<p>On s’offre un pique-nique bien mérité après ces émotions, puis on
continue dans la jolie vallée. La route est toujours excellente, et
devient de plus en plus large au fur et à mesure qu’on se rapproche de
l’entrée du parc national. Mais il se met de nouveau à pleuvoir
violemment, alors que nous atteignons le plateau avant le village
d’Alplagen. Il y a un groupe de cabanes abandonnées près d’un petit
parking, on se réfugie sous un auvent pour attendre que la pluie se
calme un peu. Deux Kirghizes viennent nous demander s’ils peuvent
partager l’auvent avec nous, bien sûr, il y a de la place ! Ils
reviennent avec 5 autres personnes, et tout le matériel nécessaire pour
faire un beau pique-nique avec des brochettes shashlyks ! Ils travaillent
au ministère de l’économie, et à ce qu’ils nous disent, aujourd’hui c’est
la fête des économistes, donc tout le bureau fait la fête. On parle
surtout russe avec quelques mots d’anglais : la chef rit très fort de se
faire appeler “boss” par ses collègues. Tout ce petit monde est très
joyeux, la vodka est sortie assez vite et même si je préfère leur
excellent jus d’abricot, on porte quelques toasts à l’amitié
franco-kirghize. Ils insistent même pour boire à la santé de Sarkozy ! On
mange plein de fruits excellents, et même une brochette chacun, puis on
les laisse faire la fête pour continuer : notre but est d’être à Bichkek
le lendemain pas trop tard, il faut donc qu’on se rapproche du village de
Kashka-Suu où on pourra prendre une marshrutka, encore à 7-8 kilomètres.
Le village d’Alplagen se compose de quelques boutiques et d’un hôtel hors
de prix (les prix ne sont pas affichés mais on nous parlera plus tard de
sommes astronomiques, plusieurs centaines de dollars pour une chambre, du
coup on sera bien contents de ne pas s’être laissés tenter par un lit
confortable…). On achète du pain, du chocolat et une bière, l’essentiel
quoi, à un magasin puis on continue vers la sortie du parc. Il y a
maintenant beaucoup plus de trafic : ce sont les Bishkekois qui vont
fêter le vendredi soir dans la montagne. On voit passer en trombe
quelques énormes Mercedes aux vitres teintées : il y a là des gens bien
différents des Kirghizes des montagnes !</p>
<p>On essaye de faire un peu de stop dans l’espoir d’arriver à Bishkek dès
ce soir mais la plupart des voitures sont pleines à craquer. On marche
maintenant sur une route goudronnée avec un trafic peu important mais
régulier. Dans une aire de repos sur le côté de la route on aperçoit une
voiture qui a roulé dans une profonde ornière, dont les occupants
n’arrivent pas à la sortir malgré leurs efforts. On se dit qu’on va
aller leur donner un coup de main pour pousser la voiture. Mais ce
n’était pas forcément une bonne idée… Le groupe se compose du
conducteur, de deux hommes qui essayent de pousser la voiture, et d’un
homme et d’une femme avachis à l’intérieur de la voiture – complètement
bourrés. On tique un peu qu’ils ne les aient pas déchargés pour alléger
la voiture, mais on ne dit rien. On commence à tirer, pousser, soulever
avec les deux autres gars pendant que le conducteur s’escrime sur
l’accélérateur mais les roues patinent et la voiture n’arrive pas à
remonter. On commence à placer des branches et à enlever les pierres pour
que les roues accrochent mieux, mais le conducteur est désagréable et
même violent, crie très fort, roule à moitié sur un de ses copains qui a
juste le temps de s’écarter . Avec un des deux autres gars qui est lui
aussi bien bourré et me jette des regards bizarres, tout ça ne me plaît
pas beaucoup… La voiture est en meilleure position que tout à l’heure
mais elle n’est pas encore sortie, mais de toute façon le conducteur
n’écoute personne donc on s’en va et on retourne sur la route.</p>
<p>Un peu plus tard, on marche tranquillement quand on entend le bruit d’une
voiture lancée à fond la caisse : c’est les cinglés de tout à l’heure qui
ont réussi à sortir leur épave de l’ornière et foncent dans notre
direction. Ils passent extrêmement vite à côté de nous – on s’est quand
même bien écartés sur le côté, mais ça fait peur – puis freinent tout
aussi brutalement pour nous parler. Ils sont super contents de s’être
sortis de ce mauvais pas, nous remercient chaudement de les avoir aidés
et nous proposent de venir faire la fête avec eux. Les deux poids morts
de tout à l’heure semblent être sortis de leur coma éthylique, du coup
ils ont tous l’air uniformément bourrés, sauf l’un d’entre eux qui avait
l’air bien inquiet tout à l’heure. Autant dire que leur compagnie ne nous
fait pas très envie, sans parler du fait de se retrouver dans leur
voiture… Mais ça a l’air de les fâcher qu’on ne veuille pas, le
conducteur se remet à crier comme tout à l’heure, ça ne nous plaît pas du
tout, du coup on explique qu’on allait camper juste à côté et on
disparaît dans les buissons sur le côté de la route. Mais on n’entend pas
la voiture redémarrer… Il s’ensuit pour nous une demi-heure de paranoïa
où on se demande ce qu’ils nous veulent au juste. On s’écarte bien de la
route et on reste un petit moment sans bouger. Puis on se convainc
qu’ils sont bien partis et on repart, mais ce n’était vraiment pas un
moment agréable. Du coup on décide de se chercher un camp loin de la
route pour éviter la présence des fêtards bourrés dans le coin. On trouve
un endroit dans un verger un peu en hauteur, pas génial parce qu’il n’y a
pas vraiment de terrain plat, mais ça ira. Un dernier lyophilisé, et au
lit ! Pendant la nuit on entend les cris déchirants d’un animal qui a
l’air tout près, c’est un peu angoissant. Le matin venu je me convaincrai
qu’il s’agissait probablement d’une espèce de corbeau, mais sur le coup
ça n’était pas très rassurant.</p>
</div>
<div class="section" id="samedi-7-juin-retour-a-bishkek">
<h2><a class="toc-backref" href="#toc-entry-16">Samedi 7 juin : retour à Bishkek</a></h2>
<p>C’était la dernière nuit sous la tente, il nous faut maintenant rejoindre
Kashka-Suu puis aller à Bishkek. Il reste quelques kilomètres à faire, on
essaye encore le stop avec les voitures qui passent mais elles sont
toutes pleines pour la plupart, comme la veille. Mais on a de la chance :
une marshrutka qui avait visiblement monté un groupe jusqu’à Alplagen
redescend vide et propose de nous emmener… jusqu’à Bishkek. On accepte,
évidemment. On propose de le payer, mais il refuse en nous disant : “mais
oui, je sais que vous avez de l’argent, mais ce n’est pas la peine”.
Super gentil de sa part. On traverse Kashka-Suu, bien contents de ne pas
avoir dû marcher jusque là, puis les faubourgs de Bishkek, et on se
retrouve en plein Bishkek à 9h du matin, soit bien plus tôt que ce qu’on
pouvait espérer.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15412630366/in/set-72157648345589451"><img alt="Bishkek bazaar" class="align-left" src="https://farm6.staticflickr.com/5601/15412630366_6390163e44_z_d.jpg" style="width: 40%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248985220/in/set-72157648345589451"><img alt="Bishkek bazaar" class="align-right" src="https://farm3.staticflickr.com/2943/15248985220_7501d41f1e_z_d.jpg" style="width: 40%;" /></a>
<div style="clear: both;"></div><p>Nous voudrions aller poser nos sacs à la guesthouse Ultimate Adventure où
nous avons réservé pour cette nuit, mais il est encore un peu tôt pour
ça. Du coup nous décidons de profiter de la gastronomie locale pour
passer le temps, et nous changer des lyophilisés des derniers jours. On
commence par quelques samsas absolument délicieux à emporter, puis quand
les cafés ouvrent à 10h on commande des salades et encore des samsas :
on va passer la journée à manger ! On marche dans les grandes avenues de
Bishkek, étincelantes sous le soleil, croisant ça et là les monuments
colossaux de l’époque soviétique : ça fait très bizarre d’être de retour
dans une grande ville. Vers 11h on se met en route vers la guesthouse :
on y retrouve Smaïl, toujours aussi sympa. Il a fort à faire car il est
un des organisateurs du Paris-Pékin à vélo qui est arrivé hier à Bishkek,
mais il prend le temps de nous demander de raconter nos balades. Puis on
se précipite sur la douche, on en a bien besoin ! Il nous reste toute
l’après-midi, qu’on passe agréablement au bazaar de Bishkek. On continue
notre recherche du meilleur samsa de Bishkek, on achète des fruits secs
et de petites boules de fromage sec (très bonnes à l’appéritif avec une
bière !) pour les ramener, ainsi que quelques souvenirs. Encore un bazaar
bien sympathique. On se promène encore un peu dans les grandes avenues,
puis on rentre à la guesthouse préparer notre départ. On avait prévu de
passer la soirée au café-karaoké d’à côté pour faire la fête avec les
Bishkekois, mais Smaïl nous propose de manger à la guesthouse. Il a
préparé un plat algérien, des crêpes fourrées d’un mélange d’herbes et de
fromage, un vrai régal. Il y a aussi sa femme américaine, sa fille, un
employé de la guest house, le médecin français du Paris-Pékin et un
cycliste allemand blessé qui a besoin d’un peu de repos. On parle
longuement de voyages et de randonnées, alternant le français, l’anglais
et l’allemand. Et on dévore les crêpes de Smaïl, le jambon et la salade
qu’il nous apporte, à tel point que ça fait bien rire les autres
convives, pourtant eux aussi affamés ! Encore une soirée très sympathique
grâce à la générosité de nos hôtes.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15412631946/in/set-72157648345589451"><img alt="Bishkek bazaar" class="align-left" src="https://farm3.staticflickr.com/2947/15412631946_5efc969185_b_d.jpg" style="width: 60%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435346432/in/set-72157648345589451"><img alt="Bishkek bazaar" class="align-right" src="https://farm4.staticflickr.com/3932/15435346432_bc07d87363_z_d.jpg" style="width: 30%;" /></a>
<div style="clear: both;"></div></div>
<div class="section" id="dimanche-8-juin-retour-a-paris">
<h2><a class="toc-backref" href="#toc-entry-17">Dimanche 8 juin : retour à Paris</a></h2>
<p>Notre vol est très tôt le matin, donc on va en taxi à l’aéroport de
Manas. C’est la fin de ces trois superbes semaines de vacances en Asie
Centrale. Il faudra y retourner, peut-être en ayant appris un peu plus de
russe, et à monter à cheval : il doit y avoir peu d’endroits où les
paysages sont si beaux et les gens aussi gentils.</p>
</div>
Asie centrale: 17 mai - 8 juin 2008: Ouzbekistan2008-06-02T00:00:00+02:002008-06-02T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-06-02:/personnal/asie_centrale_uzbekistan.html<a class="reference external image-reference" href="vignettes.jpg"><img alt="vignettes" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/vignettes.jpg" style="width: 100%;" /></a>
<p>Trois semaines en Ouzbékistan, puis au Kirghizstan… en train, bus,
marshrutka, cheval, et beaucoup à pied !</p>
<p>Photos et panoramas : Gaël | Récit, mise en page et retouches photos :
Emmanuelle</p>
<a class="reference external image-reference" href="map.jpg"><img alt="Carte voyage" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/map.jpg" style="width: 100%;" /></a>
<div class="contents topic" id="table-des-matieres">
<p class="topic-title">Table des matières</p>
<ul class="simple">
<li><a class="reference internal" href="#samedi-17-mai-dimanche-18-mai-bourg-la-reine-bukhara" id="toc-entry-1">Samedi 17 mai - dimanche 18 mai : Bourg-la-Reine -> Bukhara</a></li>
<li><a class="reference internal" href="#lundi-19-mai-bukhara" id="toc-entry-2">Lundi 19 mai : Bukhara</a></li>
<li><a class="reference internal" href="#mardi-20-mai-bukhara-samarcande" id="toc-entry-3">Mardi 20 mai : Bukhara -> Samarcande</a></li>
<li><a class="reference internal" href="#mercredi-21-mai-samarcande" id="toc-entry-4">Mercredi 21 …</a></li></ul></div><a class="reference external image-reference" href="vignettes.jpg"><img alt="vignettes" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/vignettes.jpg" style="width: 100%;" /></a>
<p>Trois semaines en Ouzbékistan, puis au Kirghizstan… en train, bus,
marshrutka, cheval, et beaucoup à pied !</p>
<p>Photos et panoramas : Gaël | Récit, mise en page et retouches photos :
Emmanuelle</p>
<a class="reference external image-reference" href="map.jpg"><img alt="Carte voyage" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/central_asia/map.jpg" style="width: 100%;" /></a>
<div class="contents topic" id="table-des-matieres">
<p class="topic-title">Table des matières</p>
<ul class="simple">
<li><a class="reference internal" href="#samedi-17-mai-dimanche-18-mai-bourg-la-reine-bukhara" id="toc-entry-1">Samedi 17 mai - dimanche 18 mai : Bourg-la-Reine -> Bukhara</a></li>
<li><a class="reference internal" href="#lundi-19-mai-bukhara" id="toc-entry-2">Lundi 19 mai : Bukhara</a></li>
<li><a class="reference internal" href="#mardi-20-mai-bukhara-samarcande" id="toc-entry-3">Mardi 20 mai : Bukhara -> Samarcande</a></li>
<li><a class="reference internal" href="#mercredi-21-mai-samarcande" id="toc-entry-4">Mercredi 21 mai : Samarcande</a></li>
<li><a class="reference internal" href="#jeudi-22-mai-samarcande-tashkent" id="toc-entry-5">Jeudi 22 mai : Samarcande -> Tashkent</a></li>
</ul>
</div>
<hr class="docutils" />
<div id="Uzbekistan" class="align-center"></div>
<script type="text/javascript" src="http://comfort-works.com/news/wp-content/plugins/javascript-flickr-badge/javascript-flickr-badge.min.js"></script>
<script type="text/javascript">
jsFlickrBadge(document.getElementById('Uzbekistan'), {
// your Flickr ID (find it here)
flickrId: '66885349@N03',
// feed type. user, group, contacts, etc.
feed: 'user',
// optional comma-delimited tags to filter by, only used with 'user' feed
tags: 'Uzbekistan',
// number of rows to display
rows: 2,
// number of columns to display
columns: 8,
// size of each thumbnail (any bigger than 75 will cause pixelization)
size: 75,
// animation to use.
// one of: vscroll, random, vscroll, shuffle, zoom, scroll, flipX, flipY
animation: 'vscroll',
// seconds each animation takes
animationSpeed: 1,
// seconds between each animation
animationPause: 20
});
</script><div class="section" id="samedi-17-mai-dimanche-18-mai-bourg-la-reine-bukhara">
<h2><a class="toc-backref" href="#toc-entry-1">Samedi 17 mai - dimanche 18 mai : Bourg-la-Reine -> Bukhara</a></h2>
<p>C’est le premier jour de nos vacances, et nous pouvons même faire une
petite grasse matinée, notre vol ne partant qu’à qu’à 14h.
Notre vol Paris -> Tashkent passe par Moscou, et nous volons avec la
célèbre compagnie Aeroflot (dont nous n’aurons pas eu à nous plaindre
d’ailleurs, malgré sa réputation un peu douteuse). Ca parle russe à CDG,
c’est amusant de reconnaître quelques mots.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15249017307/in/set-72157648344396852"><img alt="Aéroport de Moscou" class="align-right" src="https://farm4.staticflickr.com/3932/15249017307_224dd8eb83_z_d.jpg" style="width: 40%;" /></a>
<p>Le premier vol se passe sans encombre, et nous avons quelques heures de
transit à Moscou. L’aéroport de Moscou est assez étonnant, pas très
agréable au demeurant : rien n’est prévu pour s’asseoir, du coup les gens
s’entassent dans les escaliers… Quand aux écrans indiquant les vols,
ils affichent pendant la majeure partie du temps des publicités qui
laissent de temps en temps quelques brefs instants aux informations sur
les vols… Sur le vol Moscou Tashkent nous sommes assis à côté d’une
jeune femme tadjik qui étudie aux Etats-Unis et parle fort bien anglais.
Elle semble regretter que nous n’allions pas au Tadjikistan, et nous
vante avec un bel enthousiasme les beautés de son pays. Sa narration des
difficultés d’être musulman aux Etats-Unis est également assez édifiante
: en gros, beaucoup de gens la prennent pour une terroriste
potentielle…</p>
<p>Arrivés à Tashkent, la capitale ouzbèque, nous commençons à être assez
fatigués malgré la petite sieste dans l’avion. Nous sommes légèrement
anxieux à l’idée des formalités de contrôle (l’Ouzbékistan est loin
d’être une démocratie), mais tout se passe bien. Gaël a même le temps de
changer des dollars pendant que je fais la queue dans la file de
contrôle des passeports. Le fonctionnaire n’est ni aimable ni désagréable
(le flic français de CDG était particulièrement peu aimable, donc on ne
va pas se plaindre…). Nos bagages sont bien arrivés : c’est un
soulagement ! Nous remplissons un formulaire pour la douane où nous
devons déclarer les devises que nous possédons. On nous a dit qu’au
moment de quitter l’Ouzbékistan il fallait pouvoir justifier l’argent
qu’on avait dépensé et avoir des quittances pour tout. Bon, on verra
bien…</p>
<p>Nous sortons de l’aéroport, il est 5h et le jour se lève. Une nuée de
chauffeurs de taxis s’abat sur nous pour nous proposer une course.
Heureusement, le Lonely Planet nous a donné une idée des prix
“raisonnables”. Nous pouvons donc négocier la course vers la gare à 4000
soms, soit trois fois moins que le prix qu’on nous proposait initialement
(10 US dollars). Notre taxi n’a visiblement pas de licence, comme une
grande partie des taxis, c’est une toute petite voiture plus très jeune.
Nous roulons dans d’immenses boulevards déserts, des rangées de tours se
teintent en rose dans l’aube naissante. C’est mon premier aperçu d’une
ville “post-soviétique” et l’impression est saisissante : toutes les rues
sont d’une largeur inimaginables pour une parisienne, et les immeubles
sont tous gigantesques. Nous arrivons à la gare, un monument
néo-classique dont les proportions colossales peuvent étonner… quand on
sait qu’il y passe au plus une quinzaine de trains par jour ! Comme c’est
très grand, nous avons un peu du mal à trouver un guichet pour acheter
des billets. Le guide est précieux et nous apprend qu’il faut se rendre à
un petit bâtiment juste à côté de la gare. Ce bâtiment est désert (il
est 5h30 !), à l’exception d’une femme qui fait le ménage. Elle a la
gentillesse d’aller chercher quelqu’un pour nous vendre des billets.
C’est l’occasion pour Gaël d’utiliser son russe, qui est bien utile !
Nous prenons deux billets pour Bukhara sur le train de 8h, le seul de la
journée (il y a aussi un train de nuit). Nous avons eu de la chance qu’il
reste des places : apparemment c’est loin d’être le cas tous les jours.
Bien contents d’avoir réglé ce point, nous regagnons l’immense hall, sous
le regard amusé des voyageurs locaux qui nous interpellent joyeusement.
Malgré l’heure matinale il y a déjà pas mal de monde dans le hall. C’est
l’occasion d’observer avec curiosité les belles robes colorées des
Ouzbeks, ou leurs visages forcément dépaysants. Un vendeur de boissons
dort les bras croisés sur son étalage, il ouvrira son commerce un peu
plus tard. Un petit chaton blanc trottine calmement dans la gare; un
gamin se précipite sur lui et commence à jouer à la poupée en le
baladant sur ses pattes arrières. L’animal se laisse faire avec un flegme
déconcertant ! La plupart des gens somnolent. Nous dormons aussi une
heure chacun à notre tour, pendant que l’autre surveille les sacs (une
précaution que je juge après coup bien superflue !). Nous montons à
l’heure dans le train, le wagon est fort confortable avec notamment
beaucoup de place pour les jambes, ou la possibilité d’avoir de l’eau
chaude pour se faire du thé : c’est une très bonne surprise. Une
télévision passe successivement un spectacle de danse traditionnelle,
puis une série de clips world-pop d’Asie Centrale, chantés en russe, qui
pour nous ont le charme de l’exotisme… Nous sommes les seuls touristes
du train, et on est donc un peu l’attraction ! Le voyage dure 7h30, c’est
l’occasion de dormir ou de somnoler durant une bonne partie du trajet :
ce train est beaucoup plus confortable que les avions de la nuit ! De
temps en temps nous jetons un coup d’oeil au paysage : c’est pour
l’essentiel une steppe désertique assez monotone, parsemée
d’installations industrielles en plus ou moins bon état.</p>
<hr class="docutils" />
<p>Nous arrivons à Bukhara. Je nous cause une petite frayeur en ne
retrouvant pas mon passeport, qui avait glissé entre les sièges pendant
que je dormais dans une position bizarre ! Dehors il fait bien chaud. Le
guide est parfait, une fois de plus, et nous donne même le numéro de la
marshrutka (un minibus) qui peut nous emmener au centre de la ville. Il
faut noter qu’en Ouzbékistan ou au Kirghistan, les gares ferroviaires ou
routières sont rarement dans le centre ville, et qu’il faut donc prendre
un transport supplémentaire pour s’y rendre. Nous partageons la
marshrutka avec des Ouzbeks et deux touristes japonais. Elle nous dépose
près de Lyabi-Hauz, la grande place de Bukhara. Nous nous rendons à un
B&B à 2 minutes de la place, le Nasruddin Navruz. Nous y sommes
accueillis par notre hôte, un homme affable au ventre imposant. La petite
chambre qu’il nous montre à l’étage nous va très bien : un grand lit,
plein de tissus colorés, et même une climatisation que nous utiliserons
peu. Le prix de dix dollars par personne et par nuit est tout à fait
raisonnable. La maison est organisée autour d’une grande cour intérieure,
et les chambres donnent sur une galerie qui court sur les côtés de la
cour. Nous redescendons dans la cour, où notre hôte nous sert un thé vert
bien agréable.</p>
<p>Nous sortons ensuite visiter la ville, un peu anesthésiés par la chaleur
ambiante et le manque de sommeil. Nous visitons d’abord la jolie place de
Lyabi-Haus, où se trouvent plusieurs belles médersas (des universités
coraniques). Bukhara a la couleur du sable : toutes les maison sont
ocre-beige qui répond magnifiquement au bleu permanent du ciel. Mais la
place ressemble à une oasis : des arbres vénérables offrent leur ombrage
autour d’un large bassin central. C’est donc un endroit très agréable
pour s’asseoir et prendre une bière ! C’est d’ailleurs de loin l’endroit
le plus touristique de la ville, et les serveurs des chaikhanas se
débrouillent très bien dans pas mal de langues.</p>
<p>Nous entrons dans une des médersas, qui sert maintenant de vitrine à
l’artisanat local. La plupart des médersas qui nous avons vues avaient
une architecture comparable : un haut fronton flanqué de minarets, une
très grande cour intérieure, et sur les côtés plusieurs étages de
cellules qui s’ouvrent sur des coursives, et la cour. Dans cette médersa,
les cellules du rez-de-chaussée et leurs arcades abritent des petits
magasins de tissus, de souvenirs ou encore de tapis. Une des cellules a
été transformée en petit musée, et une jeune femme nous propose d’y
entrer, pour un prix modeste. Le musée est consacré à un poète de
Bukhara, Sadriddin Aini, dont l’existence mouvementée au début du 20è
siècle traduit l’instabilité politique de la région à l’époque. Notre
guide nous raconte son histoire dans un russe que Gaël n’a pas de mal à
suivre ; comme elle ajoute pas mal de <em>body language</em> à son discours, je
comprends aussi l’essentiel sans traduction. C’est un plaisir que
d’écouter un guide qui désire autant communiquer !</p>
<p>Nous reprenons alors notre marche au hasard dans la ville. Dans cet
ancien carrefour de la route de la Soie, le centre ville semble avoir peu
changé depuis des siècles. Le sol des rues est en terre battue, les murs
de torchis ou les toits en tôle, beaucoup d’enfants jouent dans les
rues. Nous retournons boire une bière sur la place Lyabi-Hauz, et nous
sommes trop paresseux pour aller dîner ailleurs : nous commandons donc
des salades (qui sont amenées sur un grand plateau où on peut choisir,
c’est pratique !) et de succulents shashlyk (des brochettes de viande
marinées). Les bières ne sont d’ailleurs pas réservées aux touristes, car
les Ouzbeks (en grande majorité musulmans) en consomment également avec
plaisir.</p>
<p>De retour au B&B, nous retrouvons dans le patio la faune habituelle des
<em>backpackers</em>, des touristes un peu bourlingueurs qui voyagent avant tout
pour le plaisir de voyager. Quelques bouteilles de vodka circulent autour
de la table, l’ambiance est détendue. Mark, un géant roux britannique
fait l’éloge de l’hédonisme et du voyage. Il nous fait bien rire avec
cette affirmation décidée “I’m not a tourist: I’m a traveller !”. La
nuance est bien difficile à définir… Dimitri d’Andijon, un Russe au
sourire permanent, essaye de m’apprendre un peu de russe et se fait de
plus en plus tendre au fur et à mesure que la vodka diminue… Mais il
est bien tard, et nous laissons ces joyeux voyageurs refaire le monde
sans nous, pour aller nous coucher.</p>
</div>
<div class="section" id="lundi-19-mai-bukhara">
<h2><a class="toc-backref" href="#toc-entry-2">Lundi 19 mai : Bukhara</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15432376031/in/set-72157648344396852"><img alt="La mosquée de Kalon" class="align-center" src="https://farm6.staticflickr.com/5601/15432376031_9e182e28fb_b_d.jpg" style="width: 100%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435579155/in/set-72157648344396852"><img alt="La mosquée de Kalon" class="align-right" src="https://farm4.staticflickr.com/3929/15435579155_da45243f4e_z_d.jpg" style="width: 40%;" /></a>
<p>Après une longue nuit de sommeil, nous somme complètement reposés. La
journée commence par un énorme petit déjeuner dans la cour du B&B :
porridge de riz au lait, biscuits, pain <em>nan</em>, confitures diverses et
petites pommes. Nous partons ensuite nous balader tant que la
température est encore fraîche. Une charmante petite mosquée abrite le
musée des tapis, où de vénérables oeuvres d’art côtoient des réalisations
modernes offertes à la vente. Nous faisons ensuite un tour dans le bazaar
: il y a de belles étoffes colorées, de la vaisselle en métal ou en terre
cuite, des sacs en feutre multicolore qui me font bien envie… C’est
clairement un bazaar pour touristes par contre ! Comme on voyage léger en
vue des randos au Kirghistan, on se contente d’admirer ! Nous arrivons
ensuite à la spectaculaire place de Kalon, où une immense mosquée et une
medressa tout aussi impressionnante se font face. Mais c’est pour son
minaret de 47 m (kalon veut dire “grand” en tadjik) que la place est
célèbre. Le monument est un véritable tour de force architectural : il
date de 1127 a n’a jamais eu besoin de restauration depuis. Gaël et moi
ne pouvons nous empêcher d’ironiser sur le pauvre muezzin qui devait
monter toutes ces marches plusieurs fois par jour… mais c’est très
impressionnant ! Il paraît d’ailleurs que Gengis Kahn avait lui aussi été
impressionné lorsqu’il avait conquis Bukahra, et contrairement à son
habitude il avait ordonné de ne pas détruire le minaret… Nous visitons
la mosquée, dont le fronton s’orne d’un magnifique ensemble d’émaux
bleus qui mêlent décors géométriques et texte religieux. La mosquée a été
prévue pour accueillir 10000 personnes : c’est immense ! Gaël prend
méthodiquement des photos tout autour de lui, comptant les réunir ensuite
dans un grand panorama : c’est beaucoup trop grand pour être pris sur une
seule photo, même avec un grand angle ! Comme à d’autres endroits, le
bleu et le turquoise des coupoles se marient magnifiquement à l’ocre des
briques.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15432376491/in/set-72157648344396852"><img alt="La mosquée de Kalon" class="align-center" src="https://farm4.staticflickr.com/3927/15432376491_2eab799834_b_d.jpg" style="width: 100%;" /></a>
<p>Nous continuons ensuite dans le bazaar, puis nous arrivons à la
forteresse d’Ark en passant par un grand terrain vague brûlant sous le
soleil de midi. Ce serait plutôt l’heure de retourner faire la sieste,
mais la forteresse est à l’autre bout de la ville par rapport à notre B&B
et nous décidons de la visiter malgré la chaleur assez étouffante. La
forteresse est une ancienne ville fortifiée dont certaines parties
remontent au 5è siècle. Plusieurs musées historiques ont été aménagés
dans les salles de la forteresse. C’est très intéressant, mais la
succession des musées est assez fatigante. Au moment où nous allons
partir, nous cédons aux invitations insistante d’un jeune homme dont
l’oncle tient un magasin d’étoffes et de souvenirs. La visite du magasin
est intéressante, mais nous nous sentons assez vite un peu gênés comme
nous ne voulons rien acheter. Le jeune homme ne semble pas nous en tenir
rigueur, et nous discutons un moment : il est étudiant en génie
électrique et voudrait bien finir ses études à l’étranger (comme beaucoup
d’étudiants que nous rencontrerons, d’ailleurs).</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15249015887/in/set-72157648344396852"><img alt="La mosquée de Kalon" class="align-right" src="https://farm6.staticflickr.com/5601/15249015887_8e8ef4d602_z_d.jpg" style="width: 40%;" /></a>
<p>Sur le chemin du B&B nous nous arrêtons dans un petit café en plein air
pour manger une soupe à la viande, du pain et de la salade, et du thé :
de la nourriture simple mais savoureuse… et très bon marché ! Nous
rentrons ensuite pour nous reposer durant les heures les plus chaudes de
la journée. Puis nous reprenons notre marche dans Bukahra, sans autre but
que de découvrir une belle mosquée au coin d’une rue, un minaret bleu
turquoise, ou un bazaar animé. Nous poussons notre marche jusqu’à des
quartiers un peu moins touristiques. Nous tombons alors sur le bazaar
pour l’alimentation où se fournissent les locaux. Nous sommes fascinés
par les empilements de fruits et légumes, les tapis de fruits secs ou
les montagnes d’épices aux couleurs saturées. Sur le chemin du retour
nous passons devant une très belle medressa. Elle ne semble pas être
ouverte à la visite, mais un vieil homme sort de la medressa pour nous
inviter à entrer. C’est le gardien, et il nous fait faire le tour des
salles du rez-de-chaussée. Il nous invite ensuite à monter à l’étage,
puis sur les toits : il est trop vieux pour nous suivre mais nous
attendra en bas. Nous faisons donc un peu d’exploration d’une cellule à
l’autre, puis sur les toits où nous avons une vue très dégagée sur
Bukhara. Nous descendons discuter un peu avec le gardien : c’est
clairement un nostalgique du communisme et de plein emploi, qui se plaint
de la dureté de l’époque actuelle par rapport à la période soviétique.
Nous avons retrouvé cette attitude chez plusieurs personnes âgées, mais
les jeunes eux semblaient tous apprécier le mode de vie occidentalisé
(tout en restant très attachés aux traditions ouzbèques ou kirghizes).
Nous lui laissons quelques billets pour la visite : c’était bien sympa
d’avoir la medressa rien que pour nous ! Nos pas nous ramènent à Kalon,
encore plus belle dans la lumière du soir. Gaël prend donc de nouvelles
photos. Puis nous errons un peu dans la partie soviétique de la ville,
avec de grands boulevards, des terrains de sport un peu partout, beaucoup
de béton… Nous nous arrêtons pour dîner dans un petit restaurant où les
locaux dînent – bon signe ! Nous goûtons avec plaisir à de nouvelles
spécialités locales : les mantis, de gros raviolis fourrés de viande et
de légumes, et cuits à la vapeur, et les samsa, des feuilletés à la
viande, proches des samosa indiens. Sans oublier une bière ! Pour
commander, nous sommes aidés par un gamin de 16 ans très débrouillard qui
fait des allez-retour avec la cuisine pour leur expliquer ce qu’on veut.
Il nous explique qu’il veut absolument améliorer son anglais car il veut
aller travailler en Angleterre. Il y serait d’ailleurs déjà s’il n’était
pas aussi dur d’obtenir un visa. Assez vite, il va chercher son cousin,
un garçon au visage doux et un peu naïf, bien différent de notre “guide”
espiègle, qui l’envoie faire les commandes pendant que lui reste
discuter avec nous !</p>
</div>
<div class="section" id="mardi-20-mai-bukhara-samarcande">
<h2><a class="toc-backref" href="#toc-entry-3">Mardi 20 mai : Bukhara -> Samarcande</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248815699/in/set-72157648344396852"><img alt="Le mausolée de Timur" class="align-center" src="https://farm4.staticflickr.com/3931/15248815699_58bd86f14f_b_d.jpg" style="width: 100%;" /></a>
<p>La journée commence à nouveau par un somptueux petit déjeuner, avec cette
fois-ci des oeufs sur le plat à la coriandre et à la ciboulette, qui sont
une pure merveille. Puis nous marchons jusqu’à l’arrêt de marshrutka pour
aller jusqu’à la gare routière. La marshrutka que nous hélons a l’air
complètement pleine, mais les gens se tassent et nous pouvons même nous
asseoir sans problèmes ! Arrivés à la gare, nous avons une mauvaise
surprise : il s’agit d’une petite gare locale et il n’y a pas de bus pour
Samarcande. Un moment, nous avons peur que les chauffeurs de taxi nous
racontent des salades pour obtenir une course lucrative, mais les locaux
aussi prennent le taxi jusqu’à la ville suivante pour prendre le bus.
Nous prenons donc un taxi avec deux autres personnes pour Gizhduvan où
se trouve la gare routière avec le bus pour Samarcande. Nous roulons sur
la seule autoroute du pays : une 2x2 voies avec sur le côté une voie en
terre battue pour les ânes et les chevaux. Encore une fois le trajet est
rythmé par la radio locale qui passe des chansons entêtantes et
rythmées.</p>
<p>Arrivés à Gizhduvan, nous montons tout de suite dans un bus à moitié
plein pour Samarcande. Le chauffeur attend que son car se remplisse pour
partir. En attendant, des vendeurs montent proposer pains et feuilletés,
puis un mendiant aveugle passe recevoir les aumônes des passagers. Au
bout d’une petite heure, le car est complètement bondé, il y a plein de
gens debout dans le couloir au milieu des bagages, mais nous avons la
chance d’être assis. Il y a derrière nous deux femmes avec qui nous
engageons la conversation, et un bébé qui a un peu trop l’envie de nous
tirer les cheveux. Nous voyageons à nouveau sur l’autoroute centrale.
Le bus fait de fréquents arrêts pour déposer des gens qui sont allés
faire leurs courses à Gizhduvan, surtout des femmes. Au bout d’un moment,
presque tout le monde est assis ! Il fait bien chaud, le car est bruyant
et nous sommes assez serrés, mais ce n’est pas désagréable non plus.
C’est surtout très amusant de voyager comme les gens du pays ! Il va sans
dire que nous sommes les seuls touristes du car !</p>
<p>Le voyage jusqu’à Samarcande dure 4 bonnes heures. Nous sommes déposés à
la gare routière, un peu à l’extérieur de la ville. Comme nous ne voyons
pas de marshrutka nous nous laissons convaincre par un taxi qui nous a
repérés dès notre sortie du bus. Il aurait en fait suffit de poursuivre
la route après un tournant pour trouver les marshrutkas ! Samarcande est
beaucoup plus grande que Bukhara ; le taxi rejoint le centre ville par de
grands boulevards modernes bordés ça et là de mosquées qui attirent notre
curiosité. Le chauffeur connaît visiblement mal le nom des rues et a du
mal a trouver le B&B où nous voulons descendre. Nous finissons par le
convaincre de nous déposer sur une grande place où nous pouvons nous
repérer et continuer à pied ; le B&B est en fait à 10 minutes à pied.</p>
<p>Le B&B est tenu par un homme au physique européen, et une femme asiatique
qui parlent tous les deux parfaitement anglais. Le B&B est plus chic que
ne l’indique le guide (et les prix ont également montés) : c’est
maintenant un <em>boutique hotel</em>, avec des chambres somptueusement décorées
de tapis et d’objets d’art de l’artisanat local. Nous avons même une
chambre avec salle de bain, un luxe auquel nous ne nous attendions pas !
J’apprécie beaucoup le tchai qu’on nous sert tout de suite pour nous
rafraîchir, avec des biscuits et des confitures, mais Gaël commence à en
avoir assez du thé ! Nous nous reposons un moment dans le magnifique
jardin recouvert d’une tonnelle rafraîchissante, puis nous partons à la
découverte de Samarcande.</p>
<p>Nous commençons la visite par le mausolée d’Amir Timur, juste à côté de
l’hôtel. Timur, ou Tamerlan en français, était un conquérant passablement
sanguinaire du 14è siècle, qui avait par exemple pour habitude de
massacrer tous les hommes d’une ville conquise, sauf les artisans qu’il
ramenait dans sa capitale de Samarcande. Tant mieux pour Samarcande, dont
l’essor date de cette période… Le mausolée de Timur est un ensemble
magnifique de briques et d’émaux colorés, bleus pour la plupart.
L’intérieur est peut-être encore plus impressionnant : dans la salle où
Timur, ses fils et ses petits-fils sont enterrés, les murs et la coupole
sont recouverts de dorures dentelées extrêmement finement.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248868140/in/set-72157648344396852"><img alt="Le mausolée de Timur" class="align-right" src="https://farm4.staticflickr.com/3936/15248868140_c9836d1c7e_z_d.jpg" style="width: 40%;" /></a>
<p>Nous nous dirigeons ensuite vers le Régistan, un ensemble de trois
médressas gigantesques disposées à angle droit autour d’une grande place
centrale. Le Régistan est l’attraction principale de Samarcande, et c’est
en effet très impressionnant. La plus ancienne des médressas a été bâtie
par Ulugh Beg, petit-fils de Tamerlan et grand scientifique, qui y
enseignait entre autres l’astronomie et la théologie. Les deux autres
médressas ont cherché à copier l’architecture de la première. Les briques
paraissent rosées dans la lumière du soir, c’est très joli. Je m’assieds
sur un banc pendant que Gaël cherche un premier plan pour ses photos du
Régistan. Il en trouve un très pittoresque en la personne d’un jardinier
qui tond le gazon… à la faux ! Pendant ce temps, un garçon d’environ 18
ans vient s’asseoir en face et se décide timidement à m’adresser la
parole en anglais. Il étudie pour devenir pilote et cherche à la fois à
pratiquer son anglais et satisfaire sa curiosité concernant la vie en
Occident : encore une rencontre très sympathique.</p>
<p>Nous faisons le tour du Registan, en réservant sa visite pour le
lendemain. L’entrée de la place est gardée par des policiers en uniforme,
il faut des billets pour entrer. Mais nous sommes rapidement repérés par
ces policiers qui nous proposent de nous faire monter dans un minaret de
la medressa d’Ulugh Beg : c’est une des manières “officielles” d’arrondir
les fins de mois des policiers, indique le guide, qui donne aussi les
prix “raisonnables” qu’on peut accepter. Nous négocions donc, puis nous
montons au minaret. C’est amusant, les escaliers sont obscurs et un peu
délabrés, et en haut la vue est jolie, sans être exceptionnelle (les
monuments sont plus impressionnants de près, je trouve !). Quand nous
redescendons, la grille à l’entrée du minaret est fermée ; nous faisons
un peu de bruit pour qu’on vienne nous chercher. Un jeune guide arrive,
il parle parfaitement français et fait semblant de chercher la bonne clé
pendant quelques minutes en riant de sa bonne blague. Nous rions avec
lui, et nous continuons à discuter une fois sortis de la tour. Il est
étudiant en français, et travaille comme interprète et guide pour des
voyages organisés très différents, de la visite culturelle de Samarcande
au trek dans le montagnes tadjikes.</p>
<p>Nous nous dirigeons après vers la partie moderne de la ville pour
chercher un restaurant. A part un resto italien qui ne nous tente guère,
nous ne trouvons pas grand chose. Nous finissons par prendre un kebab et
une bière dans un fast-food en plein air où beaucoup de jeunes semblent
se retrouver. Un long jeune homme maigre passe plusieurs fois à côté de
notre table, hésite, puis finit par engager la conversation. Il voudrait
étudier la médecine en Europe et cherche des renseignements sur le
système universitaire français, et les moyens d’avoir une bourse pour y
étudier. Nous discutons un moment, et je regrette de ne pas avoir plus
de détails à lui donner !</p>
</div>
<div class="section" id="mercredi-21-mai-samarcande">
<h2><a class="toc-backref" href="#toc-entry-4">Mercredi 21 mai : Samarcande</a></h2>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15432380001/in/set-72157648344396852"><img alt="Le Registan" class="align-center" src="https://farm6.staticflickr.com/5598/15432380001_ce7214544e_b_d.jpg" style="width: 100%;" /></a>
<p>Encore un petit déjeuner somptueux : yaourt épais, confiture de roses,
pancakes, galettes de pommes de terre… Puis on reprend la visite de
Samarcande. On fait un petit tour dans un café Internet (la seule fois
des vacances) : la café est rempli exclusivement d’enfants qui jouent aux
jeux vidéo.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435230252/in/set-72157648344396852"><img alt="Le Registan" class="align-right" src="https://farm3.staticflickr.com/2947/15435230252_1102a9551c_z_d.jpg" style="width: 40%;" /></a>
<p>Pour visiter le Régistan il y a deux possibilités : soit on est un
touriste et on achète un billet relativement cher (pour un Occidental),
soit on est du coin et on passe en glissant un peu d’argent aux policiers
à l’entrée. Le billet est de toute façon trop cher pour les salaires
locaux, et il y a beaucoup de touristes d’Asie Centrale (beaucoup plus
nombreux que les Occidentaux) : c’est un système qui fonctionne assez
bien. Nous visitons les trois médressas dans l’ordre chronologique. Ca
devait être chouette d’étudier dans des universités aussi belles ! Les
jardins à l’intérieur des médressas contribuent beaucoup à l’harmonie
générale. Dans la médressa d’Ulugh Beg, un groupe de femmes cueille des
fruits sur les arbres. La plus récente médressa accueille des boutiques
d’artisanat locaux. Nous croisons plusieurs fois les mêmes touristes qui
logent au même B&B : en un sens, c’est rassurant de se dire qu’il y a peu
de touristes et qu’on rencontre toujours les mêmes !</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248985718/in/set-72157648344396852"><img alt="BiBi Yakim" class="align-right" src="https://farm3.staticflickr.com/2950/15248985718_305de8f8f0_z_d.jpg" style="width: 30%;" /></a>
<p>Nous remontons ensuite vers le nord de la ville, vers la magnifique
mosquée de Bibi Yakim. Mais d’abord, nous allons au bazaar acheter de
quoi manger : samsa, oeuf dur pâné, et pour finir, une pâtisserie pour
Gaël. Nous marchons un peu dans la bazaar, très animé et coloré. Puis
nous visitons Bibi Yakim. C’est une mosquée aux proportions imposantes,
très verticale. La coupole aux nuances bleues variées est magnifique,
mais c’est encore l’intérieur qui me plaît le plus : on peut entrer dans
certaines salles à moitié en ruines, où des oiseaux tournoient très haut,
sous la coupole. Ca donne une petite atmosphère d’Indiana Jones…</p>
<p>Quand nous sortons, Gaël décide qu’il a encore faim et qu’il faut
retourner au bazaar (en fait ce n’est pas vrai, il voulait juste prendre
des photos :D). Nous sommes interpellés par les vendeuses de pâtisserie
que nous avions rencontrées lors de notre premier passage : c’est la fin
du marché, et elles nous proposent de venir boire un coup avec elles.
Nous nous joignons donc à elles, au milieu des étals de sucrerie. On nous
sert tout de suite de grandes lampées de vodka qu’il faut boire cul-sec.
Je transige en buvant rapidement de petites gorgées, puis en finissant
mon verre dans un grand geste théâtral qui suscite des acclamations (une
technique qui resservira !). Il y a aussi beaucoup de nourriture : de la
salade, des sucreries, et du plov, le plat national à base de riz,
légumes et mouton. Une des filles vient d’avoir un enfant, du coup les
marchandes font une fête pour célébrer l’événement. Mon verre ne reste
jamais vide, les filles le remplissent de plein de boissons diverses, pas
toutes alcoolisées heureusement. Mon vocabulaire russe est très limité
pour communiquer, ça ne nous empêche pas de bien rigoler pendant que Gaël
fait plus de conversation. Mes lunettes de soleil entre autres remportent
un franc succès et font le tour du groupe. La patronne du groupe a
environ 45 ans, c’est elle qui nous a invités. Il y a aussi une vieille
femme, sinon les autres filles ont entre 20 et 30 ans, et elles ont
toutes du caractère ! L’une d’entre elles essaye d’apprendre à Gaël
comment manger le plov avec les doigts : on roule le riz, la viande et
les légumes dans la graisse de mouton, ça fait des petites boulettes qui
sont censées tenir ensemble et se manger facilement. Pas facile, surtout
sans salir l’appareil photo ! Elles sont très curieuses à propos de la
vie en France, veulent savoir si on habite en ville, si on a une voiture,
où on travaille… A un moment, un marchand d’un stand voisin met en
marche sa sono. Les filles me disent qu’il faut danser. Je me lève,
pensant qu’on va tous danser et que je passerais à peu près inaperçue.
Mais non : elles ne veulent pas danser, mais me voir danser ! Pour
améliorer la qualité du spectacle, elles me montrent quand même quelques
mouvements de danse orientale. Sinon, c’est le fou rire généralisé… Le
plus drôle est quand un morceau de musique techno succède à la musique
orientale : la plus vieille du groupe se lève alors et se met à danser
une techno tout à fait orthodoxe… Cette rencontre très chaleureuse
restera parmi mes meilleurs souvenirs des vacances : une demi-heure de
fou rire, de langage des signes et de russe massacré (on peut en raconter
des choses avec 20 mots et des sourires !), sans oublier la vodka…</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248982888/in/set-72157648344396852"><img alt="plov" class="align-right" src="https://farm6.staticflickr.com/5598/15248982888_d0a63951c0_z_d.jpg" style="width: 38%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15248822589/in/set-72157648344396852"><img alt="On the Bazar" class="align-right" src="https://farm4.staticflickr.com/3935/15248822589_a5480f2de4_z_d.jpg" style="width: 32%;" /></a>
<div style="clear: both;"></div><p>Nous repartons de joyeuse humeur vers l’allée des tombeaux, un lieu sacré
de Samarcande où des mausolées du 13è et 14è siècle sont alignés le long
d’une magnifique allée. Beaucoup de touristes Ouzbèques viennent visiter
ce lieu : plusieurs nous demandent d’ailleurs de poser avec eux pour
leurs photos souvenir ! Chaque mausolée est une petite merveille
recouverte de faïences colorées, et l’ensemble est saisissant.</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15442377901/in/set-72157648344396852"><img alt="Alley des tombeaux" class="align-right" src="https://farm4.staticflickr.com/3935/15442377901_293fa9c988_z_d.jpg" style="width: 32%;" /></a>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15445553775/in/set-72157648344396852"><img alt="Alley des tombeaux" class="align-right" src="https://farm4.staticflickr.com/3928/15445553775_5ffc60d99c_z_d.jpg" style="width: 32%;" /></a>
<div style="clear: both;"></div><p>Nous retournons à notre hôtel par l’ancien quartier juif, pauvre et assez
délabré. Nous faisons un bref passage par le B&B pour savoir si Dennis,
notre hôte, a pu nous prendre des billets de train, puis nous allons
siroter une bière devant le mausolée de Timur, qui se colore doucement en
rose à mesure que l’heure avance. Malgré les conseils de Dennis, nous ne
cherchons un restaurant très authentique pour dîner, et nous nous
contentons du petit café en face du Régistan. C’est forcément là que
tous les touristes échouent, mais la nourriture est bonne et il y a
beaucoup plus de locaux que de touristes qui y mangent.</p>
</div>
<div class="section" id="jeudi-22-mai-samarcande-tashkent">
<h2><a class="toc-backref" href="#toc-entry-5">Jeudi 22 mai : Samarcande -> Tashkent</a></h2>
<p>Nous n’avons pas pu avoir de billets pour le train express reliant
Samarcande à Tashkent, nous allons donc prendre le train plus lent qui
part à 7h. Nous nous levons à 6h, notre hôtesse a insisté pour nous
préparer un petit déjeuner malgré l’heure. Le jour est déjà levé, et il
fait divinement frais dans le jardin pendant que nous dégustons notre
thé. Puis Dennis nous emmène en taxi à la gare : il doit parfois avoir
affaire à des touristes pas très débrouillards, car il insiste pour nous
conduire jusqu’à notre place dans le wagon ! Le trajet dure 6h, contre
4h pour le train rapide. Les grandes banquettes rouges du wagon sont un
peu dures, mais il y a beaucoup de place. Comme dans tous ces voyages où
on a le temps, nous parlons beaucoup avec nos voisins : Rita, une dame
russe très apprêtée et maquillée, Aziz, un grand Ouzbek brun, et enfin un
policier très jeune qui sourit tout le temps. Encore une fois, ils sont
très curieux et veulent tout savoir sur notre mode de vie, discutent
entre eux du prix des choses après nos réponses… Rita sort un sac de
bonbons et insiste pour que nous en prenions : je n’arrive pas à refuser,
malheureusement, car il y a sûrement de l’arachide dans l’un d’entre eux
et j’ai une réaction allergique. Je fais alors semblant de dormir un
moment pour ne plus avoir à manger de bonbons, mais je passe le reste du
voyage très fatiguée, avec un fort mal de ventre. Le guide de
conversation russe/anglais que nous avons sortis remporte beaucoup de
succès, et fait le tour de nos voisins qui cherchent des mots anglais
pour parler avec nous.</p>
<p>Arrivés à Tashkent, nous nous rendons tout de suite dans l’hôtel situé à
l’intérieur des bâtiments de la gare. Le confort est spartiate (pas de
douche, juste un lavabo à l’étage !) mais les prix sont imbattables :
8000 sums (4 euros) pour la nuit ! Il y a un gros cafard qui se balade
sur la moquette de notre chambre, mais sinon c’est tout à fait propre.</p>
<p>On part se balader dans Tashkent. C’est une ville à l’architecture
clairement soviétique, qui a néanmoins bien vieilli. Dans les rues les
gens sont majoritairement habillés à l’occidentale, contrairement à
Bukhara ou Samarcande. Une majorité des filles d’origine russe porte même
des jupes extrêmement courtes ! La ville est très verte, avec de nombreux
parcs et beaucoup d’arbres dans les rues : c’est agréable. Par contre, il
n’y a pas de petites rues, seulement d’immenses boulevards un peu
fatigants à arpenter, surtout en plein soleil ! Nous changeons de
l’argent dans une banque extrêmement moderne, puis nous nous reposons
quelques instants dans un parc, devant une martiale statue équestre de
Timur. A la sortie du parc, il y a une exposition de peintures en plein
air, posées sur le sol. Nous nous arrêtons un peu pour les regarder, et
pendant ce temps un jeune homme Uzbek se décide à nous aborder par un
“You look like foreigners…”. Indeed ! Il est étudiant en économie et
voudrait faire un master à l’étranger, mais se demande quel est le
meilleur pays pour faire ça. Nous discutons un bon moment, debout au
milieu des peintures. Nous finissons par lui conseiller de chercher avant
tout un pays où il pourra avoir une bourse pour ses études, vu le coût de
la vie en Occident, et qui est accueillant pour les étrangers (notamment
pour avoir un visa de travail).</p>
<a class="reference external image-reference" href="https://www.flickr.com/photos/gaelvaroquaux/15435590075/in/set-72157648344396852"><img alt="Le bazaar de Tashkent" class="align-right" src="https://farm4.staticflickr.com/3928/15435590075_9a1bec6247_z_d.jpg" style="width: 40%;" /></a>
<p>Nous continuons notre traversée de Tashkent. Nous passons à travers un
immense parc où les enfants se baignent dans un canal. Nous mangeons un
morceau dans un petit café, puis nous poussons jusqu’au grand bazaar de
la ville, encore en pleine ébullition à 18h. Contrairement à ce qu’on
pourrait croire, les bazaars sont des endroits très organisés, celui de
Tashkent en particulier : dans les escaliers, des petits rails ont été
installés pour permettre de monter les charrettes plus facilement, on
trouve partout des vendeurs de sacs et de cabas pour mettre ses courses,
etc. Il se fait tard et nous rebroussons chemin. Le retour est bien long
: on a traversé tout Tashkent ! En chemin nous nous arrêtons dans un
petit boui-boui pour manger des shashlyks. Le café est tenu par trois
filles russes : l’une d’entre elles prend les commandes et sert, la
deuxième prépare les bières, et la troisième reste tranquillement assise
sur sa chaise à faire des commentaires sur les passants !</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last"><strong>Suite du voyage</strong>: <a class="reference external" href="asie_centrale_kyrgyzstan.html">la suite récit, sur la page du Kyrgyzstan.</a></p>
</div>
</div>
Update on my life2008-05-05T09:35:00+02:002008-05-05T09:35:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-05-05:/personnal/update-on-my-life.html<p>I am currently changing jobs and changing countries. This is why I have
been really bad at dealing with questions on the mailing-lists,
bug-reports or feature requests.</p>
<div class="section" id="before">
<h2>Before</h2>
<p>So far I have been working as a physicist, doing atomic physics (Bose
Einstein Condensation). I studied quantum physics, mostly theory, and …</p></div><p>I am currently changing jobs and changing countries. This is why I have
been really bad at dealing with questions on the mailing-lists,
bug-reports or feature requests.</p>
<div class="section" id="before">
<h2>Before</h2>
<p>So far I have been working as a physicist, doing atomic physics (Bose
Einstein Condensation). I studied quantum physics, mostly theory, and I
did a PhD in an experimental lab, building a couple of experiments on
Bose Einstein Condensation and atom interferometry. After this, I moved
to Florence to do a post-doc also on a BEC experiment.</p>
<div class="figure">
<a class="reference external image-reference" href="attachments/gregor_in_lab.jpg"><img alt="" src="attachments/gregor_in_lab.jpg" style="width: 66%;" /></a>
<p class="caption">A colleague working on the experiment in Florence</p>
</div>
<p>This kind of work is very experimental. These experiments are monsters
that you have to keep alive doing a lot of homemade mechanics, optics,
and electronics. I thought I would love that, because I used to like
working with my hands, but I grew tired of it. I wanted to work more
with abstractions. And in addition I am computer geek, the parts of my
job I preferred were related to computers.</p>
</div>
<div class="section" id="this-summer">
<h2>This summer</h2>
<p>My contract has ended at the end of April, and I have not renewed it. I
was missing my girlfriend and wanted to come back to Paris. So now I am
jobless, living at the expense of my girlfriend. I decided to take some
time without a job, as I have the feeling I have been working without
stopping for the last few years, not having time to travel and visit the
world as I like to. We are planning a three weeks trip to Uzbekistan and
Kyrgyzstan in two weeks.</p>
<p>After this i am going to devote my summer to hacking. The big news is
that I am going to be going to the states. I will spend most of my time
in Austin, working for <a class="reference external" href="http://www.enthought.com">Enthought</a>. I am very excited about this, as I
see this as the occasion to learn more about building scientific GUIs
with Python. Building usable scientific programs is something that I am
passionate about. I will also spend some time at Berkeley, with Fernando
Perez, hopefully to work on Ipython1. I need to thank Enthought for
making this possible for me, as they are providing the money. With some
luck, this summer I will be productive on the free software side.</p>
<p>Of course right now I am battling with moving houses, fighting for
visas, trying to fall back on my feet and organize the summer. I still
don’t have my visa for the states, and it is making me nervous. I would
really hate to have to cancel my trip to Kyrgyzstan because of visa
problems with the states: I take time off work, I expect to spend it
enjoying myself, and not waiting for visas.</p>
</div>
<div class="section" id="the-future">
<h2>The future</h2>
<p>So I am quitting atomic physics. I am starting a new adventure in
something totally new for me. Starting from October, I will be working
with JB Poline and Bertrand Thirion, at <a class="reference external" href="http://www-dsv.cea.fr/neurospin/">Neurospin</a>, on neuroimaging.
This work is mostly data processing, even though it has a lot of
interplay with the physics of NMR. This is something very new for me and
I will have to discover a new field. The good news is that a lot of the
work is centered on computers, and one of the core technologies used at
Neurospin is Python.</p>
</div>
Docs using Sphinx2008-04-28T09:10:00+02:002008-04-28T09:10:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-04-28:/programming/docs-using-sphinx.html<p>After Ipython and Sympy, Mayavi is now using <a class="reference external" href="http://sphinx.pocoo.org/">sphinx</a> to build its
docs. Sphinx is very neat because it allows for high quality pdf and
html from the same restructured text source. The killer feature is that
the resulting html pages have a builtin search that works with
javascript, and …</p><p>After Ipython and Sympy, Mayavi is now using <a class="reference external" href="http://sphinx.pocoo.org/">sphinx</a> to build its
docs. Sphinx is very neat because it allows for high quality pdf and
html from the same restructured text source. The killer feature is that
the resulting html pages have a builtin search that works with
javascript, and thus works on the client without the need of a server.</p>
<p>In addition, the developer is very reactive and dedicated to making
sphinx versatile-enough to generate high-quality docs for many packages.
As a result many Python projects are switching to sphinx. First Python
itself (that’s what sphinx was created for), but now more and more. It
seems that zope is even considering it. One great side effect is that
documentation for different Python modules will be consistent, with the
same look and feel (although you can tweak sphinx output if you want).</p>
<p>We don’t have a server serving the html docs yet (it is planned, we just
need a bit of time), but you can check out the pdf generated <a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/branches/enthought.mayavi_2.1.2/docs/build/latex/mayavi_user_guide.pdf?format=raw">here</a>.</p>
Of packaging, installation and dependencies2008-04-12T15:52:00+02:002008-04-12T15:52:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-04-12:/programming/of-packaging-installation-and-dependencies.html<p>I have been struggling for the last few days trying to understand the
issues behind packaging and installing the Enthought Tool Suite. I think
have been making progress, though only in my head, no actual code or
packages so far are terribly satisfying.</p>
<div class="section" id="the-problem">
<h2>The problem</h2>
<p>If you are developing a …</p></div><p>I have been struggling for the last few days trying to understand the
issues behind packaging and installing the Enthought Tool Suite. I think
have been making progress, though only in my head, no actual code or
packages so far are terribly satisfying.</p>
<div class="section" id="the-problem">
<h2>The problem</h2>
<p>If you are developing a Python-only program, with only dependencies on
the standard library, you have no problems with packaging. You can ship
tarballs, MSI installer, eggs, … all this works.</p>
<p>However, if you want to develop a rich program that provides many
features in a closely integrated and consistent way to the user, you
will have to depend on external packages. I know that many projects work
around this by including the external dependencies inside the project,
or simply reinventing the wheel. Well this does not scale. We cannot
expect to develop a major scientific tool and community this way. Reuse
is the key to scalability, in my opinion. Thus comes the problem, how to
we ship our program?</p>
<p>The problem can be very well seen with the <a class="reference external" href="http://code.enthought.com/ets/">Enthought Tool Suite</a>
(ETS). The ETS is a suite of many different packages, all pretty much
geared towards building <strong>interactive</strong> scientific application. In
house, Enthought, the company (disclaimer: I do not work for Enthought)
uses these packages to develop domain-specific applications for
customers. They have broken up the suite in a set of small packages, to
enable assembling applications by requiring only the features you need.
This is important because if you want to use ETS’s 3D plotting package
(TVTK or Mayavi), but you want to stick with MatPlotLib to do 2D
plotting, and not use Chaco, you should be able to download only what
you need.</p>
<p>As a result the ETS is made of a set of interdependent packages. Maybe
they went a bit too far in the modularity, and there are almost 50
packages. The dependency graph looks like this:</p>
<p><a class="reference external image-reference" href="attachments/ets_deps1.png"><img alt="image0" src="attachments/ets_deps1.png" /></a></p>
<p>Just to reassure you, the next version of the ETS has a much reduced
number of packages, just because some packages where grouped, and the
dependency graph indeed is sane:</p>
<p><a class="reference external image-reference" href="attachments/ets3_deps.png"><img alt="image1" src="attachments/ets3_deps.png" /></a></p>
<p>As you can see, there is a complex dependency graph. So how do you ship
this to the user? Another problem that should not be underestimated is:
how do you make it easy for people who distribute your projects to
package this?</p>
</div>
<div class="section" id="setuptools">
<h2>Setuptools</h2>
<p>Python has no good answer for this problem, but <a class="reference external" href="http://peak.telecommunity.com/DevCenter/setuptools">setuptools</a> do go part
of the way. Dependencies in the ETS are declared using setuptools, and
installing the ETS strongly relies on setuptools.</p>
<p>Setuptools provides a way of automatically downloading dependencies.
However, it is not a full packaging system replacement. The reason I say
this is that it does not have the knowledge of a dependency graph, it
just downloads packages, introspects them to find their dependencies,
and recursively tries to satisfy them by downloading more. Phillip J.
Eby (the author of setuptools) has been quite clear that he does not
want to write an APT replacement, tough people keep getting it wrong and
making the equation “easy_install = apt for Python” (IMHO this is due
to bad communication on setuptools webpage).</p>
<p>Moreover, setuptools does not provide an easy to use API to extract all
the information it has about packages, dependencies, and download URLs.
It is thus not trivial to plug packages shipped with setuptools in an
other package manager like rpm or apt. This is why bothers me most,
because this is strongly limiting the exposure the ETS is getting in
distributions (whether they be Linux distributions, or scientific
computing “superpacks”). Recently I have had discussions with somebody
on how to ship Mayavi in a monolithic distribution he has developed. He
agreed to ship setuptools with the distribution, so now I need to give
him a list of eggs to provide. There is no obvious way to get this list
using setuptools (insert here big big rant). So I thought that an option
was to install Mayavi in a virtual environment to trac the eggs added,
and use this information. However, this person’s internet access was
possible only by login on dumbed-down servers for security reasons. So
we hit a wall. And for me this wall is a wall we keep hitting with
setuptools: setuptools does everything for you, the download, the
building the install. It does have flags to control these processes, but
it does not expose the information you need to do this without using it.
I actually think the reason it does not expose this information is that
it does not know it a priori. Looking at the code it does seem so. In
addition, the structure of the packages make it hard to do.</p>
</div>
<div class="section" id="from-packages-to-repositories">
<h2>From packages to repositories</h2>
<p>On the other side, Dave Peterson, at Enthought, has been working on a
tool to allow checking out of the ETS SVN only the projects you are
interested in. I played a bit with it, and modified it to generate the
dependency graphs. I quickly found out that I actually like this tool
much more than setuptools, even though it was pretty much using the same
concepts. It took me a while to understand what I like about the tool.
It is that it uses a map file to gather all the package and dependency
information. As a result, it has the equivalent of a dependency graph.
This makes it possible to do the operations I am interested in, eg
listing all the packages required for installing a given project without
actually downloading them.</p>
<p>The reason this is possible is that with the ETS we are not dealing with
an open set of packages, like PyPI, in which packages can come and go,
and no consistency is enforced. We are dealing with one suite of
multiple projects that are made to work with each other. The base entity
is thus a project set, on which we can make a “project map”.</p>
<p>What Dave has done works fantastically for development, I would like to
push it further for distribution. What we expose to the user can now be
a repository, in the sens of APT: a set of packages with consistent
inter-dependencies, and a way of retrieving easily this information. The
difference between the two, and the implications of the difference, is
not something I had clearly in my mind in the beginning, but it is
becoming clearer that having a repository with a project map gives a lot
of added value for distributing. I’ll see if I can reuse Dave’s work to
build such a tool, but do not hold your breath, I am not willingly in
the business of packaging, and will probably not spend enough time on
this to make it a good tool.</p>
</div>
Objects, modules and Traits and Envisage2008-04-05T13:19:00+02:002008-04-05T13:19:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-04-05:/programming/objects-modules-and-traits-and-envisage.html<p>I have been reading an article about a new language paradigm (<a class="reference external" href="http://users.encs.concordia.ca/~grogono/Erasmus/E01.pdf">Erasmus,
a modular language for concurrent programming</a>). The authors discuss
the limitations of objects in terms of modularity. To sum up their point
(and most probably distort it completely), the limitations with objects
comes from the fact that you …</p><p>I have been reading an article about a new language paradigm (<a class="reference external" href="http://users.encs.concordia.ca/~grogono/Erasmus/E01.pdf">Erasmus,
a modular language for concurrent programming</a>). The authors discuss
the limitations of objects in terms of modularity. To sum up their point
(and most probably distort it completely), the limitations with objects
comes from the fact that you can’t be sure what is modifying what:
suppose you have a method <em>foo</em> of an object <em>bar</em> that you call in a
method of an object <em>baz</em>, you cannot be sure that this method hasn’t
modified private attributes of your object <em>baz</em>, as <em>foo</em> could have
called a method of your object. This does happen in large code bases. Of
course, best practice tries to reduce this to a minimum, but this
reduces modularity, and thus limits both code reuse and concurrency (as
side effects are not well controlled).</p>
<p>Erasmus’s solution to is adopt a new container, that they call modules
rather than objects, and that are based on message passing rather than
method calls. These modules live in separate processes and can
themselves be made of more conventional code (I am extrapolating a bit
from the original article here).</p>
<p>This strikes me as being related to a pattern that I see more and more
in my code that uses Traits. The objects deriving from HasTraits have a
very easy and cheap way of coupling callbacks to the modification of
their attributes. This induces a programming style know as reactive
programming that is entirely callback-driven. In addition, this is a
nice way of ensuring that the internal state of an object is always
consistent. This is a first step to message passing and decoupling: you
no longer call methods, you just set attributes and let the object do
the rest. The limitation of this model in a large code base is that you
have to carry around references to the objects you are interested about,
and their attributes. Traits has patterns to help you do this
(delegation, namely), but it is still a limitation.</p>
<p>This is where the <a class="reference external" href="https://svn.enthought.com/enthought/wiki/EnvisageThree">Envisage framework</a> comes into play. Envisage
introduces the notion of plugins which provide extension points. These
extension points are special traits attributes that are published in a
registry (which can be application-wide, or not, in Envisage3). You can
query the registry to retrieve these extension points and contribute to
them. After that, the traits callback mechanism triggers an action in
the plugin contributing the extension point.</p>
<p>This contribution mechanism could be based on message passing between
processes quite easily (although for GUIs it breaks down, because AFAIK
you cannot assemble a consistent GUI from different widgets living in
different process space, without using some Xwindows-specific tricks).
Of course this does not give me hard guaranties of decoupling and
control of the side-effects, as a call to a plugin can induce calls to
other plugins inside it. This is where best practice comes along: core
plugins should be able to run and provide their basic functionality
outside of Envisage, as normal objects. Envisage should only be a thin
wrapper allowing them to expose this functionality and extend other
plugins. This is introducing a distinction between objects and method
calls, that do not need to be arranged in self-consistent entities and
which you use very often , and plugins and extensions contribution, that
form standalone entities and should be used more sparsely.</p>
<p>Of course Envisage cannot go too far in terms of providing guaranties
for decoupling. It gives a mechanism, best practices, could even help
plugin decoupling by having them live in different processes, but as
long as it does not enforce rules in the semantics of the language, it
cannot achieve what projects like Erasmus are trying to do. I however
think it is good to have a look at the work done in these projects to
see what we can learn.</p>
Of travels and sprints2008-04-01T02:13:00+02:002008-04-01T02:13:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-04-01:/programming/of-travels-and-sprints.html<p>This month I have traveled a bit for scientific-computing related
reasons.</p>
<div class="section" id="in-england">
<h2>In England</h2>
<p>First of all, I was speaking at the OKcon, <a class="reference external" href="http://www.okfn.org/okcon/">open knowledge conference</a>
in London, about Scientific tools in Python in general, and Mayavi in
particular. I jumped on the occasion to visit the Airbus campus in
Bristol …</p></div><p>This month I have traveled a bit for scientific-computing related
reasons.</p>
<div class="section" id="in-england">
<h2>In England</h2>
<p>First of all, I was speaking at the OKcon, <a class="reference external" href="http://www.okfn.org/okcon/">open knowledge conference</a>
in London, about Scientific tools in Python in general, and Mayavi in
particular. I jumped on the occasion to visit the Airbus campus in
Bristol. We have had some contacts with these guys, because they use
Mayavi in some of their homegrown applications, and I was curious to put
faces on friendly names on the mailing list. In addition, I was eager to
find out how they were using Mayavi and Python scientific tools in an
industrial environment, as I have never worked in another place than a
physics lab.</p>
<div class="section" id="visiting-the-airbus-campus">
<h3>Visiting the Airbus campus</h3>
<p>The Airbus visit was enlightening: the Bristol campus is a major
research facility (several thousands people) dedicated to wing design. A
good part of the work is done through simulations deployed on big
clusters. These calculations have historically been run in Fortran and
C, but apparently the engineers are switching to a mix of compiled
languages and Python. Moreover, steering of these simulations, through
mesh-design, visualization of the results, analysis of the data, is done
mainly through an interact program, ‘flightpad’, that is developed fully
in Python, using the Envisage framework to couple together a bunch of
scientific components, including Mayavi. I got to spend a fair amount of
time with the guys doing this, and it was great to see how they did it.
They have a good approach to scientific software design (loosely coupled
components, reuse of all the existing libraries), eventhough their goal
(automatic generation of Python scripts from user interaction) is way
more ambitious than anything I have in mind. I was pleased to see that
they where using Mayavi in a way completely consistent with its design,
and did not have to hack around limitation.</p>
<p>It was really very encouraging to talk with the software strategist. He
obviously completely got it as far as how an open-source model can be
profitable to a company like Airbus. See so many people using open
source tools as their main tools, as well as a manager ready to back
this position, and explaining how it can be beneficial to contribute to
an open-source project, really filled me with hope.</p>
<p>Of course visiting the Airbus campus was not only about software, it was
also about planes (I got a drive around the campus, and it is quite fun
to ride a mini cooper between to 747), and beers (reinventing the world
to make it a better place at the pub, after work). I must say there is
something special about the scientific Python community, it is the
nicest community I know (with the sailing one :->). You meet people that
you have never seen before, and you immediately feel at ease.</p>
</div>
<div class="section" id="open-knowledge-conference">
<h3>Open Knowledge conference</h3>
<p>The Open Knowledge conference was fun. Not too much like the geek
conferences I am used to, as here the focus was on the data, and not the
tools , aka the software (for instance, the big deal is when you can get
access to the complete public transport time-tables, and you can make
maps of poorly connected areas). I met <a class="reference external" href="http://www.informatik.uni-bremen.de/cgi-bin/cgiwrap/malb/blosxom.pl">Martin Albrecht</a> from the <a class="reference external" href="http://www.sagemath.org/">sage
project</a>. It was very interesting to discuss with him. I generally
consider myself as doing rather fundamental research (Bose-Einstein
condensation), but for him I was in the applied science section, because
I use math and computers to do applied things. This distinction between
applied and fundamental maths yields a distinction in the application of
the code, and therefore the way an open-source scientific project can
survive. It was very interesting to see the way sage’s development
process therefore differed from scipy’s. I think that both Martin’s talk
on sage, and mine on Python and interactive visualization had a lot of
success: the room was full of scholars, and they wanted tools to do
their work.</p>
<p>In London, I had the occasion to catch up with my brother, and Rob, a
former colleague. That was nice too (and yielded more beers).</p>
</div>
</div>
<div class="section" id="paris">
<h2>Paris</h2>
<div class="section" id="nipy-sprint">
<h3>Nipy Sprint</h3>
<p>The week after, I was attending a sprint in Paris on <a class="reference external" href="http://neuroimaging.scipy.org/">nipy</a>:
neuroimaging in Python. We were a bunch of enthusiastic scientific
Python users crammed in a small room during the day. There was the team
from Berkeley with including Jarrod and Fernando, and all their friends.
I got to make new friends, and catch up with old ones. The goal of the
nipy effort is to build a complete processing pipeline for neuroimaging
data, especially fMRI, in Python. This is a lot of work, as many
transformations are applied to the raw data to make it useful for
scientific publications. As the field matures, these transformations
pile up, and the processing pipeline gets more and more complex. There
already exists a good pipeline under MatLab (<a class="reference external" href="http://www.fil.ion.ucl.ac.uk/spm/">SPM</a>), the problem is
that, due to the poor language features of MatLab, it is a codebase hard
to extend and to modify. One of the goals of the nipy project is to make
a pluggable architecture, for researcher to be able to replace part of
the pipeline by their own code, and thus explore new methods while
comparing them to the reference one. This means that there are some
interesting software engineering problems in here (pluggable pipelines,
framework…, the kind of stuff I like), however the current focus is to
get the algorithms right, before trying to do software over-engineering.</p>
<p>The Berkeley group got an NSF grant to work on the project and has been
able to hire two developers for two years (Chris Burns and Tom Waite).
The effort is lead by Jarrod Millman, and they have put a lot of work in
making the underlying libraries better (that is improving numpy and
scipy).</p>
<p>I had difficulties contributing any useful code, as I don’t know
neuroimaging, but I had the pleasure of seeing people pick up the mayavi
API and use it to quickly build domain-specific tools for displaying
brains and activation regions. As usual this also revealed some
shortcomings in the mlab API that I plan to address ASAP.</p>
</div>
<div class="section" id="ipython-sprint">
<h3>IPython Sprint</h3>
<p>The week end after Fernando, Laurent Dufréchou, Stefan van der Waalt and
myself crashed at my parent’s place to work on ipython1 and the front
ends. My mother cooked us some fabulous food and I had a great time.</p>
<p>Unfortunately we did get as far as I would have like. The right
abstraction for talking between the ipython1 execution engine, and the
front end are not really easy to get right, as the engine is nothing
more than an abstract execution engine, that basically only has a
namespace and knows how to execute stuff in a non-blocking mode (that’s
where it gets hard: how do you know what is going on with your engine
and the commands you have sent to it? How do you deal with
introspections requests such as tab-completion or docstring
exploration). We want as little logics in the front ends as possible:
let us not duplicate tab-completion or history. This is why we are
progressively building an object, that Fernando dubbed
“InputStateManager” that is doing the impedance matching between the
front end and the engine. I am starting to believe that the best way to
connect this object (ISM) to the front end is via a callback-based
mechanism: the front-end calls the ISM methods and gives them a callback
to call when finished (for instance if running in a different thread, a
Wx frontend would pass something based on Wx.CallAfter to display the
result). That way the mechanism is very general, can adapt to
event-driven front ends or readline-based one, and knows nothing about
the front end. Of course not much code got written, because I am way too
slow, and it took me ages to figure this out.</p>
<p>We had a lot of fun, and for me the highlight of the week end was when
my girlfriend joined us to do some hacking on a really cool project
trying to use the scipy.org wiki to edit the numpy docstring.</p>
<p>Fernando has pictures of all these happy moments. and I hope he will
publish them somewhere (Fernando, get a blog :->). Next time I hope
there will be more of us.</p>
<p><strong>Edit:</strong> <a class="reference external" href="http://gael-varoquaux.info/physics/slides_okcon.pdf">my slides at OKcon</a></p>
</div>
</div>
LaTeX files of my PhD thesis2008-04-01T00:00:00+02:002008-04-01T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2008-04-01:/science/latex-files-of-my-phd-thesis.html<p class="first last">The main files of my phd thesis, to give an example of the LaTeX code used</p>
<p><a class="reference external" href="attachments/gaeltex.zip">Here</a> are the main files I use for writing
<a class="reference external" href="http://tel.archives-ouvertes.fr/tel-00265714">my PhD thesis</a> with
LaTeX. I am not publishing them on the net as a model of what to do, as
at the end I was too much in a hurry to do a good job, and I hacked
kludges all over the code (it does not compile without overflows
anymore).</p>
<p>What turned out to be very handy was the use of the <a class="reference external" href="http://www.ctan.org/tex-archive/macros/latex/contrib/memoir/">memoir package</a>. It
allowed me just enough customization while staying compact. In order to
make it work with some other packages I use, I had to hack it a bit
(horrible kludges again).</p>
<p>You need an install of the garamond fonts to build this (for epigraphs).
I use my own version.</p>
<p>Don’t e-mail me to debug the problems you get by copying the kludges in
here. This is ugly code, that I put out because people were asking for
it.</p>
How is Mayavi pronounced2008-03-26T00:26:00+01:002008-03-26T00:26:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-03-26:/programming/how-is-mayavi-pronounced.html<p>I have been traveling recently and talking to friendly Geeks I didn’t
know yet. I have been surprised to see that many people were pronouncing
“Mayavi”, “Maya-V-I”, is in “V-I”, like the old Unix editor. Maybe this
comes from the spelling “MayaVi”, that Prabhu and I recently decided to …</p><p>I have been traveling recently and talking to friendly Geeks I didn’t
know yet. I have been surprised to see that many people were pronouncing
“Mayavi”, “Maya-V-I”, is in “V-I”, like the old Unix editor. Maybe this
comes from the spelling “MayaVi”, that Prabhu and I recently decided to
avoid. Well, Mayavi is actually pronounced “Ma-ya-vee”, and it comes
from an old sanskrit name meaning magician.</p>
Numpy doc sprint in Paris tomorrow!2008-03-20T12:23:00+01:002008-03-20T12:23:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-03-20:/programming/numpy-doc-sprint-in-paris-tomorrow.html<p>We really need to get numpy 1.0.5 out. And for this release to rock, we
want to have good docs. This is why Jarrod offered to have a doc sprint
tomorrow.</p>
<p>In addition we are currently having a sprint in Paris for neuroimaging
in Python, with a bunch …</p><p>We really need to get numpy 1.0.5 out. And for this release to rock, we
want to have good docs. This is why Jarrod offered to have a doc sprint
tomorrow.</p>
<p>In addition we are currently having a sprint in Paris for neuroimaging
in Python, with a bunch of numpy developers. Some of us are going to
work on the doc sprint tomorrow. We will have a room dedicated to this.</p>
<p>It would be great if people in Paris join us. If you want to have great
fun with Python geeks and get the chance to make numpy better, send an
e-mail to Jarrod ( <millman> at <berkeley> dot <edu> ). The venue is in
Paris 6ème.</p>
<p>See you all tomorrow.</p>
Usability2008-03-06T09:54:00+01:002008-03-06T09:54:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-03-06:/programming/usability.html<p>I just wanted to echo a <a class="reference external" href="http://log.ometer.com/2008-03.html#5">very good blog post</a> about usability:</p>
<blockquote class="epigraph">
<p><strong>Users are busy not stupid.</strong></p>
<p>As you design something, ask “is this relevant to what people are
trying to do?” rather than “is this confusing?” […] It doesn’t
matter whether people <em>could</em> figure something out. It matters
whether …</p></blockquote><p>I just wanted to echo a <a class="reference external" href="http://log.ometer.com/2008-03.html#5">very good blog post</a> about usability:</p>
<blockquote class="epigraph">
<p><strong>Users are busy not stupid.</strong></p>
<p>As you design something, ask “is this relevant to what people are
trying to do?” rather than “is this confusing?” […] It doesn’t
matter whether people <em>could</em> figure something out. It matters
whether they’re interested in figuring it out - is it part of what
they’re trying to do, or an annoying sidetrack?</p>
</blockquote>
<p>Read the <a class="reference external" href="http://log.ometer.com/2008-03.html#5">original blog post</a> and
ask yourself, when designing something: <strong>“is this relevant to what
people are trying to do?”</strong>.</p>
Supporting our users under Windows2008-03-04T02:51:00+01:002008-03-04T02:51:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-03-04:/programming/supporting-our-users-under-windows.html<p>Many of our users use Windows. I don’t, I use Linux, but I completely
respect people’s choice to use the OS they want, as I expect other
people to respect my choice. As Prabhu also run Linux (and MacOS X),
this means I should sometimes roll up my …</p><p>Many of our users use Windows. I don’t, I use Linux, but I completely
respect people’s choice to use the OS they want, as I expect other
people to respect my choice. As Prabhu also run Linux (and MacOS X),
this means I should sometimes roll up my sleeves and try out Mayavi
under Windows. As my laptop came with a Windows installation that I did
not entirely nuke, I have the option of booting under Windows. For me
this is a tedious process: I don’t know anything about Windows, I loose
all my beloved programs and have to struggle with non-Posix concepts.</p>
<p>To ease my pain, and make it easier for me to run Windows, Dave Peterson
has been pushing me to run a virtual machine. VMware is able to boot an
existing partition, and I installed it tonight (trick: Ubuntu users, add
the “partner” repository). After fighting a bit with VMware and Windows,
I was finally able to start the existing windows install in a virtual
box. Then, I was stopped by a stupid activation dialog. To make a long
story short, it seems that MicroSoft doesn’t want me to use Windows on a
real machine and on a virtual machine at the same time. The canonical
solution is to buy a new Windows license, but this I won’t do. I dislike
Windows, and will not be forced to buy license. It is not (only) a
question of Money: I have already spent more than that for me free
software activities, but the idea of being coerced in spending more
money on Microsoft products just to support Microsoft products simply
doesn’t fit.</p>
<div class="line-block">
<div class="line">If somebody has a clever way of getting around this problem, and</div>
<div class="line">allowing me to legally use a virtual machine for Windows, I would be</div>
<div class="line">very grateful.</div>
</div>
<p><strong>Edit2:</strong> See also the <a class="reference external" href="http://bryceharrington.org/drupal/foss-win-paradox">blog post</a> of Bryce Harrington on the same
topic for Inkscape.</p>
Playing with filters in Mayavi22008-02-21T00:59:00+01:002008-02-21T00:59:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-02-21:/programming/playing-with-filters-in-mayavi2.html<p>Mayavi uses VTK as a rendering engine. It does its best not to force you
to learn anything about VTK, and I often forget about the infinite
possibilities of this visualization toolkit, but sometimes it can be
interesting to actually look at bit more at its data processing
algorithms to …</p><p>Mayavi uses VTK as a rendering engine. It does its best not to force you
to learn anything about VTK, and I often forget about the infinite
possibilities of this visualization toolkit, but sometimes it can be
interesting to actually look at bit more at its data processing
algorithms to make a nice visualization.</p>
<p>Lately I was trying to get a 3D view of France, using altitude
measurements that can be freely downloaded on the <a class="reference external" href="http://www.ign.fr/telechargement/MPro/produit/BD_ALTI/France_1000.zip">IGN website</a>. The
shear number of points is huge: one point every kilometer, on the whole
French territory. As a result, the brute force approach does not work,
and Prabhu hinted I could look at VTK filters to make a good use of this
data.</p>
<p>VTK, and thus Mayavi, uses a pipeline-oriented approach: the data is
loaded by a data source, and it is plotted using one or more
visualization modules. Between the source and the modules, we can insert
filters, that can process and transform the data. Playing with this
pipeline I was able to transform the set of 800 000 (yes, that’s 800
000!) altitude measurements, given as (latitude, longitude, elevation)
into an optimized mesh small-enough to display fluidly on my laptop.
Let’s see how I did it.</p>
<p>First of all I have to load the data, and I create a set of scattered
(non-grided) 3D points. This I do with numpy and mlab’s scatterscalar
function.</p>
<p>Then I want to add connectivity information to go from scattered points
to a field. This I do with a Delaunay2D VTK filter. This gives me a mesh
that I can display with the surface visualization module. But if I do
this, my computer grinds to a halt. Remember, I am dealing with 800 000
data points. It is interesting to note that VTK is a pipeline,
data-on-demand, type of rendering architecture: the delaunay filter, for
instance, will process data only as it is required by the rendering.
Thus adding a delaunay tessellation filter is a numerically very cheap
operation, as long as no visualization module is pulling the data out
(think generator/lazy-evaluation pattern).</p>
<p>The first simplification I make to the mesh is simply to remove mesh
faces outside the French border. For this I have added points outside
France with negative scalar value, and I use altitude as a scalar value
for the points given by the IGN data. I want to use the Threshold filter
to filter out the cells with scalar data equal or less than zero (this
way I also filter out the sea). As I want to act on cells rather than
vertexes, I have first to use convert the scalar data, located on the
data points, to cell data, using the PointToCellData filter.</p>
<p>Then I use the QuadricDecimation filter to simplify my mesh. This filter
finds a good approximation of the mesh with less faces. Unfortunately I
also loses track of the scalar data attached to my mesh. As I am
interested in having a scalar reflecting altitude, in order to associate
a pleasing color with it, I rebuild this scalar using the
ElevationFilter filter.</p>
<a class="reference external image-reference" href="attachments/france.png"><img alt="" src="attachments/france.png" /></a>
<p>I find the result very pleasing: the mesh simplification is very
impressive, because it yields a good rendering of the landscape with
little faces. For example, the triangles around a river are elongated
and follow it. I tried playing around with the data using numpy and
writing my own algorithms (binning, averaging, …) and I didn’t get as
good results, obviously because these algorithms have not been developed
during an evening’s hack. The resulting visualization takes a long time
to load, probably because the QuadricDecimation filter is busy doing its
work.</p>
<p>To sum up the pipeline used:</p>
<ul class="simple">
<li>Delaunay2D to create connectivity information,</li>
<li>Threshold to remove what lies outside France</li>
<li>QuadricDecimation to simplify the mesh</li>
</ul>
<p>and a few filters to do conversion/creation of different data type. The
mlab code to generate this visualization can be found in the Mayavi2
example directory (currently
on<a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/branches/enthought.mayavi_2.1.0/examples/france.py">https://svn.enthought.com/enthought/browser/Mayavi/branches/enthought.mayavi_2.1.0/examples/france.py</a>
but the branch will soon move to a tag as we do a release).</p>
<p><strong>Edit</strong>: Indeed, the release has happened and as Fred points out, the
correct link is</p>
<p><a class="reference external" href="https://svn.enthought.com/enthought/browser/Mayavi/tags/enthought.mayavi_2.1.1/examples/france.py">https://svn.enthought.com/enthought/browser/Mayavi/tags/enthought.mayavi_2.1.1/examples/france.py</a></p>
<p>I had fun zooming with this visualization and exploring France. I can
say that this doesn’t compare to well with Google Earth: these guys pull
tricks like detailed textures attached to the mesh or level of detail
adaptation of the mesh depending on the distance to the camera. Yeah,
Mayavi is a general-purpose 3D visualization software, and you can
already go pretty far quite quickly, but if you want to do better,
you’ll have to get your hands dirty.</p>
Adding simple customisation to Mayavi22008-02-05T02:37:00+01:002008-02-05T02:37:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-02-05:/programming/adding-simple-customisation-to-mayavi2.html<p>Mayavi2 is a rewrite of the original Mayavi application to make it
easier to adapt and customize.</p>
<p>Mayavi2 uses, for its full-blown application, the <a class="reference external" href="http://code.enthought.com/envisage">Envisage</a> framework.
As a result it can both use envisage plugins (such as the logger and the
python shell), and contribute to other plugins, thus providing …</p><p>Mayavi2 is a rewrite of the original Mayavi application to make it
easier to adapt and customize.</p>
<p>Mayavi2 uses, for its full-blown application, the <a class="reference external" href="http://code.enthought.com/envisage">Envisage</a> framework.
As a result it can both use envisage plugins (such as the logger and the
python shell), and contribute to other plugins, thus providing a
visualization engine.</p>
<p>The problem with a framework is that if you are not already using it, it
comes at a cost. The cost of the Envisage2 framework is well-known: it
is a bit tedious to learn. This is why Martin Chilvers (the Envisage
author) has written Envisage3, but this is another story as Mayavi2 is
currently based on Envisage2. To avoid forcing Envisage on people
wanting to use Mayavi2, we have been working on decoupling the two. As I
showed in a <a class="reference external" href="http://gael-varoquaux.info/programming/mayavi2-using-from-ipython.html">previous post</a>, Mayavi2 can now be fully used without
Envisage. But this is in the development version, and some people are
stuck with the current release.</p>
<p>Today I would like to show how one can add some very simple
customization to Mayavi2. The idea is to use the “-x” switch of Mayavi,
that allows to execute a script in Mayavi2 after it has been started.
Mayavi2 is thus started, the WxPython mainloop is running, and we can do
better than a script, we can pop up a small UI. For this I will use
<a class="reference external" href="http://code.enthought.com/traits">traitsUI</a> as I really like this library rather than raw WxPython (you
can find a tutorial for this technology on <a class="reference external" href="http://www.gael-varoquaux.info/computers/traits_tutorial">my website</a>). I will make a
small dialog that uses Mayavi2 to create a 3D visualization, giving the
user the possibility to change interactively the parameters of the
visualization:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">N</span>
<span class="kn">from</span> <span class="nn">enthought.mayavi.mlab</span> <span class="kn">import</span> <span class="n">plot3d</span><span class="p">,</span> <span class="n">clf</span>
<span class="kn">from</span> <span class="nn">enthought.traits.api</span> <span class="kn">import</span> <span class="n">HasTraits</span><span class="p">,</span> <span class="n">Int</span>
<span class="kn">from</span> <span class="nn">enthought.traits.ui.api</span> <span class="kn">import</span> <span class="n">View</span>
<span class="k">class</span> <span class="nc">MyModel</span><span class="p">(</span><span class="n">HasTraits</span><span class="p">):</span>
<span class="n">n_mer</span> <span class="o">=</span> <span class="n">Int</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span>
<span class="n">n_long</span> <span class="o">=</span> <span class="n">Int</span><span class="p">(</span><span class="mi">11</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_anytrait_changed</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">pi</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">pi</span>
<span class="n">dphi</span> <span class="o">=</span> <span class="n">pi</span><span class="o">/</span><span class="mf">1000.</span>
<span class="n">phi</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mf">0.0</span><span class="p">,</span> <span class="mi">2</span><span class="o">*</span><span class="n">pi</span> <span class="o">+</span> <span class="mf">0.5</span><span class="o">*</span><span class="n">dphi</span><span class="p">,</span> <span class="n">dphi</span><span class="p">,</span> <span class="s1">'d'</span><span class="p">)</span>
<span class="n">mu</span> <span class="o">=</span> <span class="n">phi</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">n_mer</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="n">mu</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="mi">1</span><span class="o">+</span><span class="n">N</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">n_long</span><span class="o">*</span><span class="n">mu</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">n_mer</span><span class="p">)</span><span class="o">*</span><span class="mf">0.5</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">mu</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="mi">1</span><span class="o">+</span><span class="n">N</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">n_long</span><span class="o">*</span><span class="n">mu</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">n_mer</span><span class="p">)</span><span class="o">*</span><span class="mf">0.5</span><span class="p">)</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">n_long</span><span class="o">*</span><span class="n">mu</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">n_mer</span><span class="p">)</span><span class="o">*</span><span class="mf">0.5</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">N</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">mu</span><span class="p">)</span>
<span class="c1"># Realy ugly, but so much easier than modifying in place the</span>
<span class="c1"># visualization</span>
<span class="n">clf</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">plot</span> <span class="o">=</span> <span class="n">plot3d</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span>
<span class="n">tube_radius</span><span class="o">=</span><span class="mf">0.025</span><span class="p">,</span> <span class="n">colormap</span><span class="o">=</span><span class="s1">'Spectral'</span><span class="p">)</span>
<span class="n">view</span> <span class="o">=</span> <span class="n">View</span><span class="p">(</span><span class="s1">'n_mer'</span><span class="p">,</span> <span class="s1">'n_long'</span><span class="p">)</span>
<span class="n">my_model</span> <span class="o">=</span> <span class="n">MyModel</span><span class="p">()</span>
<span class="n">my_model</span><span class="o">.</span><span class="n">_anytrait_changed</span><span class="p">()</span>
<span class="n">my_model</span><span class="o">.</span><span class="n">edit_traits</span><span class="p">()</span>
</pre></div>
<p>After the imports, the class definition is the object behind the dialog:
two integer attributes that get displayed in the dialog, and a callback
call when these attributes are modified. This callback uses Mayavi2’s
mlab scripting interface to plot a nice 3D curve. The last line pops up
the dialog that allows the user to interact with the visualization. This
is very crude, but is a simple example. If you run this code using
“mayavi2 -x”, the Mayavi2 application window appears with our
visualization, and in addition the dialog to interact with it.</p>
<p><a class="reference external image-reference" href="attachments/mayayi2-customui.png"><img alt="mayayi2-customui.png" src="attachments/mayayi2-customui.png" /></a></p>
<p>With the development version of Mayavi2, you can simply change the last
line from ‘model.edit_traits()’ to ‘model.configure_traits()’ and the
file can be run as a normal Python file: there is no need for the
Envisage framework. As a result the UI is a bit simpler, which can be
seen as a pro, or as a con, depending on what you want to do:</p>
<p><a class="reference external image-reference" href="attachments/mayayi2-noui.png"><img alt="mayavi2-noui.png" src="attachments/mayayi2-noui.png" /></a></p>
Mayavi2 in Ubuntu2008-01-26T10:10:00+01:002008-01-26T10:10:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-01-26:/programming/mayavi2-in-ubuntu.html<p>After Debian, Mayavi2 has just made it into Ubuntu Hardy
(<a class="reference external" href="http://packages.ubuntu.com/hardy/science/mayavi">http://packages.ubuntu.com/hardy/science/mayavi</a>2). From what I can
see, the deps look just good, thanks a lot to Varun for making sure the
Debian package was in shape. This means in April, it will be massively …</p><p>After Debian, Mayavi2 has just made it into Ubuntu Hardy
(<a class="reference external" href="http://packages.ubuntu.com/hardy/science/mayavi">http://packages.ubuntu.com/hardy/science/mayavi</a>2). From what I can
see, the deps look just good, thanks a lot to Varun for making sure the
Debian package was in shape. This means in April, it will be massively
easier for a lot of people to install an oldish version of Mayavi2.</p>
<p>I am quite happy to say that we did this the right way, by polishing a
Debian package first. Once this once done, getting Mayavi2 in Ubuntu was
trivial. This felt like a well-working machinery.</p>
<p>Now that a Debian package has been done and the Debian QA went over all
the fine details of permissions, license, man pages… it should be much
easier to get Mayavi2 in other distros (anybody for Fedora ?). Having a
binary package in a distro is a major bonus for the users: there is a
world between having to grab and compile the ETS to try out a program,
and being able to install it from the repos. Of course, the package will
always be a bit old and lacking the shiny features that we add in the
SVN. When I find time, I will put myself together and make debs for
Gusty. It not hard, I just have to find the time (yes, promises …).</p>
Mission accomplished2008-01-19T11:59:00+01:002008-01-19T11:59:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-01-19:/science/mission-accomplished.html<p>I defended my PhD yesterday. I am pretty happy to be done with this.</p>
<a class="reference external image-reference" href="../science/attachments/talking1.jpg"><img alt="" src="../science/attachments/talking1.jpg" /></a>
<p>After the defense, the other PhD students offered me a plastic python
(well it was a cobra, actually, but they told me to pretend it was a
Python.</p>
<a class="reference external image-reference" href="../science/attachments/gael_with_python.jpg"><img alt="" src="../science/attachments/gael_with_python.jpg" /></a>
Mayavi2 in Debian2008-01-13T09:40:00+01:002008-01-13T09:40:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-01-13:/programming/mayavi2-in-debian.html<p>Thanks to the combined efforts of Ondrej Certik, who made me do the
necessary tarballs, and Varun Hiremath, who finalized the packaging
efforts, Mayavi2 is now in Debian (
<a class="reference external" href="http://packages.debian.org/sid/mayavi2">http://packages.debian.org/sid/mayavi2</a> ). Currently, it is in testing,
but it will soon trickle down to unstable. Along with Mayavi2 …</p><p>Thanks to the combined efforts of Ondrej Certik, who made me do the
necessary tarballs, and Varun Hiremath, who finalized the packaging
efforts, Mayavi2 is now in Debian (
<a class="reference external" href="http://packages.debian.org/sid/mayavi2">http://packages.debian.org/sid/mayavi2</a> ). Currently, it is in testing,
but it will soon trickle down to unstable. Along with Mayavi2, we have
two new Debian packages: Traits, and TraitsUI, two other absolutely
great Python packages and, IMHO, fundamental technologies.</p>
<p>This is looking very good for simplifying reuse of these technologies.</p>
Mayavi2: using from ipython2008-01-04T03:38:00+01:002008-01-04T03:38:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-01-04:/programming/mayavi2-using-from-ipython.html<p>Recently Prabhu and I have been ironing the library aspect of Mayavi2
(library as opposed to application). One of the usecases we are
interested in, is interative use, via for instance ipython, <em>a la</em>
pylab.</p>
<p>Most people think of Mayavi as a big and powerful application, maybe a
bit clunky …</p><p>Recently Prabhu and I have been ironing the library aspect of Mayavi2
(library as opposed to application). One of the usecases we are
interested in, is interative use, via for instance ipython, <em>a la</em>
pylab.</p>
<p>Most people think of Mayavi as a big and powerful application, maybe a
bit clunky to script and to get to interact with other bits of code.
With the recent additions you can use mayavi just as you would use
matplotlib, to complete matplotlib’s 3D plotting.</p>
<div class="section" id="d-plotting-from-ipython">
<h2>3D plotting from ipython</h2>
<p>As Mayavi relies on wxPython (technical details, yes I know you don’t
care), to use it with ipython, you have to start ipython with <em>iptyhon
-wthread</em>. Using ipython from svn you can start ipython with both the
<em>-pylab</em> and the <em>-wthread</em> options to use both pylab and mayavi for 2D
and 3D plotting (beware of namespace mangling, <strong>don’t</strong> use <em>from
module import *</em>).</p>
<p>The matlab/pylab-like interface to Mayavi is found in
<em>enthought.mayavi.tools.mlab</em>(this will most probably change to
<em>enthought.mayavi.mlab</em>, or to something else shorter), import this
module to have familiar functions. Documentation is a bit missing for
now, (you can grab some kind of a embryo at
<a class="reference external" href="https://svn.enthought.com/enthought/attachment/wiki/MayaVi/mlab.pdf?format=raw">https://svn.enthought.com/enthought/attachment/wiki/MayaVi/mlab.pdf?format=raw</a>
), so I’ll just show an example:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">enthought.mayavi.toosl</span> <span class="kn">import</span> <span class="n">mlab</span> <span class="k">as</span> <span class="n">M</span>
<span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="o">*</span>
<span class="n">f</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">sin</span><span class="p">(</span><span class="n">x</span> <span class="o">+</span> <span class="n">y</span><span class="p">)</span> <span class="o">+</span> <span class="n">sin</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">x</span> <span class="o">-</span> <span class="n">y</span> <span class="p">)</span> <span class="o">+</span> <span class="n">cos</span><span class="p">(</span><span class="mi">3</span><span class="o">*</span><span class="n">x</span> <span class="o">+</span> <span class="mi">4</span><span class="o">*</span><span class="n">y</span><span class="p">)</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">mgrid</span><span class="p">[</span><span class="o">-</span><span class="mi">7</span><span class="p">:</span><span class="mf">7.05</span><span class="p">:</span><span class="mf">0.1</span><span class="p">,</span> <span class="o">-</span><span class="mi">5</span><span class="p">:</span><span class="mf">5.05</span><span class="p">:</span><span class="mf">0.05</span><span class="p">]</span>
<span class="n">M</span><span class="o">.</span><span class="n">surf</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span>
<span class="n">M</span><span class="o">.</span><span class="n">axes</span><span class="p">()</span>
<span class="n">M</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'Demoing mlab.surf'</span><span class="p">)</span>
</pre></div>
<img alt="" src="attachments/demoing_mlab_1.jpg" />
<p>Apart from the <em>surf</em> command, the different commands used have
equivalents in pylab. <em>surf</em> is inspired from matlab: let us continue
pylab’s work here. OK, the keyword arguments are not exactly the same,
and not all pylab features are available through the mayavi/mlab
interface. But the good news is that the objects created are VTK
objects, even though it is a bit hidden by this simplified interface.
This means that there is the power of VTK, and that you can always
modify the resulting objects to fine tune properties that you cannot
(yet) tweak with keyword arguments.</p>
</div>
<div class="section" id="modifying-the-plot-from-the-gui">
<h2>Modifying the plot from the GUI</h2>
<p>OK, as you see, we can control Mayavi without all the fuss of the user
interface. We get a really simple window that does not get in our way.
But is this still Mayavi? This clunky UI was convenient to interact with
the visualization. I can pop the pipeline up using <em>M.show_engine()</em>.
Once I have the pipeline I can easily double click on any of the items I
want to modify, and I get the usual Mayavi dialogs that are so
convenient when trying to tweak a scene, for instance to modify the
colormap:</p>
<img alt="" src="attachments/demoing_mlab_2.jpg" />
<p>This is still work in progress, but <em>mlab</em> is completely useable for
real work (I use it whenever I want to make a figure in 3D). Beware that
the API is not cast into stone (that’s a good moment to make remarks)
and that it might change. Documentation is also lacking. Don’t worry it
will come pretty soon, but I also have a thesis to defend :->.</p>
</div>
Hello world… hu I mean Planet !2008-01-02T22:41:00+01:002008-01-02T22:41:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2008-01-02:/misc/hello-world-hu-i-mean-planet.html<p>Thanks to Jarrod Millman, we now have planet.scipy . This means it is time for
me to create a blog, on top of my good old static site.</p>
Garamond fonts for LaTeX2006-10-01T00:00:00+02:002006-10-01T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2006-10-01:/science/garamond-fonts-for-latex.html<p class="first last">An easy to install version of Garamond fonts for LaTeX</p>
<p><a class="reference external" href="http://en.wikipedia.org/wiki/Garamond">Garamond fonts</a> are a large
family of fonts. At a friend’s request I modified the <a class="reference external" href="ftp://dante.ctan.org/tex-archive/fonts/urw/garamond/">URW-garamond</a> fonts to improve
kerning, add old style numbers, and make some letters prettier. These
fonts are available under the <a class="reference external" href="http://www.cs.wisc.edu/~ghost/doc/cvs/Public.htm">Aladdin Free Public License</a> , which states, if I
understand it correctly, that you can use and modify the fonts freely for
non commercial purposes.</p>
<p>Here is <a class="reference external" href="attachments/baudelaire.pdf">a pdf file</a> that gives an example
of the fonts.</p>
<div class="topic">
<p class="topic-title">Questions and suggestions</p>
<p>I made this font in 2006. Time has passed, and I have completely
forgotten the skills required to modify it. I cannot go anywhere
beyond providing the file for download. Sorry, if you send me a kind
email mentionning that the accents or the numbers are not right, I am
unable to address it.</p>
</div>
<div class="section" id="instructions-for-use-with-pdflatex">
<h2>Instructions for use with pdfLaTeX</h2>
<p>The standard procedure for installing new fonts in a LaTeX installation
is quite complicated and varies from one LaTeX distribution to another.</p>
<p>I strongly suggest that you install the fonts only in your documents
folder. This make your document portable: as long as you give the
complete folder to your colleagues, they will be able to compile it.</p>
<p>If you want to install the fonts in the TeXMF (so that all documents
compiled on your installation have access to the fonts) I assume you know
TeX well enough to perform the installation without further help.</p>
<div class="section" id="installing-in-the-current-folder">
<h3>Installing in the current folder</h3>
<p>Here is an easy way to install the fonts in your document’s folder (this
will only work if you are using pdfLaTeX):</p>
<p><a class="reference external" href="attachments/garamond.zip">Here</a> is a package to use these fonts with LaTeX.</p>
<p>Unzip <em>garamond.zip</em> in the same folder than the LaTeX document you
are working on.</p>
</div>
<div class="section" id="using-in-a-latex-document">
<h3>Using in a LaTeX document</h3>
<p>In your LaTeX file, include the package “garamond”:</p>
<div class="highlight"><pre><span></span><span class="k">\usepackage</span><span class="nb">{</span>garamond<span class="nb">}</span>
</pre></div>
<p>You also need to use the T1 font encoding:</p>
<div class="highlight"><pre><span></span><span class="k">\usepackage</span><span class="na">[T1]</span><span class="nb">{</span>fontenc<span class="nb">}</span>
</pre></div>
<p>The garamond package defines a new command <tt class="docutils literal">\garamond</tt> that switches
the font in the current group to garamond. Here is a minimal example:</p>
<div class="highlight"><pre><span></span><span class="k">\documentclass</span><span class="nb">{</span>article<span class="nb">}</span>
<span class="k">\usepackage</span><span class="na">[T1]</span><span class="nb">{</span>fontenc<span class="nb">}</span>
<span class="k">\usepackage</span><span class="nb">{</span>lmodern<span class="nb">}</span>
<span class="k">\usepackage</span><span class="nb">{</span>garamond<span class="nb">}</span>
<span class="k">\begin</span><span class="nb">{</span>document<span class="nb">}</span>
<span class="nb">{</span><span class="k">\garamond</span>
The Quick Brown Fox Jumps Over The Lazy Dog. 0123456789 <span class="k">\\</span>
<span class="nb">{</span><span class="k">\slshape</span> This is garamond slanted<span class="nb">}</span> <span class="k">\\</span>
<span class="nb">{</span><span class="k">\bfseries</span> This is garamond bold face<span class="nb">}</span> <span class="k">\\</span>
<span class="nb">{</span><span class="k">\scshape</span> This is in small caps<span class="nb">}</span> <span class="k">\\</span>
<span class="nb">{</span><span class="k">\slshape</span> <span class="k">\bfseries</span> This is slanted and bold face<span class="nb">}</span> <span class="k">\\</span>
<span class="nb">}</span>
And this is written with the latin modern fonts.
<span class="k">\garamond</span>
Here we switch to garamond.
<span class="k">\ungaramond</span>
Here we switch back to the default.
<span class="k">\end</span><span class="nb">{</span>document<span class="nb">}</span>
</pre></div>
<img alt="minimal example of a LaTeX file using garamond fonts" class="align-center" src="attachments/minimal.png" />
<p>One remark on this example: you should never, ever, use the standards
out-of-the-box T1 fonts with pdfLaTeX, they look ugly. Always include the
“lmodern” or “pslatex” package, that uses much better postscript fonts.</p>
</div>
</div>
Timing problems with a computer2006-03-20T00:00:00+01:002006-03-20T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2006-03-20:/science/timing-problems-with-a-computer.html<p class="first last">Simple experiments on real-time computing, to put in the perspective of the computer-control of an experiment</p>
<p>Computers are very versatile beasts. Physicists are tempted to use them
to do real-time signal processing and for instance implement a
feedback-loop on an instrument. If the frequencies are above 10Hz this is
not as easy as one might think (after they run at several gHz). I will
try to explore some difficulties here.</p>
<p>Remember, these are just the ramblings of a physics phD student. I have
little formal training in IT, so don’t hesitate to correct me if I didn’t
get things right.</p>
<div class="section" id="operating-systems-timing-and-latencies">
<h2>Operating systems, timing and latencies</h2>
<p>If you want to build an I/O system that interacts in real-time with
external devices you will want to control the timing of the signals you
send to the instruments.</p>
<p>Computers are not good at generating events at a precise timing. This is
due to the fact that modern operating systems share the processor time
between a large number of tasks. Your process does not control completely
the computer, and it has to ask for time to the operating system. The
operating system shares time between different processes, but it also has
some internal tasks to do (like allocating memory). All these
operations may not perform in a predictable time-lapse <a class="footnote-reference" href="#footnote-2" id="footnote-reference-1">[2]</a>, and make it
harder for a process to produce an event (eg a hardware output signal) at
a precise instant.</p>
<p>One solution to avoid problems is to run the program with a single task
operating-system, like DOS. Even when doing this you have to be careful,
as all system operations asked by your program may not return in a
controlled amount of time. The good solution is to use a <a class="reference external" href="http://en.wikipedia.org/wiki/Real-time_operating_system">hard real-time
operating system</a>, but this
forces us to use dedicated system and makes the job much harder as we
cannot use standard programming techniques and libraries.</p>
<p>I will attempt to study the limitations of a simple approach, using
standard operating systems and programming techniques, to put numbers of
the performance one can expect.</p>
</div>
<div class="section" id="real-time-clock-interrupt-latency">
<h2>Real-time clock interrupt latency</h2>
<p>The right tool to control timing under linux is the “real time clock”
<a class="footnote-reference" href="#footnote-3" id="footnote-reference-2">[3]</a>. It can be used to generate interrupts at a given frequency or
instant.</p>
<p>To quote Wikipedia: “in computing, an interrupt is an asynchronous signal
from hardware indicating the need for attention or a synchronous event in
software indicating the need for a change in execution”. In our case the
interrupt is a signal generated by the real time clock that is trapped by
a process.</p>
<p>I have ran a few experiments on the computers I have available to test
the reliability of timing of these interrupts, that is the time it take
to the process to get the interrupt. This is known as “interrupt
latency” (for more details see <a class="reference external" href="http://lwn.net/Articles/139784/">this article</a>), and it limits both the response
time and the timing accuracy of a program that does not monopolize the
CPU, as it corresponds to the time needed for the OS to hand over control
to the program.</p>
<div class="section" id="the-experiment-and-the-results">
<h3>The experiment and the results</h3>
<p>I used a test program to measure interrupt latency <a class="footnote-reference" href="#footnote-4" id="footnote-reference-3">[4]</a> on linux. The test
code first sets the highest scheduling priority it can, then asks to be
waken up at a given frequency <em>f</em> by the real-time clock. It checks the
real-time clock to see if it was really waken-up when it asked for. It
computes the differences between the measured delay between 2 interrupts
and the theoretical one <em>1/f</em>. Here is a plot of histogram of the delays
on different systems. The delay is plotted in units of the period <em>1/f</em>.</p>
<img alt="" class="align-center" src="attachments/real_time_results.png" />
<p>While the code was running I put some stress on the system, pinging
google.com, copying data to the disk, and calculating an md5 hash. This
is not supposed to be representative of any particular use, I just wanted
not the system to be idle aside from my test code. The tests where run
under a gnome session but without any user action.</p>
</div>
<div class="section" id="interpretation-of-the-results">
<h3>Interpretation of the results</h3>
<p>I am no kernel guru, so my interpretations may be imprecise, but I can
see that the results are pretty bad.</p>
<p>There is a jitter that can go up to half a period at 1kHz. Depending on
how important it is to have a narrow linewidth of your “digital
oscillator” the jitter sets a limit to the frequency where the computer
can be used as a “digital oscillator”.</p>
<p>This also tells us that an interrupt request takes in average 0.5ms to
get through to the program it targets. This allows us to estimate the
time it take for an event (for instance generated by an I/O card) to
reach a program, if this one is not running.</p>
<p>Keep in mind that this experiment only measures jitter and frequency
offset due to software imperfection (kernel: operating system related),
on top of this you must add all the I/O bus and buffer problems, if you
want to control an external device.</p>
<p>An interesting remark is to see how the results vary from one computer to
another. Quite clearly omega’s RTC is not working properly, this is
probably due to driver problems. Beta has good results, and this is
probably due to its pre-emptible kernel. The results of our computer
(digamma) are surprisingly bad. This is powerful 4 CPU computer. It seems
to me that the process my be getting relocated from one CPU to another,
which generates big jitter. Aramis is a 2 CPU (+ multithreading, that’s
why it appears as 4) box, and it performs much better. The CPU are
different, and the kernel versions are different, but I would expect more
recent kernels to fare better.</p>
<blockquote>
<strong>The take home message: do not trust computers under the milisecond.</strong></blockquote>
<p>Other sources have indeed confirmed that with a standard linux kernel, at
the time of the writing (linux 2.6.18) interrupt latency is of the order
of the millisecond. The “RT_PREEMPT” compile switch has been measured to
drop the interrupt latency to 50 microseconds, which is of the order of
the hardware limit.</p>
</div>
<div class="section" id="implications-of-this-jitter">
<h3>Implications of this jitter</h3>
<p>These histograms can be seen as frequency spectra of the signal generated
by the computer.</p>
<p>We can see that the signal created can be slightly off in frequency (the
peak is not always centered on zero). The RTC is not well calibrated.
This should not be a major problem if the offset is repeatable, as it can
be measured and taken in account for.</p>
<p>We can see that the spectrum has a non negligible width at high
frequency. This means that in a servo-loop like system the computer will
add high frequency noise at around 1kHz. It also means that the
timing of a computer created event cannot be trusted at the millisecond
level.</p>
<p>However it is interesting to note that very few events reach out of the
+/- 1 period. This means that the computer does not skip a beat very
often. It does perform the work in a reliable way, but it does not
deliver it on time. This means that if we correct for this jitter the
computer can act as a servo loop up to 1kHz. The preempt kernel performs
very well in terms of reliability, even though it is on an old box with
little computing power.</p>
</div>
<div class="section" id="dealing-with-the-jitter">
<h3>Dealing with the jitter</h3>
<p>First we could try to correct for the jitter with a software trick. For
instance we could ask for the interrupt in advance, and block the CPU by
doing busy-waiting (to ensure that the scheduler does not schedule us
out) until the exact moment comes.</p>
<p>Another option is to use an I/O device with an embedded clock, that
corrects for the jitter. For instance a hardware trigged acquisition
card. I prefer this solution as it is more versatile and scalable.</p>
<p><em>This brings us to something that seems to be quite general with real-time
computer control: buffers and external clocks. The computer has the
processing power to do the work in the required amount of time. The
buffer and the external clock correct for the jitter introduced by the
software.</em></p>
<p>Finally recompiling a kernel with the RT-preempt patch would probably
help a lot, given that it reduces the interrupt latency by two orders of
magnitudes.</p>
</div>
<div class="section" id="technical-details-about-the-experiment">
<h3>Technical details about the experiment</h3>
<div class="section" id="the-measuring-code">
<h4>The measuring code</h4>
<p>The way this work is that a small C code (borrowed and adapted from
Andrew Morton’s “realfeel.c”) asks for the highest scheduling priority it
can get, then set the real-time clock to generate an interrupt at a give
frequency. It then loops, waiting for the real-time clock (RTC). The OS
schedules other tasks during the waiting period, but when the interrupt
is generated by the RTC the OS gives the CPU back to the program. It then
compares the time delay between the last time it got the interrupt, and
this time, and stores the difference. The results are stored in a
histogram file.</p>
</div>
<div class="section" id="the-stress-code">
<h4>The stress code</h4>
<p>I have very ugly way of putting stress and the computer, so that the
kernel actually schedules other tasks. I did not put tremendous stress
on the CPU, as I want to simulate standard use cases. This is the way I
did it:</p>
<div class="highlight"><pre><span></span><span class="k">for</span><span class="w"> </span><span class="o">((</span><span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="p">;</span><span class="w"> </span>i<span class="w"> </span><<span class="o">=</span><span class="w"> </span><span class="m">10</span><span class="p">;</span><span class="w"> </span>i++<span class="w"> </span><span class="o">))</span>
<span class="k">do</span>
<span class="w"> </span>ping<span class="w"> </span>-c<span class="w"> </span><span class="m">10</span><span class="w"> </span>www.google.com<span class="w"> </span><span class="p">&</span>
<span class="w"> </span>dd<span class="w"> </span><span class="k">if</span><span class="o">=</span>/dev/urandom<span class="w"> </span><span class="nv">bs</span><span class="o">=</span>1M<span class="w"> </span><span class="nv">count</span><span class="o">=</span><span class="m">40</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>md5sum<span class="w"> </span>-<span class="w"> </span><span class="p">&</span>
<span class="w"> </span>dd<span class="w"> </span><span class="k">if</span><span class="o">=</span>/dev/zero<span class="w"> </span><span class="nv">of</span><span class="o">=</span>/tmp/foo<span class="w"> </span><span class="nv">bs</span><span class="o">=</span>1M<span class="w"> </span><span class="nv">count</span><span class="o">=</span><span class="m">500</span>
<span class="w"> </span>sync
rm<span class="w"> </span>/tmp/foo
<span class="k">done</span>
</pre></div>
<p>Three tasks running in parallel: pinging google, calculation the md5 hash
of a random chunk of bits (which also means generating it), and writing
500Mb to the disk. If the system and the network are fast enough the 2
first task finish before the last one. This is done on purpose.</p>
</div>
</div>
<div class="section" id="making-your-own-measurements">
<h3>Making your own measurements</h3>
<p>You can reproduce the histograms under linux by running the
“stresstest.sh” script given be the <a class="reference external" href="attachments/real_time_stress_test.zip">following archive</a> . The plots can be obtained by
running the “process.py” python scripts (requires scipy and matplotlib).
You may have to increase the real-time clock frequency user limit. You
can do this by running (as root) “ echo 1024 >
/proc/sys/dev/rtc/max-user-freq”</p>
<p>Send me the results dir created by the “stresstest.sh” script on your
box, I am very interested to gather more statistics.</p>
</div>
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>The jitter measurement is interesting not because it shows the absolute
limit of the technology (hard real-time OSs, like RTlinux could go much
further), but because it shows the performance achievable with simple
techniques. Looking at this data I would say that anything with
frequencies below 10 to 100Hz is fairly easy to achieve with the RTC
interrupts, anything around several kiloHertz can be done with a bit more
work, and anything above require a lot of work.</p>
<p>My current policy is to try to move to embedded devices anything with
speeds above 10Hz.</p>
<p><strong>Acknowledgments</strong></p>
<p>I would like to thank Nicolas George for enlightening discussions on
these matters, as useful questions on the purpose of this experiment. I
would also like to thank David Cournapeau for pointing me to interesting
references and to the Linux Audio Developer mailing list for more
information.</p>
<p><strong>References</strong></p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[1]</td><td>Wikipedia article on real-time computing:
<a class="reference external" href="http://en.wikipedia.org/wiki/Real-time_computing">http://en.wikipedia.org/wiki/Real-time_computing</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[2]</a></td><td>A very clear article about fighting latency in the linux kernel:
<a class="reference external" href="http://lac.zkm.de/2006/papers/lac2006_lee_revell.pdf">http://lac.zkm.de/2006/papers/lac2006_lee_revell.pdf</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[3]</a></td><td>About the RTC: <a class="reference external" href="http://www.die.net/doc/linux/man/man4/rtc.4.html">http://www.die.net/doc/linux/man/man4/rtc.4.html</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[4]</a></td><td>What this code is actually measuring is, in technical terms, the
interrupt latency, that is the time it takes for the kernel to catch
the interrupt, and the rescheduling latency, that is the time it take
for the kernel to reschedule from one process to another.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[5]</td><td>A different benchmark, that probably studies more directly the
intrinsic kernel limits than my code: <a class="reference external" href="http://lwn.net/Articles/139403/">http://lwn.net/Articles/139403/</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-6" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[6]</td><td>Another benchmark, that also benchmarks the RT-preempt patch and
shows the impressive improvements achieved with this patch:
<a class="reference external" href="http://kerneltrap.org/node/5466">http://kerneltrap.org/node/5466</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-7" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[7]</td><td>A course on real-time computing, with the lecture notes.
<a class="reference external" href="http://lamspeople.epfl.ch/decotignie/#InfoTR">http://lamspeople.epfl.ch/decotignie/#InfoTR</a></td></tr>
</tbody>
</table>
<!-- http://www.captain.at/adeos-ipipe-jitter-latency-test.php -->
<!-- vim:spell:spelllang=en_us ft=rst -->
</div>
Croisière des arts et métiers en Sicile2005-03-01T00:00:00+01:002005-03-01T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2005-03-01:/personnal/sicily_2005.html<p><strong>Du 20 au 27 février 2005, récit de Luc</strong></p>
<p>Incroyable, mais vrai : après 3 ans de louze, on a enfin réussi cette
année à s’inscrire à la croisière organisée par le BDE de l’ENSAM, qui se
déroulait en Sicile, cette année. Il y a donc eu 3 équipages …</p><p><strong>Du 20 au 27 février 2005, récit de Luc</strong></p>
<p>Incroyable, mais vrai : après 3 ans de louze, on a enfin réussi cette
année à s’inscrire à la croisière organisée par le BDE de l’ENSAM, qui se
déroulait en Sicile, cette année. Il y a donc eu 3 équipages de
normaliens : un bateau de conscrits la semaine d’avant, et deux bateaux
de vieux cons cette semaine :</p>
<ul class="simple">
<li><strong>bateau de Gaël</strong> : Émanuelle, Adel Dayarian, Céline Descamps,
Christophe (un extérieur, un ami d’Émanuelle), et moi</li>
<li><strong>bateau de Borg</strong> : Éric, Guillaume Moroz, Marguerite Martin, Élodie
Lematte, et deux extérieurs.</li>
</ul>
<p>Pour couper court au suspense, signalons tout de suite que Borg a eu un
karma insolent (enfin, tant qu’il ne s’avisait pas de toucher à son GPS:-),
tandis que l’autre bateau acumulait les tuiles.</p>
<p>À l’arrivée au port, première louze : les bateaux. Ce sont des Bénéteau
Océanis, c’est à dire des veaux prévus pour que pépé et mémé puisse promener
leur chien au moteur par calme plat. Ils remontent mal au vent, sont des
vrais savonettes, il n’y a rien de prévu pour installer des lignes de vie (à
force de tanner le loueur (et bien nous en a pris, comme on le verra
bientôt), il a fini par nous en bricoler avec un bout tendu entre les
taquets d’amarres…) l’enrouleur de génois et les bosses de ris se coincent
tout le temps… Par contre, il y a un chauffe-eau et un guindeau
électrique. Dans le même esprit, il y a un super GPS de la mort qui tue,
mais des cartes imprécises, un compas de relevement merdique, et, pour
bosser sur la carte une microscopique règle-rapporteur inutilisable. Bref,
on dirait du windows, et d’ailleurs, le loueur ne voulait pas qu’on sorte
par plus de force 5… Gaël nous fait un festival de « La voile, c’était
mieux aaaaaaaaaaaaaaaaaaaaaaaaaaavant!» pendant toute la semaine.</p>
<p>Borg ayant le plus gros équipage et la plus grosse réserve de bouffe, il
obtient un plus gros bateau que nous, ce qui lui permet de nous griller en
permanence. À noter que son bateau s’appelle « Camilla », d’où de nombreuses
vannes fines et subtiles à la VHF (« Ma choutte, ma choutte, ma choutte? De
Prince Charles, Prince Charles. »).</p>
<p>Le soir de l’arrivée, c’est « soirée cubi ». Et le cubi sicilien, il est
vraiment pas bon!</p>
<p>On part le lundi matin de Portorossa, notre port d’attache. On longe la côte
(nord) de la Sicile vers l’Est, et, le soir, nous faisons escale au port de
Capo d’Orlando. Nouvelle louze, le port n’est pas à l’endroit indiqué sur la
carte, et l’on doit arpenter la côte pour le trouver.</p>
<p>On repart le lendemain (mardi), avec l’intention d’aller dans l’ile de
Salina. La météo super précise prévoit du force 4..7. À peine sortis du
port, on se prend un vent bien bourrin. Après quelques départs au lof, Gaël
décide de rentrer au port attendre un peu que ça se calme. Et lors du
retour, première casse : un oeuillet de la grand-voile casse. Affalage
d’urgence, et retour au moteur… Après quelques péripéties, Gaël et
Émanuelle réussissent à téléphoner au loueur, qui viendra faire la
réparation (bizarrement, ils utiliseront le compas à pointe sêche pour
percer la toile…). Pendant ce temps, le vent a faibli, et Borg est parti
sans nous attendre, le vilain. Il arrivera à Salina sans encombre. Nous
partons quelques heures après lui, on se prend une alternance de calme et de
vent modérément fort (on est tranquiles avec juste 1 ris), et l’on atteint
le point marqué « X » sur la carte. Là, la trouille de ma vie : en quelques
secondes un orage monstrueux nous tombe dessus (vents de 90km/h, à ce qu’on
nous a dit après coup). On affale les voiles en urgence, et là, mégalouze :
le système pour acrocher la GV affalée à la bôme est complètement merdique
(un taud avec une fermeture éclair), et inutilisable dans ces conditions (il
y a des bouts de toile gonflés qui partent dans tous les sens). Gaël et
Émanuelle finissent par s’en sortir en enroulant des cordes autour de la GV
(ça parait rien, comme ça, mais avec le vent de chien qui fait bouger le
bateau dans tous les sens, c’était vraiment une prouesse, et puis tenir la
barre dans ces conditions était vraiment très difficile : le vent faisait
plus ou moins ce qu’il voulait avec le bateau). Une fois les voiles bien
affalées, notre situation reste précaire : le bateau n’avance plus qu’au
moteur (si il lache, on est vraiment mal), et reste très dur à manoeuvrer.
Gaël finit par prendre la décision qui s’averrera la bonne : traverser le
détroit entre Vulcano et Lipari pour se réfugier au port principal de Lipari
(point « Y » sur la carte). Le passage est stressant : le détroit n’est pas
bien large, et le vent souffle en travers, avec le risque de nous déporter.
Heureusement, ça se passe bien, et après peut-être une heure à être balotés
et trempés, on débouche de l’autre côté où l’on est un petit peu abrité, et
l’on réussit enfin à atteindre Y après un long cauchemar. Bravo à Gaël qui a
su tenir bon et faire The Right Thing dans ces conditions abominables. On a
encore eu de la chance d’avoir droit au début de l’orage où la mer n’était
pas encore trop formée. Un autre bateau s’est retrouvé dans l’orage plus
tardivement, et a galéré encore plus (ils ont eu droit à de la grêle,
d’ailleurs).</p>
<p>Le mercredi, pour se reposer, on se fait une grande marche à travers l’Ile
de Lipari. Borg nous rejoint dans la soirée.</p>
<p><strong>Jeudi</strong> : but de la journée, faire l’aller retour au Stromboli pour voir ses
éruptions de nuit. Tout d’abord, un autre bateau de la croisière a trouvé
moyen de se prendre la pendille dans l’hélice. Christophe, qui est plongeur
et a emmené son matos plonge avec Gaël pour les dégager. Puis nous partons
en milieu de journée. Le vent est d’abord faible, et Borg nous grille de
manière impressionante au démarage (ils reviennent même en arrière nous
narguer). Puis cela forci, et nous arrivons sans encombres au Stromboli au
coucher du soleil comme prévu. On fait le tour, en observant quelques jolies
explosions. Au moment de partir, Borg décide de faire un point au GPS. Au
bout de quelques secondes, le GPS s’éteint, et il s’aperçoit que ses
bateries sont à plat… Il louze un moment sur place, nous permettant de
prendre de l’avance sur le chemin du retour. Comme il n’a plus de VHF, il
fini par communiquer avec nous à coup de SMS. Dans la nuit, le vent faiblit,
et l’on est a plusieurs reprises obligés de nous en remettre au proverbe
« Quand y’en a marre, Yanmar! ». On pensait qu’on allait quand-même réussir
à arriver devant Borg cette fois-ci, mais non : c’est Éric Le Bourrin qui
barre Camilla, et il met le Yanmar à fond et nous grille à quelques minutes
de l’arrivée. Ouin bouhouhoud snif. Nouvelle louze, lors de l’amarrage,
c’est à notre tour de nous prendre la pendille dans l’hélice.</p>
<p><strong>Vendredi</strong> : Christophe plonge pour défaire la pendille de l’hélice.
On part, mais le vent nous pousse sur les amarres du bateau d’à côté, et
ce coup-ci, c’est sa pendille à lui qu’on se prend dans l’hélice. Qu’à
cela ne tienne, Christophe replonge, et on fini enfin par pouvoir
partir… La nav est courte : le but est juste d’aller mouiller à
Vulcano. On réussit néanmoins à craquer le génois (qui avait été pas mal
afaibli les jours d’avant lors d’enroulages foireux). Comble de la louze,
en affalant la GV, on se rend compte qu’elle a une belle déchirure de
10cm de long (en fait, je l’avais vue en l’affalant sous l’orage, mais on
ne l’avait pas revue après). On termine au moteur, et l’on mouille dans
une petite crique de Vulcano. La manoeuvre de mouillage est scabreuse (ou
comment envoyer une ancre en slalomant entre des corps-morts et la chaine
et la pendille du voisin…). L’après-midi, on grimpe au sommet du
volcan. On note un « Ulm torsche » gravé sur la terre du chemin, sans
doute une trace des conscrits nacszes de la semaine d’avant… En
redescendant, on ne trouve rien de mieux à faire que d’aller marcher au
milieu des fumées crachées par le volcan (alors qu’il y avait un joli
panneau prévenant qu’elles étaient toxiques), mais, apparament, c’était
surtout du H2S, donc pas de bobo. Retour au port, surprise : le youyou
de Borg, qu’ils avaient laissé à la rive est retourné tout seul auprès de
maman-Camilla qui est mouillée au milieu de la crique. On soupçonne
fortement un bateau d’archicubes de l’ENSAM, mouillé à côté de nous de
cette gentille blague. Christophe commence à se mettre pour la 3ème fois
de la journée en combinaison de plongée quand ils nous invitent à prendre
l’apéro chez eux (impressionante, leur cave), puis déposent, en partant
l’équipage de Borg sur Camilla.</p>
<p><strong>Samedi</strong> : debout à une heure impossible pour un retour au moteur sans
histoire vers Portorossa. Le soir, les organisateurs nous ont concocté
une sangria-cubi absolument imbuvable à moins de la couper avec du
nectar-multivitaminé.</p>
<p><strong>Dimanche</strong> : navion pour Paris…</p>
<hr class="docutils" />
<p>Luc, qui que merci encore à Gaël de nous avoir skippé et de nous avoir
sortis indemnes de l’orage!</p>
Raid catamaran en baie de Quiberon2004-09-12T00:00:00+02:002004-09-12T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2004-09-12:/personnal/raid-catamaran-en-baie-de-quiberon.html<p><strong>Du 6 au 10 septembre 2004. Récit de Clara</strong></p>
<p>Naviguer sur un catamaran de sport pendant une semaine, dormir sur la
plage, tracer sa route au petit matin selon les conditions météo, tirer
des bords de près de plus de deux heures.. Voilà une idée qui me tentait
bien. Pourquoi …</p><p><strong>Du 6 au 10 septembre 2004. Récit de Clara</strong></p>
<p>Naviguer sur un catamaran de sport pendant une semaine, dormir sur la
plage, tracer sa route au petit matin selon les conditions météo, tirer
des bords de près de plus de deux heures.. Voilà une idée qui me tentait
bien. Pourquoi pas ? A force de regarder des émissions d’aventure qui a
chaque fois m’obligent à me dire « pourquoi pas moi ? » L’idée de ne pas
accepter cette proposition était inenvisageable..</p>
<p>L’idée leur est venue un soir, paraît-il, dans la cuisine de l’île de
Drenec aux Glénans, après plusieurs verres de chiwawa, boisson fétiche
des îles bretonnes. !</p>
<p>Le lagon des Glénan est particulièrement accueillant pour la voile
légère, surtout lorsqu’il s’agit de se balader dedans avec une escadre de
cinq ou six bateaux qui nous suivent. Seulement, au bout d’un mois, d’un
an, ou même de plusieurs années de navigation la bas, on a envie de
sortir du fameux périmètre de navigation autorisé. Venir sur ces îles en
tant que moniteurs bénévoles a l’école de voile laisse a chacun des
merveilleux souvenirs, mais l’océan nous tente, l’envie de partir d’un
endroit pour arriver autre part, voir du paysage, faire de belles nav
sans se soucier de l’heure du goûter ou du dîner. autant de désirs qui
leur donne un soir envie de monter ce petit projet.</p>
<p>C’est pourquoi, Gaël et Olivier, les deux principaux meneurs de
l’opération se décident vite, une fois rentrés à Paris, de commencer à
organiser ce qu’ils vont appeler : Le Raid.</p>
<p>La zone de navigation est choisie sans trop d’hésitations. ce sera dans
le Morbihan. Les paysages bretons y sont particulièrement beaux, c’est
une zone connue par les membres de l’équipage, et qui peut par sa
facilité d’accès nous permettre de réaliser ce rêve.</p>
<p>Ainsi, Gaël, étudiant à l’ENS de Paris s’applique à chercher des bateaux
a louer. Il trouve son bonheur, qui sera aussi par la même occasion le
nôtre, au Nautic Club de Carnac. En effet, ce loueur de bateaux propose
des raids cata au départ de Carnac jusqu’aux îles de Houat, Hoedic et
Belle-Île. Ils disposent donc du matériel nécessaire a l’opération, et
seraient prêt à louer leur trois Nacra 570 avec l’équipement qui nous
serait indispensable tels les bidons étanches, ancre lourde, et vêtements
nautique comme les ceintures de trap, gilets de sauvetages et combinaison
isothermes pour ceux qui en auraient besoin.</p>
<p>Le niveau de l’équipage étant relativement correct, ils acceptent de nous
fournir nos bateaux, selon les conditions d’assurances relatives à notre
projet, et à un coût plutôt acceptable pour ce genre de location !</p>
<p>La date est fixée, ce sera du 5 au 10 septembre 2004.</p>
<p>C’est a peu près a ce moment là de l’histoire qu’Olivier, rencontré à
Penfret à Pâques lors de mon A2C1 cata me propose de partir avec eux, ils
leur manquaient une personne et je suis plus que ravie et motivée pour
participer au raid avec eux, bien que je ne connaissais que lui !</p>
<p>Premier rendez-vous organisé en mai chez Gaël et Emmanuelle, sa petite
amie dans leur chambre a l’ENS. On ouvre en grand la carte de la Baie de
Quiberon, et regardons les differentes possibilités de trajets. Le mieux
serait donc de partir de Carnac direction les îles. Seulement un problème
se pose : en voile légère, la législation oblige les embarcations à ne
pas s’éloigner de plus de un mile d’un abri côtier, ce qui équivaut à
1,56 kilomètres environ. Ainsi, on se retrouverai dans l’illégalité sur
une petite zone, entre la pointe de Quiberon et l’île de Houat et, en
tant que marins responsables, on ne peut pas prendre ce risque. Deux
solutions s’offrent à nous : soit se faire accompagner d’un bateau à
moteur, comme les formules que proposent le club de Carnac, soit demander
une dérogation aux « aff mar » (département des affaires maritimes) pour
avoir l’autorisation de naviguer dans cette petite zone qui nous est
interdite. Olivier se chargera donc de ça. On lui envoie nos CV
nautiques, Gaël, Olivier et moi ayant validé notre niveau A2C et
encadrant dans la fameuse école de voile des Glénans, on se dit qu’ils
n’hésiteront pas à nous accorder ce petit privilège. Il décrit notre
projet, tout en expliquant les raisons qui nous entraînent à demander
leur autorisation.</p>
<p>La réponse étant longue à arriver, on se prépare tout de même a d’autres
trajets, qui seraient de descendre la cote vers le sud, allez s’amuser
dans les courants du Golfe du Morbihan, et rester dans cette zone..</p>
<p>En ce qui concerne tout l’équipement autre que nautique, Gaël et
Emmanuelle se chargent de tout, étant habitués à partir sac au dos avec
le minimum vitale pour survivre !!! Ils s’occuperont donc de toute
l’alimentation (à mes plus grands regrets, grande gourmande que je suis)
et de l’équipement terrestre comme les bâches, les butta-gaz et autre
matériel de camping. L’idée étant que la place sera vraiment très très
restreinte et quel seul ce qui nous est vital sera embarqué sur les
trampos des catas. On n’emportera pas de tentes, en espérant que notre
vie de routard nous obligera à dormir à la belle étoile tous les soirs.
Les haches serviront à se protéger du sable pour les repas ainsi que pour
les affaires fragiles, dormir dessus si besoin, et les installer en toit
si la pluie nous tombe dessus !!!!!! Chacun est prié de venir avec le
maximum de bouts généralement récupérés dans les fonds de poches de nos
vestes de quart quand matos oblige sur nos catas lors des stages aux
Glénans. Sacs étanches sont aussi les bienvenus, permettant de
s’accrocher facilement sur les cata, et pratiques parce qu’étanche !!!!
(En effet, vu la constitution de nos embarcations, il ne valait mieux pas
craindre d’être mouillés pendant une semaine !!!!)</p>
<p>Voilà, on a presque tout :</p>
<ul class="simple">
<li>6 personnes motivées à partir pendant une semaine et a vivre dans des
conditions les plus précaires pour vivre une nouvelle expérience de
voile.</li>
<li>3 bateaux de deux nous permettant de nous déplacer</li>
<li>tout l’équipement nautique nécessaire, l’équipement de sécurité étant à
moitié fourni par le Nautic Club, le reste par Gaël (VHS portable,
cartes marines du coin.)</li>
<li>tout l’équipement de camping toujours fournis par Gaël et Emmanuelle.</li>
</ul>
<p>On tente à plusieurs reprise de tous se rencontrer, mais Eric, étudiant
aussi a l’ENS avec notre couple organisateurs est en Australie,
Guillaume, très bon ami d’Olivier étant en retard de plus d’une heure et
le petit couple pensant que c’est le WE prochain. la réunion ne dure que
peu de temps (j’ai un partiel d’histoire le lendemain. !!) et nous ne
sommes que trois !!</p>
<p>Mais tant pis, maintenant que tout est prêt, on ne va pas faire
d’histoire parce qu’on ne se connaît pas !!! et puis, ce sera aussi
l’occasion de rencontrer des gens et de faire connaissance lors de notre
périple en Bretagne Sud.</p>
<p>On s’organise par e-mails pour les derniers préparatifs comme par exemple
le moyen de locomotion Paris-Carnac. Certains seront déjà dans la région
car le bénévolats sur nos îles paradisiaques nous retiens en Bretagne
jusqu’à même fin septembre. !!!</p>
<div class="section" id="dimanche">
<h2>Dimanche</h2>
<p>Je suis sur le quai de la Gare de Rosporden près de Concarneau. On est le
dimanche 5 septembre, je viens de passer avec Olivier une semaine à
Penfret pour hiverner l’île. Je ne ressemble pas a grand chose. Mes
cheveux n’ont pas été brossés depuis plus de deux semaines, seules mes
lunettes de soleil (qui me font ressembler à une mouche nous dirait
Olivier) retiennent mes mèches salées et bouclées d’arriver dans ma
figure toute bronzée et abîmée par deux mois de sel de mer et de soleil,
je porte un débardeur blanc Petit Bateau trouvé sur Penfret 2 jours avant
et un pantalon africain rouge vif qui se noue à la taille et aux
chevilles ! Sans bien sur oublier mon éternel foulard qui me sert à la
fois d’écharpe, d’oreiller la nuit et de serviette a table mais cette
dernière utilisation n’est généralement pas voulue, c’est lui qui se
trempe tout seul dans mon café du matin ou dans la soupe du soir. Mes
pieds sont nus car je ne supporte plus d’avoir des chaussures. Mes tongs
rouges sont donc accrochées à une bretelle de mon sac a dos. Sac a dos
énorme que je traîne depuis le 1er juillet qui contient 50% de fringues
nautique : chaussons et gants pour la voile, salopette cirée, veste de
quart, combinaison shorty, top en lycra, boléro tribord. A coté se
trouve mon sac étanche remplis de ma pharmacie personnelle (je suis
toujours malade !!), et tout ce que je ne supporte pas de laisser à Paris
comme des albums photos de ma famille et de mes amis, mon téléphone
portable bien qu’éteint, mon portefeuille bien que vide ! des appareils
photos, étanches ou non pour immortaliser chaque instant de ma vie, mon
carnet d’adresse on ne sait jamais s’il faut que j’écrive à quelqu’un et
tous mes petits souvenirs matériels que je garde précieusement afin de
les laisser moisir dans mon tiroir a Paris.</p>
<p>C’est marrant, je ne sais pas vraiment ou je pars, je ne sais pas ou je
vais dormir le soir même, ni même les 5 prochains soirs. je ne sais pas
où je vais, ni vraiment avec qui. Et s’il y avait une énorme tempête et
qu’on se retrouvai coincés sur une île du genre Gavrinis sans pouvoir
rentrer à temps a Carnac ? et si on il pleut toute la semaine ?? ça va
être invivable et le camping sous la flotte, ayant fait ça 3 semaines
auparavant, je sais ce que ça donne et je n’ai pas vraiment envie de
revivre ça. et si y’avait pétole toute la semaine ? et si on démâtait en
plein milieu de l’océan atlantique trop loin des cotes pour avertir
quelqu’un parce que forcément la VHS serait tombée a l’eau, nos portables
inaccessibles aux fonds des bidons. l’espace d’un moment je me mets un
peu à flipper. et si notre raid d’une semaine devenait un véritable
cauchemar ?? on ne sait jamais a quoi s’attendre sur l’eau. il peut
devenir notre pire ennemi aussi bien qu’une source de plaisir intense.</p>
<p>A un moment, je me dis qu’il serai grand temps que je déjeune. Mais le
distributeur de billets ne fonctionne pas, alors que je n’ai que deux
euros dans ma poche. Je vais donc acheter un petit sandwich au bar qui se
trouve à coté de la gare. Seulement, un dimanche de septembre, ils ne
font plus de sandwich, normal personne ne passe par là ! Je me retrouve
donc à me nourrir grâce au distributeur automatique : un paquet de chips,
un paquet de mini cookies Bonne Maman pour le dessert, et une bouteille
d’eau, j’ai très soif et pas question d’aller boire l’eau au robinet des
toilettes !</p>
<p>Cela fait plus de 2 heures que je dors sur le quai de la gare attendant
le train qui m’amènerait alors à Auray afin de retrouver mes 5 compères.</p>
<p>Sur le quai d’Auray, j’aperçois au loin Olive, qui d’ailleurs s’est coupé
les cheveux, sa seule dread n’étant vraiment plus supportable ! Lui aussi
se traîne son sac étanche rouge et noir, qui doit faire deux fois la
taille du mien, qu’il a préalablement fixé sur une armature en fer d’un
vieux caddie ! J’en profite aussi pour remarquer qu’il a changé de tongs,
les anciennes dûment popées dans l’atelier de Penfret n’étant vraiment
plus utilisables !</p>
<p>Mais comme je n’étais restée que cinq heures dans la gare de Rosporden
avant de venir, je me suis dit qu’il fallait bien que je profite de celle
d’Auray!!</p>
<p>Alors on s’est installés sur un banc a l’intérieur de la Gare, bien au
frais car, début septembre c’est dingue comme il fait chaud en Bretagne
!!! On profite pour aller remplir nos bouteilles d’eau au bar du café de
la gare, et lire un peu les magazines de voiles que j’avais achetés
mi-août a la gare d’Avignon ! C’est d’ailleurs a ce moment la que
j’apprends que nous n’avons pas obtenu la dérogation des aff mar et donc
que Houat ne sera pas pour nous cette année !! Tant pis, la Bretagne
regorge de coins et de plages sympas ou nous serons bien installés !!!</p>
<p>Vers 17h, Gaël arrive dans sa petite voiture pour nous amener à Carnac,
où Guillaume, Eric et Emmanuelle nous attendent. Pendant le trajet on se
raconte rapidement nos expériences de catas de l’été, ayant chacun passé
la majorité de nos vacances à encadrer aux Glénans ! De toute façons, on
était la pour faire du cata, on n’allait pas parler d’autre chose que du
cata ! un peu de lasers, d’habitable. et encore !!! fallait que ça reste
dans le thème !!.</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Ca y est. le départ est proche !! Enfin ! Depuis le temps qu’on en parle.. !</p>
<p>On passe par la capitainerie du port de La Trin’ afin de jeter un coup
d’oeil a la météo. C’est bon, ils annoncent du soleil et du vent (environ
5 a 6 beauforts dans mes souvenirs) ! Mais on connaît la météo, on sait
bien qu’ils exagèrent un peu parfois histoire de prévenir les accidents !</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/001.jpg" style="width: 50%;" />
<p>On arrive alors a Carnac, et on rejoint les trois autres sur la plage.
Nous voilà enfin au complet ! Ainsi voila les 6 qui partent comme ça sur
des catas de sport afin de profiter pleinement de leurs performances de
vitesses et de sensations fortes et cela pour nous rien que pour nous six
pendant une semaine !</p>
<p>Les trois plus excités de voir la tête de nos bateaux ce sont Olive, Gaël
et moi ! A quoi ressemblent-ils vraiment ? J’ai déjà vu des nacras dans
des clubs, mais ils n’étaient pas aussi grands.. et puis, dans quels
états sont-ils ? quel est l’accastillage ? quelle est la couleur des
voiles ? des écoutes ?!!!!. autant de questions qui nous trottent dans la
tête depuis longtemps !</p>
<p>On rencontre alors Manu, responsable du Club qui nous les montre. et
ils sont bien a la hauteur de nos espérances. des coques casses vagues.
j’AdoOOOoore !!! c’est bien plus stable ! et puis, quel accastillage !
tout harken ! le chariot de GV est a bille et fonctionne tellement bien,
rien à voir avec ceux de nos éternels « 16 » qu’il faut régler avec le
pied voir même avec la main tellement ils ne bougent pas !!! et puis, des
sticks télescopiques. un réel bonheur pour sortir au trap’</p>
<p>On les testera demain matin, avant de partir, histoire de se faire une
petite idée de ce que ça va donner, et puis pour voir si on a un peu de
matos à faire. pour bien être en sécu toute la semaine ! La formation des
Glénans se reconnaît la ! Il n’est pas question de partir sur l’eau avec
un petit problème technique qui pourrait s’aggraver en pleine séance.
Surtout que là on ne rentre pas le soir même où des matos et un atelier
nous attendent, nan, on ne sait pas vraiment si on pourra réparer nos «
canots » en plein raid alors autant être surs !!!</p>
<p>On voit un peu avec le « grand chef » quel sera notre trajet, on règle
ce qu’on leur doit, on choisit nos gilets de sauvetages ; les ceintures
de trap et pour certains leurs combinaisons intégrale.</p>
<p>La honte, les gilets sont marqués UCPA, pour des membres des Glénans, on
est un peu réticent à les porter alors on opte pour l’option. on mettra
nos tops tribords par dessus !!! comme on s’équipe tous chez Décathlon,
on a tous le même !! Un vrai équipage !</p>
<p>Les ceintures de trap sont un peu usées, va falloir bien la choisir car
autant qu’elle tienne toute la semaine et qu’elle ne nous fasse pas trop
mal ! Je me souviens d’après midi aux Glénans où j’avais mal choisi ma
ceinture et j’étais rentrée avec les hanches en compote et le dos qui me
faisait tellement mal qu’impossible de me lever le lendemain matin !</p>
<p>On est un peu nerveux. moi qui à chaque fois que je pars sur l’eau en
cata, ce n’est que pour grand maximum une journée (pic nic sur le loch’
réguliers !) , ça me fait bien bizarre de me dire que la c’est pour une
semaine..</p>
<p>On observe un peu le fonctionnement du Nautic Club. C’est un point
passion plage alors n’importe qui peu louer n’importe quand n’importe
quel bateau. Les moniteurs amènent les bateaux tout bien grées au bord de
l’eau, expliquent vite fait que pour avancer faut tirer sur le bout de
ficelle qui tiens la voile et que faut s’attacher au bateau lorsqu’il
dessale. Ca nous laisse perplexe ! Et la Sécu dans tout ça ???
heureusement qu’ils vérifient avec leurs jumelles régulièrement si ça se
passe bien et qu’ils sont prêts a intervenir avec leur Zodiac !!!..</p>
<p>Mais l’heure passe et il va falloir trouver une petite plage pour dormir.
Gaël connaît bien le coin et a déjà une idée en tête. Il part donc avec
Olive et Guillaume (normal, a six dans une petite voiture genre 205 on
oublie vite surtout lorsqu’on voit la taille de nos sacs a dos !!) me
laissant sur la plage avec Eric et Emmanuelle.</p>
<p>La plage, impossible de vous dire laquelle s’était, mais une chose est
sure. c’est qu’on était les seuls ! A part que sur le parking de la plage
étaient installés des romanos, et l’idée de me dire que mon énorme sac
avec tous mes trésors dedans puisse se faire voler me faisait un peu
peur.</p>
<p>On installe alors deux haches : l’une pour dormir histoire de s’isoler de
l’humidité du sable, et l’autre pour poser les buta-gaz et autres
affaires fragiles.</p>
<p>C’est aussi a ce moment que je découvre avec le plus grand regret bien
qu’il aurai été difficile, voir même dommage d’acheter autre nourriture,
notre menu pour la semaine.</p>
<p>Gaël et Emmanuelle s’étaient occupés de tout ! Pour le petit dej on avait
des muesli (je vous expliquerai comment un matin on en a eu avec du
chocolat dedans, un grand luxe !!) avec du lait en poudre mélangé avec de
l’eau douce. Les raisins secs au petit matin. de quoi me mettre de
mauvaise humeur !!! mais apparemment je n’étais pas la seule à qui ça ne
plaise pas plus que ça de devoir manger ces petits trucs foncés le matin
!</p>
<p>Pour les déjeuners, ce sera pain, charcuterie et fromage. On achètera du
pain tous les deux jours et nos provisions de « mets d ‘accompagnements »
seront à renouveler au milieu du raid. Comme désert des fruits secs ou
des figolus.. Comme goûter, parfois du chocolat, mais généralement ça
s’arrêterai aux figolus.</p>
<p>Et pour le dîner. ma hantise ! (je n’en veux pas au petit couple,
seulement pour la bouffe, je suis très exigeante, voir même très très
très exigeante et ça m’arrive souvent de préférer ne pas manger plutôt
que de prendre ce que je n’aime pas !)</p>
<p>En fait, un jour ça serait des pattes genre cheveux d’anges (ça ne prend
pas beaucoup de place par rapport a ce qu’on consomme) ou bien du riz. Il
faut des féculents tous les jours alors voilà ! et pour faire style on ne
dîne pas tous les soirs la même chose, des sauces en poudre (goût
espagnol, gout italien (bolognaise), trappeur..) ! Le seul problème,
c’est qu’il fallait économiser l’eau douce qu’on transportait en
bouteilles accrochés à nos bateaux, alors ce qu’on faisait : 2/3 d’eau
douce et 1/3 d’eau de mer ! ce qui permettait aussi de saler notre dîner
(une salière, mine de rien ça prend la place et je le rappelle. on
n’emporte que les choses VITALES !!!!!) mais, y’a des sauces qui sont
déjà salées.. Alors là, fallait vraiment avoir faim.</p>
<p>Mais les déserts étaient bons, logique, au chocolat !!! des espèces de
crèmes chocolat en poudre qui mélangées à du lait en poudre et de l’eau
nous faisait oublier le plat précédent !</p>
<p>Pour ce qui était du service de table. 2 casseroles puisque 2 butta-gaz,
rien de plus logique ! des petits bols/assiettes en plastique, 6
fourchettes et une ou deux cuillères. !!! pour les couteaux, nos whisards
ou laguioles faisaient l’affaire !!!</p>
<p>Je suis donc installé dans mon duvet (toujours le même depuis 6 ans !)
entre je ne sais plus qui et je ne sais plus qui ! Mon écharpe me sert
d’oreiller et je me suis fait un petit trou dans le sable pour me faire
croire que je suis dans un bon lit douillet ! Mais bon, de quoi se
plaindre, certains donneraient beaucoup pour être loin du béton armé et
des bruits de voitures de Paris. On se laisse doucement bercer par les
bruits de mer pour s’endormir. demain, on part et faudra être en forme
!!!</p>
</div>
<div class="section" id="lundi">
<h2>Lundi</h2>
<p>Lundi matin, on est réveillé tôt par le lever du soleil et par l’habitude
aussi du petit dej entre 8h et 8h30.</p>
<p>Allez, on se dépêche, on range toutes nos affaires. Gaël, Emmanuelle et
Eric partent dans un 1er trajet. Avec Olivier et Guillaume on finit de
rouler la dernière bâche, de boucler nos sacs a dos et on attend Gaël qui
reviens peu de temps après avec nos croissants !</p>
<p>On passe par la Trin histoire de regarder encore la météo a la
capitainerie, on passe faire un petit tour aux toilettes (ça fait partie
de l’histoire, c’est vrai, dans les films on ne les voit jamais aller aux
toilette alors la, vous ne pourrez pas dire qu’on n’est pas normaux !!!),
et par le super marché de place (près de la maison de ma copine Lorraine
!! ça vous dit tout non ??!!!) pour acheter les derniers trucs qui nous
manque. du nutela (qu’on laissera à Carnac pour qu’on puisse en profiter
le jour du retour), du papier alu pour les butta-gaz et des lunettes de
soleil pour Guillaume (et voui, pas de chance, t’a pas la réduc des
Glénans chez Julbo à Concarneau !!!) !</p>
<p>On arrive donc au Nautic Club. On va près des bateaux et la, les mecs du
Club commencent à les gréer. Quoi ?! ne pas gréer nos bateaux nous même ?
mais c’est impensable et rapidement, deux par deux on commence
l’inventaire ! Bien sur, Olivier, Gaël et moi on le connaît par coeur ! On
commence par les coques. leur état.. pas de pops, c’est bon signe ! on
remarque juste qu’ils ont mis du Sikka là où nous aurions mis du gel
coat. on ne dit rien !</p>
<p>Ensuite, les trampos, on les resserre, pas question d’un filet qui
pandouille entre deux coques. les étais, et haubans, pas de gendarmes !
on vérifie rapidement le parallélisme des safrans, on remplace tous les
sandows pour être certains qu’aucun ne lâchera durant le raid.</p>
<p>Et, pris d’une excitation terrible avec Olive. Le WD40 !!! on en met
partout !!! sur les rails des chariots de foc et de GV, sur le stick
amovible, sur les poulies. !!!! C’est tellement bien un cata qui
fonctionne sans problème !!!!</p>
<p>On apprend alors à hooker la GV ! Habitué à la petite fourchette du 16,
ce crochet nous donnera plus d’une fois l’occasion d’énormes prises de
tête !!!!</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/002.jpg" style="width: 30%;" />
<p>Il est environ 11h et on est prêt à aller sur l’eau pour voir ce que ça
donne ! Comme tout vrai stage Glénans, on décide de faire dessaler les
bateaux pour voir quelles sont leurs limites et surtout comment se passe
le ressalage. En plein raid, pas question de dessaler, avec tout ce qui
sera attachés aux trampos, ça sera beaucoup plus dur et surtout pas très
sécu, les risques de chapeautages étant du coup plus élevés.</p>
<p>Les équipages se font naturellement. 3 bateaux. 3 A2C, Gaël sera donc
avec Emmanuelle, pas question de les séparer, Guillaume avec Olivier,
normal, ils sont potes ! et Eric avec moi, normal on ne se connaît pas
!!!!</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p><strong>WAHOUOUOUOU !!!!!!!!!!!</strong></p>
<p>Le départ entre les deux moles n’étant pas ce qui est le plus agréable à
faire ! Mais c’est pas grave, on est sur l’eau.</p>
<p>Rien à voir avec le hobbie cat 16 sur lequel j’ai navigué 6 semaines cet
été.. !!! il est carrément moins nerveux mais tellement plus stable. Au
portant, les risques d’enfournement (ma hantise) sont bien moins élevés,
ils bananent très bien (Eric me félicite d’ailleurs pour la 1ere que j’ai
faite, a mon plus grand plaisir, très bien maîtrisée !), et la vitesse
m’impressionne un peu !</p>
<p>On découvre aussi la bordure de la GV réglable a l’aide d’un petit
chariot et, pour notre plus grand plaisir (plus pour le trip que pour la
performance en elle-même d’ailleurs) de pouvoir régler son cuni au trap,
un système avec deux poulies le permettant.. !!!</p>
<p>Mon premier virement de bord n’est pas passé. ! Bon d’accord, il y avait
un peu de houle et je n’ai pas attendu d’être sur la crête de la vague
pour pousser ma barre !!!!!!! mais Olivier et Gaël me rassureront par la
suite qu’eux aussi ont manqué leur 1er ! c’est une histoire que coque
m’affirment-ils. très fiers !</p>
<p>Bien sur, Olivier pressé d’impressionner Guillaume fait son kéké et très
rapidement ils passent à la bail !!! mais le ressalage se passe bien pour
eux, je suis rassurée !</p>
<p>Mais, bien évidemment, il fallait bien nous aussi y passer ! Et il n’a
pas été volontaire ! Une banane un peu mal tenue et hop ! Nous voilà tous
les deux a la flotte ! Eric et moi remettons un certain temps à ressaler,
notre poids n’étant pas très élevé ! Je me souviens aussi du deuxième où
je n’ai pas eu le temps ni la force physique de remonter directement sur
la coque. mais heureusement Prince Eric (c’est dans un Walt Disney non
??!!!) m’a sauvé en me lançant le bout de ressalage afin de rentrer à
Carnac avec moi a bord !</p>
<p>Pour Gaël et Emmanuelle, ce fut a peu près le même topo ! Un ou deux
dessalages dans les règles de l’art !</p>
<p>On s’amuse un peu mais il faut bien déjeuner, et le temps presse ! pas
question de partir à 23h ce soir !!!!</p>
<p>On déjeune rapidement nos petits sandwichs à l’ombre de la cabane à
outils du nautic club !</p>
<p>Et puis, voilà, on commence à faire les sacs. Il y a 6 bidons, 3 gros, 3
petits, deux sacs étanches (le mien et celui d’Olive) et une pochette
Magic Marine qu’on peut accrocher sur le trampo !</p>
<p>Chacun a son bidon et on rajoute ensuite la nourriture et les affaires
communes là où il y a de la place. On part 5 jours, mais pas question de
perdre de la place. Ce sera pour chacun un pantalon (sauf Eric qui
préfère son short mais un jour il le regrettera !), un tee shirt, une
polaire, un k-way, des souvetement, et une écharpe ! Ah si, j’oubliai les
tongs, et les chaussettes !!!</p>
<p>Dans l’énorme sac étanche d’Olive on met les 6 duvets, et dans le miens
toutes les haches et les butta-gaz.</p>
<p>Seulement il manque sur un des catas un trapèze. alors on dessale nacra
500 qui ne partira pas avec nous pour lui enlever le sien pour le
remettre sur le nôtre ! Les gens du Nautic nous regardent faire ! On
dessale aussi l’autre cata afin de rajouter le trap, on en profite pour
bien resserrer la manille de capelage ! On remplace aussi des écoutes de
foc pas certaines de tenir le coup jusqu’à la fin de la semaine.</p>
<p>Ca y est, le matos est fini ! C’est le moment d’accrocher nos effets
personnels sur les catas ! Chaque duo aura sa technique bien a lui,
certains seront plus maniaques que d’autre. ! Sur le notre, notre petit
cata a nous pendant une semaine, il y a nos 2 bidons, le pack d’eau
douce et mon sac étanche ! Nous avons choisi l’option on accroche le sac
et le pack au pied du mat et les bidons sur les trampos, les bouts
passant par dessous, faisant un tour avec les haubans. tout un art !</p>
<p>Les gens du Club, aussi bien que les stagiaires ou clients peu
expérimentés nous observent depuis un petit temps ! On fait un peu
l’attraction il faut le dire ! Equipés comme des pros (des chaussons
jusqu’aux gants, lycra, combi, boléro, ceinture de trap et gilet de
sauvetage, couteaux qui pandouillent accrochés aux brettelles des bas de
cirés.), déambulant de droite a gauche depuis un ptit bout de temps
hurlant « il est où le WD40 ?? merde ya un sandow qui va lâcher faut le
remplacer ! c’est toi qui as la multiprise ? t’aurai pas vu un ptit bouts
que j’avais posé là ? mais alors elle est ou cette multiprise mon taquet
de GV est trop bas ça va m’énerver au trap. !!! »</p>
<p>Je suis certaine que les responsables ont fait des paris sur nous du
style « dans deux jours ils reviennent avec de la casse ! » et les
locataires qui passent dans le coin nous posent tout plein de question du
style « Mais, mais . ? vous partez combien de temps la ? ah bon ? mais
vous aller dormir où ? ah bon, bah c’est que vous etes des pros non ??! »
Non, juste des passionnés qui veulent profiter jusqu’au bout de leur
vacances, pour se faire plaisir pendant une petite semaine ! Et ne vous
inquiétez pas, on n’a pas lésinés sur la sécu !!!</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>17h. il est grand temps de partir pour de bon, pour de vrais, la, ça y
est de s’éloigner de Carnac au maximum !</p>
<p>Cap sur le Golfe du Morbihan, on aura largement le temps d’y arriver
avant le coucher du soleil.</p>
<p>Avec 10 noeuds de vent environ, on sera dans les temps. !</p>
<p>Au revoir les gens du Nautic Club, a Vendredi bien sur !!!! On vous tient
au courant de notre position tous les deux jours ? nan, vous vous en
fichez, d’accord, tant pis !!</p>
<p>Gaël en tête, Olivier bien sous notre vent (forcément à force de vouloir
bananer on perd sa trajectoire), Eric et moi sommes tout bien installés,
lui a la barre, moi au foc toute tranquille savourant chaque instant de
ce départ. Un bruit étrange me sort de ma rêverie. Ca vient du point
d’écoute de la GV. Je jette un coup d’oeil, rien d’anormal, je replonge
dans mes pensées quand tout a coup, OH surprise, le crochet se fend en
deux et la GV se choque en grand en moins d’une seconde ! Bah ? c’est
quoi cet accastillage qui ne tiens même pas sur un bord de près ? En
fait, Eric en l’accrochant s’était trompé de d’endroit et avait mis le
crochet dans le rail de la bordure, endroit tranchant ! Alors forcément
avec la pression. ça ne tient pas le coup !</p>
<p>Et vive le matos en pleine nav’ ! Gaël et Olivier nous rejoignent et se
mettent à la cape sous notre vent ! avec beaucoup de mal on défait un des
crochets devant servir pour le cuningham (y’en avait 2 , donc dans ma
tête, un qui ne servait à rien alors pourquoi pas utiliser celui là !!)
pour le remettre à la place du cassé ! Plus de peur que de mal, notre
réparation de fortune tiendras toute la semaine !! Vive nous !</p>
<p>On passe sous le vent de rochers, qui, dans mon souvenir doivent
s’appeler les Méabans (il y en a aussi dans le lagon) et rapidement ,
trop même , on arrive à l’entrée du Golfe ! On passe donc devant Port
Navalo. J’aperçois la maison de Vincent située à l’entrée du Golfe, alors
qu’est ce que je fais comme toute personne normalement constitué, je sors
mon appareil photo pour immortaliser cette entrée ! C’est en fin de
journée, la lumière éclaire donc merveilleusement bien la cote bretonne.
rien au monde n’enlèvera ma joie d’être sur l’eau vers les 18h30 / 19h a
l’entrée du Golfe sur un performant cata début septembre alors que
certains de mes potes ou même ma soeur, sont entrain de réviser leur
cours pour la session de rattrapages. J’ai beaucoup de chance, j’en suis
consciente mais en Juin je m’étais mis dans la tête qu’ il n’était pas
question que je remette les pieds a la fac avant Octobre.</p>
<p>Je me souviens aussi d’Olivier faisait, a sa grande habitude son kéké en
tirant des bords travers/travers devant a l’entrée alors que j’obligeais
Eric à suivre dans son sillage Gaël, n’étant pas très rassurée après
avoir entendue les histoires des Glénanais d’Arz et des dangereux
courants. Gaël connaît le coin, il sait par où passer. Je nous laisse
guider !</p>
<p>On passe donc devant Gavrinis. Cette ile représente pour moi deux choses.
La première, c’est un projet qu’on avait voulu mettre en place avec des
amis en Première. On voulait camper à la Roots pendant une semaine sur
cette ile (elle appartenait alors à l’une d’entre nous). En quelques
sorte j’étais entrain de réaliser ce rêve, avec d’autres personnes et
d’autre différence , mais ça se rapprochait. La deuxième chose a
laquelle cette ile me fait penser, c’est à une de mes peurs. En effet,
pendant la semaine de fermeture de Penfret, on dînait dans le Quart avec
le VHS branchée sur le canal 16. Un soir on a entendu la SNSM dire
qu’elle allait chercher deux plaisanciers et leur 420 sur Gavrinis, ils
s’étaient fait prendre par les courants et il n’y avait plus assez de
vent pour rentrer à Vannes. ce qui nous avait fait assez rire à ce moment
là. Au fond de moi , j’espérais qua ça ne nous arriverait pas !!</p>
<p>Gaël nous conduit donc à une petite ile, Berder je crois bien. Au moment
d’arriver à la plage , Gaël devant nous remonte ses safrans. mais, au
lieu de les faire un par un, il donne un énorme coup sur la barre de
liaison qui se casse en deux !!! Derrière, j’explose littéralement de
rire ! Pour une fois qu’il n’y a pas d’histoire de cames qui ne
fonctionnent pas. !!!!!</p>
<p>En plus juste avant de partir on avait discuté avec le type du Nautic
Club qui nous avait dit que ces barres de liaisons étaient très fragiles,
que les gens les cassaient tout le temps parce qu’ils ne savaient pas
remonter des safrans. On a l’air malin maintenant !!!</p>
<p>On commence à dégréer nos bateaux, mais on peine sur le déhookage de la
GV ! Sacré crochet ! et puis, les catas, faut les remonter assez hauts
pour que lorsque la marée sera haute, nos bateaux et nos affaires ne
soient pas mouillés. Mais y’a un truc auquel on n’avait pas vraiment
pensé, enfin, si on s’était posé la question mais nous n’avions pas
encore eu la réponse. est-ce que les catas sont lourds ?. OUI ! oui, ils
le sont ! alors technique.</p>
<p>Les deux filles à l’avant, et deux garçons au niveau des safrans !
Heureusement qu’il n’y en à que trois !! et puis, formation Glénans nous
oblige instinctivement à les installer bien parallèles les uns par
rapport aux autres !</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/003.jpg" style="width: 30%;" />
<p>Pendant que Guillaume et Olivier nous préparent un petit feu de joie,
Emmanuelle et Gaël préparent notre diner, tandis que je me balade sur la
plage, pieds nus, fumant ma petite clope.</p>
<p>Mais, au moment du diner, installés alors autour du feu, il y a quelque
chose qui commence sérieusement à me déranger. Déjà que je mange du riz
sauce paella et que je deteste la paella, je sens sur ma peau des petites
bestioles. Des puces de sable. Le truc horrible ! J’ai beau me dire fille
de la nature et tout le blabla, j’ai une phobie des petits insectes.
Comment les éviter ? je mets donc des chaussettes, comme ça je ne les
sentirais pas sur mes pieds et je grimpe sur une coque d’un cata pour
être isolée du sol. Pas question que je dorme sur le sable ! Pour moi,
mon lit sera un trampo de cata !!! Gaël a beau me dire que tant que les
insectes ne piquent pas, ce n’est pas gênant, je n’arrive pas à m’y faire
!</p>
<p>On discute un bout de temps autour du feu, jusqu’au moment où on à tous
envie de se coucher ! Notre première journée a été riche en émotions et
on ne sait pas ce demain nous réserve !</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Guillaume, Eric et Olivier dorment sur une bache, entre les deux coques
d’un cata. Gaël, Emmanuelle et moi optons pour le lit trampoline ! et
n’empêche, on y dort très très bien ! C’est décidé, ce cata aura pendant
toute la semaine le rôle de « cata-lit » ! le deuxième sera «
cata-séchage » c’est à dire celui où on fait sécher nos combis et toutes
nos affaires, tout en évitant bien sur quelles touchent le sable, et le
troisième bateau sera pour poser les affaires a ne pas mouiller, comme
par exemple la bouffe ou les habits secs.</p>
<p>Je m’endors à une vitesse éclair alors que d’autres sont encore entrain
de discuter autour du feu.</p>
</div>
<div class="section" id="mardi">
<h2>Mardi</h2>
<p>Mardi matin. On se réveille doucement, par le bruit des vagues et le
soleil dans le visage. On émerge de notre douce nuit grâce aux super
muesli aux raisins secs. ! hum, vraiment, j’adore !</p>
<p>On se fait tourner le seul tube de dentifrice, afin de garder quand même
un minimum d’hygiène ! C’est vrai que ce n’est pas cette semaine où on
prendra chacun une douche par jour. c’est plutôt au rythme d’une dans la
semaine ! Et encore.. !</p>
<p>On a donc décidé hier soir que ce matin on tracera rapidement vers la
base des Glénans à l’île d’Arz afin de pouvoir manchonner la barre de
liaison du cata de Gael et Emmanuelle !</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/004.jpg" style="width: 40%;" />
<p>Au moment où on s’apprête à partir, on croise une école de voile de la
région où naviguent des enfants sur des petits New Cat 12. Ils paraissent
minuscules à côté de nous et nous avons l’air d’être vraiment énormes !
Nos bateaux font deux fois la taille des leurs ! Et me dire que j’ai
commencé la dessus à faire du cata !</p>
<p>C’était à Sainte Marine, près de Bénodet. Tous les ans, les parents nous
inscrivaient ma soeur et moi a des stages de voile pendant une semaine.
quelle nostalgie !</p>
<p>Une fois qu’on a quitté cette petite plage très sympathique pleine de
puces de sable, on se rend compte que le vent souffle beaucoup plus fort
qu’on ne l’avait prévu ! Tant mieux on va pouvoir s’amuser et sortir au
double trap !!!! Mais, j’ai parlé trop vite..</p>
<p>Au bout d’une petite demi-heure de nav (on suit toujours Gaël qui nous
emmène à Arz) , la GV commence à s’affaler toute seule. et merde le hook
!!! Avec Eric on avait pas mal galeré a Berder pour hooker notre voile
mais je ne nous pensais pas autant incapable. ! Je m’énerve vraiment là !
On est dans un coin sympa, il y a du vent, les deux autres cata tracent
devant et s’éclatent comme des petits fous et nous on est la, au milieu
du golfe à tenter de rester bout au vent pour re-hisser la GV, du moins,
suffisamment pour pouvoir remonter au vent. car c’est surtout ça qui m’a
mise hors de moi. De la où nous sommes, il faut qu’on tienne un près
serré et qu’on enchaîne virements de bords sur virements de bord afin
d’éviter les nombreux parcs à huître et les cailloux. Gaël et Olivier
nous rejoignent, et d’un commun d’accord on décide d’aller se beacher sur
une petite plage pour rehooker notre GV comme il se doit. Eric et Gaël
mettent d’ailleurs beaucoup de temps à y arriver, on ne comprend pas
pourquoi on n’y arrive pas. S’il le faudra, ce soir on dessalera le
bateaux pour jeter un coup d’oeil a ce qu’il se passe là haut !</p>
<p>Nous sommes donc aptes à naviguer. Gaël et Emmanuelle, je le rappelle
n’ont toujours pas de barre de liaison et du coup, ils n’utilisent que le
safran sous le vent, l’autre étant remonté.</p>
<p>Mais Olivier et Guillaume n’avaient pas encore eu de petite avarie. il
fallait bien que la prochaine soit pour eux !!!</p>
<p>Peu de temps après avoir quitté la plage, ça y est. premier dessalage !!
Et de toute beauté ! Dans un parc à huître. !!!!! Sans grand étonnement,
il s’agit d’Olivier et Guillaume ! Apparemment, ils n’ont pas choqué leur
GV après avoir changé d’amure. c’est ce qu’ils ont donné comme excuse !
Très personnelle que je suis, et soucieuse de mon bien être personnel, je
me rappelle, et j’en fais part à Eric qui à l’air de partager ma crainte.
que nos duvets sont sur ce cata.. Et que ce soir je souhaiterai tout de
même être au sec.. !</p>
<p>D’un autre coté, ce qui nous fait assez flipper, c’est le chapotage. Le
sac étanche avec les duvets se trouve sur la coque en l’air et un
déplacement mal contrôlé ou trop de temps a ressaler risqueraient de
faire retourner entièrement le bateau. ce qui nous mettrai dans une très
mauvaise position. mais bon, plus de peur que de mal, on réaperçoit vite
la voile verte et bleu de leur nacra ! Ouf ! Nos duvets sont sauvés !!!</p>
<p>Cependant, l’aiguillot bâbord s’est tordu en deux, rendant impossible de
fixer la pelle. Eux non plus n’ont alors qu’un safran, le tribord !</p>
<p>C’est à ce moment là qu’on réalise que le vent est fort. sûrement 5
beaufort ! Nous croisons les escadres des Glénans. Ils sont tous avec un
ris et trois par bateaux. de quoi nous faire réfléchir un peu ! Nous
avons des bateaux plus gros que les leurs, nous n’avons pas de bosses de
ris alors comme nous dirait Manu, la surpuissance, tu la règles à
l’écoute, et nous ne sommes que deux par bateaux !!! Ça peut paraître
débile mais un sentiment de fierté nous prend. Nous nous suivons tous les
trois comme si nous faisions un concours de l’escadre la mieux tenue et
arrivons royalement sur la plage d’Arz !</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Les glénanais nous regardent un peu bizarrement ! Ils viennent d’où ces
six kékés avec leur gros nacras et leur gilet de sauvetage UCPA ??.</p>
<p>Après avoir dégréer nos bateaux et tout bien rangé pour repartir plus
rapidement après, nous allons tous les six voir le responsable de base.
Gaël le connaît bien puisqu’il à encadrer un mois cet été sur place. Pas
de problème, on sait ce qu’on doit faire, on peut se servir.</p>
<p>Gaël, Emmanuelle et Eric s’attaquent à la barre de liaison tandis
qu’Olivier, Guillaume et moi en tant que spectatrice nous attaquons à
l’aiguillot. Gaël passe tout de même un petit coup de fil au Nautic Club
pas étonnés de voir leur barre de liaison en deux et leur aiguillot
tordu. Apparemment ce sont des petites avaries assez fréquentes qui ne
les inquiètent absolument pas !</p>
<p>Le but de l’opération pour l’aiguillot, c’est de l’enlever totalement et
de mettre à la place une vis de la même taille ! La difficulté étant de
percer le morceau métallique, et c’est ce qui nous prendra le plus de
temps. Du coup, avec Emmanuelle on en profite pour faire un petit tour
aux sanitaires. Elle connaît bien la base elle aussi et me montre où se
trouvent les toilettes et les douches. Ca fait du bien de se dessaler
entièrement !</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Vers 14h, je commence vraiment à avoir faim ! Je m’installe sur les
escaliers, juste devant l’atelier et je discute avec les matos et les
moniteurs de la base ! Ils sont d’ailleurs assez enthousiaste lorsqu’on
leur parle de notre projet.</p>
<p>Les questions affluent.. ! Mais, vous êtes qui ? Ah, des moniteurs de
l’Archipel venus tripper dans le golfe ! Ah, mais c’est super sympa !!!!
on parle pas mal de nos bateaux, de notre trajet. ambiance très
conviviale, mais j’ai toujours faim !</p>
<p>Heureusement, la barre de liaison et l’aiguillot sont enfin réparés. On
s’installe sur des petites tables a l’ombre avec toutes nos provisions
pour la journée. pain, fromage et charcuterie.</p>
<p>Mais, on n’a pas trop envie de traîner, et à peine le déjeuné fini, nous
voilà prêt à remettre nos combis et tout notre attirail.</p>
<p>Sur la plage, un autre problème nous attend. Notre hook ! Après un bon
quart d’heure de mulage dans tous les sens, on observe (enfin, j’observe,
mais comme je suis sympa, je laisse la découverte à tout le groupe !!)
Que sur un des coté de l’anneau il y a un petit noeud et c’est ce qui
nous empêche de fixer correctement le hook !</p>
<p>On peut donc quitter la plage de la base d’Arz direction la plage des
Douaniers. Le principe est simple, on ne va pas loin (a peu près
3/4heures de nav), on pose tous nos bidons, sacs et matos sur la plage et
on tire des bords devant !!!</p>
<p>Ah. qu’est ce que j’aime le cata !! c’est en étant au trap à la barre que
tout a coup je me suis souvenue pourquoi j’avais passé mon été aux
Glénans. c’est parce que je trip vraiment, j’aime vraiment ça. J’adore
les sensations que l’on peut avoir lorsque l’on borde un peu plus sa GV
et que la coque au vent commence à se soulever. lorsqu’au portant on a
l’impression de glisser sur l’eau et d’aller plus vite que n’importe
quelle autre embarcation !!</p>
<p>On s’était donné comme objectif aussi de tracer les Hobbies Tiger des
Glénans !!! Lorsqu’ils sont apparus dans notre périmètre, nous nous
sommes les 3 bateaux précipités à leur rencontre pour tenter de les
dépasser et d’aller plus vite qu’eux. Au près, c’est radical, on va
beaucoup plus vite qu’eux. Par contre, au portant sous spi. nous étions
loin derrière !!!</p>
<p>Mais l’heure passe, et on ne veut pas commencer à dîner trop tard. On
décide donc de rentrer !</p>
<p>On retourne donc là où nous attendent nos affaires. Eric et moi mettons
toujours un certain temps à déhooker notre GV.</p>
<p>Gaël jette un coup d’oeil au Bloc Marine pour regarder les hauteurs des
marrées hautes et les coefs. Pas de chance, là on est en pleine marrée
basse, la plage est plate comme tout, et c’est tout la haut, a 35m qu’il
faudra s’installer !!! Et on est partis pour muler !! Qu’est ce qu’ils
sont lourds ces bateaux !!!</p>
<p>Et puis, tout à coup, Olivier et Guillaume décident de partir à la
recherche d’un bar sur l’ile afin d’acheter une ou deux petite bouteilles
! A peine partis, ils reviennent en courant, porteurs d’une très très
mauvaise nouvelle. Juste derrière la plage, il y a un marécage. Et qui
dit marécage peut aussi dire moustiques sauvages. Ils mettent pantalon et
pulls à manches longues pour éviter les piqûres.</p>
<p>Pendant ce temps, Emmanuelle et Eric s’occupent de préparer le diner,
avec Gaël on installe des bouts sur notre « cata-dodo » pour pouvoir
mettre une bâche sur notre tête au cas où il se mettrait à pleuvoir.
Mais, au bout d’une dizaine de minutes, on se fait envahir par les
moustiques. Je crois bien que je n’avais jamais vécu ça. Devant mes yeux
une cinquantaine de moustiques qui nous tournaient autours et nous
piquaient de partout. Même a travers les pantalons, même a travers les
pulls. on est tous en k-way avec la capuche, d’une main essayant de
manger des pattes bolo trop bonnes, de l’autre chassant les insectes.
C’est horrible, vu qu’on est salés et humides, ils ne décollent pas. Nous
nous mettons à courir dans tous les sens pour les éparpillés, et puis
quand on cours, ils ne piquent pas ! Je dénombre plus de trente piqûres
sur les mains, pareilles sur les jambes et une dizaine sur le visage.
Pour les autres, c’est à peu près la même chose. Psychologiquement, il
faut tenter de ne pas se gratter, penser à autre chose, se détacher de
toutes ces petites bêtes qui nous rendent vraiment la vie impossible. En
effet, pas question de faire quoique ce soit d’autre que de manger chacun
dans son coin en marchant sur la plage le plus éloignés possible des
nuages de moustiques. Eric était donc en short, et très stoïquement il
s’appliquait à écraser sur ses jambes tout ce qui s’y approchait de trop
près. Pas question de mettre nos bas de cirés trempés et salés, qui en
plus nous collerait aux jambes rendant les démangeaisons insoutenables.
Vous n’imaginez pas même pas à quel point je regrette mes puces de sable
de la veille. !!!</p>
<p>Mais, toujours pas de nouvelles d’Olive et de Guillaume. J’espérai qu’ils
reviendraient vite, surtout qu’ils devaient passer par la base des
Glénans voir si mon couteau Whisards que je m’étais offert en juillet n’y
était pas. Vers minuit, alors que nous étions déjà couchés, on les entend
arriver. Bon, ils sont seins et sauf ! D’un autre côté, on ne s’était pas
fait trop de soucis pour eux. on est sur une île pas bien grand donc
forcément un jour ils rentreraient à bon port !!! Je préfère attendre le
lendemain matin pour écouter leur récit d’aventure !!!!!</p>
<p>La nuit est agitée. Il y a beaucoup de vent et la bâche censée nous
protéger de la pluie fait tellement de bruit que Gaël décide, après des
hésitations, de l’enlever. C’est vrai qu’elle nous fouettait le visage et
tout le corps, que ça faisait un courant d’air. bref ! La bonne décision
était de l’enlever ! Tant pis, on aura un peu plus froid mais on moins on
dormira !</p>
</div>
<div class="section" id="mercredi">
<h2>Mercredi</h2>
<p>Mercredi matin, réveil doucement toujours par le bruit des vagues et le
soleil sur nos visages. On sort du lit et hop, nous voilà donc sur la
plage, face à la mer. Je prends conscience que je passe des moments
uniques, comme plusieurs fois j’en ai rêvé lorsque j’étais à Paris. Pas
de béton, pas de chaussures, pas de pollution, pas de voitures,
d’immeubles, de pigeons, de magasins, de resto de .. Rien ! Que nous, la
plage, la mer et nos bateaux.</p>
<p>Depuis le début de la semaine, la météo est vraiment impeccable ! Du
soleil et du vent, que demander de plus ?!</p>
<p>Olivier et Guillaume s’étaient donc perdu sur le chemin du retour! On
s’en doutait un peu en fait ! Ils ont omis de prendre un embranchements
et ont donc mis un peu plus de temps que prévu à rentrer au campement !
Cependant, ils nous ont tout de même ramenés trois bouteilles de bière !
Chouette alors ! Faudra bien faire attention à ne pas les casser !!!
D’ailleurs, pour en être sur, on les rangera dans le sac à duvet, pour
bien amortir les chocs !</p>
<p>Le programme de la journée : aller à Aradon pour refaire un stock de
provision. Nous n’avons plus de pain, ni de charcuterie, ni de fromage.
Après, on sortira du Golfe et selon les conditions météo, de l’état de la
mer, de l’orientation du vent et de nos envies, on avisera. Il ne faut
pas oublier que l’on est déjà mercredi et qui si on s’éloigne trop de
Carnac, on ne pourra peut être pas rentrer ! Nan mais c’est vrai, si
jamais il y a pétole ?!!!</p>
<p>On range donc nos affaires, chacun s’applique à attacher ses bidons
toujours de la même manière ! Au moins, on est sur que ça tient !!!</p>
<p>Comme à notre habitude, nous suivons Gaël, qui nous dirige vers Arradon !
Quel souvenir énorme ! Je crois que cela fait parti de mes meilleurs !
Nous étions au portant, pleine vitesse lorsque chacun son tour avons
dépassé un bateau de pêcheurs.. Ils nous ont regardé comme si l’on
descendait du ciel ! Et puis, c’est vrai que fallait nous voir ! Les
trampolines des catas sont surchargés de bidons, pack d’eau douce, de
sacs en tout genre, nous ne ressemblons plus vraiment à grand chose avec
tout notre attirail ! Et puis, à cette vitesse là, c’est vrai que les
pêcheurs avaient de quoi être étonnés !! Avec Eric, nous leur faisons un
grand signe de la main pour répondre à leur geste de politesse !!</p>
<p>Pour approcher la côte, ce n’était pas très pratique vu le nombre de
parcs à huîtres. En plus, des espèces de falaises (dans mon souvenirs !)
nous déventaient et nous avons mis du temps à accoster !</p>
<p>A peine arrivé, un petit monsieur âgé débarque sur la plage et commence à
nous parler. Il nous demande si nous sommes des amis de son petit-fils.
?! Absolument pas ! Mais, il se prend d’affection pour nous, jeunes
sportifs aventuriers et ne nous lâche plus !</p>
<p>C’est donc à ce moment là que l’on décide avec Emmanuelle et Olivier de
partir à pied pour Arradon, certains d’y trouver notre bonheur. un super
marché !!!</p>
<p>Il faut voir à quoi l’on ressemble. Emmanuelle est en combinaison
intégrale à moitié enlevée, les cheveux tout bouclés en pleine bataille,
et bien sur les supers lunettes Julbo !</p>
<p>Olive est en tong et en petit short de bain vert clair, lui aussi la
paire de lunette énormes !</p>
<p>En ce qui me concerne, j’ai pris le soin d’un peu me changer. je suis
donc pieds nus avec mon pantalon rouge, mon unique débardeur blanc
(toujours le même depuis le début !) et mes lunettes de mouches ! On fait
un bon trio !</p>
<p>Au bout d’un certain temps, nous nous décidons à faire du stop. Aussi tôt
dis, aussi tôt fait, aussi tôt pris !! Un homme du coin, environ la
quarantaine, qui allait chercher son fils à l’école ! Retour à la
réalité, nous sommes début septembre donc forcément, la plupart des
jeunes sont en cours. Pas nous ! On en a de la chance hein ? !!!</p>
<p>Il nous dépose donc au plein coeur du village d’Arradon, non loin du
super Marché ! Arrivés la bas, nous nous précipitons vers les toilettes !
On se voit dans une glace pour la première fois depuis le début du Raid,
et puis, des vrais toilettes, c’est toujours plus pratique !</p>
<p>Dans les rayons, on se fait plaisir. Bon, on prend d’abord tout ce qui
nous ait vraiment utile. Du pain « boule de mer », de la baguette pour
aujourd’hui, plein de charcuterie (saucisson secs, jambon blanc, jambon
fumé..), on se lâche sur le camembert et le comté. et puis, allé, une
petite plaquette de beurre salée, rien de tel pour remonter le moral !</p>
<p>Tout a coup, nous passons devant le rayon céréale. et nous tombons sur
des Mueslis au chocolat. aucune hésitations, on le prend !!</p>
<p>Et puis, du YOP à la fraise pour le dessert, ça nous changera un peu !</p>
<p>Voilà à peu près nos petites emplettes ! On passe aussi par un tabac pour
les clopes et les carambars !!!</p>
<p>Mais, le retour est long, il fait super chaud. On se remet au stop, tout
en avançant ! Mais, dans ce sens là c’est moins pratique. Peu de voitures
passent et personne ne veut de nous. Sauf un, le même qu’a l’aller ! Son
fils est dans la voiture, et apparemment ça lui fait plaisir de nous
avoir dans sa voiture ! On se raconte nos vies !</p>
<p>Nous arrivons comme des princes et étalons notre succulent déjeuné ! On
reconnaît les vrais bretons gourmand avec l’épaisseur de beurre salé sur
les tartines de pain !! Quel plaisir ! En plus, le petit monsieur de tout
à l’heure nous à offert du Breizh Cola et nous a proposé de venir remplir
nos bouteilles d’eau vide ! Plutôt sympa ! Et puis, j’en profite alors
pour utiliser ses sanitaires, proposés avec tant d’affection !</p>
<p>On ne sait pas trop où on va ensuite mais la première chose à faire est
de sortir du golfe ! Je suis d’ailleurs toute fière d’avoir été en tête
avec Eric alors que Gaël et Olivier étaient bien derrière ! La sortie
n’est pas toujours des plus évidente et pour une première fois, je trouve
que l’on s’en ait très bien tirés !</p>
<p>Cap sud-est là où le vent nous portera ! On s’en fou un peu, il fait trop
beau, il y a plein de vent, on trip.</p>
<p>Avec Eric, on ne parle pas beaucoup sur notre bateaux. Généralement on
profite, on trip, enfin moi oui, lui, je ne sais pas !</p>
<p>Je regarde les paysages lorsque je suis équipière, j’aime bien sortir au
trap et restée calée sur ma coque, tout en m’appliquant à maintenir
l’équilibre du bateau ! Et puis, j’embète un peu Eric .. borde un peu,
lofe, tire ta barre, TIRE TA BARRE, nan, pas autant, re lofe s’il te
plait, oui ça y est, là on est bien.. ! Généralement on n’est pas
vraiment d’accord, mais on ne va pas se disputer pour ça ! Tant pis,
chacun ses réglages !</p>
<p>Et lorsque je suis à la barre, soit je commente tout ce que je fais, tout
ce qu’il se passe sur le bateau et autour du bateau. ! « waahooo t’a vu
la vague qu’on vient de se bouffer en pleine gueule, attend, là on va
aller remonter au vent pour grier Gaël sur le bord suivant.. !!!! ».
Parfois, par contre, je me tais, je ne parle plus, je sourie et je me
concentre sur ce que je fais. Mon creux de GV ressemble à quoi ?
L’équilibre.. ? Je borde, je choque, je sors au trap, je rentre. ! Je
m’amuse comme une petite folle, je trip, je me fais plaisir, je profite à
fond de ce que je suis entrain de faire !!!!</p>
<p>!Des beaux bords de prés au double trap. Un réel bonheur ! Nous sommes
surtout content d’avoir quitté le Golfe et de pouvoir faire de longs, de
très longs bords !</p>
<p>Et puis, c’est tellement drôle de voir la tête que font les gens que
l’ont croise sur l’eau ! Surtout quand les trois nacras à la suite leur
passe devant, tous au double trap, tout ça parce qu’on est tribord amure.
! Ils nous regardent parfois amusés, parfois très étonnés ! Une fois, un
vieux monsieur sur son First 31.7 nous a demandé si on avait besoin
d’aide.. nous étions à la cape entrain de décider de notre route ! C’est
vrai que la distance de sécurité n’a pas toujours été respectée et que
nous avons pu nous éloigner un peu trop. !</p>
<p>Au bout d’un certain temps, lorsque nous nous étions bien éloigné du
Golfe, on décide d’aller jeter un coup d’oeil à la carte, et d’aller sur
une petite plage pour goûter !</p>
<p>On débarque comme des stars, les trois bateaux en même temps, devant une
plage pleine de touristes !</p>
<p>Comme à notre habitude, on se précipite à la capitainerie , qui bien sur
est fermée ! Tant pis, on demande à un inconnu sur la plage le nom de
l’endroit où nous sommes ! Nous avions bien avancés .</p>
<p>Mais, comme nous ne pouvons pas tout avoir, on se rend compte qu’au
milieu de notre coque bâbord, il y a un trou d’au moins deux centimètres
sur un ! Le Bad ! et merde et merde ! On est maudit c’est pas possible !
les casses se suivent les unes aux autres . On a du faire ça en partant
de la plage d’Arradon. La mer était très basse et on a du porter le
bateau dans l’eau afin de sortir de la mini rade ! Et on a du raser un
caillou de trop près. !</p>
<p>Olivier et Guillaume prennent en charge le pansement grâce à la pharmacie
qu’il nous avait préparés ! Un petit morceau de plastique, du pansement
étanche. voilà de quoi tenir jusqu’au port du Crouesty.</p>
<p>Pendant tout ce temps là, je m’amuse avec deux gosses de deux/ trois ans
trop mignons ! Ils jouent sur les trampos, font style de border leur
voile. ! je fonds littéralement, deux petits anges blonds aux yeux bleu !
Bien sur je me fais charrier par les autres. ! Clara, celle qui n’a
encadré que des Jujus, Clara et Penfret. !!!!!</p>
<p>On revient donc sur nos pas. De quoi nous mettre un peu de mauvaises
humeurs ! On se pose sur une petite plage avant de rentrer vraiment dans
le port. Emmanuelle , Eric et Gaël vont donc, tout de néoprène vêtus, en
ville pour voir ce qu’ils peuvent acheter pour étanchéifier la coque. Pas
le temps de faire du gel coat, ou quoi que ce soit dans ce genre !</p>
<p>Pendant ce temps là, avec Guillaume et Olivier, on profite de cette
petite pause imprévue pour sortir nos petites bières du sac des duvets !
Assis sur les rochers, on discute tranquillement, on profite ! Ils
partent d’ailleurs visiter les alentours. Du coup j’allume mon téléphone
portable, j’ai pas mal de coup de fil à passer. Tout d’abord je souhaite
prendre des nouvelles de ma grande soeur qui n’est pas au top de sa
forme, et puis, il faut aussi que j’appelle la base des Glénans à
Concarneau pour une histoire d’encadrement la semaine suivante.</p>
<p>Ils reviennent les mains vides, le seul magasin qui aurait pu les aider
était fermé, il faudra revenir tôt demain matin. Du coup on décide de se
trouver une petite plage pas trop loin. Je propose celle de Kerjouano,
tant qu’à faire, aller là où j’ai de bons souvenirs ! Et puis, de toutes
les façons, on ne peut pas aller très loin, le soleil est entrain de se
coucher et on a intérêt à arriver avant lui !!!! Cependant, le vent est
totalement tombé. Je suis à la barre, tout est si calme, cela nous change
des autres nav’ ! Eric se pose sur la coque sous le vent , tout allongé
pour dormir . Du coup, avec Gaël on s’amuse à essayer de se doubler !
Tous les coups sont permis ! Je relève le safran au vent, je me mets sous
le vent, choque le cuni pour obtenir un meilleur creux.. !! et je trace
tout le monde !!!</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/005.jpg" style="width: 40%;" />
<p>L’arrivée à la plage se fait tout en douceur, sur un fond de soleil
couchant, tout autour de nous est rose et violet, c’est tellement beau
qu’avant même de remonter les bateaux je me précipite sur mon appareil
photo !</p>
<p>Et puis là, c’est le fou rire, j’en peux plus de ces plages bretonnes où
tout est permis ! Hier c’était l’invasion de moustiques, le jour d’avant
des puces de sable, et là, des nudistes ! Faut s’arrêter zut !!! Morts de
rire on remonte nos bateaux sur l’air de « c’est à bâbord qu’on mule le
plus fort/ C’est à tribord. » et n’empêche, c’est dingue comment ça passe
plus vite !</p>
<div class="line-block">
<div class="line"><br /></div>
</div>
<p>Et ça y est, chacun se met à ses activités. Les arrivées sont toujours
dans le même ordre. Avec Guillaume on se précipite sur nos clopes, avant
de faire quoi que ce soit d’autre ! Et puis, on se change tous. Avec
l’habitude, on met tous nos affaires sur le même cata, chacun ayant sa
technique pour éviter le sable, ne pas trop se montrer, accrocher ses
lunettes et ses gants pour être sur de les retrouver le lendemain. !</p>
<p>Et puis, on s’active pour s’installer. Pendant qu’Emmanuelle aidé de deux
ou trois personnes s’occupe de préparer le repas, Gaël s’applique avec
les autres à installer les bouts pour le cata dodo afin qu’une bâche
puisse être mise en ultime vitesse s’il se met à pleuvoir.</p>
</div>
<div class="section" id="jeudi">
<h2>Jeudi</h2>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/006.jpg" style="width: 40%;" />
<p>C’est fou ce que ce sable est fin, c’est fou comme c’est paradisiaque de
se lever le matin, de courir jusqu’à la mer pour se mouiller le visage et
se réveiller, c’est fou comme j’aime marcher pieds nus sur ce sable.</p>
<p>Gaël et Emmanuelle sont au Crouesty pour pouvoir réparer notre bateau.
Ils sont partis tôt ce matin. Nous émergeons tranquillement, mais la
seule idée de se dire que ce matin on a des mueslis au chocolat facilite
mon extraction de mon duvet ! Quoi que.. En y repensant bien, je sortais
de mon duvet qu’après avoir petit déjeuner. j’aime me le faire apporter
!!!!</p>
<p>Ils reviennent vers 10heures avec tout ce qu’il nous faut. On
confectionne un joli pansement qui apparemment à l’air de bien
fonctionner. On en profite pour en mettre sur d’autres petits trous qui,
on ne sait jamais, pourraient abîmer la coque.</p>
<p>Pour le départ , c’est toujours aussi le même schéma. Avec Eric on n’est
pas très perfectionnistes sur la manière d’accrocher nos bidons alors que
d’autres sont des plus maniaques. Nous mettons toujours plus de temps à
hooker notre voile (c’est due à un détail technique sûrement, il n’y a
pas d’autres explications !!) . Certains s’habillent avant d’avoir
greer, d’autres après. Mais il y a une chose qui est sure. C’est que
c’est toujours Olivier le dernier ! Il prend son temps tout en ayant
l’air d’être débordé ! C’est très marrant à voir !</p>
<p>Toujours pas de trajet bien définis, ont descend toujours un peu. Les
deux autres ont pris la météo au port ce matin, rien de quoi s’inquieter,
au contraire. Bonne nav le matin, soleil et vent ! Il y a même un peu de
mer, comme ça on peut profiter à fond des coques casses vagues !!!</p>
<p>On déjeune sur une petite plage, je ne sais où.La première où nous avions
voulu aller était pleine de déchet alors on en a choisit une autre. Je me
souviens juste qu’on est partis en trip sur des sketchs de comiques et
pendant tout le déjeuner on a raconté n’importe quoi !!!</p>
<p>Mais il fait tellement beau et le vent nous appelle ! Pas question de
s’attarder ! Belles marches arrières pour quitter la plage . Et on trace
plein travers ! Un bord de travers de plus d’une heure et demi ! Double
trapèze obligé si on veut contrôler la surpuissance tout en avançant le
maximum !! Avec Eric on alterne barreur/équipier même s’il comprend
rapidement que s’il ne me laisse pas la barre rapidement je le fou à la
flotte !!! En plus, on passe dans des parcs à huître et le but c’est de
toucher le plus de drapeau possibles ! Pour Olivier et Guillaume, ils
s’amusent à bananer au dessus de chaque drapeaux.. ! On s’amuse aussi à
couper les méduses en deux avec nos pelles de safrans !!! le seul
problème c’est que du coup la pelle remonte (heureusement il n’y a pas de
cames à relever !!!) et que lorsque c’est celle sous le vent, on doit
ralentir !</p>
<p>Quelle vitesse et quelles sensations..</p>
<p>Au bout d’un certain temps, on se met tous les trois à la cape pour
choisir de ce qu’on va faire. Nous sommes à l’entrée de la Vilaine. Ca
fait un peu loin demain pour rentrer sur Carnac si jamais il n’y a moins
de vent que prévu. Alors, comme à notre fâcheuse habitude, on revient sur
nos pas !!</p>
<p>Comme il n’est pas tard, on fera comme à Arz.</p>
<p>Le retour est encore plus trippant que l’aller. On oscille entre le
travers et le grand largue. La mer nous donne du fil à retordre. On est
complètement trempés ! J’hésite toujours pour l’utilisation de mes
lunettes de soleil. soit je les mets mais à cause des gouttes je ne vois
plus rien, soit je les relève mais les gouttes d’eau salée dans les yeux
ça fait mal, surtout avec les lentilles !!! c’est dingue le nombres de
petits détails qui peuvent nous changer la vie ! C’est comme la ceinture
de trap ! A partir du moment où elle commence à te faire mal, plus
question de sortir équilibrer ton bateau ! Le pire, c’est quand t’a envie
de faire pipi et que t’a tout plein de manoeuvres à faire. Certains
n’hésitent pas à sauter à l’eau mais bon, dans les combinaisons. c’est
pas toujours ce qu’il y a de plus agréable, surtout lorsque c’est une
intégrale !</p>
<p>On est dans l’anse de Suscinio. Comme on ne connaît pas les cotes, on y
va tout doucement afin d’éviter de se prendre encore un caillou ! bon, en
même temps avec seulement 70cm immergés, et encore, ce sont les safrans,
on ne craint pas grand chose !!!</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/007.jpg" style="width: 40%;" />
<p>Notre arrivée est toujours aussi impressionnante ! les gens nous
regardent débarqués ! C’est qui ces jeunes qui viennent de nul part ?
qu’est ce qu’ils viennent faire ici ?!</p>
<p>On pose nos affaires et hop, on repart. Enfin, pas tous, y’a un cata qui
reste à terre . La nav de cette aprèm était crevante je l’avoue alors
seul les motivés repartent. Je monte alors avec Olive. Je veux
perfectionner mes bananes ! Mais le vent est tombé et le cata ne se
soulève presque pas. Du coup ça m’énerve et il me laisse au bord de
l’eau. Il repart en solo.</p>
<p>Lorsqu’on a finit de remonter le dernier bateaux. on s’aperçoit. qu’ils
ne sont pas alignés !!! Mais, qu’est ce qu’il nous arrive ?? Nous n’avons
pas alignés nos bateaux !!! Alors là ! On se relâche totalement ! Déjà
qu’on dort dessus et que l’on est constamment assis sur les coques à
terre, si là on n’aligne plus nos bateaux, c’est notre fin ! Bon,
heureusement le reste du rangement est niquel ! Les écoutes de foc sont
dans la bail à mouillage. !!!</p>
<p>Du coup, chose que je n’avais que très peu faite depuis le début, je
prépare le diner ! Toute seule ! Comme une grande ! Et puis, comme c’est
notre dernier diner, on peut se faire plaisir ! On aura le choix entre
deux sauces pour notre riz ! Grand luxe ce soir ! Sans oublier notre
mousse au chocolat, et des fruits secs !</p>
<p>Ce soir, c’est la fin, alors on peut se coucher tard et se permettre
d’être un peu fatigués demain alors on se raconte nos vies, on discute
bateaux, réglages, exploits. !!!</p>
<p>Et puis, on est quand même un peu nostalgique. Cette petite semaine va
bientôt toucher à sa fin, c’est bien dommage !</p>
<p>Je crois que Guillaume, Eric et Olivier sont partis voir ce qu’il se
passait au loin sur la plage. on pouvait apercevoir un feu de djeunes
cools qui devaient sûrement avoir guitares et tout ce qu’il s’ensuit !</p>
<p>Mais, Emmanuelle et moi ne faillerons pas à nos habitudes. toujours les
premières couchées !</p>
</div>
<div class="section" id="vendredi">
<h2>Vendredi</h2>
<p>Vendredi. Le réveil est plus douloureux que d’habitude. Première nuit où
il a plu, mais du coup nos affaires nautiques sont trempées. L’idée de
devoir mettre mon lycra et ma combi trempée me met de très mauvaise
humeur ! et en plus on rentre à Carnac là ? mais c’est quoi ce délire ?
c’est déjà fini ?? mais c’est abuser ! moi je veux continuer, je veux
garder mon ptit cata que j’aime tant, je veux arriver ce soir sur une
nouvelle plage. En même temps, je ne sais pas où je dors ce soir. Je
devrai rentrer à Concarneau pour partir encadrer samedi. Chouette alors !</p>
<p>On regarde la carte avant de partir. Ca ne sera pas compliqué du tout de
rentrer, au contraire ! Comme le vent a tourné, on sera partis au près et
on reviendra au près !</p>
<p>Heureusement, la météo ne nous a pas oublié du tout. Cinq beaufort bien
établis, des petits creux d’1m parfois. Le retour va être marrant !</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/008.jpg" style="width: 40%;" />
<p>On quitte la plage, il est 11h. Je sais que c’est ma dernière nav sur ce
bateau, que c’est fini. Mais autant profiter jusqu’au bout !</p>
<p>Un bord de près qui dure plus d’une heure et demi.. Et vas-y qu’on se
prend des vagues dans la figure, dans les mollets lorsqu’on est au trap,
que le bateau gîte, gîte, mais les sensations sont énormes. Je galère un
peu parfois à border ma GV parce que mes muscles ne sont pas assez
performants ! Et puis, de toute les façons, j’ai décidé de monopoliser la
barre durant toute la nav alors voilà, faut que je tienne bon ! Mais je
trip tellement !</p>
<p>Plus on se rapproche de la Trin et de Carnac, plus on croise des
voiliers. Au loin on aperçoit une régate, ou des entraînements je sais
pas. Sous notre vent il y a un beau trimaran. Comme je ne reconnais pas
la voile, je ne prends pas la peine de descendre sur lui voir qui c’est !
De toutes les façons Olivier et Guillaume s’en chargent !</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/raid_catamaran/009.jpg" style="width: 40%;" />
<p>Au bout de deux heures et demi de nav intense , on arrive à Carnac. Mais
comme je suis finalement une fille sympa, c’est Eric qui est à la barre
!!!!!!!!!</p>
<p>Et puis voilà, tout est rapide, tout s’enchaîne. !!!</p>
<p>On arrive au Nautic Club, personne n’est là, normal il est fermé !
Pendant que Gaël, Eric et Guillaume se chargent de défaire toutes nos
affaires, on va avec Emmanuelle et Olivier chercher de quoi se nourrir
pour le déjeuner. On se ballade donc à Carnac comme si nous étions sur
notre cata. Nous n’avons même pas enlevé nos gilets de sauvetage !!!!!</p>
<p>On prend plein de pain parce qu’on n’a pas oublié le Nutela laissé pour
le retour ! Et puis, une petite bouteille de cidre pour fêter tout ça !!!</p>
<p>Seulement, il pleut ! Alors on s’abrite dans une petite cabane en bois du
club, là où les gens se changent généralement !</p>
<p>Comme nous avons les cata jusqu’à 16h à peu près, Gaël et Eric
ressortent . Moi j’ai un peu la flemme, il pleut comme pas possible, je
viens juste de me changer et de mettre des fringues sèches. Je n’ai pas
trop la motivation. Et puis, quel intérêt de tirer des bords devant
Carnac ? Aucun ! S’il s’agissait de partir pour une nouvelle destination,
je n’aurai pas hésité, mais là, nan, ça ne me tente pas !</p>
<p>Je parcours tout Carnac pour trouver l’office du tourisme. Il me faut un
car jusqu’à Auray. Les parents d’Eric viendraient alors nous chercher
pour nous amener à Concarneau. Eric à de la famille là bas, et moi je
dois retourner aux Glénans pour finir une semaine de D1 et pour valider
mon pass formation !</p>
<p>C’est trop glauque, il pleut depuis que nous sommes rentrés.</p>
<p>Lorsque les gens du Nautic Club reviennent, ils sont intéressés par notre
Raid, nous demande ce qu’on à fait, par où on est passé. et sont surtout
impressionnés par nos performance de matos !!la barre de liaison est
mieux réparée que ce qu’ils avaient fait auparavant ! c’est pourquoi elle
avait lâché d’ailleurs..</p>
<p>Bref !</p>
<p>Le temps passe. Il pleut toujours. Gaël, Emmanuelle et Guillaume rentrent
sur Paris et ne veulent pas partir trop tard. Ils déposent du coup
Olivier à Vannes qui lui part encadrer à l’ile d’Arz samedi matin. On
reste tous les deux avec Eric.</p>
<p>On a passé notre semaine assis l’un à coté de l’autre sur notre bateau,
et puis nous voilà assis toujours à coté à l’arrêt de bus.</p>
<p>Il est 17 heures.</p>
<p>On met environ 20 minutes pour aller jusqu’à Auray en Navette.</p>
<p>On dîne dans le seul resto de la gare.</p>
<p>Ses parents viennent nous prendre il est 23h. J’aurai donc encore passé
cinq heures dans une gare à attendre ! Bon, la je ne suis pas seule, mais
comme on est crevé, les discussions sont vite coupées par des bâillements
!</p>
<p>Ses parents me déposent à la base. Je croise pas mal de gens que je
connais ! J’arrive dans la chambre que j’ai l’habitude de prendre. Pour
un vendredi soir à minuit, il n’y a vraiment personne ! Mon lit habituel
est libre, et il y a même à coté une paire de tong que j’avais trouvé à
Penfret et que j’avais laissé là avant de partir pour le raid.</p>
<p>Je me dépêche de sortir toutes mes affaires trempées afin qu’elles soient
sèches demain.</p>
<p>Je me prends une bonne douche bien chaude et avec plein de pression..</p>
<p>Je m’endors encore bercée par la semaine fantastique que je viens de
passer.</p>
</div>
<div class="section" id="de-retour">
<h2>De retour</h2>
<p>Voilà, je suis maintenant à Paris, il fait bien froid, et même pire, il
pleut. !</p>
<p>Cette petite semaine m’a beaucoup apporté. Même si ce n’était que cinq
jours, ça fait du bien de partir comme ça, un peu à l’aventure, sans trop
avoir prévu ce que le jour d’après sera fait. En même temps, le principe
aussi de se sport est de dépendre entièrement des conditions extérieures
et naturelles.</p>
<p>Elle m’a beaucoup appris aussi sur un plan technique car, comme le dirait
un de mes formateurs, à chaque fois que l’on pars sur l’eau, on apprend
quelque chose, on se perfectionne. C’est totalement vrai. Sur nos longs
bords de plus de deux heures, on a vraiment le temps de rechercher le
meilleur réglage possible.</p>
<p>Mais, ce que je retiendrai le plus, ce sera tous nos trips, que ce soit
sur l’eau, lorsqu’il y avait 25 noeuds comme lorsqu’il n’y en avait même
pas 5. et à terre, toute notre petit rituel afin d’être sur d’être bien
installé pour la nuit.</p>
<p>Je referai ça avec grand plaisir, sur les mêmes bateaux sûrement, avec
les mêmes personnes aussi, mais pourquoi pas pendant un peu plus
longtemps, histoire de vraiment profiter à {sc fond} de ce que l’on
aime vraiment !</p>
<p>Voilà la petite morale de l’histoire, je m’arrête là sinon je serai
capable de vous raconter les vacances dans le Jura il y a quatre ans, ou
encore ma petite saison aux Glénans, ou même pire, ma vie à la fac !!!!</p>
</div>
Voyage en Islande2004-08-01T00:00:00+02:002004-08-01T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2004-08-01:/personnal/islande.html<p><strong>Du 23 juin au 22 juillet 2004, récit d’Emmanuelle</strong></p>
<div class="section" id="introduction">
<h2>Introduction</h2>
<p>Comment Gaël et moi avons-nous eu la drôle d’idée de partir un mois en
Islande ? Eh bien, en plus il paraît que l’idée vient de moi, et ça doit
être la lecture de bouquins sur les volcans …</p></div><p><strong>Du 23 juin au 22 juillet 2004, récit d’Emmanuelle</strong></p>
<div class="section" id="introduction">
<h2>Introduction</h2>
<p>Comment Gaël et moi avons-nous eu la drôle d’idée de partir un mois en
Islande ? Eh bien, en plus il paraît que l’idée vient de moi, et ça doit
être la lecture de bouquins sur les volcans qui ne tarissent pas de
photos époustouflantes sur ce pays actif volcaniquement, c’est le moins
qu’on puisse dire. Ou bien peut-être l’envie de partir dans un pays du
Nord, peuplé de légendes, de Vikings et de trolls. Ne serait-ce que vers
le 20 juin, nous nous retrouvons avec deux billets pour Keflavik, un
guide Lonely Planet que nous n’avons pas pu lire autant que nous
l’aurions souhaité à cause des exams qui viennent de se finir, et une
très forte envie de partir !</p>
</div>
<div class="section" id="mercredi-23-juin">
<h2>Mercredi 23 juin</h2>
<p>Après un vol sans encombre, nous arrivons à l’aéroport de Keflavik vers
17h. De l’avion nous apercevons notre premier champ de lave parsemé de
petites fleurs violettes que nous prenons d’abord pour de la lavande. A
la sortie de l’avion nous nous attendions à nous faire contrôler nos
cartes d’identité mais nous sortons de l’aéroport sans aucun contrôle (ça
nous arrange plutôt car en théorie l’importation de comté ou de coppa est
interdite… ). L’Islande semble avoir renoncé à contrôler ses
frontières. Nous prenons la navette pour Reykjavik, ce qui nous permet au
passage d’aider un groupe de Français vraimant paumés et ne parlant pas
un mot d’anglais… La navette nous dépose directement au camping, qui
s’avère être relativement cher (20 euros pour deux). On monte la tente
avant de chercher un supermarché pour compléter notre approvisionnement :
nous n’avions droit qu’à 3kg de nourriture par personne dans l’avion. Nous ne
trouvons pas le Bonus, chaîne de supermarchés mythique dont il sera
beaucoup question dans ce récit donc nous nous rabattons sur un 10-11
(aux USA c’est 7-11, ici les supermarchés n’ouvrent que très tard. Par
contre ils sont souvent ouverts le dimanche, ce qui est bien pratique.).
Il n’y a pas de gaz dans les supermarchés, par contre on en trouve dans
quasiment toutes les stations-service. Il pleut à verse, conformément à
l’idée que nous nous faisons du temps islandais. Heureusement nous avons
pu monter la tente quand il faisait encore beau : ici le temps change
très vite ! Une fois l’approvisionnement fait, nous rentrons au camping
faire un gros dîner de baked beans, de poisson fumé dans une sauce sucrée
(pas mauvais, mais c’est dommage qu’ils le sucrent), et pour finir un pot
de skyr. Le skyr a joué un grand rôle dans la vie de Gaël pendant ce mois
et je dois admettre que c’est fort bon, alors pour le lecteur curieux, le
skyr est une sorte de fromage blanc très compact et onctueux à la fois,
ça nourrit bien et en plus c’est plein de protéines ! J’avais repéré dans
des guides que le skyr était un produit islandais bon et bon marché, so
we gave it a try et ce n’était que le premier d’une longue série ! Mais
n’anticipons pas. Nous allons nous coucher dans la tente : il fait encore
jour car il y a tout au plus une heure de nuit à cette époque de l’année.
Moi ça ne me gêne pas mais Gaël a du mal à dormir. Il faut dire que ça
fait bizarre d’entendre les oiseaux chanter quand je me réveille un
instant pour changer de position !</p>
</div>
<div class="section" id="jeudi-24-juin">
<h2>Jeudi 24 juin</h2>
<p>Aujourd’hui nous avons prévu d’aller en bus vers Landmannalaugar, au
centre du pays, mais de nous arrêter avant près du volcan en activité
Hekla et de tenter l’ascension. Nous nous réveillons à 6h. Il a plu et la
tente est mouillée. Nous avalons un petit déj’ original composé du reste
de harengs et de skyr puis nous nous dirigeons vers le terminal BSI (la
compagnie de bus locale), à 3/4 d’h de marche du camping. Le bus est très
haut et $4times4$ pour franchir les gués : un vrai bus pour aventuriers
! Le chauffeur est marrant, un peu looseur. Il ne paye pas de mine, a un
anglais relativement mauvais, mais veut tout nous expliquer (nous avons
d’autorité choisi les places juste derrière le chauffeur pour mieux voir
le paysage) et il joue du levier de vitesses en même temps que du
portable. Nous suivons d’abord la côte du sud-ouest, relativement peuplée
: c’est une région fertile où se trouvent également des serres chauffées
à l’énergie géothermique qui font de l’Islande un grand producteur de
bananes ! Puis nous nous enfonçons dans l’intérieur des terres où nous
apercevons des paysages lunaires : de grandes plaines noires avec le
mont Hekla qui se dresse dans le fond. En arrivant près du volcan, le
chauffeur propose un arrêt photo et nous voyons que la montagne est
couverte de neige jusqu’à une altitude très basse donc nous préférons
renoncer à l’ascension et continuer jusqu’à Landmannalaugar. Ce site est
connu pour ses sources chaudes entourées de montagnes de rhyolite aux
couleurs insolites. Du bus nous admirons le paysage volcanique : les
montagnes semblent sortir de nulle part car la lave a aplani leurs
contreforts en se déversant. Elles ont des couleurs étonnantes : oranges,
brunes, et la mousse a un vert fluo très impressionnant, sur lequel les
dentelles de neige en altitude sont de toute beauté. Après quelques gués
qui ne posent aucun problème au chauffeur, nous arrivons dans la plaine
de Landmannalaugar. Nous nous installons dans la hutte, le camping
n’étant pas très très tentant car le vent est fort et il s’est mis à
pleuvoir. Après un petit pique-nique nous partons en promenade dans le
temps de chien : fort vent et pluie à la limite de neige. Nous marchons à
travers les collines vertes et noires qui recèlent plusieurs lacs dans
leurs creux, dont un dans une ancienne caldeira. Les pentes de la
caldeira sont indigos, rouges, violettes, brunes : c’est superbe et si nous
apprenons que ce lac porte le nom de “lac laid” en islandais, c’est parce
que les paysans n’appréciaient pas d’y perdre leurs moutons ! Nous
l’admirons avec la capuche serrée par tous ses cordons : il y a beaucoup
de vent. Cette sauvage beauté aurait sa place dans un épisodes des
<em>Celtiques</em> de Corto Maltese ! Nous ne croisons personne. Au retour
le froid et l’humidité m’engourdissent les doigts sans que je m’en rende
compte. A la hutte je cours me plonger dans les sources à côté de la
hutte (et je cours vraiment parce qu’être en maillot de bain dans ce
temps n’est pas vraiment agréable !). Quand on est dedans, qu’est-ce que
c’est bon ! Mes pieds engourdis me piquent au début, mais au bout de
quelques minutes je me sens vraiment très bien. L’eau est environ à 38°.
Dans le bain j’ai la compagnie de 25 petits Anglais assez surexcités à
l’idée d’être en Islande…</p>
</div>
<div class="section" id="vendredi-25-juin">
<h2>Vendredi 25 juin</h2>
<p>Nous avons prévu de passer cette journée à Landmanalaugar pour nous
balader dans le coin. Au réveil (7h), un petit coup d’oeil par la fenêtre
: tout est enneigé ! Et il fait un temps de chien : une pluie/neige
battante… Le groupe d’Anglais devait faire la traversée jusqu’à
Þórsmörk mais ils y renoncent avec sagesse : les gamins ont environ
15/16 ans. Quant à nous nous avons compris que si on attendait la fin de
la pluie pour se promener on n’allait pas se promener beaucoup… On
sort donc en direction d’un sommet pour voir comment ça se comporte sur
les hauteurs. Nous sommes suivis par quelques personnes bien
encapuchonnées mais qui renoncent assez vite. What a shame ! Nous
grimpons à flanc de la montagne, et assez vite nous sommes sur la crête.
La pluie abondante nous arrive dessus horizontalement car il y a beaucoup
de vent… surtout sur l’arête ! Je trouve qu’il y a vraiment beaucoup de
vent mais comme j’écris ce passage le jour d’après (voir la suite !) en
réalité c’était très honnête ! C’est fatigant en tout cas car il est plus
difficile de respirer quand le vent vient de face. Ma prochaine veste
sera <em>vraiment</em> imperméable et aura une capuche semblable à une
armure ! Arrivés au sommet local, nous préférons rentrer plutôt que de
faire un grand tour à patauger dans la neige. Nous redescendons beaucoup plus
vite que nous sommes montés ! Le terrain est assez friable, composé de
terre, de cailloux : pas grand chose ne pousse (la veille par contre nous
marchions sur des collines moins hautes recouvertes d’une mousse
moelleuse qui permettait de dévaler leurs pentes sans se faire mal). Nous
arrivons à la hutte en même temps que 4 jeunes (Allemands et Hollandais )
qui sont descendus de Hfratinnusker, la hutte suivante dans notre
parcours. Ils sont très mouillés et bien crevés. Nous les interrogeons :
“How was it ? -Bad. How bad ? Very bad. “. Les gardiennes nous avaient
informé que personne ne pouvait monter à l’étape suivante ce jour-là :
trop de neige et surtout très faible visibilité. Les 4 ont bien galéré en
pateaugeant dans la neige parfois jusqu’aux hanches. Les gardiennes
rassurent par radio Fanney, la gardienne du haut qui a accepté de les
laisser descendre, mais se fait du souci pour eux. Quant à nous, je convaincs
Gaël d’aller se baigner dans les sources chaudes où nous passons un
moment très agréable. Puis pique-nique et une petite sieste (je dors
comme une marmotte !). Et on repart en balade. Nous traversons d’abord
un très “joli” champ de lave : de gros blocs moussus qui se dressent les
uns à côté des autres. On y imagine bien une armée d’elfes et autres
gobelins se cachant dans les anfractuosités entres le statues de lave.
Nous arrivons dans une grande plaine, en fait un cirque entouré de
montagnes aux couleurs fantastiques : toute la gamme des
rouge-orange-bruns, du violet et un vert incroyable. La plaine est
sillonnée de quelques petites rivières : je me mouille les pieds et je ne
suis pas contente ! Nous rencontrons des bébés sources chaudes : de
petites mares bouillantes et crachotantes avec une dizaine de bulles.
C’est très mignon. A un endroit il y a un véritable vivier de sources
chaudes, et ce chaudron géothermique est d’un rouge impressionnant. A
peine plus loin une mare de boue d’un gris maquette bout furieusement.
Nous passons à côté d’une belle cascade, et nous voudrions rentrer par
une arête mais je fatigue (pas mal de vent et ça monte raide) et nous ne
sommes pas sûrs de trouver un chemin donc nous reprenons le même chemin
qu’à l’aller.</p>
<p>Nous dévorons notre riz cantonnais et un groupe de Français en voyage
organisé nous offre des côtes de mouton qu’ils n’ont pas mangées (5!) :
quel régal inattendu !</p>
</div>
<div class="section" id="samedi-26-juin">
<h2>Samedi 26 juin</h2>
<p>Sûrement notre journée la plus dure.</p>
<p>Réveil à 8h (car nous attendons une météo à 9h). Il fait toujours aussi
moche mais le temps s’est réchauffé. On se tâte, la gardienne de
Hfratinnusker ne veut pas qu’on lui envoie de randonneurs mais pour nous
c’est ou y aller ou reprendre le bus. Finalement on part avec l’idée de
faire demi-tour et prendre le bus si ça ne passe pas. La qualité de notre
équipement et notre expérience ont un peu rassuré les gardiennes de la
hutte. Le début de la marche est magnifique. Nous retraversons le champ
de lave, puis le chemin passe de vallon en vallon avec à chaque fois des
couleurs merveilleuses dégradées en bandes sur les flancs des montagnes,
des sources chaudes et des bancs de brume qui sentent le soufre ! Il
pleut toujours bien sûr, mais ce coin est d’une beauté indescriptible.
Nous arrivons sur de hauts plateaux désertiques où rien ne pousse. Il
commence à y avoir beaucoup de vent et j’ai du mal à gravir certaines
pentes assez raides où le terrain s’éboule sous mes pieds. Arrivés à une
rivière, on se pose la question : “on continue ou pas ?”. Vu qu’on a fait
la moitié d’après Gaël, et même plus d’après mes souvenirs de la carte,
j’ai envie de continuer. Il est vrai que je ne me doutais pas que la
suite serait aussi dure. Le vent forcit et forcit : j’ai du mal à tenir
debout, et à penser à autre chose qu’au vent. Une sangle de mon sac bat
et me fouette le visage : pas très agréable. Heureusement Gaël me
l’arrange “en vol”. Périodiquement nous devons vider nos gants censément
waterproof mais qui se remplissent d’eau par le haut : ce n’est pas très
grave mais dès qu’on veut toucher à sa capuche ou au sac un flot d’eau
glacée vient mouiller la polaire : à éviter ! Plusieurs fois on se confie
les gants quand on les enlève : il vaut mieux se concentrer car avec ce
vent lâcher un gant c’est le perdre = gros problème, les mains sont ce
qui se refroidissent le plus vite. Après chaque manoeuvre de vidange
je me livre à quelques exercices pour bouger les mains en chantant
intérieurement la comptine “Jean petit qui danse” pour me motiver.
Heureusement on marche vite et je n’ai pas froid. On atteint vite la
neige et en même temps la visi baisse beaucoup : il est souvent
impossible de voir les poteaux qui marquent le chemin. Heureusement nous
voyons les traces de ceux qui sont descendus hier : ce serait beaucoup plus
dur sinon. Elles ont été un peu soufflées, mais ça permet d’avancer de
poteau en poteau. Mais que de vent ! Les cheveux de Gaël sont trop courts
et ne restent pas dans son élastique : à un moment il me demande de
passer devant car je vois mieux que lui. La neige est assez peu profonde
mais très mouillée et on enfonce parfois pour se retrouver les pieds dans
15 cm d’eau ! Mais je préfère la neige lisse aux rares endroits où nous
avançons sur de la terre et de gros cailloux : avec le vent j’ai du mal à
contrôler où je mets les pieds et je dois parfois faire quelques
entrechats pour éviter des rochers. A un moment nous perdons la trace. Le
brouillard est très dense : tout est blanc autour de nous. Nous essayons
dans une direction : Gaël me demande de rester à un endroit et s’avance
plus loin ; je crois comprendre qu’il me demande de le rejoindre. En fait
il agitait seulement les bras pour se maintenir en équilibre dans le
vent. On s’assied alors quelques instants pour une pause carte +
instruments. Il fait drôlement froid quand on ne bouge pas. On repart en
suivant la direction indiquée par la carte grâce à la boussole de Gaël.
Ce sont peut-être les pas les plus durs car on ne sait pas où on va.
J’avoue avoir pas mal douté alors : le vent hurle et me fait tomber tous
les 100m, j’essaye de penser très fort au refuge. Mon envie de sucrerie a
brusquement disparu ; je m’aperçois que je ne veux qu’une chose : avancer
sans m’arrêter. Au bout de quelques centaines de mètres j’aperçois dans
la brume la silhouette fantômatique d’un poteau : on a retrouvé le
chemin. Quel soulagement ! Nous pouvons alors suivre les poteaux : il y a
toujours autant de vent, mais c’est quand même beaucoup plus motivant de
savoir qu’on va dans la bonne direction. A un moment Gaël me fait un
grand signe d’encouragement : il a vu la hutte entre deux bancs de brume.
Moi je ne l’ai pas vue et je n’ose pas y croire. Je ne la vois que très
tard, quand elle est tout près si bien qu’elle a vraiment l’air énorme.
Euphorie intérieure. Nous nous déshabillons dans l’entrée du refuge sous
l’oeil goguenard et pas très content de Fanney, la gardienne de la hutte,
qui avait expressément demandé qu’on ne lui envoie personne en raison de
la météo. Elle se rassure un peu en voyant notre équipement et nous offre
une bonne tasse de chocolat chaud : ça fait du bien. Pendant ce temps je
me change des pieds à la tête : ma veste n’est pas très étanche et mes
vêtements sont trempés. Je suis assez crevée mais j’essaye de ne pas le
montrer pour que Fanney ne soit pas trop fâchée ! Je me jette sur les
sucreries qui m’avaient fait tellement envie durant la marche et nous
mangeons un petit pique-nique. Il y a quatre randonneurs dans le refuge
qui voudraient bien redescendre à Landmannalaugar mais ils sont assez mal
équipés. Fanney qui est vraiment adorable les enveloppe dans des sacs
poubelle et du grey tape pour leur confectionner une armure à peu près
étanche. Nous leur prodiguons quelques conseils : au moins ils pourront
suivre nos traces. Nous passons le reste de l’après-midi à lire, manger
et discuter avec Fanney qui nous raconter ses tribulations avec les
touristes inconscients et mal équipés. Et puis plouf dodo !</p>
<div class="section" id="la-version-de-gael">
<h3>La version de Gaël</h3>
<blockquote class="epigraph">
Nous décidons de tenter le coup : je voudrais savoir ce que cette
montagne a de si terrible. Le début du chemin est superbe : champ de
lave, fumerolles de soufre et montagnes colorées. Puis nous débouchons
sur les sommets : un relief très rond, des collines noires avec une
superbe mousse verte. La brume s’abat sur nous. Le vent souffle fort et
nous refroidit. La progression n’est pas très dure mais fatigante, à
cause du vent et du terrain qui part sous nos pieds. Le chemin est très
bien marqué mais au fur et à mesure que nous montons la brume s’épaissit
et la neige s’accumule. Les traces des quatre qui sont descendus avant
nous sont bien visibles et nous facilitent la progression. Emmanuelle a
l’air fatiguée et je lui propose avec insistance que nous fassions
demi-tour : je crains que cela devienne de plus en plus dur. Elle n’est
pas découragée même quand je lui dis que nous n’avons fait que la moitié
du chemin (alors que nous avions fait les deux tiers). Le vent m’arrache
l’élastique qui tenait ma queue de cheval, je continue avec les cheveux
dans les yeux. On voit à peine à vingt mètres. Le vent est si fort que
nous avons du mal à tenir debout. Il a fini par souffler les traces que
nous suivons : nous perdons toute trace du chemin qui disparaît sous la
neige. Les piquets sont tombés ou cassés par le vent. Nous nous abritons
derrière nos sacs pour regarder la carte : 1080m, l’altitude de la hutte.
Alors nous faisons 300 m de SSW au compas, droit dans le jour blanc, puis
Emmanuelle aperçoit des piquets. Nous revoila sur le chemin. 200m plus
loin j’aperçois la hutte par une trouée dans la brume. Nous y sommes !</blockquote>
</div>
</div>
<div class="section" id="dimanche-27-juin">
<h2>Dimanche 27 juin</h2>
<p>Au réveil nous constatons qu’il y a encore beaucoup de brouillard. Au moins il
ne pleut pas. nous passons la matinée à attendre une éclaircie, bien au
chaud dans notre petit refuge. Je suis un peu malade : visiblement mon
estomac a du mal à se remettre des émotions d’hier. Nous partons peu
avant midi et nous prenons congé de Fanney avec émotion : elle
est vraiment géniale, pendant la matinée elle nous a montré sur plusieurs
cartes les endroits d’Islande que nous devions visiter. Il y a pas mal de
vent, mais moins qu’hier. Toujours beaucoup de neige, avec parfois des
accumulations dans les creux qu’il faut escalader. Toujours du brouillard
aussi, mais nous suivons des traces de pas. On descend peu à peu,
quelques éclaircies dans la brume laissent entrevoir des flancs de
collines multicolores (rouge, violet, vert gris) agrémentés de sources
chaudes, fumerolles, langues de neige bleutées et curieusement sculptées
par le vent. Hfratinnusker signifie mont des obsidiennes. Le paysage est
censé être magnifique, pour ce qu’on voit c’est vrai… mais on ne voit
pas grand chose ! Sous nos pieds, une sorte de boue grise collante : nous
sommes au paradis du géologue ! Nous amorçons alors une longue descente :
nous quittons la piste des hautes terres. Sur le côté, quelques traces de
mousse : quelle végétation luxuriante ! Le temps est toujours très
pluvieux mais la visi devient meilleure : normal, nous sommes bien
descendus. Nous rencontrons à nouveau des rivières, dont une qu’il va
manifestement falloir traverser… à gué ! Nos chaussures sont déjà
mouillées de toute façon. Ce premier passage de gué n’est pas très
agréable pour moi : le sol est irrégulier et je glisse à cause du
courant. Je me retrouve quasiment à quatre pattes dans le sens du
courant, accrochée à une grosse pierre et un peu paniquée. Heureusement
le gentil Gaël vient me sauver et m’aide à rejoindre l’autre rive. J’ai
de l’eau partout jusqu’à la ceinture, mes chaussures font floc-floc et je
mets à marcher TRES vite pour ne pas avoir froid. De jolies petites
collines vertes bordent le chemin. Nous atteignons la hutte d’Alftavatn
mais nous décidons de continuer jusqu’à la hutte suivante, 5 km plus
loin. Nous traversons un petit gué facile, puis nous arrivons à une
rivière beaucoup plus large au courant assez fort : nous faisons une
chaîne pour avancer. Je trouve ça beaucoup plus sécurisant : chacun
avance à son tour. C’est bien trempés et fatigués que nous arrivons à la
hutte où se trouve déjà Martin, un autre français… à côté d’un poêle à
gaz bien chaud ! Le gardien du refuge, un solide fermier, passe prendre
le prix de la nuitée : il est accompagné par ses deux fils, deux bambins
en ciré et bottes qui nous sourient avec un air angélique, puis se
mettent tout d’un coup à se tomber dessus à bras raccourcis sans cesser
de sourire. Nous passons le reste de l’après-midi à discuter avec
Martin qui a pas mal bourlingué et a un régime assez spécial : dès qu’il
a fini de marcher il se câle l’estomac avec un sachet de quatre assiettes
de soupe puis s’endort avant d’avoir faim ! Nous on préfère manger : on
doit tenir un mois quand même. Nous assistons à l’arrivée d’un groupe de
12 cavaliers, qui se montrent discrets dans la hutte.</p>
</div>
<div class="section" id="lundi-28-juin">
<h2>Lundi 28 juin</h2>
<p>Au réveil : il y a du ciel bleu ! Je n’en crois pas mes yeux. Quel
bonheur ! Nous nous préparons lentement. Les reliefs du festin des
cavaliers font envie. Nous allons voir les chevaux : ils sont super
mignons ! Aujourd’hui nous avons une grosse étape : 21 km jusqu’à
Þórsmörk. Nos chaussures sont toujours trempées de la veille, il ne fait
pas très chaud mais ça fait du bien de marcher sous le soleil ! Un
premier gué est en fait très facile, contrairement à ce que Martin nous
avait raconté : ça me rassure. Puis nous traversons un désert de sable
noir avec sur les côtés de belles collines vertes aux sommets enneigés.
Le pauvre Gaël a une ampoule, nous nous arrêtons pour qu’il se soigne.
Nous arrivons à Botnar, normalement la fin d’une étape, mais nous sautons
la hutte. Nous nous arrêtons cependant pour faire un bon pique-nique
étendus dans l’herbe : ça faisait longtemps que nous n’avions pas
pique-niqué dehors en raison du temps. Puis nous repartons en enchaînant
une deuxième étape. Nous commençons par suivre assez longtemps les gorges
encaissées (la pierre rouge bordeaux répond au vert de la mousse) d’une
rivière tumultueuse. Heureusement il y a un pont pour la traverser. C’est
très beau et très sauvage, des oiseaux blancs planent dans les
profondeurs de la gorge où quelques moutons improbables font de petites
taches blanches. Puis nous descendons dans des vallées, nous marchons
assez longtemps me semble-t-il. Le soleil fait place à un peu de pluie,
mais il revient et nous arrivons… dans une forêt ! C’est assez rare en
Islande pour qu’on le signale : la blague ici est “Que fais-tu quand tu
es perdu dans la forêt en Islande ? Réponse : tu te lèves !”. Mais de
jolis bouleaux très verts s’agitent doucement dans le vent. Le dernier
obstacle est la traversée d’une rivière glaciaire : comme nos chaussures
ont séché nous la traversons en Teva (nos sandales). Ca fait froid aux
pieds ! Nous arrivons à la hutte où nous retrouvons le groupe d’Anglais
qui nous font raconter notre traversée et ne sont pas loin de nous
considérer comme des héros ! Nous avons fait 51 km en trois jours et pas
dans les meilleures conditions… Ce soir nous dormons sous la tente pour
faire quelques économies : en plus il fait beau.</p>
</div>
<div class="section" id="mardi-29-juin">
<h2>Mardi 29 juin</h2>
<p>Nous avons terminé hier la randonnée classique Landmannalaugar-Þórsmörk : nous
voulons à présent continuer jusqu’à Skogar, sur la côte Sud de l’Islande.
Pour cela il faut emprunter un passage entre deux glaciers.</p>
<p>Il fait moins beau quand nous nous réveillons, et nous nous dépêchons de
plier la tente. Il y a du vent. Gaël prend un énorme petit déjeuner. Ce
matin nous avons un peu de mal à nous mettre en route. Nous devons
d’abord traverser la “vallée de Thor”, une vallée boisée où la rivière se
divise en plein de bras. C’est superbe vu du haut, on a l’impression que
le delta se prolonge jusqu’à la mer. Il nous faut d’ailleurs traverser
cette rivière : un pont permet de traverser une partie à pied sec, mais
il y a encore d’autres bras à traverser à pied. Nous nous mettons pieds
nus pour ne pas mouiller nos chaussures qui sont sèches : mauvaise idée
car la rivière est très froide et les cailloux font mal aux pieds. Nous
remontons la vallée avant d’entamer une forte montée dans les montagnes
pour trouver la passe entre les deux glaciers. Le chemin est joli : les
nombreux versants sont recouverts de mousse très verte. A la fin de notre
pause déjeuner nous sommes rejoints par une famille allemande que nous
avions déjà vue à Landmannalaugar et qui avait pris le bus jusqu’à
Þórsmörk. Ils recyclent notre aire de pique-nique ! Ca commence à monter
très raide : je fatigue et il y a pas de mal de vent. Nous arrivons sur
un grand plateau désertique et caillouteux ; il y a un vent du tonnerre.
Nous n’avons pas de temps à perdre : il faut avancer vite, même si le
vent rend la progression difficile. Le temps se gâte. Au bout du plateau
nous avons une vue superbe : devant nous se dresse un magnifique rocher
creux, d’un côté une enfilade de vallées qui plongent, sillonnées d’une
multitude de petites rivières, de l’autre le glacier descend en langues
bleutées et tourmentées qui finissent en cascades jaillissant sur la
terre noire. Le tout avec un vent terrible. Nous regretterons très fort
de ne pas avoir pris de photos mais il y avait vraiment trop de vent. En
plus nous devons maintenant gravir une forte montée, sur un terrain
caillouteux qui glisse, il faut vraiment arracher chaque pas à la
montagne. Nous ne voyons plus les Allemands : ont-ils fait demi-tour ?
Nous luttons beaucoup durant cette longue montée : c’est épuisant car
nous voulons aller vite. Après la grande montée, il y a une série de
petites montées et descentes que la carte n’est pas assez précise pour
indiquer : c’est frustrant car nous pensions en avoir fini de grimper.
Heureusement, il y a un peu moins de vent et même un rayon de soleil qui
fait miroiter les champs de neige sous nos pieds. Nous voyons deux
maisons au loin : l’une ressemble à une hutte touristique et l’autre à un
refuge de secours mais la carte dit le contraire. Nous allons donc en
direction de ce que la carte dit être la hutte. Plus on se rapproche plus
ça a l’air d’être un abri, on a même peur que ce soit fermé mais
heureusement c’est ouvert et il y a même quatre jeunes islandais, enfin
plutôt trois plus un sac de couchage qui a l’air habité : ces jeunes
voulaient faire la traversée dans le sens inverse que nous mais l’un
d’eux est tombé malade et ils attendent les secours. Une route de jeep
mène jusqu’à la hutte mais les secours ont crevé et mettent du temps à
arriver. Pendant ce temps nous nous installons : je vais chercher de la
neige à faire fondre car il n’y a pas d’eau. Bonne surprise : cet abri
non gardé coûte moins cher que le camping de la veille. Par contre la
carte était fausse et ce ne sera pas la seule fois… Nous mettons nos
sacs à l’étage où il fait plus chaud, je m’endors comme un bébé pendant
que Gaël dessine.</p>
</div>
<div class="section" id="mercredi-30-juin">
<h2>Mercredi 30 juin</h2>
<p>Le vent a soufflé toute la nuit, faisant trembler les vitres,
heureusement quand on se lève il est moins fort que la veille. Le muesli
froid a un goût très cartonneux. On se met en route. Aujourd’hui, nous
n’avons que de la descente. Le chemin suit d’abord une piste de
4x4 moyennement intéressante avant d’arriver à une tumultueuse
rivière où ô bonne surprise il y a un pont pour les piétons (non marqué
sur la carte). Un sentier suit le cours de la rivière. Un panneau
uniquement en anglais le recommande pour ses “many beautiful waterfalls”,
en effet à certains détours de la rivière l’eau et les rochers forment un
spectacle grandiose. Je suis assez fatiguée, la marche n’est pas dure
mais le terrain est assez accidenté. Mais le paysage est très beau. Nous
arrivons à Skógarfoss, ou chutes de Skógar, site annoncé par la présence
de nombreux touristes, appareil photo en bandoulière. Plusieurs volées
d’escaliers nous permettent de perdre les dernières dizaines de mètres
pour arriver au niveau de la mer. Nous pouvons alors admirer les
impressionnantes chutes, très belles en effet. C’est ici que se termine
notre traversée depuis Landmannalaugar : il va nous falloir un peu de
repos… Nous décidons de faire du stop jusqu’à Kirkjubaejarklaustur
(essayez de prononcer !). Il fait beau. Nous nous postons à la sortie de
Skógarfoss, mais la plupart des gens y vont et les autres ne semblent pas
vouloir nous prendre. On attend. Après un certain temps, surprise, c’est
un bus de touristes qui s’arrête pour nous prendre ! C’est inhabituel. Il
s’agit d’un bus de Genevois avec un guide très sympa qui annonce à la
cantonnade : ils viennent de Paris, on peut leur parler français ! Il
fait chaud dans le car, on est très confortablement assis derrière le
guide et je savoure l’instant. Mais le temps se dégrade et arrivés à Vik,
où le car fait une pause déjeuner avant de repartir dans l’autre sens, il
tombe une pluie battante et il y a du vent. Nous avalons rapidement notre
déjeuner dans la station service qui représente le coeur du bled puis
nous nous remettons au bord de la route avec le pouce levé. Hélas nous
sommes trempés en 5 minutes, intérieur des chaussures y compris : pas très
agréable, surtout dans le vent très fort qui nous oblige à tenir nos
sacs. Personne ne semble vouloir nous prendre : certaines voitures,
pleines par exemple, nous font un signe d’encouragement mais pleins de
grosses 4x4 vides passent sans nous jeter un coup d’oeil.
Appremment l’Islande n’est pas un pays très propice au stop ; c’est
décevant car nous avons eu de très bonnes expériences dans ce domaine
dans d’autres pays. Trempés et glacés nous nous réfugions à nouveau dans
la station service pour nous concerter. Nous décidons d’aller à l’auberge
de jeunesse de Vik. Malheureusement quand nous y sommes, il n’y a plus
qu’un lit ! Coup dur pour le moral. Nous retournons piteusement faire du
stop à notre ancien emplacement, mais toujours en vain et c’est toujours
aussi fatigant, même si cette fois nous avons eu soin de mettre notre
pantalon étanche. Nous avons le moral dans les chaussettes. Au bout d’une
heure et demi je n’en peux plus : nous rentrons dans la station service.
Nous ne savons pas trop quoi faire : il y a d’autres hôtels mais hors de
prix (il faut quand même réaliser que l’auberge de jeunesse coûte 25
euros par personne…) : on pourrait planter la tente mais sous cette
pluie diluvienne… En désespoir de cause nous retéléphonons à l’auberge
de jeunesse en espérant qu’ils ont eu des annulations. Non, mais la
patronne (qui a sûrement pris pitié de nous) nous dit qu’en fait on peut
prendre le lit et mettre un matelas par terre. Ouf ! Comme ce n’est pas
encore l’heure de se rendre à l’auberge, j’achète un yaourt à la fraise à
Gaël et je me prends un café chaud, le tout pour nous consoler et nous
réchauffer un peu. Nous sommes crevés, mais soulagés d’avoir un toit. A
l’auberge de jeunesse, un jeune croate sympa nous montre la chambre et on
s’installe. Frigorifiée, je fonce sous la douche que Gaël a la galanterie
de me laisser. Ca fait du bien. C’est la première douche depuis une
semaine, je la règle pour qu’elle soit brûlante, et ce n’est pas du luxe
de me démêler les cheveux ! Nous mangeons rapidement puis nous nous
couchons tôt. Arrivée dans la chambre d’un couple qui deviendra mythique
pour nous : le gros porc et la haridelle en ciré jaune. Lui ressemble à
un bûcheron scandinave avec un énorme ventre, un gros ours mal léché ;
elle super sèche en ciré jaune, très grande avec des cheveux gris
impitoyablement tirés. A 5h du matin les ronflements du gros porc me
réveillent : c’est tout un concerto avec des interruptions, des trémolos,
des gémissements : le pire est encore quand ça s’arrête. En plus de
l’autre côté de mon matelas sa nana ronfle aussi, quoiqu’avec moins
d’imagination : on a des ronflements en stéréo !</p>
<p>Ah oui, pour information Vik est l’endroit d’Islande où il pleut le plus
! On a bien galéré pendant quelques heures…</p>
</div>
<div class="section" id="jeudi-1er-juillet">
<h2>Jeudi 1er juillet</h2>
<p>Au réveil nous décidons d’aller dévaliser le supermarché du bled pour se
faire un petit déjeuner de fête. En plus il nous faut du ravitaillement
pour les jours suivants. Notre menu : café ou thé + 6 oeufs + pain grillé
au fromage + skyr au miel : on mange bien. L’estomac satisfait, nous
allons faire une balade au pied des falaises noires de Vik dans l’espoir
d’apercevoir des macareux ou puffins en anglais. Le temps est typiquement
breton : petit crachin + vent. De gros rouleaux déferlent sur l’austère
plage de sable noir : c’est beau mais ça ne donne pas envie de se
baigner. Plein d’oiseaux nichent dans la falaise, surtout des simili
mouettes qui font l’aller-retour entre la mer et leur nid où les petits
piaillent. Et au-dessus de ces nids Gaël aperçoit un puffin ! C’est un
tout petit oiseau très coloré qui bat très vite des ailes. Nous sommes
tout contents et après le premier nous en voyons plein d’autres.
L’Islande est connue pour être le paradis des ornithologues (je n’ose pas
dire des oiseaux car le temps n’est pas très clément). Nous retournons à
l’auberge de jeunesse chercher nos sacs et nous allons prendre le bus
BSI à notre chère station-service ! Heureusement le temps est un peu
meilleur qu’hier. On avale vite fait une boîte de poisson mariné dans la
sauce aux oignons : miam ! Le bus s’arrête quelques minutes au parc
naturel de Skaftafell, au pied du plus grand glacier d’Islande, le
Vatnajökull. Nous voyons le bout du glacier, tout crevassé. Le coin a
l’air sympa mais il est très touristique et conçu principalement pour des
promenades à la journée, donc nous avons choisi de ne pas nous y arrêter
(contrairement à la majorité des touristes). En lisant attentivement les
horaires des bus nous avons trouvé une combine qui nous permet, grâce à
un changement de bus, de passer 40 minutes à Jökulsárlón où le
Vatnajökull descend jusqu’à la mer. Des icebergs se détachent et
flottent quelque temps dans un lagon glacé et irréel avant d’être
suffisamment fragmentés pour pouvoir passer le goulet qui mène à la mer.
L’endroit est très impressionnant ; en arrivant avec le bus on voit tout
d’un coup une dent bleue qui dépasse du remblai sur le côté de la route,
ça fait une drôle d’impression. La pluie et la brume donnent une
atmosphère désolée à cet endroit étonnant ; on devine au loin le glacier
au-delà du lagon où traînent les blocs bleutés attendant leur départ pour
l’océan. Nous reprenons un autre bus, tout rouge et tout petit ! Les bus
les plus luxueux se trouvent du côté de Reykjavik, la population est
moins riche dans le reste du pays. Nous avons un chauffeur à la noble
barbe blanche : beaucoup mieux que le précédent ! Nous arrivons à Höfn, ville
portuaire importante (1900 habitants !) où le camping est sympa et pas
trop cher. Un groupe de motards allemands hauts en couleurs y fait la
cuisine. Nous allons faire quelques petites courses, et je trouve un
magasin de sport pour remplacer mon bandeau North Face que je crois avoir
perdu : heureusement, je le retrouverai peu après. Le repas est suivi
d’une petite douche (deux douches en deux jours, quelle décadence !),
puis dodo dans notre belle tente.</p>
</div>
<div class="section" id="vendredi-2-juillet">
<h2>Vendredi 2 juillet</h2>
<p>Notre petit déj’ est agrémenté d’un bon gâteau marbré. Nous reprenons le
bus avec le même chauffeur barbu. Après Höfn la Ring Road (l’autoroute du
pays) se transforme en dirt road. Nous n’allons pas très loin : au bout
de 30 km on se fait à dropper à Sfatafell, une ancienne ferme qui fait
aussi auberge de jeunesse, au bord d’une réserve où nous voulons aller
randonner. Nous cherchons des informations sur le chemin jusqu’au
Snaefell (un volcan qui est aussi la plus haute montagne d’Islande en
dehors des glaciers). A côté de l’auberge de jeunesse se trouve une
petite église entourée d’un cimetière très paisible, avec plusieurs
espèces de beaux arbres (c’est rare ici). A l’auberge de jeunesse un
fermier très sympa nous apprend qu’un bus part pour la première hutte du
chemin dans une demi heure, ce qui nous gagnerait pas mal de temps (en
plus, nous ne croulons pas sous la nourriture)… mais pour 2000
couronnes par personne ! C’est cher et on n’arrête pas de dépenser ici.
Nous voyons avec effroi le couple ronfleur au ciré jaune de Vik sortir de
l’auberge de jeunesse. Heureusement ils ne vont pas au même endroit que
nous. Enfin, nous décidons de prendre le bus quand même, en compagnie
d’une joyeuse bande d’Islandais (il y a beaucoup moins de touristes dans
cette région) qui vont passer la nuit à la hutte. La route est évidemment
non goudronnée. Notre bus 4x4 et notre chauffeur sont
impressionnants : ils traversent des gués difficiles, manoeuvrent sur
des routes très raides à flanc de côteau, c’est l’aventure ! Le paysage
dans lequel nous avançons est très beau. Géologiquement, c’est de la
rhyolite dont les couleurs rouges se mêlent au vert des mini-forêts et au
bleu-gris des rivières : on se rattrape de ce qu’on n’avait pas pu voir à
Landmannalaugar à cause de la brume. On grimpe de plus en plus à travers
de très jolies vallées, puis le bus s’arrête. Nous sommes sur un
surplomb, le bus ne peut pas aller plus loin car la hutte est de l’autre
côté d’une tumultueuse rivière (il y a un pont pour les piétons). Nous y
descendons. En chemin nous croisons les gardiennes qui vont chercher leur
approvisionnement apporté par le bus. Nous les attendons à la hutte car
nous comptons sur elles pour obtenir des informations. Tiens d’ailleurs
on ne nous a rien demandé de payer dans le bus ! Pour une fois que
quelque chose est gratuit en Islande… La hutte est jolie, mais un peu
froide. Les gardiennes et un de leurs amis à l’allure de berger arrivent
et se préparent un solide repas de bacon, oeufs et baked beans : miam
!!!! Elles allument également le poêle à gaz, ce qui est agréable. Pour
un peu je me mettrais à ronronner. Elles sont très gentilles. La chef
nous donnes des infos sur la route, rectifie la carte qui est
grossièrement fausse à certains endroits, prend nos noms par sécurité,
puis elle nous accompagne pendant un bout du chemin. C’est une petite
balade tranquille jusqu’à la hutte suivante. Au début nous suivons la
vallée colorée rouge-grise à travers un joli bois de bouleaux, puis ça
monte assez sec vers des hauts plateaux d’où nous apercevons les sommets
enneigés environnants, des langues de glaciers, plusieurs lacs de
montagne : c’est très joli. Au bout d’un plateau nous voyons notre hutte
dans un creux au bord d’un lac. Un dernier effort : une rivière à
traverser en Teva juste à côté de la hutte d’Egilsell. Nous investissons
la jolie hutte, toute petite et proprette avec même du gaz et du
chocolate mix ! Nous faisons un festin : 400g de pâtes + de la soupe + un
chocolat chaud. Gaël a mal au dos : il porte beaucoup et il a peut-être
fait un faux mouvement auparavant.</p>
</div>
<div class="section" id="samedi-3-juillet">
<h2>Samedi 3 juillet</h2>
<p>Et pour se mettre en forme le matin, encore un chocolat chaud et même du
muesli chaud ! Et en route pour une bonne étape. J’ai beaucoup regardé
mes pieds aujourd’hui car nous progressions sur des rochers et des
cailloux coupants donc il fallait faire attention, mais sinon je crois
que le paysage était joli ! Et il n’a pas plu aujourd’hui.</p>
<p>Au début, nous traversons de verts pâturages au bord de lacs de montagne
(les sol est assez boueux et mes chaussures sont assez vite très sales),
puis nous arrivons à des collines de pierre. Il paraît que des rennes
vivent ici (ça doit être assez rude) mais nous n’en avons pas vu. Nous
traversons une rivière à gué juste avant le déjeuner, sinon pas de
difficulté particulière. Nous avons pu voir des langues de glacier de
très près, c’était sympa. Ce soir, nous sommes à Geldingafell dans une
jolie petite hutte au bord d’un ruisseau, on va se faire un bon repas et
j’espère que demain on arrivera à faire la traversée ! Nous sommes au
pied du Snaefell, belle montagne blanche et conique qui nous contemple,
droit et majestueux comme il se doit. Il y a souvent des nuages autour du
sommet, mais ils se sont dégagés à un moment !</p>
</div>
<div class="section" id="dimanche-4-juillet">
<h2>Dimanche 4 juillet</h2>
<p>Au programme : la traversée du glacier.</p>
<p>Au réveil il y a plein de brouillard. Nous ne sommes pas contents car le
chemin n’est pas marqué. Le petit déjeuner est agrémenté de thé chaud au
miel. Nous passons d’abord pas mal de temps à chercher un gué convenable
pour la rivière juste à côté de la hutte puis nous avançons en traversant
une succession de rivières, les unes en Teva, les autres en faisant du
BTP, c’est-à-dire en cherchant l’endroit le moins large puis en balançant
de grosses pierres près de la rive pour encore rétrécir le pas. Ca marche
plutôt bien. La marche est assez longue, sur le coup de midi nous
arrivons en vue de la langue de glacier à traverser. Le brouillard se
dissipe très vite. Nous pique-niquons : je me bourre de pain et de gouda.
Pour monter sur le glacier il faut d’abord contourner de profondes gorges
où coulent des rivières glaciaires : ça fait donc pas mal de détours. Ces
gorges sont belles, à un endroit plein de cascades vaporeuses tombent sur
une paroi de toutes les couleurs. Gaël repère un endroit pas mal pour
monter sur le glacier, mais les dernières dizaines de mètres avant le
glace sont une boue grise-verte dans laquelle nous enfonçons jusqu’aux
chevilles : nos chaussures se recouvrent d’une belle croûte de boue ! La
traversée du glacier dure environ une heure, elle est assez facile : la
glace est pleine d’aspérités donc on y avance très bien. Il faut sans
cesse sauter par-dessus des mini-crevasses : c’est rigolo pour les 30
premières, un peu fatigant ensuite. Le glacier est sillonné par plein de
petites rivières qui se jettent dans d’énormes trous en spirale bleutées
: mieux vaut ne pas tomber. Mais cela nous a quand même fatigués : nous
faisons une pause brownie de l’autre côté du glacier, non sans avoir
recouvert nos chaussures d’une nouvelle couche de boue ! Il fait très
beau. Nous obliquons vers une plaine sillonnée de petits ruisseaux et
peuplées d’oies sauvages et de petits loirs. Nous nous choisissons un
joli terrain de camping entre deux montagnes près d’un petit ruisseau, le
repas se compose de semoule, poisson séché et chocolat. C’était une très
belle journée, fatigante car longue. Il faut que j’arrête d’écrire, Gaël
va avoir besoin d’un massage.</p>
</div>
<div class="section" id="lundi-5-juillet">
<h2>Lundi 5 juillet</h2>
<p>Au réveil nous sommes fatigués. Heureusement le soleil descend
progressivement la pente de la montagne et chauffe bientôt la tente. Le
petit déj est acrobatique, avec juste la bouche et les mains qui
dépassent de la tente pour ne pas la salir, puis on y va. Le chemin est
facile, avec juste quelques petites rivières à traverser. Nous croisons
une famille d’oies, et un peu plus loin nous voyons un des oisons
sauvages qui est visiblement perdu : il était très mignon, j’espère
qu’il aura retrouvé sa famille. Il fait toujours très beau. Il n’y a que
10 km à faire pour arriver à la hutte du Snaefell, mais nous sommes
fatigués. Une fois arrivés à la hutte, le gardien est gentil et un peu
surpris de nous voir arriver : il est là depuis deux jours seulement. Par
contre il nous donne une mauvaise nouvelle : il n’y a pas de bus qui
vient ici, contrairement à ce que nous croyons. Il va falloir marcher 12
km pour atteindre une route assez fréquentée et faire du stop. Mais avant
nous faisons une pause déjeuner à l’intérieur de la hutte en lisant le
guestbook où nous complétons la saga d’un groupe que nous avions suivie
de hutte en hutte : ils parlaient notamment d’un renne Rudolf qui venait
leur piquer leurs bières… Les guestbooks sont une lecture très
rigolote. Nous pouvons même boire une tasse de Marshmallow Lovers, le
gardien est sympa. Puis c’est reparti pour 12 km sur une piste de
4x4 qui tourne beaucoup, avec un passage de gué en Teva pour faire
bonne mesure. Enfin, une 4x4 que nous avions croisée dans l’autre
sens revient et nous prend en stop ! C’est amusant, car la 4x4
est assez crade de l’extérieur, les passages de gué sont assez
dévastateurs pour ça, mais à l’intérieur elle est nickel : sièges en
cuir, bois de rose… il y a même le costume du cadre sup accroché sur un
côté… ils sont sympas de prendre deux randonneurs sales comme nous !
Nous avons terminé hier la randonnée classique Landmannalaugar-Thórsmörk : nous
C’est un couple sympa, ils nous offrent même des bonbons. Ce sont des
Islandais qui visitent leur pays. Nous arrivons sur la route “très
fréquentée” : on doit bien croiser une vingtaine de voitures à l’heure.
Mais ce n’est pas si mal. Le trafic est dû à la construction d’un barrage
pour alimenter une grosse usine : les Islandais sont partagés à propos de
ce chantier colossal. Le voyage en 4x4 de luxe est très agréable !
Nos sympathiques conducteurs nous déposent à une intersection. Nous
attendons un peu (au soleil !) puis nous sommes à nouveau pris par un
Islandais et son amie grecque qui nous emmènent jusqu’à Egilsstadir. En
chemin nous passons à côté d’une forêt de conifères : j’avais souvent
ironisé quand nous voyions un sigle pique-nique - la table avec le sapin
à côté - car je n’avais encore jamais vu de conifères en Islande, mais
apparemment il y en a quelques uns ! Ils en sont d’ailleurs très fiers à
Egilsstadir. Nos conducteurs nous déposent au camping (ils ont tout
compris !) où nous plantons la tente avant d’aller faire les courses au
Bonus, le supermarché pas cher. C’est un vrai concept en Islande que le
Bonus, et la présence d’un Bonus dans une ville semble réjouir ses
habitants. Il en va de même pour les randonneurs affamés. On fait la fête
: avec 500 g de viande hachée, un oignon, et deux boîtes de baked beans,
voila un ragoût qui nourrit bien, arrosé d’un litre de jus de tomate et
suivi de skyr à la fraise et d’un gros gâteau au chocolat et à la crème.
Ouf ça fait du bien ! On s’endort vers 11h : pour une fois nous ne sommes
pas couchés à 8h !</p>
</div>
<div class="section" id="mardi-6-juillet">
<h2>Mardi 6 juillet</h2>
<p>Je me réveille plus tôt que la montre de Gaël : je dors beaucoup ces
derniers temps ! Nous voulons prendre un bus vers 11h pour Borgafjördur,
un fjord de la côte est, donc nous avons tout le temps. Le petit déjeuner
est la suite du festin d’hier : muesli au chocolat, skyr et gâteau. On
mange tout le temps et beaucoup, mais on a tout le temps faim ! Nous avons
lavé nos T-shirts la veille : c’était nécessaire, mais c’est désagréable
d’enfiler un T-shirt froid et mouillé. Pour se réchauffer nous allons
consulter cartes et prospectus à l’accueil du camping. Le bus est bien en
retard, il sert aussi de poste, d’approvisionnement. Arrivés à
Borgafjördur, nous nous installons au camping où nous sommes seuls puis
nous partons visiter le village. Une petite église typique en bois peint
abrite un retable d’un grand peintre local, naïf mais beau : il
représente le Christ prêchant. Le dessin des corps est souple, les
couleurs osées : j’aime bien. Nous faisons rapidement le tour du village, avec
notamment une maison traditionnelle au toit de mousse et aux murs rouges
absolument charmante, maintenue en l’état par sa propriétaire depuis des
dizaines d’année. Un bateau de pêche est amarré sur le quai : ici on
pêche du cod, qui sont envoyés en Espagne sauf les têtes qui vont au
Nigeria ! Puis nous partons nous balader dans les collines au bord du
fjord. Un banc de mouettes se laisse descendre avec le courant le long
d’une rivière où elles pêchent, jusqu’à arriver à la mer où elles
recommencent alors. Le fjord est entouré de jolies montagnettes rocheuses
aux couleurs doucement rouges. C’est une journée de repos, nous n’avons
pas très envie de marcher et nous préférons lézarder sous le soleil
éclatant. Vers le soir deux Islandais bien équipés arrivent à “notre”
camping pendant que nous essayons de faire une sauce carbonara sans lait,
puis le fermier qui possède le camping passe nous faire payer, typique
dans sa salopette verte.</p>
</div>
<div class="section" id="mercredi-7-juillet">
<h2>Mercredi 7 juillet</h2>
<p>Nous nous réveillons super tôt (5h30) car nous devons prendre le bus à
8h. Le bus fait le tour du village pour prendre des colis, déposer des
gens : c’est le seul lien avec Egilsstadir. De retour à Egilsstadir à
notre grande déception le Bonus est fermé : pas de skyr avant midi. Gaël
est en train de devenir boulimique ! On part visiter la mini-ville :
beaucoup de maisons plates sans grand intérêt, mais une église à
l’architecture très moderne, à l’intérieur clair en bois, haute, avec
orgue et piano Steinway. Certaines maisons en bois sont plus jolies (et
appartiennent certainement à des gens riches !), avec des balcons en bois
sculptés, et l’inévitable driveway avec la grosse Toyota 4x4. Un
peu désoeuvrés, nous partons sur un sentier dans les bois, le long
d’une jolie rivière, et nous atterrissons sur un parc pour enfants désert
où nous pouvons même faire de la tyrolienne ! C’est un endroit agréable
pour passer le temps, et nous assistons aux exploits de petits Islandais
sur la tyrolienne. Puis nous repartons, bien synchronisés avec l’heure
d’ouverture du Bonus où nous refaisons des courses. Nous faisons encore
un énorme repas avec un pain de 700g, du fromage et du mills kaviar (une
pâte à tartiner à base de poisson) et pour finir du gâteau et du skyr,
l’alliance des deux étant très réussie. Puis nous prenons le bus (avec
plein de Français) pour le lac Myvatn. Ce coin est censé être très beau
mais infesté de moucherons. Nous traversons de grandes plaines rocheuses
et sableuses, mais en fait je n’en vois pas tellement que ça car je suis
prise d’une irrésistible envie de dormir ! Nous arrivons et nous
installons au camping local, il y a un peu de vent et quasiment pas de
mouches, ouf ! Puis nous cherchons une carte pour notre prochaine rando,
à Askja, mais nous n’en trouvons pas d’assez précise. Nous cherchons
aussi un moyen de nous rendre à Askja : il n’y a pas de ligne de bus mais
un voyage organisé y va : peut-être qu’on pourra s’arranger avec eux. On
prépare la journée suivante (autour de Myvatn) et nous assistons un peu
effarés à l’arrivée d’un groupe de Français très bruyants et sans-gêne (un
groupe de Nouvelles Frontières) qui envahit l’espace autour de notre
tente. Nous faisons un bon repas de… pâtes (étonnant !) à la sauce
basilic-tomates, mais ça ne suffit pas à Gaël qui se jette sur le pain !
Jusqu’où ira-t-il ?</p>
</div>
<div class="section" id="jeudi-8">
<h2>Jeudi 8</h2>
<p>Ce matin nous avons même du muesli chaud. Comme les Français restent
encore quelques jours au camping et que nous avons peur pour la tente
(ils shootent dans les tendeurs), on la transporte toute montée à un
autre endroit du camping (le groupe nous avait d’ailleurs proposé de le
faire, un peu à la méthode US on est nombreux on fait du bruit on
s’installe et au besoin on vous déplace, mais nous n’enfonçons pas nos
sardines au marteau !). Puis nous partons vers le volcan Krafla pour une
longue balade. C’est une très belle journée avec plein de soleil :
j’attrape même un coup de soleil ! Nous partons dans de jolies petites
collines vertes où nous croisons un troupeau de chevaux indolents, puis
au bout d’un certain temps on arrive dans le champ de lave d’une des
nombreuses éruptions du Krafla, la dernière ayant quand même duré de 1975
à 1984 ! Il y a différents types de lave : elle est parfois très très
noire et brillante, sans aucune trace de végétation ; à d’autres endroits
il y a quand même un peu de terre et de mousse. Le paysage est rigolo :
des collines qui ont été à moitié recouvertes par l’éruption dépassent
de la lave comme des périscopes verts. Nous déjeunons d’ailleurs au
sommet de l’une d’elles, cherchant le maximum de vent pour avoir le
minimum de mouches. Nous apercevons en contrebas dans le champ de lave
quelques stupides moutons qui s’y sont aventurés et bêlent lamentablement
jusqu’à trouver une touffe d’herbe qui leur fait oublier leur fâcheuse
posture. Puis nous continuons à sauter de bloc de lave en bloc de lave.
Ici la terre est craquelée, fissurée, torturée, la lave se tord en
grosses cordes à certains endroits, à d’autres ce sont des arêtes
coupantes qui se dressent brutalement en diagonale, dignes de <em>La
mer de glace</em> de Friedrich : le paysage est absolument lunaire. Nous
passons également par un cirque volcanique dont les couleurs - noire pour
la lave, rouge et jaune à cause du soufre - brillent au soleil. Nous
débouchons sur un parking et nous prenons la route qui mène à
Viti (l’enfer en islandais !), un cratère rempli d’eau turquoise
absolument splendide. La plupart des touristes s’y arrêtent 5 minutes en
car pour venir jeter un oeil dans le cratère. Ce serait vraiment dommage
de ne pas faire le tour : de l’autre côté du cratère il y a beaucoup
d’activité géothermique, en partie exploitée par une usine qui produit de
l’électricité, et il y a même des mini-piscines d’eau turquoise, des
mares de boue grise bouillante, et des vapeurs sulfurées qui donnent des
couleurs jaune citron, orange et bleu pastel aux roches environnantes.
Très chouette.</p>
<p>Le retour est un peu long (en tout ça fera une journée de plus de 30 km).
Nous passons à côté de la centrale électrique. Au sommet d’une colline,
nous surplombons une enfilade de poteaux haute tension qui se dressent
sur la terre noire : c’est assez impressionnant. On se paume un peu, puis
on retrouve le chemin : c’est l’heure de la pause Twix. Dopée au Twix, je
trotte de plus belle pendant un certain temps, puis nous rejoignons la
grande route. J’ai vu la veille des gens se baigner dans un lagon d’eau
turquoise et ça me fait très envie mais au bord c’est marqué “Hot spots -
Bathing forbidden” : je suis très déçue. Pourtant au syndicat
d’initiative on nous a dit qu’ils avaient aménagé des bains. Comme nous
voyons plein de voitures se diriger dans une direction nous les suivons,
en effet plus loin une pancarte annonce “Myvatn nature baths”. En y
arrivant on s’aperçoit que c’est payant : 1000 couronnes par personnes,
c’est exorbitant. Snif ! On y renonce. Sur la route du retour une
gentille dame et sa fille nous prennent en stop et nous disent qu’elles
se baignent dans le lagon “dangereux” : cette pancarte n’existe sûrement
que depuis l’apparition des bains payants, GRRRRRRRRRR. Elles nous
déposent au supermarché où nous faisons le plein de skyr et de pâtes pour
la prochaine rando. Au tourist information l’employé téléphone pour nous
à la compagnie de tours guidés qui peut nous emmener à Herðubreið et
négocie pour nous un prix tout à fait raisonnable : bonne nouvelle. Au
camping on se gave de chili beans et… de skyr bien sûr.</p>
</div>
<div class="section" id="vendredi-9-juillet">
<h2>Vendredi 9 juillet</h2>
<p>Nous nous réveillons tôt (5h30) pour aller prendre le bus à 8h. Nous
avons droit au début du tour guidé jusqu’à Herðubreið. Le gentil
guide Stefan Petrus a un grand sourire et parle français pas mal du tout.
Il fait beaucoup de blagues de gentil organisateur, mais au milieu
distille quand même quelques infos intéressantes sur les différents types
de lave, etc. Il nous montre une série de cairns marquant une route
ancestrale et nous fait remarquer qu’ils sont orientés et indiquent une
direction : en cas de brume (et il y en a dans ce pays) ça aide à trouver
le cairn suivant. La route de terre est très accidentée, le car roule
bien lentement. On fait un arrêt pour boire “la meilleure eau du monde”,
recommandée par le meilleur guide évidemment, dans une rivière, puis le
guide nous raconte l’histoire d’Evin le hors-la-loi “in the old days”. On
s’arrête à Herðubreið, la “montagne aux larges épaules”. Le groupe s’en
va, nous on reste là ! Les rangers nous disent que la hutte est quasi
pleine ce soir : eh bien on va camper, ça sera moins cher. Un peu
désoeuvrés, nous étudions les cartes en comptant les minutes avant
l’heure du déjeuner où nous nous jettons sur le pain (les levers à 5h30,
ça fait des matinées très longues pour l’estomac !). On fait une petite
sieste sur le matelas de la pièce commune. Puis on va faire une petite
promenade : nous commençons par visiter la cabane du pauvre Evin, le
fameux hors-la-loi qui s’était réfugié dans une cabane ici et affirmait y
avoir passé l’hiver le plus dur de sa vie. Il n’a pas dû en effet passer
un hiver folichon : la cabane est un minuscule trou moussu au milieu des
blocs de lave. Heureusement Evin avait réussi à se cacher pendant 20 ans
et à être pardonné selon la coutume islandaise qui voulait qu’un
hors-la-loi soit à la merci de n’importe qui : il pouvait être tué sans
que son meurtrier soit poursuivi ; mais au bout de 20 ans il était
amnistié. Puis nous suivons la rivière autour de laquelle il y a une
oasis de végétation au milieu du désert de lave. Nous faisons un détour
par l’aéroport de secours (on regarde bien à gauche puis à droite avant de
traverser la piste les enfants…). Nous retournons à la hutte où nous
déplorons le manque de lecture. Des méchants ont installé leurs affaires
sur nos matelas de sieste, donc je m’endors sur la table… cf. dessin de
Gaël. Puis c’est à mon tour de dessiner, pendant que rentre le groupe de
Hollandais qui occupe la hutte. Une dame très gentille, Saskja, qui parle fort
bien français (elle a été jeune fille au pair en France) et le cuistot
nous offrent du café et un verre de vin. Nous allons installer la tente
puis nous revenons cuisiner nos pâtes pesto en discutant avec le groupe,
surtout avec Saskja et avec Armand, un homme sympathique, la trentaine,
qui a travaillé en France : c’est bonne ambiance. Ils mangent beaucoup
plus que nous et quand je vois disparaître vers la cuisine un plat
quasiment fini mais pas tout à fait je ne peux m’empêcher de demander :
“Are you going to throw this away ? “. La nana rigole et nous laisse
finir le plat : j’espère ne pas l’avoir trop choquée, mais on a trop faim
! Puis skyr et gâteau et au dodo. Ce fut une journée calme.</p>
</div>
<div class="section" id="samedi-10">
<h2>Samedi 10</h2>
<p>C’est un réveil en douceur. On meurt de chaud dans la tente ce matin.
Nous partons vers 10 h après avoir discuté avec une famille de Français
qui a emmené son Espace avec le ferry. Le problème, c’est que la batterie
n’est pas très haute et qu’après le gué il s’échappait une fumée bien
blanche de leur voiture… On dit aussi au revoir aux gentils Hollandais.
Il fait un soleil de plomb et on doit transporter plein d’eau pour
économiser les réserves du refuge suivant, mais la ranger se radoucit en
voyant que nous ne sommes que deux, et pas un groupe qui va pomper toutes
les réserves (comme si on avait l’air de voyager en groupe !). Nous
commençons par nous rapprocher de la grosse montagne Herðubreið à
travers le champ de lave et de sable noir, comme ça nous pouvons admirer
de plus près ses fissures, ses névés et ses cascades, puis nous la
contournons en passant par un terrain pas désagréable pour marcher (un
mélange de sable et de terre où pousse une maigre végétation bien
courageuse). Nous faisons un sort au reste du gâteau pendant la pause
déjeuner : le prochain sera à Akureyri. Nous repartons sous le soleil,
atteignant bientôt un nouveau champ de lave. Au loin Kverkfjöll, la porte
du glacier, offre sa promesse glacée : le paradis bleuté. Hélas nous ne
nous y rendront pas. Nous nous dirigeons cependant vers d’étranges
formations rocheuses qui me plaisent beaucoup. Je fatigue un peu : les sacs
sont lourds et le mien frotte dans mon dos car mon T-shirt est trop court
(classique chez moi…). C’est très étrange de marcher dans le désert de
lave : le silence y règne comme dans une espèce de Far West, tout est
noir sauf quelques pousses en fleurs, et nous sommes entourés de
montagnes enneigées. Nous apercevons la hutte dans le creux d’une
montagnette bien rocailleuse : youpi ! Et on s’installe tranquillement.
Le guestbook remonte à 1978, ça fait beaucoup d’histoires à lire ! Je le
passe à Gaël qui y déniche le passage de Virginie et Fabrice Orgogozo (le
frère de notre ami Benjamin et sa femme) ! Quant à moi je lis la Holy
Bible : ça fait du bien d’avoir de la lecture. Nous explorons un peu le
coin, puis nous faisons un gros dîner grâce à des pâtes au parmesan
laissées par un généreux donateur, ainsi que notre semoule, de la soupe
et du chocolat. C’était une belle journée, un peu fatigante, demain c’est
encore du champ de lave et j’espère que ce ne sera pas trop crevant : on
moins on aura moins de nourriture à porter ! Il n’y a pas d’eau au
refuge (c’est le désert), à part l’eau de pluie collectée dans des bidons
qu’il faut économiser : je suis toute sale et j’ai sur le visage une
croûte de crème solaire et de sable du plus bel effet. Non Gaël, pas de
photo !</p>
</div>
<div class="section" id="dimanche-11">
<h2>Dimanche 11</h2>
<p>Au réveil il y a plein de brouillard : zut. Je mange mon muesli chaud
dans les restes de pâtes et de soupe (eh oui, pas de vaisselle pour
économiser l’eau !) et je prie très fort le dieu des lentilles pour ne
pas attraper de conjonctivite. Quand on part le brouillard se dissipe peu
à peu et il fait vite même assez beau. Le haut du Herdurbreid reste
cependant pris dans un capuchon de nuages. On marche d’abord un peu dans
la lave, puis à flanc de colline (ce qui permet d’éviter d’escalader les
blocs de lave). Nous croisons des rochers aux formes acérées, des
paysages de canyon gris et ocres. De jolis cirques rocheux par-ci par-là.
La montagne à notre droite, la lave à notre gauche, nous marchons dans du
sable et des caillasses : à un moment tout devient gris. Que du gris. Je
ne me sens pas très bien : j’ai l’impression que chaque pas résonne ma
gorge alors que mes pieds n’amortissent rien. Très désagréable. Nous
profitons de la pause pique-nique pour assassiner un demi pain de mie, le
fromage qui va avec et un sachet d’abricots puis on repart. La marche
est toujours pénible pour moi, mais pour Gaël ça va mieux qu’hier (il
avait mal au dos). A un moment on quitte ce flanc de collines grises pour
s’enfoncer dans le champ de lave ; là il faut sauter par-dessus un petit
bloc de temps en temps mais la progression est en fait assez facile et
surtout il y a des couleurs ! Noir de la lave, vert de l’herbe, et plein
de pierre ponce rouge, noire ou dorée. J’aime beaucoup la pierre ponce. Et ô
miracle les couleurs me redonnent la pêche : je me sens mieux et je
gambade comme un cabri. Après un petit mont tout noir et rouge nous
rejoignons la route de jeep qui mène à Dreki (la hutte d’Askja). Une
pause barre de muesli s’impose. Nous aimerions faire du stop sur cette
route, malheureusement il n’y a pas de voitures donc nous finissons à
pied. Mauvaise surprise : il faut traverser une rivière avant d’arriver.
La lecture de la carte m’y avait psychologiquement préparée mais Gaël
n’apprécie pas. Et en plus la carte est fausse il y a une deuxième
rivière à traverser. Heureusement nous étions restés en Teva. Nous
arrivons à la hutte qui est pleine : nous allons être obligés de camper,
ce qui en temps normal ne nous dérangerait pas mais le sol est très dur,
sans le moindre brin d’herbe : que de la pierre ponce qui risque d’abîmer
la tente. Nous la montons dans le vent en enfonçant les sardines qui
percent certaines pierres ponces puis on va s’abriter dans la hutte. Peu
après arrive un groupe de Français qui fait un voyage thématique sur les
volcans (c’est un bon endroit pour ça !), accompagnés par leur très
sympathique guide-photographe-géologue. On discute un peu, puis je les
regarde avec envie préparer la cuisine : nous avons bien essayé de
tromper notre estomac avec un bouillon cube, mais nous avons très faim. Ils
préparent notamment un apéritif de fromage roulé dans du bacon, le tout
grillé, qui me met l’eau à la bouche. Ils m’en proposent un, je ne dis
pas non, et comme peu de personnes se reservent j’ai droit à un deuxième
et même un troisième ! Sur ce Arnaud rentre (il était en train de
s’occuper du barbecue) et nous invite carrément à dîner : c’est la fête !
La première étape du dîner est quasi initiatique pour un touriste en
Islande : du requin pourri à l’arrière-goût très fort, qu’on fait passer
avec une bonne gorgée de schnaps. Le chauffeur très jovial du groupe
(brosse blonde, moustache et grand sourire) menace de ne pas ramener les
membres du groupe s’ils ne mangent pas. C’est moins dégueu que ce que je
pensais et le schnaps n’est pas mauvais. La suite du festin : salade au
thon, gigot et pommes de terre. Le groupe est aux petits soins pour nous,
ils sont vraiment tous très gentils. Une DRH de la Poste nous donne
assiettes et couverts, un ingénieur rigolo nous fait passer la crème et
le bleu. On a même du vin. Puis le dessert de surmjölk (une sorte de lait
caillé) à la confiture de myrtilles. Arnaud est très intéressant, il nous
parle de ses voyages, des volcans en Sicile, aux USA, au Japon, de
comment gérer un groupe de touristes. Il nous laisse sa carte : j’espère
qu’on le reverra. Nous allons nous coucher dans la tente en essayant
d’être le plus léger possible pour ne pas l’abîmer. C’était super cool ce
repas ; d’un côté on l’avait un peu cherché en essayant quelques
allusions discrètes du genre “on a 8 jours d’autonomie donc on a pas beaucoup
à manger” etc., pas très fin mais trouver de la nourriture qu’on ne porte
pas est devenu notre nouveau sport qui occupe le temps en dehors des
randos (ce n’est d’ailleurs pas facile car aujourd’hui les gens imaginent
difficilement que deux jeunes gens bien portants et relativement aisés
aient envie qu’on leur donne de la nourriture). D’un autre côté nous
n’avions fait aucun appel du pied de ce type à Arnaud qui a proposé de
lui-même de nous inviter. Et vive la tradition médiévale de raconter ses
aventures en échange d’un repas !</p>
<p>Si certains trouvent que je parle plus de bouffe que de paysage dans ce
journal, c’est normal : on a tout le temps faim, et pendant que j’admire
les paysages en marchant, j’ai intérieurement des visions de cassoulet,
de côtes d’agneau… pas incompatible avec l’harmonie d’un paysage !</p>
</div>
<div class="section" id="lundi-12-juillet">
<h2>Lundi 12 juillet</h2>
<p>Il fait très chaud au réveil. Nous replions la tente car la nuit
prochaine nous dormons dans la hutte pour l’économiser. Nous nous
sommes réveillés bien tard, car c’est notre journée off sans sacs pour
visiter Askja et sa caldeira. Nous prenons congé du groupe et du bel
Arnaud, tous toujours aussi sympas avant de se faire un bon petit
déjeuner dans la hutte. Puis on part en montant vers la caldeira d’Askja,
un volcan actif qui pourrait bien détruire la hutte un de ces jours… Le
chemin offre des paysages lunaires, chevauchant une arête qui surplombe
des cirques tout à fait lunaires (Apollo 13 s’est entraîné à cet endroit
!), au milieu de montagnes de toutes les couleurs, les rouges et noires
étant celles que je préfère. Arrivés en haut, je découvre la caldeira
avec émerveillement : un grand lac d’un bleu profond scintille, entouré
de pentes enneigés ou colorées. C’est magnifique sous ce soleil radieux.
Nous faisons un rapide pique-nique : aujourd’hui nous faisons peu
d’effort donc nous nous limitons en nourriture. Puis nous descendons au
fond de la caldeira pour atteindre un petit cratère secondaire : c’est
Viti, l’enfer. Le cratère aux fumerolles soufrées abrite un lagon
turquoise au milieu de ses pentes abruptes, de ces roches ocres, jaunes
ou vertes. On peut s’y baigner : l’eau est chaude grâce à l’activité
géothermique. La tradition est de se baigner tout nu mais nous avions
prévu les maillots. Ce n’est pas si chaud que ça (22 °) à cause de la
fonte des nieges, mais moi je trouve ça très agréable… contrairement à
Gaël qui râle comme un pou <em>“j’aime pâs l’eau froide !”</em>. Ah la la c’est
pourtant pas souvent qu’on peut se baigner dans un lagon d’eau turquoise
chauffé par un volcan au fond d’un cratère.</p>
<p>A certains endroits le sol est hyper chaud et on se brûle les pieds !
Quand on sort de la flotte par contre ça fait froid ! Nous remettons
vite nos fringues qui garderont une bonne odeur bien soufrée jusqu’au
prochain lavage. Pour me réchauffer je vais admirer les fumées soufrées
jaunes, grises et vertes.</p>
<p>Nous remontons la pente bien raide puis nous allons au parking où nous
faisons du stop pour éviter 8 km de marche jusqu’à la hutte. Nous sommes
assez vite pris par une énorme Defender que le père conduit avec un tout
petit gamin sur les genoux qui tient le volant, un autre gamin à côté et
sa femme étendue sur un matelas dans le fond. Ce sont de sympthiques
Danois qui ont beaucoup aménagé leur voiture pour voyager. La nuit, les enfant
dorment dedans ! Les passages de gués sont plus rigolos en voiture qu’à
pied. De retour à la hutte, nous faisons une lessive salutaire de nos
pantalons, puis comme le groupe annoncé ne vient pas (zut, pas de
nourriture supplémentaire ! Par contre on va être tranquille.), nous
faisons notre cuisine en compagnie d’un couple d’universitaires islandais
gentils et intéressants. Ils vont même chercher un annuaire dans leur
voiture pour nous trouver l’adresse du Bonus d’Akureyri ! Puis dodo bien
au chaud.</p>
</div>
<div class="section" id="mardi-13-juillet">
<h2>Mardi 13 juillet</h2>
<p><strong>C’est Gaël qui raconte</strong></p>
<blockquote class="epigraph">
Aujourd’hui c’est notre plus longue étape : 10 h dans le topo !
Heureusement la gardienne de Dreki nous amène en voiture jusqu’à la
caldeira : 8 km et 500 m de montée. Le chemin n’est pas indiqué et il
faut traverser un champ de lave et longer les montagnes qui bordent la
caldeira jusqu’à une petite passe. A partir de là le chemin sera indiqué.
Ma carte est très imprécise, et la navigation avec elle n’est pas aisée :
lorsque je prendre un relèvement sur l’île du lac de la caldeira je
m’aperçois qu’elle n’est même pas sur la carte. Ce n’est pas grave, je
pense déjà avoir repéré le col la veille. En fait je vais beaucoup trop loin
dans la caldeira. Nous montons là où j’imagine que se trouve la passe
mais aucun piquet ! Impossible de savoir si nous sommes trop à gauche où
trop à droite. Je décide de franchir le col pour essayer d’apercevoir des
piquets de l’autre côté… mais de l’autre côté il y a une longue pente
noire qui se perd dans la brume sans aucun signe distinctif. De plus le
compas fait des choses bizarres ; Arnaud m’avait bien parlé des anomalies
magnétiques près des volcans. Après un petit moment de découragement nous
décidons de descendre sans chemin : ce sera plus long et fatigant mais
nous trouverons la hutte. Nous perdons 400 m dans un interminable
pierrier de pierres ponces bien tranchantes et finalement j’y vois
suffisamment clair dans le paysage pour nous situer sur la carte. 100 m
sous nous passe une piste de jeep que nous pouvons longer pendant une
dizaine de kilomètres pout arriver à la hutte. Heureusement la vallée est
superbe : canyons et châteaux de pierre de toutes les couleurs. Une pause
figue et c’est reparti au trot. Mon dos me fait mal donc je trace. 2h
plus tard nous voici à la hutte où une vieille Land Rover est garée. Ce
sont deux Suisses Allemands, un père et son fils. Ils sont venus en
Islande pour pêcher. Nous avalons un véritable festin (400g de pâtes,
100g de poisson séché, 50 g de chocolat et un instant pudding !). Puis le
père nous offre du whisky et nous parlons de pêche au saumon dans les
rivières d’Islande.</blockquote>
<p><strong>Le petit mot d’Emmanuelle</strong></p>
<p>C’est la warden pas très fut fut mais très gentille qui nous donne un
lift bien utile. Nous démarrons dans le champ de lave non marqué. Sous
nos pieds, de très belles pierres de toutes les couleurs, noires aux
reflets métalliques et dorés : on en prend quelques échantillons. Après
c’est comme Gaël le raconte : on s’est paumés mais on a atterri dans une
vallée vraiment superbe, façon canyon du bout du monde ou expédition sur
Mars. Le paysage s’étendait à perte de vue, mais il était varié… et il
n’y avait personne ! Les Suisses du refuge étaient sympas (absolument
passionnés de pêche, il fallait les entendre parler du combat avec le
saumon !), mais il a fallu que j’attende de lire leur provenance sur le
guest book pour comprendre qu’ils parlaient allemand ! Après j’ai
commencé à comprendre des bouts et ça allait mieux. Et qu’est-ce que
l’instant pudding à la vanille était bon !</p>
</div>
<div class="section" id="mercredi-14-juillet">
<h2>Mercredi 14 juillet</h2>
<p><strong>Encore Gaël</strong></p>
<blockquote class="epigraph">
22 km de piste de jeep, dans un désert plutôt plat et barbant. Nous
partons sous la pluie à une bonne allure. Emmanuelle n’est pas en très
bonne forme et mon dos me fait mal, raison de plus pour aller vite. Nous
croisons trois voitures dans la journée. La route, d’abord droite dans
des étendues plates déprimantes de sable gris finit par entrer dans un
champ de lave pahoehoe. Finalement l’embranchement pour Botni : encore 8
km. La piste sinue comme c’est pas possible dans le champ de lave
pahoehoe, en bordure d’une étendue de lave aa. Enfin j’aperçois la hutte
et je fonce tout droit jusqu’à elle. Nous avons mis 5 h ! Bonne moyenne.
Comme quoi le mal au dos ça fait avancer.</blockquote>
<p><strong>Le mot d’Emmanuelle</strong></p>
<p>On a marché à fond la caisse ! Normal, il pleuvait et la route de
4x4 n’est pas passionnante : du gris à perte de vue. Ca allait
mieux quand nous avons obliqué dans le champ de lave : le paysage était
plus varié. Et en arrivant à la hutte, j’ai trouvé un bouquin ! C’est
<em>The great Gasby</em> de Fitzgerald : très agréable de bouquiner après
une journée de marche. D’ailleurs quand je l’ai fini Gaël me l’a piqué et
a lu jusqu’à minuit !</p>
</div>
<div class="section" id="jeudi-15-juillet-la-sortie-le-marathon">
<h2>Jeudi 15 juillet : la sortie - le marathon</h2>
<p>Il fait beau quand on se réveille, c’est agréable. Au programme : 15-16
km faciles jusqu’à une ferme qui marque la fin de la rando “officielle”
mais qui débouchent sur une route où on se demande quel est le trafic
pour faire du stop, jusqu’à Akureyri si possible.</p>
<p>En fait le début n’est même pas si facile que ça car le chemin est très
mal marqué et on se paume en allant beaucoup trop à l’est. On croit suivre le
bon chemin qui n’est en fait qu’une route de 4x4 donc on
l’abandonne pour piquer sur une grosse ferme qu’on aperçoit au loin. On
quitte le champ de lave pour des collines et des prairies verdoyantes ; à
ce moment les rivières qui avaient coulé sous la lave jaillissent en
sources claires et abondantes : très agréable pour se désaltérer. La
progression dans la prairie est facile mais c’est long car nous avons
fait un gros détour en prenant le mauvais chemin. On arrive enfin à la
ferme, une grosse exploitation au bord d’un petit lac. De nombreuses
voitures garées nous font espérer d’être pris en stop. On pique-nique,
avec filets sur la tête en raison des mouches, et on repart en pensant au
Bonus d’Akureyri. Il n’y a personne sur la route et nous sommes chargés.
A chaque fois que nous dépassons une ferme ça nous donne une chance
supplémentaire d’être pris en stop mais ça n’arrive que tous les 5
kilomètres ! Gaël a mal au dos et au tendon d’Achille : c’est très
pénible pour lui de marcher et nous faisons de fréquentes pauses. On se
fixe un objectif : un pont qui permet de rejoindre une route un peu plus
fréquentée, semble-t-il. Deux voitures peu chargées nous dépassent en
ignorant notre pouce à cran d’arrêt. Enfin quelqu’un nous
prend : il ne va pas très loin mais nous dépose quand même de l’autre
côté du pont. Il travaille à une guesthouse qui fait aussi camping et
nous envisageons de dormir dans son camping si personne ne nous prend en
stop. Mais à partir de là les choses s’accélèrent. Un couple de Français
dans une minuscule 4x4 rouge s’arrête, contemple nos sacs d’un air
dubitatif : “Vous croyez que ça va rentrer ?”. Oui, on a très envie d’y
croire ! Et on entasse dans leur coffre leurs bagages, nos sacs… et
puis nous ! Nous sommes très comprimés, je ne vois pas Gaël qui est caché
derrière son sac mais nous sommes super contents car comme ça nous allons
pouvoir aller jusqu’à la route numéro 1 ! Ils nous racontent leurs
vacances et en retour je leur fais le récit de nos aventures. Ils sont
très sympas : ça fait plaisir de rencontrer des Français comme ça à
l’étranger. Ils nous déposent à Godafoss, site avec de très belles
cascades à étages que nous admirons avant de nous remettre sur le bord de
la route. A priori nous avons moins de chance d’être pris sur la route
numéro 1 car il y a un bus - en effet beaucoup de voitures nous dépassent
-, mais soudain deux filles islandaises s’arrêtent et youpi ! elles vont
jusqu’à Akureyri. La voiture est petite, on s’y entasse comme on peut et
je m’abandonne à un doux sentiment d’euphorie pendant qu’ABBA sur une
vieille cassette pourrie résonne comme un chant de victoire. En arrivant
par la route nous voyons apparaître le fjord où se niche Akureyri, la
deuxième ville du pays (avec 20000 habitants !). Les filles nous déposent
au camping, on s’installe, assez fatigués, et puis on mange ! Le Bonus
doit être fermé à cette heure mais après une bonne quantité de semoule je
vais acheter du Skyr pour Gaël dans un autre supermarché.</p>
<p>Sur la route au beau milieu de nulle part, je ne pensais pas qu’on serait
à Akureyri ce soir. En tout cas, on a marché plus de 40 bornes
aujourd’hui !</p>
</div>
<div class="section" id="vendredi-16-juillet-journee-de-repos-a-akureyri">
<h2>Vendredi 16 juillet : journée de repos à Akureyri</h2>
<p>Nous nous réveillons assez tard ; au petit déjeuner nous avons même du
lait frais. J’entreprends un shampoing assez acrobatique dans le lavabo,
et une lessive, puis on part visiter Akureyri. Nous passons d’abord à
l’office du tourisme où nous achetons la carte de Kjölur, la prochaine
région où nous voulons aller. On a de la chance : ils ont une
relativement bonne carte. L’office du tourisme est grand, bien chaud,
avec des chaises confortables : je me promets d’y revenir écrire des
cartes. Mais une fois nos informations glanées, voici l’objectif numéro 1
de la journée : le BONUS !</p>
<p>En chemin nous visitons la boutique de 66 North, la marque islandaise de
vêtements techniques. Il n’y a rien de très original, mais c’est fait pour
un temps rude ! Au Bonus on fait une razzia de bonnes choses : 1.5 kg de
skyr (une bagatelle…), deux gâteaux et surtout, 900 g de saumon mariné
à l’aneth qu’on peut manger cru ! Vivement le festin du soir ! En
attendant on rentre au camping et on se fait un énorme déjeuner avec
notamment du beurre : que c’est bon le beurre ! Je pense à l’avance aux
immenses aux immenses tartines beurrées que je vais avaler en France. On
essaye aussi un fromage au poivre pas mauvais. Après vient la difficile
tâche de transférer le lait en poudre dans sa bouteille. C’est du lait de
bébé car ici ils n’ont pas de lait normal en poudre. C’est assez compliqué
de verser et Gaël et moi nous énervons. Je vais écrire mes cartes à
l’office du tourisme et j’en profite pour aller sur Internet pour
regarder les résultats de Marc-Antoine. Gaël me rejoint peu après pour
une balade dans la ville : d’abord la jetée avec quelques pubs à l’air
sympa, la cathédrale et le centre-ville puis des quartiers résidentiels
avec de belles maisons. Dans un petit jardin public où poussent des
arbres, une plaque explique que c’est ici que les habitants d’Akureyri
ont remarqué que les arbres, ces choses bizarres qui résistent mal au
climat islandais, pouvaient être beaux… On rentre au camping et le
festin commence. Le saumon est une révélation : quelle joie de mordre à
pleine dents dans cette chair tendre et savoureuse ! Avec les herbes de
la marinade, c’est sublime. On ne finit pas tout à fait les 900g, on en
garde pour le petit déj comme ça nous avons encore faim pour du skyr et
du gâteau. On a beaucoup mangé !</p>
<p>Akureyri est de loin la plus jolie ville islandaise que nous ayons
visitée.</p>
</div>
<div class="section" id="samedi-17-juillet">
<h2>Samedi 17 juillet</h2>
<p>C’est un petit déjeuner de fête : le reste du saumon, puis du bon muesli
au sésame. Nous allons ensuite prendre le bus qui fait Akureyri -
Reykjavik par Kjölur, une des rares routes intérieures du pays. On passe
par plein de jolies vallées (à moutons, bien sûr), et au bout de
quelques heures nous arrivons à Hveravellir où nous allons passer deux
jours. J’ai dormi une bonne partie du trajet ! On s’installe puis on part
en balade sans sacs à dos. Nous passons d’abord à côté d’un bain chaud
naturel amménagé en nous promettant d’y revenir puis nous suivons un
mini-sentier qui fait le tour des merveilles géothermiques de
Hveravellir. Très chouette : tout est concentré au même endroit, et on
passe d’une source bouillante à un gros cône qui siffle en émetant un
panache de fumée (je le baptise immédiatement la cocotte-minute), le tout
au milieu de couleurs pastel acidulées à cause du soufre. On prend plein
de photos puis on arrive à une petite bergerie typique au toit en herbe
absolument charmante. J’y prends la photo “officielle” de Gaël
l’aventurier. Il y a un sentier qui semble continuer là où on veut aller
mais le film touche à sa fin alors nous rentrons à la tente prendre une
pellicule avant de partir par l’autre bout de la boucle que nous voulons
faire. Après quelques kilomètres près de la grande route, peu
intéressants mais où nous croisons tout de même des traces absolument
gigantesques de pattes d’oiseau (une quinzaine de cm, nous n’avons pas
rencontré la bête mais nous n’en mourrions pas d’envie), nous piquons sur
le côté pour suivre une piste de cheval au bord d’une colline. Il fait
très beau, l’éclairage est très joli, au loin nous voyons de superbes
calottes glaciaires et les pics des Kerlingarfjöll, et c’est très
agréable de marcher dans l’herbe. Une fois la colline dépassée, nous
atterrissons dans le champ de lave où le sentier est marqué par
d’immenses cairns très verticaux, dont l’assemblage de pierres tient on
ne sait trop comment : de vrais ouvrages de maçonnerie. Ces silhouettes
fantômatiques rythment la lande de lave. La carte indique une grotte,
nous la cherchons quelque temps mais en vain. Nous nous dirigeons alors
vers le sommet d’un grand plateau qui monte en pente douce. Il est 7h et
je commence à être fatiguée, mais la lumière est très belle : d’habitude
nous nous promenons plus tôt et nous ne pouvons pas l’apprécier. Comme
tout est très plat il est assez difficile de voir exactement où est le
sommet mais on finit par y arriver. Il correspond au cratère d’un volcan.
Gaël part faire le tour de la caldeira et explorer tous les petits
cratères, moi je commence par me reposer avant d’y jeter aussi un
coup d’oeil. Quand Gaël revient, il est tout content d’avoir vu un
immense faucon, une espèce en voie de disparition d’après des fans
d’ornithologie, qui pique sur les touristes car il a peur pour ses
petits. Nous redescendons alors sur le plateau vers le camping, sans
difficultés, et nous allons cuisiner nos pâtes dans la hutte, agrémentées
ce soir-là de skyr et d’un délicieux gâteau pain d’épices du Bonus, le
“krydbraud”. Quand nous sortons de la hutte, nous pouvons voir notre
premier coucher de soleil du mois. Tous les nuages sont roses, c’est très
beau. Je vais me coucher pendant que Gaël escalade courageusement la
colline voisine pour prendre des photos.</p>
<p>Et encore une journée où on a marché plus de 25 km (nous sommes arrivés
vers 22h au camping) !</p>
</div>
<div class="section" id="dimanche-18">
<h2>Dimanche 18</h2>
<p>Pas de réveil ce matin : c’est les vacances. On prend notre petit déj
avant d’aller tester le bain chaud. Il y a du soleil mais un peu de vent
et psychologiquement ce n’est pas évident de se déshabiller ! Le bain est
divinement agréable : on peut choisir sa température en se rapprochant de
l’arrivée d’eau chaude ou d’eau froide, et même choisir la température
globale en déplaçant le tuyau d’eau chaude pour contrôler la quantité qui
rentre dans le bain. Des pierres permettent d’être confortablement assis
dans l’eau mais au bout d’un moment je m’assieds sur le bord en laissant
tremper mes jambes pour refroidir un peu. Comme d’habitude, on rencontre
des Français, dont une famille assez gratinée : le père avec un look
Johnny cheveux décolorés, boucle d’oreille, la mère à l’accent bourge pas
croyable, la grand-mère qui ronchonne et last but not least la fille, une
grande blonde qui déclare qu’avant de partir elle jettera son maillot de
bain et son jeans car ils sentent vraiment trop le soufre…</p>
<p>C’est très agréable quand on sort : on n’a pas froid ! On pique-nique,
puis on part se promener en direction d’une petite hutte de montagne.
Nous suivons une route de jeep, mais le paysage est très joli. Comme
d’habitude, il y a un soleil radieux. La marche est facile, nous nous
dirigeons vers une jolie vallée aux couleurs vertes et ocres. Au bout de
2h30 nous arrivons à la hutte, qui est vraiment idéalement située : dans
le creux d’une charmantes vallée aux diverses nuances de verts, le long
d’une rivière argentée qui serpente dans le creux de la vallée, elle est
surplombée par plusieurs glaciers qui se terminent en d’innombrables
crevasses. L’endroit me plait beaucoup : si je reviens en Islande j’y
passerai bien une nuit. Il y a même quelques moutons autour. Nous voyons
arriver la fameuse famille de Français qui va y passer une nuit. Ils sont
affolés de voir qu’il n’y a pas de robinet dans la hutte ; Gaël leur
explique que l’eau du ruisseau convient très bien… On tout cas ils ont
bien raison de venir ici : c’est un truc sympa à faire en famille. Nous
grimpons le sommet voisin pour admirer le paysage, en particulier les
glaciers avant de rebrousser chemin. Et encore une journée à 25 km ! Nous
arrivons vers 21h au camping, nous allons dans la hutte pour faire nos
pâtes et là nous tombons sur un groupe de Canadiens tellement bruyants
qu’on les prend d’abord pour des Américains, sans compter un plus petit
groupe d’Italiens naturellement haut en couleurs : ça fait pas mal
d’animation dans la cuisine. Certains Canadiens font semblant (ou pas)
d’être bourrés pour mieux draguer, pendant ce temps nous engloutissons
nos 400g de pâtes suivis d’instant pudding… Nous remettons au lendemain
matin notre projet de bain nocturne en raison de la horde bruyante qui
s’y précipite. On a bien fait, car il paraît qu’ils ont laissé le bain
jonché de clopes et de canettes : pas cool.</p>
</div>
<div class="section" id="lundi-19-juillet">
<h2>Lundi 19 juillet</h2>
<p>Le matin, un stupide mouton bêle lamentablement et nous réveille. Mais on
se rendort. Après le petit déj’, bain en compagnie d’une joyeuse bande
d’Italiens. Aujourd’hui nous voulons aller de Hveravellir à
Kerlingarfjöll : on fait les sacs et on va faire du stop mais les travaux
de couture de Gaël et le long bain nous ont retardés donc nous n’avons
pas beaucoup de temps avant le bus. Comme le stop ne marche pas, on prend le
bus qui n’est d’ailleurs pas très cher. Le bus nous dépose à la jonction
avec une route menant aux Kerlingarfjöll, que nous prenons joyeusement.
Un gué est marqué sur la carte et il n’a pas l’air gentil, mais un
cyclo-touriste nous rassure : il n’y a qu’un seul gué et il est facile.
Le paysage est varié, au fond le massif très découpé des Kerlingarfjöll
avec une montagne en forme de Toblerone. La marche est facile et on prend
notre temps. Nous traversons un petit gué facile, comme ça on peut dire
que nous sommes de vrais aventuriers, et en fait le gros gué marqué sur
la carte est maintenant un pont, au-dessus de jolies gorges d’une rivière
puissante. Sur le dernier kilomètre un homme nous prend en stop dans sa
4x4, il le propose d’ailleurs de lui-même : il fait partie du
personnel des huttes et du camping. Nous nous installons au camping et je
vais me reposer dans la tente bien chaude en attendant l’heure du hot
bath : de 18 h à 21 h, ils ouvrent un bâtiment avec des douches bien
agréables, et après on plonge dans un bain chaud, des sortes de cuves où
on se sent merveilleusement bien. Dans les vestiaires je rencontre la
grand-mère française déjà mentionnée : elle me dit que tout le monde est
nu dans le hot pot. Je ne sais pas si elle a voulu me faire un canular ou
si elle voulait juste se changer en paix, mais en tout cas j’ai bien fait
de mettre mon maillot de bain ! Un groupe d’Allemands est en train de
boire de petits verres du schnaps local (celui qu’on boit après le requin
!), ils sont gentiment pompettes et c’est très drôle de comprendre leurs
blagues sans qu’ils s’en rendent compte. Ils partent et Gaël et moi
profitons alors de notre bain privé, maintenu à température idéale par le
patron qui est aux petits soins. Les gens étaient d’ailleurs tous très
gentils ici ; je conseille l’endroit à tous, le camping restant à prix
raisonnable et pour les plus fortunés il y avait un restaurant et de
jolies huttes…</p>
</div>
Balades dans les crevasses2003-05-12T00:00:00+02:002003-05-12T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2003-05-12:/personnal/balades-dans-les-crevasses.html<p>Cool week end, malgré le mauvais temps. Pour une fois j’étais
ailleurs que dans Fjordland. J’essaie de me tirer de Fjordland pour voir
le reste de cette fabuleuse ile (c’est dûr). Je crois que je ne mettrais
même pas le pied dans la deuxième région classée patrimoine …</p><p>Cool week end, malgré le mauvais temps. Pour une fois j’étais
ailleurs que dans Fjordland. J’essaie de me tirer de Fjordland pour voir
le reste de cette fabuleuse ile (c’est dûr). Je crois que je ne mettrais
même pas le pied dans la deuxième région classée patrimoine mondial, elle
est un peu loin.</p>
<p>En fait le plan était d’aller un peu au nord, dans la chaine principale
des “Southern Alps”, et de gravir une chouette montagne pour redescendre
dans une vallée trés sauvage. La météo était totalement foireuse, donc
nous nous disions que nous ne pourrions jamais atteindre l’objectif
principal, mais un petit groupe de participants ne voulais absolument pas
passer la nuit en ville, donc quatre des huit participants d’origine
sont partis en voiture, et non dans un horrible minibus. Il y avait Lara,
une Kiwi avec une bonne expérience alpine, Meg, une Américaine de Chicago
qui avait fait “Turret Ridge” avec moi, et Brian, un américain dégourdit
en Nouvelle Zélande déjà depuis deux mois, et puis moi.</p>
<p>Nous sommes partis en retard, avec un coffre bien plein de vêtements
cahuds et imperméables. J’ai somnolé jusqu’à ce que nous arrivions à la
base de notre objectif, vers une heure du matin. C’était au milieu de nul
part, sur une route qui traversait les montagnes par un col assez bas.
Soudain Lara fait signe de s’arrêter sur une aire de pic-nic : nous
prévoyons d’établir le camp ici car il y a une zone plate. Au moment de
sortir de la voiture nous nous prenons conscience de la pluie
torrentielle qui s’abat sur la voiture. Il faut se secouer pour saisir
dans le coffre une deuxième polaire, un imperméable et une lampe torche,
puis la “tent-fly” et les piolets, qui font d’excellents pics pour la
tente. Devant la pluie Brian sort sa propre tente et je me réjouit
d’avoir pris mon sac de bivouac. Quand j’arrive à l’endroit où Meg et
Lara essaient de tendre la fly-tent le sol est couvert d’une fine couche
d’eau. C’est du gravier dans lequel j’ai du mal même les piolets. Je
décrète sur le champ que si personne n’y voit d’inconvénients je dormirai
dans la voiture. Au bout de quelques minutes de bataille avec la fly-tent
qui forme une poche pleine d’eau Lara suggère que nous allions nous
abriter dans l’aire de pic-nic dix kilomètres plus loin. Nous y trouvons
deux tables abritées par de petits toits, malheureusement la pluie est si
forte que le sol un béton et les bancs sont trempés, le seul endroit sec
est le dessus des tables. Brian et moi nous installons dans la voiture,
laissant les deux filles sur les tables. La voiture de Brian n’est pas
étanche et goutte sur lui. Pour ma part je passe une assez bonne nuit.
Quand je me reveille le matin il pleut toujours. Les vitres de la voiture
sont couvertes de condensation et je peux voir que Brian se tient debout
sous un des abris. Je m’habille lentement, enfilant un pantalon étanche
et mes guêtres avant de sortir. Nous petit-déjeunons sous un abris puis
préparons nos sac avant de nous rendre au pied du sentier qui part vers
le refuge où nous comptons passer la nuit. Nous devons traverser un petit
torrent, mais quand nous arrivons sur ses berges c’est une tumultueuse
rivière qui nous attend. Le courant est plus rapide que nous ne pouvons
marcher, et nous ne sommes pas certain de la profondeur au milieu. Nous
passons un bon bout de temps à hésiter : nous pouvons probablement tenter
la traversée, mais en avons nous réellement envie ? Brian suggère que
nous allions aux grands glaciers (le Fox Glacier et le Franz-Jozef
Glacier), car de toute façon dans les conditions que nous avons nous
n’irons pas jusqu’au sommet, et nous voilà repartis sur la route, la
voiture gouttant à travers une superbe vallée qui s’élargit jusqu’à
devenir un estuaire gigantesque flanqué de montagnes aux pentes raides
qui disparaissent dans les nuages. Il y a deux heures de route pour aller
jusqu’aux glaciers mais quand nous arrivons il pleut toujours. Nous
n’avons bien sûr pas la carte de la région mais Lara est déjà allée au
refuge juste sur le coté du Franz-Jozef, “Castle Rock”. Nous déjeunons
planqués dans la voiture puis marchons quelques kilomètres jusqu’au bas
du glacier.</p>
<p>C’est une masse impressionnante de glace qui part des névés qui bordent
le Mt Cook et descend jusqu’à la côte Ouest, où nous nous tenons en ce
moment. Des palmiers de la “rain-forest” bordent presque le bas du
glacier ! Malgré la pluie diluviène des groupes de touristes accompagnés
par des guides montent et descendent sur le glacier. Ils sont équipés de
petits crampons ridicules dont le dessous ressemble à une sci à métaux !
A l’avant du groupe un guide manie une gigantesque pioche pour tailler
des marches dans la glace. Tous les jours ils empruntent le même chemin
qui sinue entre les crevasses ou les traverse à l’aide de pont et
d’échelle. Nous déballons nos gros crampons, nos casques et nos piolets ;
le tout assorti avec nos gros sac à dos nous vaut des regards admirateurs
des touristes et amusés des guides. La montée le long des chemins taillés
par les guides est une vraie promenade de santé. Il pleut des cordes et
l’eau ruisselle le long de la glace, formant de petites rivières et
disparaissant dans des trous dans la glace. Nous sortons des chemins
battus à la hauteur d’un éboulis que nous gravissons jusqu’au refuge. La
montée est très jolie, avec le glacier en dessous, une immense chute
d’eau sur la gauche et la vallée disparaissant dans le brouillard sur la
droite. Vers la fin la pluie a redoublé d’intensité et juste au moment où
je commençais à en avoir marre le refuge est apparu dans la brume. Le
soir plus personne ne se moquait de le taille de mon blouson en duvet :
j’étais le seul à me balader encore dans le refuge alors qu’ils étaient
tous planqués au fond de leurs duvets !</p>
<hr class="docutils" />
<p>Le lendemain matin nous pouvions voir le glacier qui apparaissait et
disparaissait 400 mètres sous nous, continuant de plus en plus raide et
crevassé vers les grandes étendues plates couvertes de neiges éternelles
d’où il part. Ces névés brillent faiblement à travers les nuages. De
temps en temps nous apercevons même les montagnes derrière.</p>
<p>Nous prenons notre temps : la visibilité ne permet pas de continuer plus
haut et nous comptons redescendre pour monter un peu de l’autre coté sur
le flanc de la vallée. Sur le glacier nous nous baladons sur les chemins
taillés dans la glace et par hasard je découvre que l’un d’entre eux
s’engage dans une crevasse. Si les guides emmène des groupes d’une
vingtaine de touriste là dedans nous pouvons y aller sans danger. C’est
ainsi que nous nous sommes promené dans des centaines de mètres de
crevasse, prenant une quantité astronomique de photos. J’ai même emprunté
un deuxième piolet pour partir à l’assaut d’une pente ; c’était un peu
limite avec mon équipement (un des deux piolets était ridiculeusement
léger et n’ancrait pas bien, et les crampons étaient un tout petit peu
petits pour une telle utilisation), mais la chute ne craignait pas du tout.</p>
<hr class="docutils" />
<p>Aujourd’hui je suis aller rendre l’équipement à midi au Tramping Club.
Tout le monde se racontait le week end et j’ai appris par Adrian qu’ils
avaient eut de la neige sur Secretary Island ! On peut pas tout faire. Je
crois que je n’irais jamais sur cette ile, mais j’ai une bonne idée de ce
à quoi elle ressemble.</p>
Mes vacances de pâques : 10 jours dans l’arrière pays2003-05-07T00:00:00+02:002003-05-07T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2003-05-07:/personnal/mes-vacances-de-paques-10-jours-dans-larriere-pays.html<img alt="" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/panorama.jpg" style="width: 100%;" />
<p>Tout a commencé il y a deux semaines, je me demandais ce que j’allais
faire pendant les vacances de pâques. L’hivers approche ici, et un type
normal resterait à Dunedin, une petite ville bien sympa. Bon étant donné
que je ne suis pas vraiment un type normal vous …</p><img alt="" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/panorama.jpg" style="width: 100%;" />
<p>Tout a commencé il y a deux semaines, je me demandais ce que j’allais
faire pendant les vacances de pâques. L’hivers approche ici, et un type
normal resterait à Dunedin, une petite ville bien sympa. Bon étant donné
que je ne suis pas vraiment un type normal vous imaginez bien que ce
n’étaient pas mes projets. Je comptais remonter vers le nord, où il fait
plus chaud, et aller voir de jolis lacs.. Mais j’ai reçu cet e-mail par
l’intermédiaire de la mailing list du “Tramping Club” :</p>
<blockquote class="epigraph">
<p>James Scott and I are heading to Fiordland for 10 days over
Easter, in the remote country in the back of the Special Takahe
Area. We are looking for two more people to share fun, pain, costs
and gear.</p>
<p>Our intended route: Boat-ride with Adrian to Bradshaw Sound,
Gear Arm. Camelot River and Cozette Burn. Robin Saddle Hut
(apparently the remotest hut in NZ). Irene River, Large Burn and
Doon River. Boat ride out on Middle Fiord of Lake TeAnau. We’ll try
and climb both Coronation Peak and Mt. Irene on the way. Yes, we
are both aware it’s an ambitious plan, but we are excited about it
- aim for Mars, and you’ll land on the moon.</p>
<p>Dates: leaving on Thu night, April 17th; back on Sunday, April 27th</p>
<p>Cost estimate: $250 - $300 each, including boat rides on
Doubtful Sound and Lake Te Anau, ride to Te Anau and back, mountain
radio hire, food</p>
<p>trip level: madness</p>
<p>skills required: off-track tramping, river crossing,
bush-bashing, alpine (rock + snow), being able to smile when life
is shit</p>
<p>equipment needed: ice-axe (1), sense of humor (LOTS), crampons
(2), thirst for adventure (LOTS), harness (maybe). A bivvy bag is
strongly recommended - we will NOT carry a tent.</p>
<p>what we can guarantee: a trip that is going to be wild, rough,
hard, in a unique remote area</p>
<p>what we cannot guarantee: your survival (particularly if your
name is Bob Smith)</p>
<p>If you are interested, get in touch ASAP with me or James
(<a class="reference external" href="mailto:james.scott@paradise.net.nz">james.scott@paradise.net.nz</a>, 471 0096) - sorry we’ll have to be
kind of picky on this one.</p>
<p>Ciao</p>
<p>Danilo Hegg</p>
</blockquote>
<p>Je précise que je connais Danilo et qu’il est totallement givré,
que je connais aussi James, et qu’il est moins givré mais que
récement il a dû se faire évacuer par hélicoptère parce que son
compagnon avait perdu son sac dans un torrent à des jours de marche
de toute contrée hospitalière. Sachant ceci (et donc que ce n’était
pas un e-mail juste pour crâner) je me suis dépéché de répondre, et
j’ai été jugé assez fou pour être accepté.</p>
<p>Le week end suivant je restais en ville, pour la première fois depuis des
mois, j’ai donc été chargé d’aller chercher la radio : une petite boite
dans une poche en toile cirée jaune, avec une antenne de vingts mêtres à
dérouler.</p>
<p>La veille du départ je reçois un e-mail de Danillo m’invitant à le
joindre pour aller faire les courses, mais ce soir là j’avais convié tout
le labo à un dinner chez moi pour fêter mon anniversaire. La règle tacite
dans la maison est que lorsque l’on organise une fête les différents
locataires peuvent inviter quelques uns de leur amis, c’est ainsi que la
moitié des convives ne savaient même pas quel était le “birthday kid”. Au
milieu de la foule hétéroclyte invitée par mes colocataires, charmants
mais un peu dégentés, les physiciens de labo essayaient d’établir une
atmosphère familière afin de ne pas se sentir etrangers. La bière
favorite des post-docs, qui traine si souvent dans la salle commune du
département de physique était au rendez-vous, mais Dieu merci mes
colocataires avaient une bonne provision de vin. J’ai reçu une carte avec
sur le devant “Happy birthday to a person who has that certain ‘Je ne
sais quoi’ “ et derrière “That’s french for ‘a card with no money in it’
“. Avec la carte venait un gigantesque oeuf de Pâque que nous avons
partagé, et dedans il y avait des pastilles de chocolat, choisies parce
que je pouvais les enmener avec moi dans la brousse.</p>
<p>La fête s’est vite tassée et le labo est parti se coucher, laissant
derrière lui Andrew, un de mes colocataires, ivre mort, grattant sur sa
guitare tout en m’expliquant lentement à travers sa barbe fournie sa
philosophie, une version simplifié du “Carpe Diem”. Il fallait voir cette
masse de plus de 100 kilos, qui a voyagé à travers toute l’Asie dans
des conditions inimiginables, débiter ses âneries, tout à fait conscient
qu’il est sou, mais encore droit comme un ‘I’, comme un clodo parisien,
interpelant l’assistance.</p>
<p>Sur ce Danillo a débarqué : il m’apportait dix kilos de nourriture, et
deux kilos de fuel, le portage ayant été réparti à l’aide d’une balance
chez lui.</p>
<p>Le lendemain j’arrivais en retard au labo, avec un sac à dos immense,
qui, je dois avouer, m’effrayait un peu. Nous avons fini les gateaux pour
le thé, sauf le gateau au chocolat, qui était déjà dans mon sac, et deux
heures plus tard j’étais dans la voiture de James, partant pour
Fjordland.</p>
<hr class="docutils" />
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/gael-pac.jpg" style="width: 30%;" />
<p>Le bateau d’Adrian était mouillé sur le lac Manapouri, mais il a fallut
faire la navette entre le bateau et la station service à Te Anau pour
remplir le reservoir, puis hisser à bord 150L dans des bidons. A la
station service la pompiste s’amusait beaucoup. Surtout quand nous avons
entrepri de séparer à l’odeur les bidons d’essence, destinés au bateau,
des bidons de diesel, destinés au camion qui attendait de l’autre coté du
lac pour amener le bateau jusqu’à la mer.</p>
<p>Une fois le bateau plein James est parti à Te Anau attendre la
deuxième voiture : la sienne resterait là-bas pour servir au retour. Pour
ma part je restais sur la jetée, à attendre dans la nuit, au milieu de nul
part. Mes vêtements chauds étaient dans mon sac à dos, sur le bateau,
mouillé à vingt mêtres de là, et l’attente a été glaciale : si je restais
en place plus de dix minutes je me mettais à grelotter. Le froid me
gagnait lentement et au plus profond de moi même.</p>
<p>Finnallement des phares ont surgit sur cette route déserte, et toute
la clique était réunie. Adrian a amené le bateau au ponton, et nous
avons embarqué pour la traversée du lac Manapouri. La pleine lune
permettait de naviguer sans difficulté, et les montagnes dominant le lac
étaient superbement éclairées, la neige qui les courronait luisant
pâlement dans l’obscurité. Engourdi par le froid, malgrés la veste en
duvet que j’avais emprunté avant de partir, je tentais de m’arracher à la
somnolance qui me prenait : même sur un plan d’eau calme le bateau
d’Adrian rebondit violement sur la moindre irrégularité. Ce qui devait
arriver arriva : une seconde d’inattention, et ma tête rebondit sur le
capot ; le gout de sang dans ma bouche me tient éveillé pendant quelque
temps, mais mes yeux se ferment à nouveau, et je m’enfonce une deuxième
fois les dents dans la lèvre inférieure. Heureusement la traversée est
terminée. Un hall d’embarquement pour les touristes qui traversent
pendant la journée est ouvert et chauffé : de ce coté du lac il n’y a
rien, c’est un avant poste dans la brousse, et les portes n’ont même pas
de verrous. Nous nous couchons sur la moquette, bien à l’abris du vent.
C’est ma dernière nuit sous un toit pendant un moment, et je compte bien
en profiter.</p>
<hr class="docutils" />
<p>Le lendemain matin nous nous levons dès les premières lueurs du jour :
le plus tôt nous pouvons commencer à marcher le mieux. Mais il faut
encore amener le bateau au fjord. Le gros 4x4 d’Adrian est garé non loin
de la jetée et nous plaçons le bateau sur la remorque. Une petite demi
heure de voiture nous amène au fjord par une route sinuant au fond d’une
gorge. “Doubtful sound” s’étend devant nous dans la lueur du petit matin.
Le vent souffle fort et le bateau tappe sur les vagues. Le paysage est
superbe : des montagnes d’un rocher rendu vert par la végétation
surgissent de l’eau de chaque coté du bateau. Régulièrement des vallées
les fendent, des gorges partent à l’assault des cimes, taillées comme à
coup de hache. La fameuse “Secretary Island” se dresse devant moi. Située
à l’entrée du fjord, loin de tout et pourtant accessible en quelques
heures, elle constitue l’une des destinations favorites d’Adrian, qui y a
trainé les membres les plus fous du club. J’en ai entendu parlé depuis
longtemps : elle est partiellement couverte d’une brousse basse et
extrèmement dense, formée d’arbustes secs et agressifs, la “sub-antartic
scrub”. Il paraît que la progression est terriblement difficile. J’y vais
le week end aprés mon retour de ce périple. Pour l’instant je ne vois
d’elle que des paroies presques verticales couvertes de “rain forest” ;
le reste de l’ile est sur un plateau qui domine le fjord de plusieurs
centaines de mêtres.</p>
<p>Nous nous engageons dans la branche du fjord où Adrian va nous déposer.
Au fond la “Camelot river” vient se jeter dans la mer Tasmienne formant
un long estuaire plat et vaseux, qu’Adrian remonte habilement le long des
veines de la rivière. Il s’engage surprenement loin dans la rivière,
dépassant les étendues de vase de l’estuaire pour remonter l’une des
branches de la rivière, à peine deux fois plus large que son bateau. La
forêt forme un toit au dessus de l’eau et il ne faudrait pas beaucoup
pour que je me crois en amazonie. Malgrés tout Adrian doit arréter son
bateau : des tronc d’arbres submergés barrent le passage. Il nous
débarque et nous souhaite bonne chance. Il part naviguer pendant une
semaine avec sa copine. Pour eux, comme pour nous, ce sera le dernier
contact humain extérieur pendant plus d’une semaine.</p>
<p>Il faut vite nous mettre en marche, et nous commençons par lutter
contre la boue. Les sacs à dos trés lourds n’arrangent pas la situation,
et à un moment je m’enfonce jusqu’à la taille, ce qui déclanche un fou
rire général, même si je suis absolument incapable de sortir tout seul.
La brousse n’est pas trés dense et plus loin dans la vallée nous
rencontrons des sentiers de rennes, certainement pas d’humains : il y a
en moyenne un groupe qui passe par an ici. Les rennes font d’excellents
sentiers, qui malheureusement se perdent assez souvent, et ne vont pas
toujours là où nous voulons aller. De plus les rennes engembent ou
sautent les troncs en travers, tandit que pour nous, bipèdes lourdement
chargés, se redresser aprés avoir rampé sous un arbre est la croix et la
bannière quand on a 25 kilos sur le dos. Je découvre que le piolet, que
j’avais choisi parmis les plus longs, content d’avoir réussi à en avoir
un à ma taille, dépasse de mon sac, et se prend dans les branches et les
lianes ; ce n’est pas pour rien que les Néo-Zélandais aiment les piolets
courts. La journée passe vite et vers 6 heures nous sommes dans un
marécage, naviguant au dessus de la boue par l’intermédiaire de troncs
d’arbres tombés, ou sautant d’une racine à l’autre. Cependant quand nous
essayons d’en sortir nous nous heurtons à un enchévêtrement de racines et
de mousse formant un plancher irrégulier à travers lequel on passe
facilement à travers. Nous décidons de retourner dans le marécage pour
tenter de rejoindre la rivière. Là nous trouvons des plages de galets et
quelques centaines de mêtres plus loin les chutes à coté desquels nous
avions prévu de camper. Il était temps : la lumière du jour
disparaissait.</p>
<p>La “fly-tent” est rapidement mise en place, et pendant que Dave
cherche du bois sec et Danillo prépare le dinner, James et moi déroulons
et tendons l’antenne de vingts mêtres de long de la radio. A 7h30 la base
va nous appeller, comme tous les soirs, pour que nous leur donnions notre
position. C’est l’occasion de prendre la météo, et de recevoir ou envoyer
des messages. Lorsque la voix laconique de l’opérateur se fait entendre
il fait nuit depuis longtemps et nous somme sous la “fly-tent”, dans nos
sac de couchage, la radio posée à coté. “This is IB-Base Christchurch,
calling field stations, now calling IB twenty six, IB two six.” IB 26
est l’indicatif d’Adrian, et nous apprenons qu’ils sont sortis du fjord
et passent la nuit dans un petit fjord plus au sud. Les appels
s’éternisent : comme la météo est bonne et que c’est le debut des
vacances il y a un nombre incroyable ( 20 ? ) de groupes dans l’arrière
pays. Finallement “Now calling IB one forty nine, IB one four nine.”
C’est nous, et James répond, articulant lentement : “IB Base, this is IB
one four nine, we copied the weather forecast, have no messages and are
currently located at the entrance of the Cozette burn, that’s the Cozette
burn, over” - “IB one four nine, could you please repeat your location ?
Over”, l’opérateur n’a pas la moindre idée d’où est la “Cozette burn”,
personne ne va jamais aussi loin. Une fois le rapport radio terminé James
et Dave démarrent le feu. Je trouve même le courage de sortir de mon sac
de couchage et de courrir pieds nus jusqu’au feu. Nous ne veillons pas
longtemps, mais nous apprécions tous pleinement ce moment privigié où on
sent vibrer Fjordland autour de soi dans le calme de la nuit. Avant de me
reglisser dans mon sac de couchage j’examine mon petit doigt de pied : il
y a quelques jours de là je me suis arraché l’ongle et je crainds un ongle
incarné. Il est rouge et probablement inflamé, mais n’a pas l’air trop
laid. Je mets à tout hasard la paumade que j’ai acheté à Gore en
prévision de tels problèmes ; ce n’est pas ce que j’aurais voulu, mais je
n’ai pas réussi à me faire comprendre de la pharmacienne.</p>
<hr class="docutils" />
<p>Le lendemain matin Danillo nous reveille avant les premières lueurs du
jour. Il a déjà préparé le porridge et nous le mangeons sans sortir du
sac de couchage : il fait noir comme dans un four mais nettement plus
froid. Je suis de mauvaise humeur : je déteste devoir faire mon sac à la
lumière de ma lampe frontale : “Danillo, this IS too early !”.
Heureusement le temps que nous plions la tente et l’antenne il fait
suffisement jour pour que nous marchions sans lampe. La progression reste
assez facile, même s’il faut traverser la rivière plusieurs fois. Le
problème n’est pas trop que cela nous mouille jusqu’à la ceinture, c’est
désagréable mais on s’y fait presque, le problème est plutôt que le
courant est élevé, le fond est irrégulier, et avec nos sac à dos une
chute peut être fatale. A chaque fois que je lève un pied pour l’avancer,
le pauvre bipède que je suis se retrouve en équilibre instable sur un
unique appui ; c’est pourquoi lorsque le courant est vraiment trop élevé
nous nous regroupons, formant une ligne parallèle au courant, et, épaule
contre épaule, chacun avance un pied à son tour. Bien sûr des rochers ou
des trous viennent briser la ligne ainsi formée, mais cela permet de
passer avec plus de facilité.</p>
<p>Alors que nous suivons péniblement le sentier de lestes rennes,
grimpant au dessus de troncs d’arbres, ou passant en dessous, Danillo
s’éfondre soudain, et pousse un grand cri. D’où je suis je ne peux rien
voir à cause des fougères, mais James et Dave, juste derrière lui se
penchent sur lui. Il gémit à terre, mais apparement ne semble pas blessé
trop gravement. Il a glissé du tronc qu’il escaladait et c’est cogné à la
tête. Il ne saigne pas, n’a pas de nausée, se sent en possession de tous
ses moyens, et se plaind juste d’une grande douleur là où il s’est cogné.
Nous écartons donc le risque d’une commotion cérébrale et repartons aprés
une demi heure de pause. Cet épisode nous rapelle que ici le moindre
incident peu tourner au drâme et que nous ne pouvons nous permettre de
prendre des risques inutiles.</p>
<p>Le soir nous somme encore dans la “Cozette burn”, mais à coté du
“Hidden lake”, au pied de “Coronation peak”. Le pic se dresse au fond de
la vallée, féroce, le sommet est entouré de falaise pour autant que nous
pouvons voir, mais Danillo nous dit qu’il a le récit de la première
expédition et connait donc le chemin. Nous le questionnons un peu à
propos de cette expédition, et nous apprenons que les personnes en
question sont restées pendant quatres semaines avant de trouver un passage,
recevant leur avitaillement par hélicoptère. Nous n’avons qu’une journée,
et ce sera une longue journée.</p>
<hr class="docutils" />
<p>Le matin lorsque Danillo nous réveille je suis ravi de voir que la lune
est levée ; je n’espère bien sûr pas voir le soleil de si tôt. Nous
laissons la tente et une grande partie de notre équipement et nous
lançons dans la brousse à la lumière de nos frontales. Progresser dans la
brousse n’est déjà pas bien facile, mais la nuit c’est encore plus
impressionant : on a l’impression d’être enfermé dans une cage infinie de
branches, et à chaque pas que l’on fait une nouvelle branche, un nouveau
rocher, une nouvelle liane surgit. Nous gagnons un torrent que nous
remontons lentement. La brousse est si compacte que à chaque fois que
nous tentons de quitter le lit du torrent nous sommes refoulé. Il nous
faut donc patauger de caillou en caillou, franchissant des petites chutes
sur le coté, grimpant encore et encore sur des rochers glissants et
couverts de mousse. A la lumière de ma lampe je distingue Danillo qui
passe lentement au dessus d’un rocher pour se retourner et redescendre :
cela ne passe pas. Danillo m’impressionne par sa détermination, il est
nettement moins agile que nous, tombe souvent et progresse avec beaucoup
de difficulté, mais il repart constement à l’assaut, malgrés un sac
encore plus lourd que le notre, et particulièrement mal équilibré.</p>
<p>Le jour se lève lentement et dans ma tête je me demande en avançant si
ma vénération va d’abord au soleil, ou à mon sac de couchage. Nous
quittons enfin le torrent pour un éboulit qui s’élance droit vers la
falaise, sous laquelle nous allons traverser pour rejoindre le torrent et
tenter de passer au dessus de la superbe chute de 70 mètres qui nous
barre le passage. Nous passons vite au dessus de la brousse, mais
j’apperçois que les flancs de la montagne sont couvert de la fameuse
“scrub”. Arrivée en haut de l’éboulit il faut s’engager dedans. Je ne
vois aucun passage, les arbustes forment une masse continue et uniforme,
leur branches sèches et dénuées de feuilles s’entremélant. James se
lance, se frayant un passage avec les bras. Nous nous regardons et
lentement nous préparons à nous engager à sa suite quand il réapparaît :
Il a vu un chamois un peu plus haut, il doit donc y avoir un passage.
Nous découvrons en effet juste sous la falaise des marques de sabots qui
nous guident à travers la brousse. Nous longeons la falaise si haut que
nous sommes situés au dessus de la chute que nous devons franchir. A un
endroit où la falaise est particulièrement petite James se retourne vers
moi : nous pensons la même chose, le rocher est bon, il est loin d’être
vertical, et est parcouru de beaucoup de failles, le passage sera facile
ici. Dave, qui est un trés bon grimpeur est d’accord avec nous mais
Danillo veut faire le tour. Il part donc vers la chute tandit que nous
sortons la corde et les harnais. Le début ne présente aucune difficulté
et nous ne nous encordons même pas, mais, étant en tête, je me trouve
bloqué lorsque la faille que je suivais est envahie par une petite
source. Nous nous arrétons sur une plateforme en dessous, et Dave prend
la tête, tandis que Jame l’assure à partir d’une sangle passée autour
d’une piton rocher. Dave progresse lentement, mais avec aisance, et
beaucoup de précaution. Il place deux sangles pour s’assurer avant de
disparaître de notre champ de vision derrière une avancée rocheuse. Les
deux sangles, qu’il avait pourtant bloqué par des pierres, finissent par
sortir de leur encoche : si il tombe il parcourera deux fois trente
mêtres avant d’être arrété par la corde. James et moi nous nous regardons
: Dave n’a rien remarqué. Nous choisissons de ne pas lui dire, c’est un
grimpeur sûr et il ne prendra pas de risques ; de toutes façons il ne
peut pas vraiment redescendre. Finallement Dave nous appelle : Go ahead.
Je pars, encordé au milieu de la corde, gravis les difficultés, puis
libère la corde pour James qui grimpe en bout de corde. De là où je suis
la sortie est facile et je suis rapidement sur le plateau, où Danillo
nous attend en riant : son chemin était clairement le plus court, mais
nous sommes content d’avoir eu une jolie vue, et d’être sortis de la
brousse. De plus Dave, qui transporte la corde, se réjouit de l’avoir
utilisé au moins une fois.</p>
<p>Il nous faut alors contourner la montagne pour s’engager dans une
vallée se terminant par un long pierrier à partir duquel nous nous
grimpons sur une arrète menant au sommet. En chemin nous tombons sur un
groupe de Kea. Ces perroquets des montagnes sont remarquablement
intelligents et vivaces. Leur jeu favori est de voler de la nourriture
ou de l’équipement aux randonneurs. James m’a raconté qu’une nuit, alors
qu’il dormait à l’extérieur d’un refuge plein, un Kea se glissait
derrière lui pour picorer son matelat de sol. Il l’a chassé de nombreuses
fois, courant derrière l’animal, qui ne se donnait même pas le mal de
voler, mais sautillait pour lui échapper. Et la satannée bestiole
revennait toujours, choisissant bien sûr d’approcher derrière lui. Il a
fini par aller dormir sur le sol déjà encombrée du refuge, rejoind par un
autre malheureux qu’un Kea empéchait de dormir en sautant continuement
sur son sac de bivouac. Alors que nous gravissons un à un les blocs dont
les pentes sont jonchées, ces oiseaux se posaient à coté de nous et
riaient, se moquant problablement de nous, pauvres humains, condamnés à
marcher alors qu’eux peuvent survoler le terrain sans difficulté.</p>
<p>Dave, James et moi arrivons au col rapidement. On apperçoit la vallée
jusqu’au lac qui surplombe la chute d’eau. Sur le coté, au dessus de
l’éboulit que nous avons gravit, se dresse un glacier, qui se continue en
une chute qui plonge directement au fond de la vallé. Dieu merci le
glacier est un petit, et il fond juste avant la falaise, nous n’avons
donc pas marché sous des chutes de serracs, comme on en voit tant autour
du Mont Cook, mais sous une superbe cascade de glace. Danillo est
toujours dans la pente et je suis surpris de voir les deux autres
s’engager sur l’arrète : il est onze heures et je commence à avoir faim.
J’avale deux barres de céréal et je me lance à leur suite. Vingt minutes
plus tard Dave déclare qu’il ne peut plus avancer et doit manger. C’est
avec plaisir que je sors le nutella. Nous cassons la croute en vitesse et
repartons quand Danillo nous rejoind : nous avons encore du chemin à
parcourir. Arrivés au glacier Dave James et moi chaussons nos crampons
tandis que Danillo décide de le contourner sur l’arrète. C’est fabuleux
de marcher sur la neige, lisse et régulière. Le glacier se prolonge sous
le sommet jusqu’à une autre arrète, mais aucune des deux arrètes, ni la
falaise sous le sommet ne nous tentent. Nous nous arrétons à un petit col
où le glacier rejoind l’arrète que nous gravissions. Nous attendons
Danillo, bien décidés à ne pas nous lancer dans les paroies qui nous
attendent. Danillo est de notre avis : le chemin pris par la première
expédition lui semble difficile et dangeureux, mais il veut descendre le
goulet au sommet duquel nous nous tenons, pour rejoindre un plateau qui
nous permettera de traverser vers une troisième arrète qui sur la carte
n’a pas l’air trop difficile. Nous nous laissons convaincre et nous nous
engageons lentement dans le goulet, un par un, car nous libérons des
pierres qui dévallent dangereusement la pente. Danillo part devant, il
veut absolument arriver au sommet. Nous autres sommes content de profiter
du soleil et de la vue fabuleuse : on voit loin dans Fjordland, une
succession de pics pointus, d’arrètes étroites et de vallées encaissées.
Arrivés à l’autre arrète nous décidons que c’est une impasse et nous
arrétons pour attendre le retour de Danillo et casser la croute.</p>
<p>Danillo ne reviend cependant pas bien vite et nous partons voir ce
qu’il devient, laissant les sacs derrière. Nous le trouvons en train de
faire méthodiquement des marches dans un glacier de neige froide avec ses
crampons, ce qui nous fait bien rire : non seulement la paroie à coté du
glacier est facile à grimper, comme nous lui montrons, mais en plus
tailler des marches dans une neige aussi froide, où les crampons
accrochent trés bien nous semble ridicule, mais clairement Danillo n’est
pas à l’aise en terrain alpin. Il s’entête cependant et se lance à
l’assaut de la falaise qui le sépare du sommet. Nous avons laissés la
corde avec les sacs et ne pouvons rien faire pour lui. Nous sommes bien
décidé à ne pas le suivre : il est temps de rentrer, aucun d’entre nous
n’a une envie folle de descendre la nuit tombée. Je m’assoie sur un
rocher, au soleil, et j’enlève mes chaussures pour les faires sécher. Mon
petit doigt de pied m’inquiète : j’en tire une bonne quantité de pus,
même si je ne peux localiser l’endroit où l’ongle s’enfonce dans la
chair. Au bout d’une heure nous voyons réapparaître Danillo, qui nous en
veut de ne pas l’avoir suivi avec la corde. Selon lui nous aurions pu
faire le sommet. C’est vrai qu’avoir grimpé “Coronation peak”, un sommet
en haut duquel seulement une vingtaine d’alpinistes se sont baladés
aurait fait joli sur ma liste de course, mais cela n’aurait rien apporté
à ma journée, que j’estime trés réussite.</p>
<p>Nous entamons la descente au pas de course, trainant Danillo essouflé
derrière nous, le plus bas nous nous trouvons que le soleil se couchera
le mieu cela sera. Nous contournons la chute d’eau par la brousse,
descendant une pente si raide que nous avons l’impression de nous
suspendre d’arbre en arbre. Quand je pense que ce pauvre Danillo a dû
monter par là. Nous sortons de la “scrub” avant de devoir allumer les
lampes : il ne reste plus que l’éboulit et le torrent à descendre, ce qui
nous prend deux heures. Au fur et à mesure que nous approchons de la
tente je me rend compte que nous n’avons aucun moyen de la repèrer.
Danillo prétent qu’elle est à coté d’un ruisseau, et qu’il suffira donc
de le suivre mais il y a une dizaine de ruisseaux, et nous pouvons passer
à vingts mêtres de la tente sans la voir. Je commence à être passablement
en colère contre cet inconscient. Au bout d’un moment il paraît clair que
nous avons dépassé la tente sans la voir et nous nous divisons en deux
groupes, Dave et James, et Danillo et moi, pour ratisser la forêt. Nous
cherchons une aiguille dans une botte de foin. Je regrette de ne pas
avoir pris ma doudounne : dès que nous nous arrèterons de marcher nous
nous refroidirons, nous avons peu de chances de passer la nuit si nous ne
trouvons pas nos sac de couchage. Danillo est persuadé qu’il peut trouver
la tente juste par son instinct et parcourt la forêt de façon
désordonnée, ce qui a le don de m’exaspérer encore plus. Je le force à
s’arréter et j’établi le contact avec les autres en criant : nous devons
nous organiser et coordonner notre action. La communication n’est bien
sûr pas évidente et encore là Danillo se montre incapable, coupant James
alors qu’il parle et donnant des indications imprécises. Il faut
comprendre que la brousse atténue les sons, et parler avec quelqu’un deux
cents mêtres plus loin demande un protocole semble à parler sur une radio
: si la personne en question n’est pas immobile et concentrée elle ne
comprendra pas le message. Nous décidons donc de choisir deux ruisseaux et
de les remonter jusqu’à ce que nous soyons certains d’avoir dépassé la
tente, puis de se décaler, de choisir deux nouveaux ruisseaux et de
recommencer. Nous balayons ainsi deux paires de ruisseaux, mais ils
sinuent tellement que nous n’avons pas une idée clair de la situation, ni
d’où nous marchons. Je décide de regrouper tout le monde et demande aux
autres de rester où ils sont. Il faut alors les trouver, leurs lampes
sont bien sûr invisbles. Pour cela je marche pendant vingt mêtres puis
crie “James ?” - “Here” - “OK”, et c’est repartit pour vingt mêtres.
Soudain je distingue un fil tendu en l’air. Un fil ? Dans un endroit
comme ceci cela cloche vraiment ; “Freeze every body !” je crie. Et je
suis le fil. C’était l’antenne de la radio, j’étais passé à dix mêtres de
la tente sans la voir. Vous pouvez imaginer ma joie quand je crie “I have
the fly !”. Nous nous sommes vite mis dans nos sac de couchage sans un
mot et avons mangé notre dinner silencieusement. Je ne sais ce que
Danillo pensait mais nous autres étions furieux contre lui à cause de son
obstination à vouloir faire le sommet, et son manque d’organisation. Il
me paraît la moindre des choses de repérer la tente dans un cas comme
cela, ne serais-ce qu’avec des relèvements, mais monsieur ne transporte
même pas de compas. Je pense que à ce moment nous avons pris la décision
tacite que Danillo ne dirigeait plus le groupe et que nous agirons selon
notre évaluation de la situation, qu’il le veuille ou non.</p>
<hr class="docutils" />
<p>Quand Danillo nous réveille le jour suivant il fait déjà jour.
Apparement nous avons eut le droit à une grasse matinée. Nous l’avons
mérité : nous avons marché 14 heures la veille. Un “weka” furte autour
de la tente. Cet oiseau indigène ressemble à une sorte de poule. Il ne
peut pas non plus voler et est tout à fait inoffensif mais il adore
chipper tout ce qu’il peut trouver. Nous essayons de lui faire peur mais
ce n’est pas évident sans sortir du sac de couchage. Le weka finit par
trouver un ver de terre qu’il extrait entier du sol, au bout d’une longue
bataille dont le but évident était de ne pas couper le ver en deux en le
tirant par une extrémité. Il s’enfuie alors avec sa prise, nous laissant
savourer notre porridge tranquille. Les rations sont clairement trop
petites. Non seulement je pourrais facilement en avaler le double, mais en
plus dès dix heures la faim me tenaillait. Il n’y a cependant rien que
l’on puisse faire : nous devons nous limiter sur la nourriture. La bonne
nouvelle est que ce soir nous serons à la hutte : “Robin saddle hut”, que
le topo annonçait comme la hutte la plus reculée de Nouvelle Zélande. Pour
cela il nous faut franchir un col, puis traverser à mi pente de l’autre
coté pendant quelques kilomêtres jusqu’à la hutte. Une longue journée
mais la perspective d’une bonne nuit me motive.</p>
<p>Au bout de quelques kilomêtres de brousse nous arrivons à une chute
d’eau bordée de chaque coté par des barres rocheuses couvertes de
brousse. Il n’y a aucun moyen de la contourner, il faut se lancer dans la
falaise. Mais ce n’est pas une falaise sèche et grimpable comme celle que
nous avons gravit la veille : c’est une dalle de roche lisse et couverte
entièrement de plusieurs centimêtres de mousse. A travers chaque trou
dans le rocher sortent des arbustes variés. Danillo et James essaient
plusieurs voies sans succés avant que James trouve une faille
prometteuse. L’ascension est particulièrement pénible : nous nous
agrippons à tout ce qui pousse, le rocher étant trop lisse et mouillé
pour fournir un appuit, et nous hissons presque verticalement. Pour
courronner le tout nous sommes condamné à passer au milieu de
“bush-lawyers” : les ronces de la brousse. James est devant et nous
encourrage : oui cela passe, oui la falaise continue plus haut. Nous en
sortons finallement et rejoignons le torrent pour le remonter, content de
trouver un terrain aussi facile. Nous somme tous d’accord que c’est le
passage le plus dûr de brousse que nous avons jamais franchi, et je crois
qu’aucun d’entre nous n’a une grande envie d’en franchir un plus dûr.
Nous avançons sans nous arréter. Je sens la fatigue des jours passés,
gravissant sans cesse de nouveaux rochers. Danillo est lent et nous
devons régulièrement nous arreter pour l’attendre. Au cours de l’un des
arrêts j’ouvre les chocolats que j’ai eut pour mon anniversaire : j’ai
vraiment trop faim. Nous trouvonns un éboulis sur la gauche qui nous
permet de passer au dessus d’une autre grande chute, contournant les
falaises qui l’entourent. De plus il nous permet de passer au dessus de
la “scrub”, qui ne subsiste pas bien haut, heureusement. Je me sens
faible, il faut que je mange. Je force les autres à s’arréter sur un
petit plateau, au soleil, à coté d’un ruisseau. Nous nous alongeons dans
les hautes herbes, dégustant le déjeuner avec plaisir. De temps en temps
nous nous levons pour marquer notre position pour Danillo qui progresse
lentement. Le paysage est fabuleux : des falaises en arc de cercle nous
séparent du col, fendues par une chute d’eau au milieu. La montagne est
couverte, entre quelques éboulis, d’une herbe jaunâtre qui paraît dorée
au soleil. Jusqu’ici la météo a été fabuleuse même si la radio nous
annonce une déterioration.</p>
<p>Nous repartons à l’assaut du col. Arrivé en haut il faut redescendre de
l’autre coté, ce qui est plus dûr car d’en haut on ne voit pas les pentes
et on ne sais jamais où descendre, comment éviter les barres rocheuses.
Nous nous engageons dans un goulet étroit et raide, couvert d’herbe.
Désescalader ce mélange d’herbe glissante et de rocher mouillé est plutôt
désagréable et nous envisageons en riant de mettre nos crampons. Je rie
encore plus quand je vois Danillo glisser lentement pendant quelques
mêtres sur un plan d’herbe. Je trouve que c’est une manière
particulièrement inélégante de le descendre et je compte bien faire mieux.
Mais arrivé à cet emplacement mes pieds glissent soudainement et je
pars dans la pente, gagnant de la vitesse rapidement. J’évite Danillo
mais je ne parviens pas à m’arréter, la pente est trop glissante, je
rebondit sur mes pieds, roule deux trois fois et réussis à agripper de
l’herbe pour m’arréter. Je suis sur le ventre, la tête en bas. J’ai
descendu vingts mêtres et Danillo au dessus de moi est toujours en train
d’hurler. Dave me demande si je vais bien. Oui, tout est OK… Je porte
ma main à ma tête. Ce n’est pas que de la boue que j’ai sur le visage, il
y a du sang. Il vient de dessous ma lèvre inférieure. Cela me fait mal,
mais ce n’est pas un bon critère. Dave me dit que ce n’est pas profond
mais sort tout de même un pansement. Mon coude aussi me fait mal, mais je
vois bien vite que c’est un détail. Je repars lentement, tremblant. La
pente ne me semble plus trés amusante. Au bout de quelques minutes je me
rend compte que ma hanche me fait aussi mal. Sous ma ceinture j’ai une
jolie entaille. C’est relativement profond. Rien qui ne m’inquiète
réellement, mais la ceinture de mon sac à dos appuie là. Cela va être
douloureux et cela ne va pas cicatriser de si tôt. Je descend le reste de
la pente trés lentement. J’ai compris que l’herbe ici était la seule
chose qui tenait et je saisis de grande touffes à pleines mains avant
chaque pas.</p>
<p>D’où nous sommes nous pouvons voir un petit plateau avec quelques
marres qui longe la montagne, formant un trou dans la brousse. Nous
descendons vers se plateau et Danillo, qui choisit un chemin différent du
notre, Dieu sait pourquoi, nous dit qu’il nous y retrouvera. Lorsque nous
pénétrons dans la brousse nous découvrons avec horreur que c’est un
mélange de brousse classique et de scrub, le tout saupoudré de gros
rochers aussi hauts que nous. Il nous faut une bonne heure pour franchir
la fine bande de brousse qui nous séparait du plateau, et nous découvrons
qu’il y a en fait plusieurs plateaux. Bien sûr pas de Danillo et nous
nous époumonons pendant une heure avant de le retrouver. Il est tard ; il
paraît clair que nous n’arriverons pas à la hutte aujourd’hui. De plus
nous sommes absolument épuisés. James est le premier à l’avouer mais nous
en sommes tous conscients. Pour aller à la hutte il nous faut franchir
quelques kilomêtres en traversant sur une pente raide couverte de scrub.
La hutte est cachée par un éperon rocheux. Danillo semble vouloir tenter
l’ascension de “Mount Irene”, au pied du quel la hutte est situé le
lendemain, j’insiste donc pour que nous essayons de continuer un peu. De
plus je n’aime pas l’idée de camper sur le plateau, l’eau des marres ne
m’inspire pas confiance et il n’y a aucun abrit contre le vent. Dés que
nous quittons le plateau nous heurtons une couche uniforme de “scrub”. La
pente est trés raide et la seule progression possible est en s’aggripant
aux arbustes avec les mains pour faire tenir les pieds en opposition. Au
bout de cents mêtres nous décidons qu’il est vain d’éssayer de continuer
ainsi : chaque mêtre est épuisant et nous n’avançons que trés lentement.
Je me porte volontaire pour aller examiner le torrent que nous
surplombant à la recherche d’un emplacement plat pour établir la tente.
Descendre cinquante mêtres jusqu’à la rivière n’a pas été évident, mais
remonter, lorsqu’il a été clair qu’il n’y avait aucun endroit favorable,
a été un vrai calvaire. L’unique moyen était de se hisser, presque
entièrement à la force de ses mains, les pieds battant pour prendre
appuie sur un tronc, un arbuste, n’importe quoi, de toute façon la pente
n’était que branches et épines. Quand j’arrive à la tente, que les autres
on déjà dressé sur le plateau que nous venions de quitter, je n’ai que la
force de me glisser dans mon sac de couchage, gromellant que le jour
suivant allait être dûr. La tente, que je devrais décidement appeller
bache, avait était tendue grace aux batons de marche que Danillo
transporte sur son sac et aux piolets. Piolets qui avaient d’ailleurs
servit à faire un peu de “jardinage” sous la bache. La tente n’est
clairement pas assez tendue et je ne peux même pas m’assoir dans mon sac
de couchage. Si il y a du vent cette nuit elle s’envolera.</p>
<hr class="docutils" />
<p>Je ne peux même pas prétendre que j’ai mal dormit, malgrés le vent
glacial qui balayait la tente : pour la première fois j’ai entièrement
fermé mon sac de bivouac sur ma tête. Le matin la condensation était
gelée sur la tente. Mes chaussettes et même mes chaussures étaient aussi
gelées. Le départ a été psychologiquement dûr. Sachant ce qui m’attendais
je me sentais assez misérablement. Mes pieds n’arrivaient pas à se
rechauffer et comble de l’horreur, la “scrub” était gélée, je me tractait
donc à travers la paroi sur des buissons épineux qui me maintenaient
froid et humide. La traversée a duré trois heures pendant les quelles
nous avons été continuement dans le froid à l’ombre de l’éperon rocheux.
Il nous a fallut passer au dessus, dans des pentes telles que nous avons
dû hisser à plusieurs reprises le sac de Danillo aux passages délicats.
En haut de l’éperon on appercevait le lac à coté du quel était la hutte.
Quelle vue délicieuse. Haut dessus trônait “Mount Irene”, que Danillo
étudiait déjà pour trouver une voie vers le sommet. Les autres n’on pas
voulu s’arréter pour manger et l’intégralité de mes chocolats a à peine
suffit à me faire tenir jusqu’à la hutte où j’ai finallement pu entamer
mes crackers. Je crois que pour la première fois de ma vie j’ai été en
hypoglycémie. La faim me tenaillait et me rendait faible. J’ai apparement
besoin de manger plus que les autres, et dans une expédition comme celle
là c’est un problème. Heureusement j’avais prévu des déjeuners beaucoup
plus gros que les autres qui en riaient il y a quelques jours mais
maintenant me regardaient avec envie avaler une demi boite de crackers.
C’est là que James a sorti des oeufs de Pâques pour tout le monde. C’est
impressionant ce qu’on les apprécie dans un tel endroit.</p>
<p>Le miel était délicieux, le soleil me chauffait agréablement, assis
sur des pierres, pieds nus devant la hutte. Cette fabuleuse hutte, grande
de trois mêtres sur trois mêtres me réchauffait le coeur. Un tel abri
dans un endroit aussi reculé, perché dans de superbes montagnes, dans des
paysages indéscriptibles avait quelque chose de féérique. Le livre de la
hutte indiquait que personne n’y avait été depuis mars de l’année
dernière. Datant de 18 ans le livre ne contenait qu’une trentaine
d’entrées, principalement des zoologistes à la recherche des Takahes, une
espèce de perroquet en voie de disparission. Il y avait aussi quelques
histoires folles comme celle de cet Allemand, parti de la pointe sud de
l’ile, remontant vers le nord à travers l’ouest, sauvage et désert. Cela
ne lui avait pris que 23 jours pour arriver jusqu’ici ; comme le faisait
remarquer quelqu’un dans le livre “This man must be a machine !”. Un
autre voyageur était tombé d’une arrète et s’était réfugié dans la hutte
pour une semaine “I laid and suffered here for a week while I recovered
from my injuries”. Presque toutes les entrées faisaient remarquer combien
cette hutte était fabuleuse, et beaucoup mentionnaient une météo execrable
: “foul weather”. Pour l’instant le soleil brillait mais des nuages
s’amoncellaient autour des sommets.</p>
<p>Danillo n’envisageait plus de gravir Mont Irene ce jour même. De
toutes façons il aurait été seul. J’étais épuisé et je continuais à tirer
du pus de mon doigt de pied. Mes mains étaient déchirées par la “scrub”
et mes avants bras lacérés. Tout mon corp réclamait une pause, et les
autres semblaient d’accord. J’étais un peu préocuppé par mon pied :
j’avais marché pendant quatres jours avec de mauvaises chaussures, deux
semaines au paravant, et j’avais attrapé ce que certains ont
diagnostiqués comme une infection osseuse au talon. Je connais cela car
c’est ce qui a tourmenté ma malléole au début de l’année. Je n’avais pas
encore réalisé ce que j’avais au début du voyage, mais la douleur
croissait lentement et la fin du périple promettait d’être difficile. En
attendant j’étais dans un endroit fabuleux et le repos inattendu dont je
bénéficiais me permettais d’en profiter pleinement. Dave avait piqué une
tête dans le lac mais nous étions trop frilleux pour le suivre, malgrés
un besoin réel d’un bain. La hutte avait des ressources inattendues : non
seulement il y avait une bonne provision de livres, ce qui est assez
classique, mais en plus il y avait de la nourriture, visiblement amenée
par un groupe déposé en hélicoptère. Je ne citerais pas ici les dates de
péremption de ce que nous avons consomé, de peur de vous affoler, mais
certains produits étaient tout à fait mangeable. Le soir nous avons eut
double ration de pâtes, ce qui est passé sans difficulté.</p>
<p>Le vent s’était levé et la nuit a été terriblement ventée. La petite
hutte tremblait et gemissait sous les bourrasques, ce qui faisait que nous
l’aimions encore plus. J’avais choisi parmis les quatres couchettes une
située en haut, sous la partie du toit en taulle transparente. A notre
surprise le toit était bien étanche, mais le vent s’engouffrait dessous
juste à la hauteur de ma tête et je me suis retrouvé a ne laisser que mon
nez dépasser de mon sac de couchage.</p>
<hr class="docutils" />
<p>Au cours de la nuit j’entendais le temps empirer et je me demandais
quels étaient les plans de Danillo pour la suite. Nous nous sommes levés
vers neuf heures et personne ne parlait de quitter la hutte. C’était le
22 avril, l’anniversaire de Danillo. Il y avait de la farine, du sucre,
de l’huile, de la canelle, et même du chocolat en poudre dans la hutte.
De plus il y avait un réchaud avec du kerosene. Dave et moi, les deux
affamés de service, nous sommes donc attelés à la tache de faire un
gateau. Comme substitut de four nous avons utilisés une marmite que nous
avons suspendu au dessus du réchaud. Notre gateau était plat et nous
avons pu le retourner pour le faire cuire convenablement des deux cotés.
Je l’ai trouvé trés réussi ! Aprés le déjeuner nous avons fait un défillé
de mode avec les habits trouvés dans la hutte puis nous nous sommes
plongés dans de la lecture. J’avais trouvé dans la hutte un Steinbeck que
j’adore et le mauvais temps était une bénédiction. Nous ne pouvions
cependant rester indéfiniment dans ce refuge car nos provisions étaient
limitées et il y avait encore beaucoup de trajet avant de rejoindre le
lac Te Anau où nous pourrions être évacués. Entre le lac et nous se
dressait la “Takahe restricted area”. Cette région où on ne peut pénetrer
sans autorisation du DOC est le refuge des Takahes. Nous avons contacté
le DOC par radio pour leur demander l’autorisation de sortir par là, au
lieu de notre itinéraire prévu, mais à ma grande surprise ils ne nous
l’ont pas accordé, ce qui nous posait un serieux problème. Aucun d’entre
nous n’avais rééllement envie de franchir encore deux cols et trois
vallées de notre route initial, vu la difficulté des terrains que nous venions de parcourir.</p>
<p>Nous avons choisit un nouvel itinéraire de sortie, contournant la “Takahe
restricted area” par le sud, passant deux jours sur les crêtes avant de
s’engager dans la “Gorge burn” que je connaissais pour y avoir passé un
week end. Le danger d’un tel itineraire était que du mauvais temps sur
les crêtes ne pardonnerait pas.</p>
<hr class="docutils" />
<p>Le matin nous a apporté un ciel couvert mais sans pluie ni grand vent.
Le repos nous avait revigoré et nous sommes partis à l’assaut de Mont
Irene. A mi chemin James nous abandonne, nous conseillant de s’entendre
avec Danillo sur une heure limite de descente. Dave et moi décidons de ne
pas s’arréter pour déjeuner, nous vengeant sur notre provision de
gouters pendant les pauses pour attendre Danillo. En effet nous foncions
car il était hors de question de descendre de nuit. L’ascension n’a pas
posé de véritables difficultés. Les nuages avaient hélas entouré le sommet,
réduisant la visibilité à quelques dizaines des mêtres et nous forçant à
laisser des kerns dans les pierriers pour marquer la descente. Au sommet
une minuscule cahutte abritait de l’équipement radio compliqué ;
probablement un relais utilisé par les zoologistes du DOC, mais nous
avons été bien surpris de trouver cela au milieu de nul part. Mont Irene
est le plus haut sommet de Fjorland au sud de Milford, c’est donc un
endroit stratégique pour de tel équipement. Malgrés cela nous n’avons été
que le quatrième groupe à le gravir au cours des dernières 18 années.</p>
<hr class="docutils" />
<p>Le lendemain il nous faut quitter cette fabuleuse hutte. Un fort vent
froid balaye les montagnes et au bout d’une heure nous perdons Danillo
qui a décidé de tenter une différente approche. Nous passons vingt
minutes à l’attendre au milieu d’une crète, nous refroidissant
terriblement. Encore une fois je suis furieux contre lui, même si je
ne le laisse pas paraître. Dans un tel vent tout arrêt implique un
refroidissement trés rapide et dangeureux. Il est classique dans de
telles conditions de se retrouver avec de la transpiration gelée sur les
vêtements. Le vent en question vient directement de l’Antartique et la
situation me rappellait des descriptions que j’ai lu des régions
sub-polaires, où le vent est l’un des pires ennemis de l’homme. Ce matin
là c’était clairement ma préocupation principale. Heureusement quand nous
arrivons sur les crètes il a nettement diminué, et nous sommes au dessus
des nuages, bénéficiant donc de la chaleur du soleil. Un paysage
grandiose nous attend : quelques montagnes grises aux paroies quasi
verticales dépassent d’une mer étincellente de nuages tandis que la crête
sur laquelle nous nous trouvons est couverte d’une fine dentelle de gel.
Les quelques brins d’herbe sont blancs de neige sans pourtant être couchés.
Une ravine descend quasiment à pic de l’autre coté, son flanc à l’ombre
est bleutée et brillant : une couche de cristaux de glace recouvre le
rocher.</p>
<p>L’air se réchauffe vite au fur et à mesure que le vent se calme et nous
pouvons nous arréter pour déjeuner avec la valley de l’Irene à nos pieds.
Je sors enfin le gateau au chocolat et au brandy que je transporte depuis
le début. Je serais content de me débarrasser de ce poid qui me fait
envie depuis le début. Les autres se moquent de moi “This is a guy who
doesn’t bring his camera but carries a chocolat cake in his pac”.
N’empèche qu’ils l’ont apprécié, mon gateau au chocolat. Cette balade
sur les crètes est de toute beauté et la progression est
particulièrement rapide. Le soir nous établissons la tente à coté d’une
marre, derrière un col. Le sol est un peu en pente et j’ai la chance
d’avoir un petit rocher où je peux poser mes pieds pour me caler. J’ai
tout de même dormit en révant que j’étais debout.</p>
<hr class="docutils" />
<p>Au lever le soleil dépasse à peine de l’horizon tumulteux et un ciel
plein de couleurs se reflète dans la marre. Celle-ci est d’ailleurs
partiellement gelée : le froid est le prix à payer pour avoir une telle
vue au levé. Laver ce satané de porridge dans une eau à 0 degrés est un
vrai supplice et je dois m’y reprendre à trois fois, réchauffant mes
mains entre temps, mais c’est moi qui transporte la casserolle donc je
suis motivé.</p>
<p>Nous continuons sur les crètes, mais aujourd’hui nous sommes un peu
plus bas et le terrain est moins alpin. Nous sommes même obligés de
descendre un peu pour éviter un pic trop raide. C’est l’occasion de
regoutter à la “scrub”. Dans la remontée mon infection à l’os du pied me
tourmente vraiment. Depuis quelques jours elle rend toute montée pénible.
Je suis content que la fin approche. Danillo ne peut s’empécher de partir
à l’assaut du pic ; mais nous continuons, au dessus d’une crète et
jusqu’à deux lacs qui marquent l’entre de la vallée que nous suivrons
pour sortir. La descente dans la vallée se termine avec la corde : nous
n’avons pas la carte et n’avons pas vraiment choisi le meilleur chemin.
Nous profitons du calme et du soleil sur le bras de terre entre les deux
lacs en attendant Danillo. L’endroit où nous nous tenons est le point le
plus proche de Mont Irène qu’a atteind la dernière expédition qui voulait
tenter son ascension : bloqués par la neige ils ne sont jamais sortis de
la vallée.</p>
<p>Nous nous enfonçons à nouveau dans la brousse. C’est presque agréable
de changer de terrain. La progression semble facile aprés les éboulis et
la scrub. Nous campons à coté de la rivière. C’est le dernier jour et
nous finissons les légumes et la viande lyophilisés ! C’est une vraie
fête à coté d’un feu de camp. C’est en conversant autour de ce feu que
nous nous rendons compte combien la forêt ici est sûre : pas d’anacondas,
de crocodiles, d’ours… Les seuls dangers sont les arbres, les rivières
et les rochers. J’apprend que les seuls mammifères de Nouvelle Zélande
sont des chauves souris. En fait presques toutes les espèces indigènes
sont des oiseaux.</p>
<hr class="docutils" />
<p>Le dernier jour ! Nous nous levons tard, huit heures ! Il nous faut
juste contourner le lac boomerang et nous sommes en terrain connu.
Malheureusement ce lac est typique d’un lac de Fjorland : coincé dans une
vallée avec des paroies à pic il en occupe toute la partie plate. Il faut
donc le contourner à travers d’amusantes pentes sur lesquelles il paraît
aberrant de trouver de la forêt tellement elles sont raides. En deux
emplacement le sol a cédé, et la forêt c’est retrouvée au fond du lac,
laissant le rocher nu. Traverser ces falaises glissantes est un peu
vertigineux, surtout que les prises sont nombreuses, mais restent entre
nos mains. Je suis blasé par les différents passages exposés des derniers
jours et je passe donc sans trop de difficultés, mais il faut transporter
le sac de Danillo qui met une demi heure avant de se décider à traverser.
Le résultat est que nous sommes terriblement en retard : nous avons
rendez-vous avec un bateau à l’embouchure de la rivière, sur le lac Te
Anau à 16h, et si nous loupons le bateau nous sommes condamnés à dormir
sur place et à utiliser la radio pour le rapeller. Nous avons bien sûr
gardé des pâtes et du porridge dans cette éventualité mais personne
n’a une envie folle de dormir à coté du lac Te Anau pour le plaisir.</p>
<p>Une fois le lac contourné James et moi connaissons le chemin. Nous
prévenons Danillo que le rythme va être élevé et que nous tenons
absolument à arriver à l’heure. Une sucession de sentiers de rennes nous
amène rapidement ( quelques heures de marche effreinée ) au dessus du lac
Te Anau. Nous accusons cependant la fatigue des derniers jours et ce
sprint nous a crevé. La brousse ne nous semble pas aussi facile que ce
dont nous nous souvenions. Nous tombons tous souvent. Je me souviens de
cet arbre, tombée parallèlement à un torrent encaissé. Pour le passer
James c’était mis debout dessus avant de se laisser descendre deux bons
mètres dans le ruisseau, s’accrochant aux fougères. Je l’ai regardé en me
disant que c’était risqué, mais à moins de se retourner et perdre en
temps fou à désescalader c’était la seule solution. Le ruisseau était
bien sûr plus profond qu’il ne paraîssait et en descendant je suis tombé
sur l’avant de la cheville, rajoutant une tache rouge sur mon pantalon
déjà coloré. J’avais à peine fait dix mêtres à la poursuite de James qui
cavallait devant que j’entend un choc sourd et un juron lancé avec un fort
accent américain. C’est Dave qui a tenté de désescalader. Le flanc du
ruisseau a laché sous ses pieds et il est tombé trois mêtres en arrière
sur le dos. Heureusement les sacs à dos sont encore bien pleins, et cela
amorti.</p>
<p>Enfin nous nous retrouvons sur la plage du lac Te Anau, a nous faire
dévorer par les sandflies en attendant le bateau. Danillo, qui connait le
type nous avait promis qu’il serait en avance mais nous ne voyons
personne. Loin sur la droite un bateau avance lentement vers le fond du
fjord, rasant la côte. D’ici quelques temps il sera sur nous. Danillo lui
fait des grands signes et finallement le bateau accélère et vient
accoster à coté de nous. Ce ne sont que des plaisanciers, mais ils nous
proposent de nous ramener à Te Anau. Notre bateau n’est toujours pas là
et nous craignons que le type n’ait confondu “Gorge burn” et “George
burn” à la radio. Dans le doute nous acceptons l’offre, aprés beaucoup
d’hésitations, nous disant que nous croiserons forcement le bateau dans
l’étroit fjord s’il arrive en retard. Nos hôtes sont un jeune couple
d’australiens et leur ami propriétaire de la vedette, un local. Nous les
amusons beaucoup, avec nos habits couverts de boue, nos crampons et nos
piolets, et notre sourire béat à l’idée de retrouver la civilisation.
Oui, dix jours c’est long, mais c’était absolument génial. Des bières
apparaissent par magie tandis que la vedette fonce vers Te Anau. A mis
chemin le premier reservoir de carburant se vide : il faut commuter vers
le deuxième. C’est là que nous réalisons qu’une autre vedette nous
poursuit depuis le fond du fjord : c’est notre bateau. Il était
tranquilement en train de pécher au fond du fjord quand il a vu l’autre
vedette charger ses passagers. Tout est bien qui fini bien, nous
remercions nos hôtes et nous embarquons sur l’autre bateau. Le pilote est
un kiwi bourru qui s’avère sympatique si on sait bien le prendre.</p>
<p>A la cale j’ai la bétise de sauter à l’eau pour l’aider à mettre le
bateau sur la remorque. Les autres attendent qu’il soit sorti de l’eau.
Je ne veux plus mettre mes chaussures mouilées et à Gore, où nous nous
arrétons pour un “fish and chips” je me balade pieds nus dans le froid.
Qu’importe, je suis heureux. A la station service il y a une balance pour
peser les bonbonnes de butane. Je me suis pesé dessus au départ et je
tiens à comparer. J’ai perdu cinq kilos ! Je vous rassure, je les ai
regagné depuis.</p>
<hr class="docutils" />
<p>Hier nous nous sommes retrouvés chez Danillo pour regarder les diapos
que James et Danillo ont pris. J’ai amené un gateau au chocolat et au
brandy. Depuis j’ai emprunté les diapos de James, dans l’espoir de
trouver un scanneur de diapos dans l’unniversité. Je me les suis projeté
seul dans la salle de conférence de physique. C’était fabuleux ;
absolument crevant mais si c’était à refaire je me relancerai dedans tête
baissée. Le week end dernier, celui où j’étais censé aller sur Secretary
Island j’ai annulé ma participation, et je suis resté en ville pour que
mes pieds guérissent. Un week end en ville et j’ai l’impression d’être
inactif, oisif, je suis impatient de repartir, je ronge mon frein.
Heureusement ce week end je pars dans la vallée “Makaroa”, nous allons en
terain alpin, avec probablement un bivouac sur la neige. Cela va être
froid mais mon dos se plaind que le matelas de ma chambre est trop
confortable, mes yeux se plaignent des rideaux que je vois en me
réveillant… Dormir dans le froid et marcher dans la brousse me
manquent.</p>
What the local newspaper writes on the French2003-04-10T00:00:00+02:002003-04-10T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2003-04-10:/personnal/what-the-local-newspaper-writes-on-the-french.html<p><em>In the “Otago daily times”, lost in NZ’s South island</em></p>
<a class="reference external image-reference" href="https://gael-varoquaux.info/personnal/attachments/nz/Juliet.jpg"><img alt="Enorme !" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/Juliet.jpg" style="width: 100%;" /></a>
<p>Sans commentaires (autre que j’ai trouvé cela très bon)</p>
La “Milford Track”.2003-04-08T00:00:00+02:002003-04-08T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2003-04-08:/personnal/la-milford-track.html<p>Ce week end j’ai fait la plus célebre rando du monde. Il s’agit du
“Milford Track” ; j’en avais entendu parlé même en France et il est
tellement célébre qu’il avait fallut que je le réserve dès mon arrivée
ici. Il y a bien longtemps (cela veut …</p><p>Ce week end j’ai fait la plus célebre rando du monde. Il s’agit du
“Milford Track” ; j’en avais entendu parlé même en France et il est
tellement célébre qu’il avait fallut que je le réserve dès mon arrivée
ici. Il y a bien longtemps (cela veut dire que je ne connais pas la date)
deux explorateurs, Sutherland et Mackinnon, remontant la vallée du
“Milford Sound”, le fjord le plus encaissé de Nouvelle Zélande, on
découvert les “Sutherland falls”, chutes qui on été crut les plus hautes
du monde pendant bien longtemps. Elles ont attiré tellement de monde que
un chemin a été taillé allant du lac Te Anau jusqu’à Milford (qui n’était
alors accessible que par la mer). C’est la génèse du “Milford Track”,
batisé “Greatest walk on earth par un poète Londonnien du début du
sciècle. Maintenant je dois avouer que c’est une gigantesque attraction
touristique ; Dieu merci le nombre de randonneurs est limité à une petite
centaine par jours (il faut quatres jours pour le parcourir).</p>
<p>Quelques jours avant de partir pour le “Milford Track” j’ai découvert
que l’une de mes colocataires, Louise, avait été guide dessus. Elle m’a
fillé tous les tuyaux, et m’a entre autre motivé pour traverser un col
sauvage pour y aller, et non prendre le traine-couillons ( = bateau ) à
travers le lac pour y aller. J’avais entendu parler de cette passe, mais
elle était réputée difficile.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/eglinton.jpg" style="width: 40%;" />
<p>Je me rends donc en stop à Te Anau où je déclare au bureau du DOC mon
intention de passer par “Dore Pass”. On cherche à me décourager : la
météo est dégeulasse (un affreux front froid se profile à l’horizon), et
la passe est alpine. Je ne me laisse pas décourager car j’ai des amis qui
l’ont fait il y a trés peu de temps (pas pour se rendre sur le Milford,
néanmoins). Je fais donc du stop jusqu’au pied de la passe, traverse une
première rivière (j’espère que vous n’imaginez quand même pas qu’il y a
des ponts) puis plante ma tente en prévision d’une pluie torrentielle.
J’utilise une technique local trés efficace : une toile extérieure de
tente (tent-fly) tendue entre des arbres comme premier abris, et mon sac
de bivouac dessous. Comme prévu le déluge arrive au milieu de la nuit,
mais malgrés la pluie je suis quasiment certain d’avoir entendu des kiwis
s’appeller dans la brousse. Je me suis tellement obsédé par cette pluie
que j’oublie de parler de la vallée : “the Eglinton valley”. C’est une
vallée coincée entre les montagnes à pic de Fjordland, et les chaines
montagneuses de la région de “Mont Aspiring”. Au fond coule l’Eglinton,
disparaissant dans la brousse et réapparraissant sur de petites plaines.
Je l’ai traversé à la sortie de la brousse, devant moi une étendue
d’herbe sauvage, puis les montagnes de Fjordland, un mur, véritablement,
avec une courte vallée montant vers la passe qui disparaissait dans la
brume : 1350m, rien à l’échelle des Alpes, mais ici les conditions sont
plus rûdes.</p>
<hr class="docutils" />
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/glade-valley.jpg" style="width: 40%;" />
<p>Le lendemain matin je me suis donc mis en marche à travers une brousse
trampée qui me mouillait encore plus que la pluie elle même. Le chemin
était indiqué, bonne surprise : même si on me l’avait dit je n’y croyait
qu’à moitié. Il montait trés raide et au bout de trois heures je sors de
la brousse. Il me faut traverser un torrent. Le problème n’est pas le
froid, il ne descend même pas d’un glacier et donc n’est pas si gelé que
cela, mais le courant. La plus grande cause de mort dans l’arrière pays
Néo-Zélandais est la traversée de rivière. Bon visiblement je ne suis pas
mort. Des batons marquent les chemins ; l’hiver ils dépassent de la
neige.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/dore-pass-stream.jpg" style="width: 20%;" />
<p>Il pleut des cordes et la visibilité diminue au fur et à mesure que
j’avance. Je n’ai toujours pas vu le col. Toute la vallée dégouline d’eau
: des chutes d’eau rebondissent des deux cotés et plus loin une bonne
dizaine de torrent descendent du col. C’est absolument superbe, même si
les conditions sont un peu effrayantes. La pente se fait de plus en plus
raide. La visibilité et à peine d’une centaine de mètres et au bout d’un
moment je perd les marques du chemin de vue. Je continue donc dans le
même direction : droit dans la pente. Ma progression est trés lente : la
pente est assez instable et je dois lutter pour trouver des appuis
corrects, de plus monter une telle pente m’essoufle et m’oblige à
m’arréter tous les vingt mêtres pour reprendre mon souffle et chercher le
chemin. Je dois escalader une chute d’eau. Le sol est un mélange de
rocher d’herbe et de pierre. Le tout est rempé et trés glissant, mais
l’herbe tient bien et comme souvent dans Fjordland on s’en sert de prise
de main. Je vois le col qui se profile au dessus de moi. Ou tout du moins
un col : je commence à être certain que ce n’ai pas le bon chemin, mais
je n’ai pas du tout l’intention de redescendre cette chute d’eau si je
n’en ai pas l’aboslue nécessité.</p>
<p>Arrivé en haut je ne vois absolument rien plus haut ! La pente est trés
raide de l’autre coté. Je m’abrite derrière un rocher, plus de la pente
que du vent ou de la pluie, et je sors mon compas et ma carte.
L’orientation de la crète me situe un peu. De toute façons pour trouver
des pentes plus douces il faut que je parte sur ma droite, d’aprés la
carte. Des trous se forment dans la brume et j’apperçoit un peu plus loin
: il y a indéniablement une falaise devant moi, elle n’était pas sur la
carte, mais les cartes ici ne sont pas d’une précisions absolue. Je me
dirige donc lentment sur ma droite, descendant quand le terrain le
permet, mais je comprend vite qu’il faut que je retrouve le chemin si je
veux sortir. Je pense appercevoir la passe au dessus de moi, mais
toujours aucune marque du chemin. La visibilité a augmenté et je devrais
maintenant pouvoir appercevoir les marques du chemin. Je me remémore le
topo que j’ai consulté avant de partir : le chemin part sur la droite
aprés le col pour éviter des falaises. Ce n’est que trop tard que je
comprend qu’il monte pour éviter les falaises en question : elles me
barrent le passage et je suis obligé de grimper une pente
inconfortablemnt raide. Finallement j’apperçois un poteau : j’ai retrouvé
le chemin. Je ne peux m’empécher d’être soulagé. La suite est sans
difficulté : le chemin n’est pas trop mauvais ( à l’échelle de Fjordland
) et je suis sortie trois heures plus tard : il ne m’aura fallut que 7
heures pour franchir la passe, comptant une heure où j’étais perdu. Le
DOC comptait 8 heures, je ne sis pas mécontent de moi.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/dore-pass-side.jpg" style="width: 20%;" />
<p>J’arrive donc aprés une descente à travers la brousse et une autre
traversée de rivière à “Glade House”, la première hutte du Milford Track.
Elle n’est pas pour moi : elle accueuille les randonneurs avec guides :
un hélicos maneuvre pour amener des canettes de “Speight : Pride of the
South”, la bière locale. Par la fenêtre je vois des chambres doubles avec
douches chaudes.. Heureusement la pluie à cessé et je peux me mettre en
marche sur le chemin le plus large que j’ai vu dans Fjordland jusqu’ici :
on peu marcher à deux de front. Au bout d’une heure je suis à ma propre
hutte. C’est un luxe fabuleux comparé aux huttes d’arrière pays
auxquelles je suis habitué. Elle accueuille quarante randonneurs qui ont
reservé il y a des mois. Ils viennent de partout dans le monde (il y a
même trois kiwis !) A sept heure une petite bonne femme absolument
chamrante se présente comme la gardienne et nous fait un petit discourt
sur la sécurité et l’entretient de la hutte. Elle s’encquiert aussi de ma
présence. Cela fait plaisir de voir que dans ce pays si on disparaît une
équipe de secours partira bien vite (j’avais spécifié comme “panic day”
le lendemain).</p>
<hr class="docutils" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/clinton-river.jpg" style="width: 30%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/clinton-valley.jpg" style="width: 30%;" />
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/mackinnon-mountains2.jpg" style="width: 20%;" />
<p>Je me couche tôt et savoure une nuit fabuleuse (aprés un trés bon
repas, bien sûr). Le lendemain je me lève le plus tard possible: il n’y a
que 4 heures de marche à plat jusqu’à la prochaine hutte. Il pleut des
corde pendant les deux dernières heures et je m’amuse à admirer les mines
dépités des touristes etranger arrivant à la hutte. Ils laissent bien sûr
leurs chaussures dehors, une trés bonne idée en principe, mais dans les
montagnes il y a un perroquet trés joueur, le “Kea”, qui aime voler ce
qu’il peut. Un tel Kea sévit un peu plus tard mais n’arrive pas à
s’envoler avec le baton de marche qu’il avait saisit. Le materiel rentre
bien vite. Encore un bonne nuit mais le matin je pars dés le levé du
jour. Je veux franchir la passe Mackinnon tôt pour avoir du temps de
l’autre coté. Le brouillard est épais et arrivé à la passe on ne voit
rien. De plus le vent est gelé et je sors vite mes trois polaires et mes
gants. Je me cache pendant deux heures dans un petit refuge au col en
attendant que cela se dégage, mais il n’y a rien à faire et je repars
donc vers la vallée. Les Sutherland falls m’attendent. Le soleil est
apparu et c’est un impressionnant get de blanc fracassant qui s’offre à
moi. Je passe derrière, comme Louise me l’avait conseillé. C’est
magnifique de voir cette gerbe devant soit. Le bruit est assourdissant,
et bien sûr je suis trempé, mais qu’importe !</p>
<hr class="docutils" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/lake-mintaro.jpg" style="width: 30%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/lake-mintaro-2.jpg" style="width: 30%;" />
<p>Troisième hutte, troisième nuit, puis petite marche pour sortir de la
vallée. En chemin je sors de sous les fourrés des kayaks dont Louise
m’avait indiqué l’emplacement. Sous les yeux ébahis des mes compagnons je
me promène sur le lac Ada, mais il faut vite que je reparte : un bateau
nous attend pour nous faire traverser le Fjord et nous enmener à Milford.</p>
<p>Le fjord est impressionant : les montagne plongent à pic dans l’eau. On
a la sensation que tout est soit plat est calme comme l’eau sur laquelle
on glisse, ou tourmenter et vertical, comme ses falaises impossible où
s’accrochent malgrés la pente des arbres tordus.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/milford-sound.jpg" style="width: 40%;" />
<p>Un bus nous ramène à Te Anau à travers une route construite il y a
cinquantes ans. C’est encore une nouvelle face de Fjordland que je
découvre : des montagnes qui partent d’une ancienne vallée glacière, avec
sa forme en U si particulière, pour dominer avec des pics, des glaciers,
des cols, bref tout ce qui fait une montagne digne de ce nom, comme on
peut en voir dans les alpes.</p>
<p>La richesse de ce pays, sa variété m’impressionne. Je suis tombé
ammoureux de Fjordland : ses vallées cachées, ses lacs oubliés, ses cols
infranchissables, ses pics pointus, ses prairies inattendues, ses
rivières tumultueuses, ses chutes rugissantes… Tant de paysages
différents, et pourtant membre d’un tout formidable, indompté et
surhumain. J’ai appris que cette région était classé patrimoine mondial
de l’Unesco, j’espère que cela lui permettera de garder sa magie.</p>
Le “RouteBurn track”2003-04-04T00:00:00+02:002003-04-04T00:00:00+02:00Gaël Varoquauxtag:gael-varoquaux.info,2003-04-04:/personnal/le-routeburn-track.html<p>Il y a dans ce chouet pays quelques chemins (“track”) qui vont
d’un ilot de civilisation à un autre à travers la brousse. Ce sont les
seuls chemins réelement marqués et parcourus. Parmis ceux-là quelques-uns
sont trés célébres, ce sont les “great walks”. Il faut les réserver et le …</p><p>Il y a dans ce chouet pays quelques chemins (“track”) qui vont
d’un ilot de civilisation à un autre à travers la brousse. Ce sont les
seuls chemins réelement marqués et parcourus. Parmis ceux-là quelques-uns
sont trés célébres, ce sont les “great walks”. Il faut les réserver et le
camping sauvage n’est pas permis à leur voisinage. Ces deux derniers week
end j’en ai fait deux.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/route-burn.jpg" style="width: 20%;" />
<p>Tout d’abord il y a une semaine le “Routeburn track”, qui descend de
Glenorchy vers Fjordland. Ma sortie d’alpinisme programmée pour se week
end est tombée à l’eau alors je me suis joins à un groupe de physique
atomique qui allait sur le Routeburn (des jeunes, bien sûr, dont Stephan
et Cluso). Nous avions en fait deux groupes, parcourant le chemin dans
chacun des sens et échangeant les clef des voitures au milieu. Comme je
savais que le rythme ne serait pas trop effrayant j’ai organisé un
concours de dinner avec Cluso, qui allait dans l’autre sens : nous nous
retrouvions au camping le soir et celui qui faisait le meilleur dinner
gagnait, bien sûr il s’agissait de cuisiner à nous deux pour le totalité
des deux groupes : 8 personnes. Je suis donc parti avec un sac à dos
monstrueux : lait frais, vin, boites de conserves, légumes frais…</p>
<p>La rando elle même a été trés jolie : une jolie vallée, puis un jolie
col, avec un petit sommet à coté du col et une vue jusqu’à la mer
Tasmienne… J’ai des photos (pour une fois j’ai pris mon appareil), mais
elles ne sont pas encore scannées.. Le dinner a été monstrueux, on avait
même trop de vin (car Stefan en avait aussi pris), et on a bien dormit
aprés.</p>
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/harris-mountains.jpg" style="width: 35%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/lake-harris.jpg" style="width: 35%;" />
<hr class="docutils" />
<img alt="" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/hollyford-valley.jpg" style="width: 80%;" />
<hr class="docutils" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/divide.jpg" style="width: 25%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/routeburn-falls.jpg" style="width: 25%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/fjordland-moutains.jpg" style="width: 25%;" />
Tunnel beach2003-03-26T00:00:00+01:002003-03-26T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-26:/personnal/tunnel-beach.html<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/tunnel-beach-rocks.jpg" style="width: 25%;" />
<p>Hier je suis retourné à tunnel Beach. Je voulais aller en au de Mt
Cargill pour voir le coucher de soleil et les nuages envahir la montagne,
mais à 4 heures les nuages étaient déjà là, donc j’ai pris mon vélo (et
mon appareil photo, cette fois) et j …</p><img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/tunnel-beach-rocks.jpg" style="width: 25%;" />
<p>Hier je suis retourné à tunnel Beach. Je voulais aller en au de Mt
Cargill pour voir le coucher de soleil et les nuages envahir la montagne,
mais à 4 heures les nuages étaient déjà là, donc j’ai pris mon vélo (et
mon appareil photo, cette fois) et j’ai foncé vers tunnel beach. J’ai été
me placer au bout de la petite presqu’ile, cinq mêtres au dessus des
vagues qui venaient se briser sur la falaise. On voit d’abord l’onulation
venir du large. Au fur et à mesure qu’elle approche elle grandit, prend
la forme d’une vrai vague, puis d’un rouleau. Enfin elle brise : elle se
dresse, se courbe, un peu de blanc apparait au sommet, et se propage vers
la droite. Trés vite c’est toute la vague qui est blanche, grondante, se
précipitant vers la falaise sous moi. Je dis se précipitant, mais tout se
passe assez lentement. La vague, avec ses tonnes d’eau, s’écrase contre
la pointe, montant le long du rocher, puis se brisant dans l’air,
laissant de petits morceaux d’écume qui retombe dans l’eau tandis que
cette explosion blanche avance le long de la falaise, ricochant entre les
parois, donnant naissance à d’autres vagues qui viennent aussi se briser
sur les rochers à tel point qu’on observe un feu d’artifice de blanc, feu
d’artifice qui retombe bien vite pour laisser la mer tout aussi calme…
jusqu’à la prochaine vague.</p>
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/tunnel-beach-cliffs.jpg" style="width: 40%;" />
<img alt="" src="https://gael-varoquaux.info/personnal/attachments/nz/tunnel-beach-peak.jpg" style="width: 40%;" />
A travers la brousse de Fiordland : la “Gorge Burn”2003-03-25T00:00:00+01:002003-03-25T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-25:/personnal/a-travers-la-brousse-de-fiordland-la-gorge-burn.html<p>Ce week-end : Fiordland, un voyage organisé par le “tramping club”
composé d’une quinzaine de sortie. Pour ma part j’avais choisi celle
d’Adrian, car c’était celle qui s’enfonçait le plus profondement dans
l’étendue sauvage qu’est Fiordland.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/te-anau-dock.jpg" style="width: 40%;" />
<p>Vendredi soir je m’installe donc à l …</p><p>Ce week-end : Fiordland, un voyage organisé par le “tramping club”
composé d’une quinzaine de sortie. Pour ma part j’avais choisi celle
d’Adrian, car c’était celle qui s’enfonçait le plus profondement dans
l’étendue sauvage qu’est Fiordland.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/te-anau-dock.jpg" style="width: 40%;" />
<p>Vendredi soir je m’installe donc à l’avant d’une camionette remplie de
5 Américains, 1 Suisse, 1 Suédois, et Adrian et moi. Comme d’habitude les
américains sont super sympas, super sûr d’eux, parlent trop fort, et
trimbalent de gigantesques sac à dos avec des poches et des sangles
partout. Dinner dans Gore puis arrivé à coté du lac Te Anau vers onze
heure. Nous commençons à planter la tente mais nous sommes délogés par
les flics, donc nous partons plus loin et dormons dans un coin paumé. La
nuit a été trés froide, et je n’avais pris que mon sac de couchage
léger..</p>
<p>Le lendemain matin nous mettons le bateau d’Adrian à l’eau et
traversons le lac Te Anau pour aller nous poser au fond du fjord du Sud
du lac. Il fait trés beau et pénétrer dans cette langue d’eau entourée
par des pentes quasi verticales couvertes d’une végétation luxuriante me
fascine tout autant que la première fois. J’ai l’impression de péntrer
dans un autre monde, un monde où l’homme est une fourmi, un monde
totallement vierge.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/fowler-valley.jpg" style="width: 40%;" />
<p>Dés que nous posons pied à terre nous sommes assaillis par les
sandflies, un seul échappatoire : se mettre en marche. Il n’y a pas de
chemin : nous sommes dans une région qui ne voit pas souvent l’homme.
Adrian est cependant assez doués pour trouver les chemins qu’empruntent
les cerfs. Le groupe est trés grand pour une brousse aussi dense, et
les autres chefs de groupe ont pariés que nous n’arriverons pas à
destination. Adrian me demande de rester à l’arrière pour faire le serre
file : nous ne devons jamais nous perdre de vue.</p>
<p>La brousse est trés sèche : nous ne nous mouillons pas a son contact,
la mousse qui couvre les racines n’est pas trempé, il n’y a quasiment pas
de boue. C’est trés rare ici, il pleut 9 mètres par ans. La progression
est lente, 1 km/h, mais c’est normal. Nous remontons une gorge,
contournant plusieurs lacs. Le but est d’arriver à un col, 600m plus
haut. Pour cela nous montons lentement le long de la rivière jusqu’à un
replat au niveau du lac “boomerang”, puis nous partons sur le coté, dans
la pente vers le col.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/up-gorge-flat.jpg" style="width: 40%;" />
<p>Le topo est alors trés imprécis, la pente raide et les sentiers de cerf
disparaissent. Nous nous hissons avec les mains grace aux arbustes qui
dépassent de la pente. A l’avant ils doivent se battre pour se frayer un
chemin. Je vois des sacs à dos et des pieds au dessus de moi, qui
disparraissent dans la brousse. S’il se met à pleuvoir, nous aurons
besoin d’une corde au retour. Au bout d’une cinquantaine de mètres nous
tombons sur une barrière rocheuse. Nous la longeons mais elle s’étend
loin, nous n’en voyons pas le bout. Nous avons encore une heure de jour,
mais je n’aime pas beaucoup l’idée de camper dans une brousse aussi
épaisse et il faudrait trouver un endroit suffisement plat pour nos
grosse fly tents. D’un commun accord Adrian et moi décidons de
redescendre au lac Boomerang. A la descendre il nous faut contourner une
autre barre rocheuse et qui est apparu sous nous alors que nous longions
la première.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/gorge-burn-creek.jpg" style="width: 30%;" />
<p>Nous installons les tentes prés du lac et nous commençons à ramasser du
bois pour faire un feu, activité assez rare dans Fiordland. Je pars en
reconnaissance pour essayer de trouver un chemin vers la passe. Je
m’engage le long d’un ébouli qui monte à l’assaut d’un pic avoisinant, sur
le coté je trouve le lit d’un torrent avec peu d’eau, il se sépare et une
branche se dirige vers la passe. Je suis alors au dessus de la corniche
qui nous a arrété, et j’apperçois à travers les arbres le jour, sur ma
droite, cinquante mêtre au dessus. Cela me semble une possibilité, mais
il est trop tard pour que j’aille plus loin.</p>
<p>De retour au camp nous nous faisons un excellent repas : Adrian
travaille dans le département de nutrition, et je ne suis pas certain que
cela soit un hasard. Le feu de camp nous éclaire et la nuit n’est pas
froide, nous pouvons donc rester debout longtemps et avoir une soirée
agréable. Lorsque je vais au lac pour me laver les dents je découvre un
payasage silencieux, baigné de la douce lumière de la lune. La lumière du
feu, et sa chaleur, crééent comme des murs qui nous aveuglent et nous
font perdre conscience du calme et de la magie de la nature qui nous
entoure.</p>
<p>La nuit a été confortable et le lendemain matin je bénissais la
moustiquaire de mon sac de bivouac, regardant enmitouflé dans mon sac de
couchage les autres essayer de dormir sous les attaques des sandflies. Il
a plut, certe, mais nous n’avons pas été mouillé.</p>
<hr class="docutils" />
<p>Nous partons, abandonnant les sacs sous une tente. Je mène la marche,
retraçant mon chemin de la veille. J’ai un peu d’inquiétude quand à la
suite : mon itinéraire va-t-il aboutir ? La brousse se révèle peu dense,
et nous tombons vite fait sur des sentiers de cerfs. En une demi heure
nous sommes au dessus des arbres et dans la vallée qui mène au col : les
difficultés sont passées. La vallée est de toute beauté : un torrent
coule au fond, reliant plusieurs petites mares. Au dessus une grande
plaque rocheuse s’étend sur plusieur centaines de mêtres, quasiment
verticale, jusqu’à un sommet. On apperçoit le lac boomerang en bas, et
derrière une autre pente boisée, entrecoupée de barres rocheuses, qui
mène à un petit lac enfermé par des falaises à pic.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/group-fowler-pass.jpg" style="width: 40%;" />
<p>Le col lui même est situé au confluant de trois vallées. On peut voir
le long d’une d’entre elle profond dans les étendues sauvages de
Fiordland. Le payasge est un mélange de rochers, de neige, d’herbe
jaunâtre, de lacs, torrents, cascade, et bien sûr, plus bas, de forêts
luxuriantes. Les appareils photos fusent, mais nous n’allons pas rester
là longtemps : comme nous n’avons pas pris les sacs nous sommes tous en
t-shirt avec un simple coupe vent au dessus. Un amerloque gigantesque,
vétu d’une veste technique orange d’une marque hors de prix dégaine un
appareil photo digne d’un film de science fiction et déclare qu’il veut
être pris en photo avec “the crazy frenchman” ? Depuis que j’ai trouvé un
chemin à travers le rempart de brousse je suis devenu la mascotte de ses
joyeux lurons du nouveau monde.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/down-gorge-burn.jpg" style="width: 40%;" />
<p>La descente se fait sans difficulté, par un chemin similaire à celui
emprunté lors de la montée. Arrivé au lac je presse tout le monde de faire
son sac : les sandflies, absentes pendant la nuit, nous font sentir leur
présence. Adrian reprend la tête de la file, et moi la queue, position
qui me plait car je suis au calme, et nous partons d’un bon pas. Il y a
beaucoup de chemin à faire. Au bout de quelques heures de marche nous
terminons nos provisions : un vrai festin. Le même américain, Kevin, se
retrouve systématiquement devant moi, à l’arrière du groupe. Il passe
son temps à tomber dans les différents trous qui s’ouvrent entre les
racines et les pierres et n’arrive pas à suivre le rythme. Heureusement
la tête s’arrète fréquement, en quète d’un passage au dessus d’un arbre
tombé ou d’un torrent. J’ai tout de même hâte d’en finir : Kevin est
visiblement épuisé et il va finir par se faire mal en tombant.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/lake-te-anau-hills.jpg" style="width: 40%;" />
<p>Nous arrivons enfin à la plage, que nous pouvons longer pour retomber
sur le bateau. Nous embarquons en catastrophe, désireux d’échapper aux
éternelles sandflies. Au bout de quelques centainess de mêtre Adrian
arrète le bateau et plonge dans l’eau. Les autres font de même, mais moi
je suis trop poule mouillée. J’attends que nous repartions et je vais
m’installer sur le capot à l’avant. C’est le retour à la civilisation, de
l’autre coté du lac Te Anau. Ce lac magique marque la séparation entre un
monde dûr, sauvage et indompté, et les plaines de Central Otago qui, bien
que désertes et inhospitalières, sont placés sous le reigne du genre
humain.</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/fowler-pass-top.jpg" style="width: 70%;" />
Le nord de l’ile2003-03-20T00:00:00+01:002003-03-20T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-20:/personnal/le-nord-de-lile.html<p>Vendredi j’avais terminé mes bobines, je les avait câblées et testées,
et j’allais passer à l’optique. Cependant il me manquait des miroirs
pour faire passer les faisceaux, et il fallait les commander, ce qui
signifiait que les prochaines semaines allaient être assez relax, passées
à faire de …</p><p>Vendredi j’avais terminé mes bobines, je les avait câblées et testées,
et j’allais passer à l’optique. Cependant il me manquait des miroirs
pour faire passer les faisceaux, et il fallait les commander, ce qui
signifiait que les prochaines semaines allaient être assez relax, passées
à faire de la lecture, des petits boulots et des petits calculs. Quand
j’ai réalisé cela je suis reparti sur mes grands chevaux : mon projet
pour le week end était tombé à l’eau, pourquoi ne pas le remplacer par
une virée au avec un week-end prolongé.</p>
<p>C’est ainsi que je me suis retrouvé le samedi matin sur le bord de la
route quittant Dunedin vers le Nord, le pouce levé à regarder les
voitures défiler. J’avais un gros sac à dos : l’hivers approche, la radio
n’arrête pas de le répéter. Je suis déposé par un couple dans un petit
bled sur le highway d’où j’ai une superbe vue sur les voitures qui
dépassent à 100 à l’heure. Après une heure je suis pris par deux minettes
de 20 ans, puis c’est une dame ayant la quarantaine, un jeunot dans une
bagnole de sport puant le shite, puis un anglais qui m’amène de l’autre
coté de Christchurch, apportant ainsi la solution à une question qui me
préoccupait : comment traverser Christchurch en évitant de faire vingts
kilomètres à pieds pour aller d’un bout de l’agglomération à l’autre. Il
se fait tard et j’ai encore beaucoup de chemin à parcourir, mais
heureusement je suis vite pris par un Maori. C’est mon premier Maori. Ca
voiture est dégueulasse, remplie de bouteilles de lait vides et de peaux
de bananes. Lui même est un gros type qui se meut avec difficulté, ne
connaissant qu’un adverbe : “bloody” qu’il case à tout grain de sel dans
son discours. Il me propose de m’emmener là où il va, ce qui n’est pas
réellement sur ma route mais comme j’ai oublié ma carte du nord de l’ile
dans la précédente voiture nous devons nous arrêter à une station
d’essence pour m’en acheter une autre. Je choisis de le quitter et de
continuer ma route comme je l’avais imaginé à l’origine. Il me dépose à
l’embranchement, au milieu de la campagne du Canterburry, campagne qui se
désertifie rapidement au fur et à mesure que on s’éloigne de
Christchurch. Une voiture de sport s’arrête assez vite et un type au
crâne rasé me propose de m’enmener “50 km plus loin”. Il ne parle pas
beaucoup mais j’apprend que l’endroit où il quitte le “highway” est sa
maison de vacance. L’endroit en question s’appelle “Domett”, et apparaît
sur ma carte. Je ne prétendrai pas que c’est la plus petite localité que
j’ai vu ici (la plus petite apparaît aussi sur ma carte est est composée
d’un croisement entre de routes et d’un bar), cependant ce n’est pas
grand : un restaurant à vendre, dont on peut admirer les photos sur
internet, et une petite maison de l’autre coté de la route, à vendre sur
internet aussi. Une petite route s’éloigne vers la montagne, où disparaît
la voiture qui m’a déposé.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/bivy.jpg" style="width: 30%;" />
<p>Il est 6h ; j’abandonne le stop, infructueux, à 7h30 pour chercher un
endroit où passer la nuit. Les deux cotés de la route sont occupés par
des prés entourés de fil de fer barbelé avec de temps en temps un petit
bosquet. Je finis par sauter par dessus une barrière pour aller me cacher
dans un bosquet. Comme je n’ai pas assez d’eau pour me faire du riz ou
des pâtes j’improvise vite un repas froid à l’aide de mes boites de
conserve et je me couche à l’abris de mon sac de bivouac.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/kaikoura-truck.jpg" style="width: 30%;" />
<p>Le lendemain matin je me commence dès huit heures à faire du stop, au
milieux de nul part, à regarder les voitures descendre la colline puis me
dépasser à toute allure, au bout d’un certain temps je me demandais
vraiment comment j’allais me sortir de ce petrin quand un gros camion me
fait trois rapides appels de phares, lance un gros coup de son puissant
klaxon, et s’arrête à coté de moi avec un grand vacarme de suspension et
de freins à air comprimé. J’ouvre la porte, grimpe les haute marches,
pusse mon sac à dos devant moi, et pénètre dans une gigantesque et
confortable cabine où un Maori souriant m’accueuille. C’est Rangi ; il
transporte de la nourriture de Christchurch jusqu’à Auckland, et va
prendre le Ferry à Picton. C’est justement l’endroit où je me rend, je
vais donc passer quatre heures, confortablement juché dans cette haute
cabine d’où je peux admirer le paysage. Car le paysage vaut le coup :
nous traversons tout d’abord des petites chaines de montagnes séparées
par de grands plateaux où les quelques rivières creusent des petits
canyons. Des moutons érent dans ce pays sec et désertique. Puis la route
émerge des montagnes pour longer la cote. Comme d’habitude le pacifique
vient se brisser violement sur les spectaculaires rochers des nombreux
caps que nous doublons, entre lesquels s’étendent de longues plages où
les lames defèrlent sur un fond turquoise. La route sinue à travers ce
magnifique paysage, coincée entre les montagnes, qui tombent souvent à
pic dans l’eau, et l’océan.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/malborough-sounds.jpg" style="width: 40%;" />
<p>Nous quittons l’océan pour replonger dans les montagnes, et finallement
Rangi se tourne vers moi : “Well Gaël, this is Picton”. J’étais ravis
d’être arrivé. Je suis allé au centre ville, sur la plage au bout de
Queen Charlotte’s Fjord, j’ai cassé la croute, acheté une carte détaillée
du coin qui m’a permis de décider où je voulais aller, et me voilà
reparti sur la route. Un touriste australian me prend en stop. Il
s’arrête à tous les tournants de la route (c’est vrai que les fjords sont
beaux, mais j’en ai vu de plus beau). Heureusement je ne vais pas loin :
je lui demande de me déposer au niveau d’une petite route de graviers :
elle mène au bout d’une péninsule de laquelle je devrait avoir une belle
vue. Ma chance s’arrête là : aucun des touristes qui prennent cet
embranchement ne veux me prendre en stop. Tant pis, je ne verrais pas le
bout de la péninsule, mais je ne vais certainement pas rester à moisir
ici. Je prend donc mon sac et marche le long de la route jusqu’à ce que
je trouve un chemin qui monte vers la crête centrale de la péninsule. Ce
faisant je croise quelques uns des touristes qui avait bien évités de me
voir lorsque je faisais du stop ; j’espère qu’ils se sentaient gênés dans
leurs belles voitures.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/malborough-sounds-sun-down.jpg" style="width: 30%;" />
<p>Le chemin que je prend grimpe dur. Je suis bien plus au nord que
Dunedin, et il fait nettement plus chaud. Je monte donc sous un grand
soleil, à travers un paysage dévasté par les engins de bucherronnage :
de la forêt de pins (visiblement replanté il y a quelques dizaines
d’années), il ne reste autour de moi que des souches. Les engins ont
remodelé la montagne : les chemins ne correspondent clairement pas à ma
carte, ceux que je vois ne mênent nul part et ont été créés juste pour
évacuer les troncs d’arbre. J’arrive à la crête sans difficulté mais je
n’ose m’engager dans les chemins qui descendent vers l’autre coté de la
péninsule, où je comptais passer la nuit : j’ai trop peur que ce soit des
impasse. Je continue donc le long de la crête, jusqu’à que le chemin
devienne de plus en plus petit et que je quitte la zone où la forêt à été
rattiboisée. J’arrive à ce qui est probablement une tranchée laissée par
un engin en reconnaissance. Elle descend droit dans la pente pendant
cinquante mètres puis s’arrête à un replat : l’engin a dû faire demi. La
nuit est proche et cet endroit me plait : il est loin de tout, me donne
une belle vue sur les fjords, et surtout il est orienté dans l’axe de la
lune, qui va être pleine, et du levé de soleil.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/malborough-sounds-evening.jpg" style="width: 40%;" />
<p>Comme la nuit va être douce je ne me presse pas pour faire mon diner
(encore des boites, il n’y a pas d’eau sur le crête). Je peux donc le
savourer à la lueur de la pleine lune. Je trouve qu’un beau paysage
éclairé par la lune acquière une intensité particulière. C’était
certainement le cas des fjords qui s’étendaient sous mes yeux. J’oriente
mon sac de bivouac de telle sorte que l’ouverture donne sur la pente et
me permette d’admirer le paysage en dormant, par la moustiquaire.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/malborough-sounds-sunset.jpg" style="width: 40%;" />
<p>Le lendemain matin ma montre me réveille juste avant le lever du
soleil et je contemple le paysage qui prend progressivement de nouvelles
couleurs. Je me décide à prendre l’un des chemins qui j’avais boudé la
veille, et me choix se révèle le bon : il s’agissait bien de l’ancien
chemin descendant de l’autre coté de la péninsule. Il me permet de faire
le lien avec une randonnée célèbre : le “Queen Charlotte’s track”. Le
chemin court le long du fjord, dans de la brousse indigène, une superbe
forêt à mis chemin entre nos forêts du midi et une forêt tropicale. Le
chemin est désert et je marche vite : il faut que je sois sorti pas trop
tard pour pouvoir commencer mon trajet de retour vers Dunedin. Assez
soudainement je me mets à croiser une myriade de touristes ; je n’ai
compris que plus tard que il était possible de faire le chemin dans
l’autre sens, allant de gite en gite, les bagages transportés par
autobus. Le chemin croise la route : il est temps que je le quitte et que
je recommence mon périple en stop. Sur cette même route qui m’avait
arrêté la veille je suis pris immédiatemant par un petit vieux qui
conduit une grosse 4x4. C’est un local qui c’est mis dans la tête
d’éliminer une liane parasite. Il la repère de loin, arrête sa voiture et
se lance dans la brousse avec son desherbant à sa scie. Je l’aide un peu
car il n’a pas froid aux yeux quand il s’agit de se lancer dans un tas de
ronce mais a toujours du mal à en sortir. Il m’explique que quelques
jours auparavant il était tombé dans un trou, une lianne autour du pied,
et était resté suspendu la tête en bas pendant vingt minutes avant de
s’en sortir. Il m’offre du thé et du délicieux pain de sa femme. Nous
progressons lentement sur la route et croisons incessamment de gros
camions chariant des troncs : le bucheronnage continue.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/queen-charlotte-sound.jpg" style="width: 40%;" />
<p>Je quitte ce charmant petit père pour une bande de jeunes qui m’offrent
une bière au bled local, et au bout de quelques heures je suis de nouveau
sur cette côte grandiose. J’avais l’intention de dormir sur la plage mais
il pleut ; je m’arrète donc dans un gite.</p>
<p>Dans le gite je rencontre “Maude” (même si je ne peux prétendre être
Harold) : une bonne femme de soixante et quelques ballets qui possède
en tout et pour tout un vélo et ce qu’elle peut transporter avec. Elle
travaille l’été dans une ferme dans le sud, et l’hiver dans une autre
ferme dans l’ile du nord. Elle passe le printemps et l’automne sur les
routes, avec son vélo. Pas mal pour une sextagénaire.</p>
<p>Le lendemain j’ai continué mon périple en stop, restant bloqué plus
de deux heures à deux doigts du but ; cela m’a flanqué une sacrée trouille
de ne pas arriver à Dunedin pour la nuit. Mais je suis arrivé sans
problème à la tombée de la nuit.</p>
Un Week end à “Paradize”.2003-03-10T00:00:00+01:002003-03-10T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-10:/personnal/un-week-end-a-paradize.html<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/adrian.jpg" style="width: 30%;" />
<p>C’était le premier week end organisé par le club de l’année. Il y avait
120 personnes inscrites. Le départ a été un basard monstrueux. Il y avait
une bande hétéroclyte d’ahuris se baladant au point de rendez-vous : des
gros barraques crânant avec des vestes “Mountain Hardware” ou …</p><img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/adrian.jpg" style="width: 30%;" />
<p>C’était le premier week end organisé par le club de l’année. Il y avait
120 personnes inscrites. Le départ a été un basard monstrueux. Il y avait
une bande hétéroclyte d’ahuris se baladant au point de rendez-vous : des
gros barraques crânant avec des vestes “Mountain Hardware” ou “The North
Face”, ce qui les identifiait clairement comme des amerloques ; il y
avait des minettes avec des sacs de couchages loués en jean blanc, et
puis bien sûr les ahuris habituels : Danillo avec son T-shirt rentré dans
son pantalon de survêtement, Adrian, qui avait mis une jolie chemise
bleue, la présidente du club, Joe, qui fait une tête de moi et est deux
fois plus large que moi… Des camionettes de location attendaient sur
les cotés et finallement le bus est arrivé. La foule a commencé à faire
la queue pour charger les sac dans les coffres. Pour ma part je n’étais
pas pressé : j’avais plutôt envie de prendre une camionette que le bus.
J’ai bien faillit ne pas avoir de place du tout : à force d’attendre il
ne restait plus de place pour mon sac dans le coffres du bus (qui étaient
chargés à en déborder), et je m’étais fait doubler pour les camionettes.
Heureusement Adrian avait gardé une place pour ses bières, qu’il a bien
voulu me céder. Je me suis donc trouvé à l’avant d’une camoniette bien
spacieuse à coté de ce bon gros géant, avec derrière dix touristes, la
pluspart américains, et douzes sacs empilés entre les sièges. Nous avons
fait une escale avant de sortir de Dunedin pour charger un lecteur de CD
et des disques. La propriétaire à dû sortir par la fénêtre pour aller les
chercher, la porte étant bloquée par un tas de sac.</p>
<p>Adrian bien sûr ne prenait par la route la plus courte, mais la plus
jolie : un détour par le nord de “Central Otago” vallait largement le
cout. Derrière les américains buvaient du Gin qu’il mélangeaient à du
Sprite, mais n’ayant pas de verres, le mélange devait se faire dans leur
bouche. Au fond deux hollandaises observaient cela d’un air affolé tandis
que Adrian et moi nous regardions d’un air amusé.</p>
<p>Nous avons traversé les collines de central Otago juste avant la tombée
de la nuit. Ce paysage semi desertique parcouru de ravines était la
raison de notre détour. Nous nous sommes arrété pour acheter à manger
dans un pub surgi au milieu de nul par, où nous avons triplé la
clientèle. Les locaux, des fermiers, étaient ravis de nous voir : un
groupe des touristes américains les distrayait plus que le match de rugby
hebdomadaire. Aprés une bière à la santé des locaux nous voilà repartis.
J’avais évité les burgers, préférant des tartines de fromage sur le pain
de je venais de me faire cuire.</p>
<p>Nous roulions depuis dix minutes quans Adrian quitte soudain la route
principale pour pour s’engager sur un chemin de terre sinuant dans une
ravine. Il se retourne pour annoncer le plus vieux pont de Nouvelle
Zélande encore en activité. En effet nous nous engageons, roulant au pas,
sur un vieux pont de bois traversant une gorge dont nos phares
n’éclairaient pas le fond. Dix minutes aprés nous voilà de retour sur la
route.</p>
<p>Nous sommes arrivé à Paradize vers minuit et demi. Quelques
camionettes éclairaient une dizaine de gigantesques “fly tents” tandis
que tout le monde s’afférait à étendre son sac de couchage sous une de ces
bâches. J’ai pris mon sac et ai été m’étendre dans mon sac de bivouac un
peu plus loin : pas moyen de m’entasser avec cette foule. Bien sûr deux
heures plus tard les derniers arrivés plantaient une “fly tent” presque au
dessus de mon sac de bivouac.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/beech-forest.jpg" style="width: 40%;" />
<p>Le lendemain matin je pars avec vingts autres à l’assaut de “Turret
Ridge” : 1300 mêtres de dénivelés, dont 700 dans une forêt qui dégénérait
par fois en brousse. Mais au dessus de la “bush line” la progression à
été encore plus dûre : une broussaille jonchés de “spagnard”, une sorte
de cactus, nous empéchait d’avancer. Elle a progressivement laissé place
à un pierrier. La pente était si raide que les pierres tenaient à peine.
D’aucun prétende que cela permet de l’estime à 37 degrés. Elle s’étendait
uniforme du chaque coté pendat quelques kilomêtres et débouchait
directement sur l’arrête.</p>
<p>Nous nous sommes reposé sous un grand soleil, admirant la vue sur
Paradise avant d’entamer la descente. Je n’étais pas pressé de
redescendre alors je suis resté derrière avec les lambins. Au bout d’une
heure de descente dans la forêt le groupe était complètement dispersé :
il n’y avait pas de chemin, bien sûr, et il fallait se frayer un passage
en écartant les arbres et en enjambant les troncs pourris qui jonchaient
le sol. On ne voyait pas à vingt mêtres et nous regrouper n’a pas était
une mince affaire. Nous avons même perdu un participant qui est descendu
tout droit, passant à travers les plus horribles fourrés. C’est la que
j’ai découvert que je commencais à aimer cette progression : je sautais
littéralement dans la pente, utilisant mon élan pour repousser la
végétation et m’aggripant avec les mains de chaque coté aux arbres pour
me ralentir.</p>
<p>A sept heures nous étions en bas et de retour au camp pour un bon
repas. Une fondue au chocolat était prévu mais le temps que l’on mange le
reste il faisait nuit et nous avons décidé d’éviter un massacre.</p>
<hr class="docutils" />
<p>Le lendemain matin les quelques fous de kiwis qu’il y avait sont allés
se baigner dans un lac juste à coté. J’ai décidé d’en faire le tour. Il
m’a fallut une heure pour arriver au bout du lac, soit deux kilomêtres.
Il y avait une petite plage de galets où j’ai savouré la moitié de mon
déjeuner avant de battre en retraite devant les sandflies. Pour finir mon
tour de lac il m’a évidement fallut traverser une rivière, et je n’en
suis pas tellement sorti plus sec que ceux qui avaient été se baigner.
J’ai appris en rentrant que il y avait un pont un peu plus haut.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/from-dunstan-range.jpg" style="width: 30%;" />
<p>Pour le voyage du retour je m’étais réservé la même place qu’à l’aller,
à coté d’Adrian mais j’ai accepté de la céder à l’une des hollandaise
qui avait le mal de voiture. Cette fois Adrian a décidé de traverser une
petite chaine de montagne non par la route mais par un chemin
d’exploitation. Nous nous somme donc engagé dans une gorge qui montait
vers la passe la plus haute de Nouvelle Zélande (certainement pas plus
d’un millier de mêtres). Tous les kilomêtres une barrière bloquait la route
et l’hollandaise était obligé de descendre pour l’ouvrir et la refermer
derrière nous. En haut de la route une autre camionette du club nous
attendais. Quand nous sommes arrivé ils ont démarré au quart de tour,
nous laissans la route couvert d’un nuage de poussière. Adrian à
cependant réussi à leur passer devant alors qu’ils ouvraient une
barrière, leur rendant la pareille. Les pauvres camionettes de locations
rebondissaient dans tous les sens sur le cahuts de la route que nous
avalions à tout vitesse. Au bout de quelques barrières nous nous sommes
organisés : la première voiture s’arrétait, ouvrait la barrière et laissé
passer l’autre, puis fermé la barrière, récupérait son passager et
repartait.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/dunstan-range.jpg" style="width: 40%;" />
<p>De l’autre coté des montagnes la route que nous avions quitté
continuait paisiblement dans la vallée. A dix heures Adrian me déposait
devant ma maison, me promettant que la prochaine fois que nous partirions
ensemble la brousse serait plus épaisse.</p>
Tunnel beach et chocs culturels culinaires2003-03-06T00:00:00+01:002003-03-06T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-06:/personnal/tunnel-beach-et-chocs-culturels-culinaires.html<p>Ce soir j’avais la pèche comme c’est pas croyable: dans ces cas je
prend mon vélo et je pédale comme un taré. Je voulais visiter la
“Kaikorai valley” mais je l’ai traversé trop vite, alors j’ai continué
jusqu’à la côte. Là je suis tombé sur …</p><p>Ce soir j’avais la pèche comme c’est pas croyable: dans ces cas je
prend mon vélo et je pédale comme un taré. Je voulais visiter la
“Kaikorai valley” mais je l’ai traversé trop vite, alors j’ai continué
jusqu’à la côte. Là je suis tombé sur la route menant à “Tunnel beach”,
comme on m’avait beaucoup parlé de cette plage j’y suis allé. Assez
plutôt fascinant : la côte est faite d’une jolie falaise jaune dans
laquelle la houle incessante de l’océan pacifique a creusé des motifs les
plus étranges. Il y a une petite presque ile avec un pont naturel sous
lequel s’engouffrent les vagues. On decend à la plage elle-même par un
tunnel en pente grand comme un petit homme où des marches ont étés
façonnées. La plage est longue d’une vingtaine de mêtres, au fond d’une
petite crique, encombrée par une dizaines de rochers grands commes des
baleines. On y est coincé entre la mer qui rugit et des grandes falaises
entièrement lisses. C’est vraiment géant d’avoir un endroit comme cela à
une demi heure de vélo de la maison.</p>
<p>Aujourd’hui c’est mon jour de cuisine, et je ne voulais pas me fouler :
omelette au champignons et pomme de terre à la poelle. J’avais mis un de
mes CD de jazz ringuard pour faire la cuisine (Miles Davis et Charlie
Parker), à la place de la radio qui est en marche 24h sur 24. Quand les
autres sont arrivés ils ont poliment laissé le CD jusqu’au bout, mais
“The Edge FM” est vite revenu aprés. Je ne suis pas mécontent de ma
cuisine de ce soir : les pommes de terres étaient cuites et les trois
ognions entiers que j’avais mis dedans n’étaient pas de trop, l’omelette
n’étais pas mauvaise, compte tenu du manque de crème fraiche, et de la
poelle tordue. Mais bon, tu sais bien, je les aime baveuses ! Personne
n’a bronché mais Marc c’est fait des saucisses tandit que Louise
finissait de cuire l’omelette à l’aide du micro-onde, pour en faire une
pâte blanchâtre qui lui a alors convenu, et que Grant se faisait
rechauffer du maïs. Ils n’ont même pas goutter à mon omelette ! Quand je
pense qu’il y a ici des restaurant où on paye une somme ahurissante pour
avoir de la cuisine française ! La première fois que je me suis fait du
pain je l’ai mis à disposition de tout le monde. Il n’était pas
absolument génial, mais certainement meilleur (et de loin) que les pains
d’ici (des pains mis sans gout). Personne n’y a touché, l’aspect ne leur
plaisait pas. Personne sauf Tim, l’américain, qui ne s’en n’est pas
plaind. Lorsque je fais un essaie de couscous, je commence par me heurter
à un problème de vocabulaire : “couscous” c’est comme cela qu’ils
appellent la semoule, et ils n’imagent pas que cela puisse etre un plat.
Moi cela heurte mon respect pour la cuisine magrébienne. Je leur fais
donc un couscous, avec du boeuf car nous n’achétons par d’agneau pour
faire des économies, et quand il s’est agit de rajouter les raisins secs
on m’a arrété : je ne savais pas ce que je faisais, je n’avais pas de
goût ! Le pire reste la ratatouille, qui a été décrétée trés bonne aprés
l’ajout d’une sauce barbecue et de cotage cheese :-< . Pour ma part je
suis content de ma cuisine, je n’obtient jamais exactement ce que je
veux, mais c’est toujours plutôt bon ; les autres, eux, ont décidé que
mes goûts étaient dégeulasses, et qu’il n’y avait rien à faire. Et
pourtant j’apprécie leur cuisine, elle est juste trés différente de la
cuisine Européenne. M’en fout : je mange bien, et c’est l’important.</p>
Le Mont Cook2003-03-05T00:00:00+01:002003-03-05T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-03-05:/personnal/le-mont-cook.html<p>Cela me trottait dans la tête depuis quelque temps : une superbe photo
du Mont Cook trone au dessus du bureau de la secrétaire du département de
physique. La photo est impressionante par sa taille, mais aussi par
l’empillement de glacier, de falaises, de couloirs d’avalanche… qu’elle
montre …</p><p>Cela me trottait dans la tête depuis quelque temps : une superbe photo
du Mont Cook trone au dessus du bureau de la secrétaire du département de
physique. La photo est impressionante par sa taille, mais aussi par
l’empillement de glacier, de falaises, de couloirs d’avalanche… qu’elle
montre. Je voulais voir ce montre en réalité, me balader autour, mais
d’ici c’est loin. Soudain la semaine dernière j’ai décidé qu’il fallait
absolument que je profite des derniers jours de beau temps avant que
l’hiver ne vienne. Je me sentais enfermé dans mon labo, alors que un
soleil fabuleux brillait dehors. J’en ai parlé à Chris qui a était tenté.
Quelques e-mails à des amis du tramping club m’ont fournit des
renseignement sur ce qui était faisable dans le coin, et je me suis
procuré une carte et un topo. C’était devenu un besoin urgent : partir
pour le Mont Cook, et vendredi j’ai soudain décidé que c’était pour ce
week end : la météo était bonne sur tout l’Otago et il fallait en
profiter. Chris s’est désistéé : il était retenu pour le week end, avec
lui partait toute chance d’avoir une voiture. Que cela ne tienne, j’irais
en stop.</p>
<p>Je suis passé à la réunion du tramping club le midi pour prendre du
materiel : ils avaient de bons crampons et piolets, cela peut toujours
être utile. Je retourne donc au labo avec le piolet attaché à mon sac et
les crampons sanglés sur le coté, décidé à foutre le camp le plus vite
possible.</p>
<p>Malheureusement quelques savants fous ont inventé un truc fabuleux qui
s’appelle une pompe à sublimation de titane. Le principe est trés simple
: il y a dans le système sous vides des filaments de titane que l’on
chauffe par un courant électrique. Si le courant a la bonne intensité le
titane sublime et va constituer une fine couche sur les paroies du
système sous vide. Il peut alors adsorber toutes les impuretés et laisser
un trés bon vide. Sauf qu’il faut déterminer le courant à injecter pour
atteindre la sublimation. Si on en met pas assez on chauffe juste le
filament qui libère une tonne de saletés et on peut voir la pression
augmenter. Si on le chauffe trop on le fait fondre. En théorie quand on a
juste le bon réglage on peut voir la pression augmenter quand on le
chauffe, puis chutter juste aprés. En pratique il n’y a pas de jauge de
pression dans mon système, c’est la jauge de la pompe que j’ai utilisé
pour le vider qui sert. Bref il fallait rendre la pompe aprés le week
end, et donc terminer ce boulot le jour même. Alors j’augmentais
lentement le courant, lançais la procédure, surveillait la pression (qui
chutait forcement aprés la séquence de chauffage à cause de la pompe), ne
voyait rien de probant, et donc réaugmentais le courant. Et ce pendant
l’aprés midi. Rien ne se passait, la pression ne chutait pas de façon
significative et en parlant avec les autres j’ai appris qu’il n’y avait
pas un groupe au monde qui comprennait comment sa pompe à sublimation de
titane marchait réellement. Celui d’Alain Aspect détiend le record de
filaments fondus : 8, ils ont une bonne idée du courant nécessaire pour
faire fondre le filament, mais pas de la limite de sublimation. Au fur et
à mesure que j’augmentais le courant on a commencé à me suggère de faire
fondre le filament exprés, pour calibrer les deux autres de rechange (qui
attendent bien au chaud dans la chambre sous vide à coté du premier).
Vers 5 heures je n’étais toujours pas partir. Andrew avait quitté son
bureau et nous regardions le courant monter et la pression faire le yoyo.
Pour ma part je dansais sur place, consultant le topo du Mont Cook entre
chaque cycle. Arrivé à un certain point Andrew et moi nous somme mis
d’accord que aller plus haut fairait fondre le filament, et donc qu’il y
aait forcement sublimation, et que moi je partais en courant, que je ne
revenais pas lundi, et qu’il libèrerait la pompe lundi et fermerait le
système. Bien sûr mon départ était grillé, mais cela me laissais le temps
de faire mon sac et de me coucher tôt. J’ai même pu me faire une grosse
miche de pain pour le week end.</p>
<hr class="docutils" />
<p>Le lendemain matin j’étais à la sortie de Dunedin, le pouce levé. La
première personne qui m’a pris m’a déposé quelques kilomètres plus loin,
dans un endroit où personne ne s’arrétait. Puis j’ai été pris en stop par
un jeune “derry farmer”, bref un cow boy. Il conduisait à fond une vielle
voiture à boite manuelle (ce qui est rare ici) en écoutant du rock à la
guitare électrique grinçante. Personnellement j’aime plutôt ce genre de
situation : cela donne de l’authenticité au paysage.</p>
<p>Il m’a déposé à un embrenchement vers “central otago”. Deux minutes
plus tard une autre personne était déposée 50m en avant de moi et
commençait elle aussi à faire du stop. Nous avons était pris par la même
voiture : un couple de kiwis dans une 4x4 qui m’ont déposés au milieu de
nul part car c’est là qu’ils allait (ainsi que l’autre autostoppeur). Je
me suis demander si j’allais attendre longtemps, en bas de cette côte,
sur cette route droite pendant des kilomêtres, à travers des champs
déserts. Heureusement j’ai été pris un stop par un australien avec lequel
j’ai conscienceusement refait le monde pendant deux heures. Nous avons
traversé des paysages superbes : des collines ou presque rien ne pousse à
cause de la secheresse, et qui sont recouvertes de petits groupes de
rochers, se dressant bizzarement dans l’air. Je me suis promis que je
retournait là juste pour le paysage.</p>
<p>J’ai été déposé à coté d’un lac, et j’ai savouré mon déjeuner dans un
soleil radieux, qui commencait à me donner des coups de soleil lorsque
j’ai repris ma position sur le coté de la route. Cette fois c’est un beau
kiwi torse nu au sourire iluminé qui c’est arrété. Il revenait d’une
semaine d’ouverture de voies d’escalade un peu plus loin, mais s’arrétait
car le mauvais temps approchait. Je n’ai pas voulu le croire pour ce qui
s’agissait du mauvais temps. Nous avons traversé de superbes montagnes
desertes, où je me suis aussi promis de revenir. Il m’a laissé dans un
village typique du centre de l’ile du sud : une intersection entre deux
routes avec des maisons de part et d’atres des routes. La voiture
suivante était conduite par un hollandais auquel j’ai expliqué pour la
quatrième fois de la journé (et pas la dernière) ce qu’était la
condensation de Bose Einstein. Le dernière bout de chemin (100km ne
menant nul part ailleurs qu’au Mont Cook) a été parcourru en compagnie
d’un écossais.</p>
<p>Je suis donc arrivé à “Mont Cook Village” à 17h, juste à temps pour
passer au bureau du DOC afin de prendre la météo (moyenne, mais pas
catastrophique), l’état de la montagne (pas de neige fraiche depuis une
semaine, la neige à eut chaud et s’est bein stabilisé), et de décliner
mes intentions pour le week end (afin qu’ils puissent envoyer les secours
si je ne reviends pas).</p>
<p>Une petite marche de deux heures sur un bon chemin m’amène à l’abris
où je comptais passer la nuit. C’est une petite cahutte sans porte à coté
d’un ruisseau. Il y a une table et des bancs, qui se sont révélés un peu
étroits pour dormir et je n’ai compris qu’aprés coup qu’ils n’avaient pas
été conçut pour cela. Alors que le soleil se couchait j’ai pu profiter
d’une jolie vue du mont Cook de loin. Mon objectif était de remont la
vallé jusqu’au “Hooker glacier” puis de le longer au dessus des murs de
la morraine pour passer au dessus de “Ball Pass”, à coté du Mont Cook,
afin de redescendre sur le “Tasman Glacier”, de dormir dans le “Ball
shelter” au dessus des mur de la morraine du glacier Tasman, puis de m’en
retourner. Pendant la nuit le vent c’est levé et j’entendais que les
raffales rabattaient quelques précipitations sur les murs de mon abris.
Le leendemain matin j’étais surpris de ne pas trouver de la neige partout
autour de moi. Mais non, le Mont Cook était caché par les nuages, mais le
col que je visais était encore visible, et il n’avait clairement pas de
neige fraiche dessus. Je n’avais donc aucune excuse pour ne pas y aller
(la pluie n’en était pas une).</p>
<p>Cependant j’ai n’ai jamais trouvé le gués indiqué par mon topo.
L’endroit où il suggérait de passer la rivière ne me tentait pas du tout,
et je n’ai eut la réponse que aujourd’hui : une recherche sur internet
m’a révélé que le chemin actuel passe quelques kilomètres plus bas dans
la vallée. Je me suis donc dit que j’allais suivre le chemin jusqu’à la
hutte “Hooker”. Une fois arrivé au glaier il m’a fallut une demi pour
trouver le gués pour fanchir un premier ruisseau, puis il a fallut se
hisser à l’aider des mains en haut du mur de la morraine. A ce stade
j’étais couvert de boue et trés pressé d’arriver enfin à du véritable
alpinisme. J’ai perdu le chemin 6 fois avant d’abandonner. Cette horreure
passait un dessous et au dessus de mur de la morraine tous les 500m. Je
dois avouer que j’ai décider devant une descente vers le glavier
particulièrement moche que la hutte n’en vallait pas le coup.</p>
<p>Je suis rentré au village pour signaler mon abandon, puis j’ai tenté
mon coup dans l’autre vallée qui permet d’approcher le mont Cook. Il
pleuvait des cordes et un fort vent ballayait les vallées. Au fond on
pouvait voir les plaines de “central Otago”, toujours ensoleillées.</p>
<p>Il y a 8km de route avant d’arriver au Tasman Glacier, et je n’avais
pas une envie folle de les faire à pied, je faisais donc du stop au fur
et à mesure que je progressé vers le glacier. La seule voiture qui c’est
arrété allait dans l’autre sens. Elle était pleine de matériel
d’alpinisme en piteux état et le barbu qui la conduisait m’a conseillé de
faire demi tour, me prévoyant un temps horrible. Bien sûr je ne l’ai pas
écouté, j’étais bien convaincu que je saurais voir le danger et me
planquer avant la tempête. De toute façons ma carte indiquait un trés bon
chemin jusqu’au Ball shelter. Il est vrai que le jour diminuait trés
vite, et ce à cause des nuages, et non de l’heure. Je pressais le pas, il
me restait une vingtaine de kilomètres si personne ne me prenait en stop.
Personne ne m’a pris et j’ai marché pendant pas mal de temps. A la fin de
route carrosable j’ai repéré un abris, présent aussi sur la carte,
c’était un bon point de chute en cas de problème.</p>
<p>Le chemin se prolongeait entre le mur de la morraine, qui dépassait de
cinquante mêtres sur ma droite, et la chaine de sommets séparant les deux
vallées. Il continuait à perte de vue, sinuant et monotone, me redant
fou à force. Je pressais le pas, j’avais le sentiment d’étouffer dans
cette atmosphère grise, sur ce chemin de pierre, bordé par des pierres et
des falaises, dans un paysage où la végétation était presque entièrement
absente. Finallement mon imagination délirante crois distinguer une forme
blanche au loin. Je ne rêve pas, il y a bien une trace de l’activité
humaine qui brise la monotonie, mais ce n’est pas l’abris, juste une 4x4
blanche garée : elle ne peut aller plus loin, des grosses pierres barrent
le chemin. Et pour cause, quelques dizaines de mètres plus loin le mur de
la morraine s’est effondré et la route plonge droit vers le glacier. Un
petit chemin a été creusé sur le coté et je m’engage d’un pas pressé
dessus : le vent se lève brusquement. Le chemin monte et descend le long
du mur de la morraine. il devient étroit et le bourrasque me
déséquilibre. Il me tarde d’atteindre le refuge. Soudain je n’en peux
plus : le refuge n’est pas loin, et il me serait facile de continuer,
mais le retour le lendemain matin me fait peur : les conditions seront
pires. Peut-être ai-je manqué de courage, peut-être ai-je fait preuve de
sagesse. Tout ce que je sais c’est que le “Ball shelter” était proche et
confortable, tandis qu’il m’a fallut suivre cette infernale route encore
pendant deux heures, avecla nuit qui tombait, pour arriver à un autre
abris de picnic.</p>
<p>Je me suis arrété pendant un moment pour contempler le glacier qui
s’étendait à mes pieds. Gris comme tout le reste, bien sûr, à cette
altitude il est couvert par des pierres et on apperçoit pas la glace mais
un relief lunaire. Derrière des pics infernaux, sans couleurs,
sétendaient vers les nuages. Cela me paraît un paysage que convient tout
à fait au “Mordor”, le pays du mal de Tolkien. Sous cet éclairage gris
l’immense vallée toute entière respirait le mal et la méchanceté.</p>
<p>Arrivé à mon abris j’ai constaté qu’il avait deux ouvertures sans
portes, le vent le traversé donc de part en part. Je me suis installé
dans un coin, protégé de la pluie et du vent. Conçut pour abriter les
touristes qui pic-nicquent l’abris éait munis de beaucoup de fenêtres, ce
qui me permettait de goutter pleinement à la tempète. J’ai dormit sur le
sol, sous un banc. Bien au chaud, j’entendais le vent hurler à la mort et
j’avais la sensation bizarre que les éléments qui se déchainaient autour
de moi n’étaient pas vraiment arrétés par mon abris, mais me laissaient un
surcis. Le lendemain matin j’ai pus voir que la pluie avait pénétré dans
l’abris sur plusieurs mêtres, portée par le vent entre les deux portes.
Je me suis mis en marche sur cette route infernale, à moitié admiratif
de cette beautée cruelle de la nature, à moitié abattu par la pluie et le
vent. Quand une 4x4 c’est présenté, descendant la route, j’ai levé le
pouce de façons automatique. J’ai été pris en stop et je l’ai presque
regretté : de l’intérieur d’une voiture on ne peut complétement apprécier
la violence sublime du paysage.</p>
<p>Le reste du trajet vers Dunedin a été à la fois banal et génial : j’ai
expliqué la condensation de Bose Einstein à trios autres personnes ;
discuté de la guerre en Irak avec un américain et deux Israeliens ; j’ai
écouté de la techno dans une grosse voiture à boite automatique conduite
par un gros kiwis au regard fuillant ; j’ai joué avec un gosse alongé à
l’arrière d’une camionnette remplie de tout le nécessaire pour vivre
pendant tout l’été ; j’ai été abandonné au prés d’un lac sur une route
déserte par un marchand ambulant, me promettant de repasser me prendre
une demi heure plus tard… mais je n’y était pas : j’étais à coté du
lac, me baignant enfin de soleil et tartinant le reste de mon pain dans
un joyeux et dernier pic-nic.</p>
<p>En dehors des montagnes le temps était si beau qu’arrivant à Dunedin je
me suis précipité à la plage, pour me rendre compte que l’eau était si
froid que je n’arrivais pas à laisser mes pieds dedans. Ce soir je me
suis couché tôt, mais j’ai révé de la “Ball pass”. Si je trouve le temps
j’y retourne, j’ai maintenant des informations précises sur l’itinéraire
et je sais ou trouver une météo pour le Mont Cook. De plus j’aimerais
quand même l’appercevoir à moins de dix kilomètres.</p>
Une rando de folie dans les Fjords2003-02-15T00:00:00+01:002003-02-15T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-02-15:/personnal/une-rando-de-folie-dans-les-fjords.html<p>Voici le récit d’une rando dans “Fjordland”, un endroit assez
exeptionnel…</p>
<hr class="docutils" />
<p>Mercredi soir, 7h, j’attendais sur le balcon, en train de lire le
Otago Daily Times, le canard local, avachit dans un profond fauteuil, en
tenue de combat : sandales confortables, pantalon brun avec beaucoup de
poches, et fourrure …</p><p>Voici le récit d’une rando dans “Fjordland”, un endroit assez
exeptionnel…</p>
<hr class="docutils" />
<p>Mercredi soir, 7h, j’attendais sur le balcon, en train de lire le
Otago Daily Times, le canard local, avachit dans un profond fauteuil, en
tenue de combat : sandales confortables, pantalon brun avec beaucoup de
poches, et fourrure polaire, bien sûr. J’attendais donc une fourgonette
tirant un bateau, le type qui m’avais contacté par mail m’avait dit qu’il
tirerait un bateau. Bien entendu ils étaient en retard, et plus qu’un
peu. Finallement je vois une vielle 4x4 jaunâtre surgir au coin de la
rue, derrière elle une petite vedette de 6 metres en alu. La 4x4 s’arrète
loin de la maison mais j’avais déjà saisi mon sac. Il en sort un grand
type avec un sourire ahuri mais bronzé et un T shirt “Otago University
Tramping Club” vaguement rentré dans son jean. Quand je parviens à son
niveau j’apprend qu’il s’appelle “Danillo”, et nous sommes parti vers la
4x4. Non, bien sûr il n’y a pas assez de place dans le coffre, mon sac
part dans le bateau et moi je me dirige vers la porte du passager pour
monter derrière. Malheureusement il y a quelqu’un et nous nous regardons
pendant un instant à travers la vitre, indécis. Puis il ouvre la vitre et
se présente “A-a-adrian”. Effectivement c’est le type avec qui j’avais
pris contact, mais il a un volant dans les mains : il n’y a rien a faire,
j’ai beau essayer trés fort de le changer, ils conduisent à gauche ici.
Je fais donc le tour penaud et me hisse dans une voiture légèrement sâle
et particulièrement encombrée par toute sorte de matériel. A l’arrière il
y a une troisième personne, Craig, qui pèse au premier coup d’oeil ses 90
kilos et n’a pas l’air commode. Danillo est clairement le Bon, mais je ne
sais lequel de Adrian ou Craig est le truand.</p>
<p>Nous sommes donc partis et d’aprés la conversation je comprend que
Craig et Danillo vont ête déposer pour faire l’avitaillement tandit que
Adrian va faire le plein d’esence. Je décide de rester avec Adrian et
passe devant. La conversation débute difficilement. Il ne me faut pas
longtemps pour comprendre que ce géant à l’air un peu gauche est trés
sérieusement bègue, mais son visage reflète une grande intelligence
alors que sa bouche happe vainement l’air à la recherche de syllabes et on
sent qu’il a beaucoup plus a dire qu’il n’en articule. A la station
service il remplit son réservoir puis vérifie systématique la pression
des pneux, remorque y compris.</p>
<p>Arrivé au supermarché il ne nous faut pas longtemps pour trouvé les
deux autres : Craig déambule tranquillement derrière le cadi dans une
grosse polaire tandis que le grand et maigre Danillo fait dans
allez-retours rapides d’un rayon à l’autre. Nous repartons avec une
provision plus que raisonnable de riz, pattes, muesli et lait en poudre.
Avant de quitter Dunedin nous nous arrétons pour prendre Tim, qui nous
attend sur le bord de la route avec son gros sac à dos, ses cheveux
mi-longs et sa barbe pointue. Puis nous fillons vers le sud. C’est
l’aventure : je ne suis encore jamais allé à plus de 20km de Dunedin.</p>
<p>Craig dégaine son portable : des affaires importantes à régler avant
le week end. Je ne peux m’empécher de préter l’oreille : quelque chose
est tombé au fond d’une clairière… “Oui, c’est plutôt inaccessible mais
va falloir que tu amène le camion là… une vrai boucherie, un bazard pas
croyable, il faut que tout soit nettoyé ce lundi”. De quoi parle-t-il ?
La conversation dure bien cinq minutes pendant lequels Craig discute d’un
nettoyage et de sa difficulté : “Il va falloir ramasser les morceaux…
Je me demande bien comment il a pu arriver là… Ils ont vraiment fait un
travail de cochons”. C’est décidé, Craig doit être le truand, mal rasé,
la mine patibulaire, pas trés bavard (à part pour décrire une scène de
… massacre ?). Je ne saurai jamais de quoi il s’agissait, mais
j’apprendrais plus tard que Craig dirige une petite exploitation
forestière et est spécialiste en travaux lourds de bucheronnage.</p>
<p>Une fois le téléphone raccroché Craig et Adrian échangent quelques
mots dont je ne saisis pas le quart. Craig est assis au plus profond de
son siège et Adrian tenant le volant d’une main est retourné vers lui. De
temps en temps il jète un coup d’oeil à la route et corrige sa
trajectoire. Heureusement il n’y a pour ainsi dire aucune voiture sur la
route.</p>
<p>Quelques dizaines de kilomètres plus tard Craig fait signe de
s’arréter : il est déjà passé là dans la journé et doit 3 $ à quelqu’un :
il n’avait pas de monnaie. Nous traversons de longues vallées encerclées
par de petites montagnes éclairées par le rose du soleil couchant. Entre
quelques fermes éparses on peu voir des champs claircemés de moutons.
Craig fini par sortir un livre de science fiction et la conversation
meure. Pour ma part je m’endore, comme d’habitude en voiture.</p>
<p>Quelques heures plus tard nous entrons dans Gore, une ville précédée
par sa réputation de trou paumé, de peuplade de fermiers mal dégrossis :
“southern land, southern people”, et fiers de l’être. Nous nous
arrétons juste pour faire le plein. Pendant qu’Adrian vérifie la pression
des pneux nous nous dégourdissons les jambes. La jeunesse de Gore est
présente, elle boit de la bière depuis de grosses voitures américaines.
Des garcons habillé de jeans nous interpèlent : “You’ve got a cool boat
here.” “How much do you ask for it ?”. 5 minute plus tard Gore est
derrière nous.</p>
<p>Qand je me reveille nous arrivons à Te Anau où nous nous inscrivons
sur un livre du DOC précisant notre itinéraire, notre date de retour et
la date à partir de laquelle il fallait déclencher les secours si nous
n’étions pas de retour.</p>
<p>Il est 2h du matin quand nous arrivons au lac. Une cahute nous permet
de poser vite fait notre sac de couchage et nous nous lavons les
dent à la lampe frontale. Tim décide qu’il fait beau et qu’il préfère
dormir dehors.</p>
<hr class="docutils" />
<p>Le lendemain Craig est le premier debout. Nous découvrons la
cahutte à la lumière du jour. Il s’agit de la première “Back Country Hut”
établie, le premier refuge de Nouvelle Zélande. A l’heure actuelle il ne
reste plus que des murs et un toit. Tout le monde sort sa gamelle pour le
petit déjeuner. Je n’ai pas pris la mienne à la suite d’un malentendu,
alors je verse une grande quantité de muesli et de lait en poudre dans
une casserole et j’emprunte la cuillère de Craig. Viens le moment de
faire les sacs, et je m’apperçois que mes compagnons n’ont pas acheté que
des rations légères… Puis tout est chargé dans la bateau ; Adrian nous
distribue des gilets de sauvetage et Craig recule la voiture jusqu’à ce
que le bateau soit à flot. Nous somme soulagé d’apprendre qu’il ne va pas
falloir se mouiller pour monter à bord : Adrian va se mettre au ponton.
Une fois la voiture garé et tout le monde à bord nous larguons les
amarres. Les trois place assises sont prises à bord et je pose à
l’arrière. Le bateau file doucement mais je contemple le moteur d’un air
soupconneux : 115 chevaux sur une barcasse d’un tel poid cela ne doit pas
faire beaucoup de bien quand cela part. En effet Adrian ne faisait que
chauffer le moteur et il augmente graduellement la puissance à mesure que
la “mer” devient plus rude. Le lac fait 88km de long et 10 de large ce
qui est largement suffisement pour qu’une houle raisonnable est le temps
de se former. Nous nous trouvons vite à ricocher sur le vagues, Adrian
debout aux commandes et les autres agripés sur ce qu’ils trouvent,
rebondissant contre la coque à chaque choc. Il nous faut bien une demi
heure pour traverser le lac et s’engager dans le fiord qui nous
intéresse. Adrian s’amuse à raser la cote et à passer en des iles. De
superbes montagnes plongent à pic dans le fiord avec ici et là une chutte
d’eau ou une falaise. Finallement nous arrivons à une petite plage au
fond du fiord et, le moteur relévé, le bateau aterrit doucement dans le
sable.</p>
<p>Nous sommes donc arrivés sur une petite plage on fond d’un fjord de
lac Te Anau. Je saute à terre avec une ammarre et je la passe autour d’un
arbre pendant que les autres déchargent les sacs.</p>
<p>La forêt est extrèment dense, on ne voit que à une dizaine de mêtres
dedans, mais elle n’est pas trés haute, 4 mêtres au plus. Une fois les
sacs déchargés nous partons avec nos sac pour les amener sur l’autre lac
(Lake Hotchinson) tandis que Adrian va mouiller le bateau (à la
Morbihannaise, j’apprendrais plus tard en discutant avec lui). Un
triangle orange sur un arbre le long de la plage signale le début du
chemin et nous nous engagons dans la forêt. En quelques instants nous
somme au plus profond d’une “rain forest” (je n’ai pas trouvé de
traduction).</p>
<p>J’aimerais pouvoir décrire cette superbe forêt : “the native bush”
comme on l’appelle ici. C’est clairement une forêt d’un climat
extrèmement humide qui n’a jamais été souillé par l’homme à part sur
quelques rares chemin. Il y a du vert partout : les troncs d’arbres
montent du sol au feuillage dans un désordre pas croyable, coudés,
sinueux, se croisant et s’enchevêtrant. Sur le sols toutes sortes de
plantes rivalisent pour couvrir le moindre centimètre de terre laissé nu
par les racines protuberrantes des gros arbres. De la mousse verte couvre
tout : troncs, racines, blocs de pierre, branches mortes. Des branches
pendent des lianes vertes formées de mousses, ou même par fois de
véritables lianes. Il n’y a essentiellement pas de buissons au sens où
nous l’entendons, tout est couvert d’un mélange varié d’espèces. Il y a
même des petites palmiers qui nous arrivent aux genous.</p>
<p>Le chemin n’a rien à voir avec ce que nous appellons chemin. C’est
une bande de un demi mêtre de large où il n’y a pas de végétation. Il est
cependant traversé par des racines couvertes de mousse et trés
glissantes. De plus il suit un tracé ératique pour éviter la myriade
d’obstacles que créé la forêt. Par moment il passe sous un arbre tombé,
et il faut se baisser trés bas pour passer avec le sac à dos, d’autres
fois c’est au dessus du tronc qu’il faut passer. Il faut se glisser entre
deux troncs serrés, escalader un amas de pierres et de racines…</p>
<p>Pour l’instant le chemin n’est pas trop mauvais et en un quart
d’heure nous arrivons à l’autre lac. Là un hangar trone au milieu de la
brousse et des rails en bois descendent jusqu’à l’eau. Dans le hangar il
y a un bateau, et Adrian a la clef du hangar. Malheureusement le moteur
n’est pas fournit avec, il faut donc retourner à la plage chercher un
petit hors bord, enfin si on peut considérer que 15 chevaux c’est petit.
Craig et moi choisissons un long morceau de bois au quel nous attachons
le moteur. Nous pouvons donc entamer à nous le trajet vers le lac
Hotchinson, avec lourde branche nous sciant l’épaule à chaque pas. Il va
de soit que le trajet n’est pas vraiment facile, d’autant plus qu’il faut
placer ses pieds avec soin pour ne pas tomber. Danilo nous escorte, exité
comme une puce, proposant de l’aide que nous refusons, tout en lui
promettant qu’il portera le moteur au retour. Une fois le moteur de
l’autre coté nous descendons le bateau, amarrons le moteur, chargeons les
sacs, et nous voilà partis.</p>
<p>Pendant tout ce temps les “sandflies” nous harcellent. Les “sandflies”
sont de toutes petites mouches qui, comme les moustiques, ont besoin de
notre sang pour se reproduire. Ces petites bètes sont trop lente pour
pouvoir nous suivre quand on marche, ne peuvent mordre à travers les
vétements et mettent pas mal de temps à trouver un emplacement qui leur
conviennent dans notre belle chair. De plus leur morsure n’est pas trés
douloureuse, et tant qu’on est dans la brousse on est bien trop occupé
pour beaucoup se gratter. Ces mouches ne sont donc en soit pas trop
méchantes. Mais le problème est qu’il y en a des miliers. Danilo lui a
une solution : il se met un filet sur la tête. Le seul produit chimique
efficace pour les éloigner est tellement actif qu’il ne faut pas en
mettre sur les vétements synthétiques : il perce un trou dedans. La
légende Maori veut que le dieu qui a créé les fjords ait créé les
sandflies pour que les humains ne puissent pas goutter au paradis sur
terre : les fjords étaient trop beau, il fallait une punition pour
pouvoir y accéder.</p>
<p>Sur le lac un fort vent souffle dans le sens opposé à la marche de
notre bateau, les sandflies disparraissent donc bien vite. Mais le ciel
s’est couvert et le bateau tappe sur les vagues, me trempant
complètement. De plus il se remplit assez vite et il faut éccoper. Arrivé
de l’autre coté je suis aussi mouillé que si j’avais pris une douche, et
je grelotte. Je me mets bien vite en route pour la première hutte, dix
minutes plus loin : la marche me réchauffera sûrement. Arrivés à la hutte
nous décidons de continuer jusqu’à la suivante avant de prendre notre
déjeuner. On me préviend qu’il y a pont un peu plus loin, on se
retrouvera là bas. Effectivement aprés cinq minutes de marche j’arrive à
la rivière. Trois cables métalliques sont tendus à travers et nous
traversons les uns parés les autres, attendant que le précédent ait fini
sa lente progression dans le vide : les cables ne sont pas conçus pour
supporter plus d’une personne. Le reste du trajet n’est que brousse et
rochers. Le chemin est indiqué par de triangles oranges trés bien
positionnés mais par moment pas visibles à plus de dix mêtres. Le chemin
se confond avec la brousse trés aisement et si on le perd on a peu de
chances de le retrouver. Par moment les fougères sont tellement denses
que l’on apperçoit même pas ses propres pieds.</p>
<p>Nous devons passer le long d’un lac. C’est toujours en moment pénible
dans Fjordland : les montagnes plongent directement dans le lac et l’on
doit faire une traversée sur une pente trés raide et tout de même
encombrée par la végétation. Chaque obstacle doit être contourné en
montant une dizaine de mêtres, s’agrippant avec les mains aux racines et
aux fougères, pour les redescendre juste aprés. De plus je ne sais
pourquoi il y a tendance à y avoir plus d’arbre tombés sur de telles
pentes. Lorsqu’un arbre est tombé trop récement et que le chemin n’a pas
encore été modifié il faut se frayait un chemin dans la brousse. Il vaut
mieux généralement essayer de passer au dessus : dans le trou laissé par
la souche il n’y a pas de végétation, c’est donc plus facile d’avancer.</p>
<hr class="docutils" />
<p>Deux heures plus tard nous arrivons à la hutte du lac Thompson. C’est
l’occasion d’un bon repas, mes compagnons ont vu large sur la nourriture,
cela tombe bien j’étais affamé. Ces huttes sont de petites maisons en
bois avec une unique pièce. Celles que nous avons visitées avaient huit
couchettes, une table dans un coin, un poele ou une cheminée, et un petit
placard. Dans le placard on trouve toutes sortes de choses que les gents
ont laissé : des livres souvent, des denrées non périssables, des
couverts.. Dans la première hutte j’avais récuppéré un bol, dans la
deuxième une cuillère, et le problème des couverts avait été résolu. Au
retour je les ai bien sûr laissé dans la dernière hutte. Il y a aussi
dans chaque hutte un livre où chaque groupe laisse le nom de ses membres,
son activité et son intention de route. Un tel livre sert lorsque un
groupe est porté manquant, ce qui arrive relativement souvent. Nous les
avons rempli méticuleusement à chaque hutte devant laquelle nous
passions. Pour passer la nuit dans une hutte il faut laisser des tickets
dans une boite, tickets acheté en ville dans n’importe quel bureau du
DOC. Je n’avais bien sûr pas de ticket mais à mon retour j’ai acheté un
pass annuel, 30 euros, cela ne me paraît pas du vol.</p>
<p>Aprés le déjeuner nous reveillons Tim qui s’était immediatement
endormi et nous repartons à l’assaut de la montagne : la prochaine hutte
est de l’autre coté d’une passe, au fond du “George’s sound”, un fjord de
la mer de Tasman, soit à dix heures de marche d’ici, et au bout du
chemin. Nous n’avons bien sûr pas l’intention de marcher dix heures, mais
nous voulons nous placer au pied de la passe, pour avoir moins de chemin
le lendemain.</p>
<p>Nous passons de nouveau au dessus de la rivière, mais cette fois elle est
dans des gorges profondes et étroites. Suffisement étroites pour que nous
aillons droit à un vrai pont. Du pont on peut admirer une superbe chute
d’eau d’une dizaine de mêtres qui tombe d’un bassin creusé dans le rocher
vers le fond des gorges. Fjordland est un pays trés raide et trés humide,
il y a donc des chutes d’eau absolument partout.</p>
<p>Le chemin se met à grimper sec, et je suis vite obligé de m’aider de
mes mains. Le temps passe vite et lorsque vers 5h nous appercevons sur
replat un endroit sympatique pour bivouaquer nous nous arrétons, craignant
qu’il n’y en ait pas d’autre avant la passe. Là je vois apparaître la
“tente” emportée pour le groupe. Les kiwis appellent cela une “fly
tente”, j’appelle cela une bache tendue entre les arbres par des cordes.
Heureusement j’ai pris mon sursac de bivouac, je serais au sec, moi au
moins. Nous sommes prêt de la rivière mais, assez curieusement il n’y a
pas de sandfly.</p>
<p>Nous nous faisons un succulent repas : du riz avec diverses legumes
frais, un régale par rapport à ce que je mange d’abitude en rando. Pour
le dessert je découvre les “instant puddings” : des poudres qui,
mélangées à de l’eau donnent des crèmes de gouts varié. Puis Craig et
Adrian se mettent en tête d’allumer un feu. Bien sûr tout est humide
alors ce n’est pas évident, même avec les allumes feux de Craig. Résultat
ils préchauffent leur bois sur le réchaud, et finissent par réussir à
allumer un feu, qui dégage une fumée trés chargée en vapeur. La nuit
tombe et je me couche, bien emittouflé dans mon sursac. Je dois dire que
j’ai passé une excellente nuit, bien à l’abrit dans mon sac de bivouac,
lui-même sous la bache tendue par mes amis “tramper”.</p>
<hr class="docutils" />
<p>Le lendemain matin ils étaient debout bien avant moi : il n’avaient pas
de moustiquaire pour les protéger et les sandflies ont attaqué dès le
matin. A genoux sous la bache nous avons pliés nos sac de couchage et
puis nous avons dégusté du mueslie au lait en poudre à coté de la
rivière. Je crois qu’il n’y a pas eut beaucoup de paroles échangées :
nous savions ce que nous avions à faire et étions pressés de nous mettre
en route pour échapper aux sandflies.</p>
<p>Nous avons donc repris le chemin qui disparaissait sous la végétation.
Mes chaussures étaient encore humide de la veille mais je n’étais tout de
même pas trés enthousiaste à l’idée de plonger mes pieds dans la boue.
Danillo, qui était derrière moi, me regardait d’un air amusé sauter d’un
bout de bois à l’autre tandit qu’il marchait vaillament au milieu des
flaques d’eau. Bien sûr mes tentatives de rester sec étaient vaines, et
au bout de quelques glissades j’étais aussi trempé que Danillo. J’avais
tout de même eut le temps de faire une raison et je me suis mis moi aussi
à marcher au milieu des flaques. Ce n’était que pour me rendre compte que
l’on arrivait à un marécage, et que dans certaines “flaques” on pouvait
s’enfoncer jusqu’aux genoux. Au delà de la sensassion désagréable une
telle mésaventure ralentie considérablement la marche, car il faut alors
se saisir d’une main ou d’un arbre pour se tirer de la vase. La
progression dans un marécage se fait donc sur les tronc d’arbres morts et
sur les racines. On est en équilibre sur un réseau de bois pourrit et
recouver de mousse. Cela se termine obligatoirement par une baignade mais
qu’importe, le végétation est tellement humide qu’on est trempé des pieds
à la tête rien qu’à la toucher.</p>
<p>D’aprés la carte nous nous approchions du “deadwood lagoon”. Drole de
nom” ! Mais j’ai bien vite compris l’origine du nom : sur une étendue
d’eau sale et de boue se dressaient des squelettes d’arbres morts, dont
seuls le tronc et les branche principales restaient. Ce n’était
certainement pas macabre, juste un peu etrange.</p>
<p>Aprés le lagoon la carte indiquait une montée directe droit dans la
pente jusqu’à la passe. Nous n’y avons bien sûr pas coupé. Ce n’était
rien de terrible, la passe elle-même n’étant qu’à 700m, mais les mains
ont été mises à contribution pour se hisser de racine en pierre tandis
que le sac à dos frottait sur quelques troncs d’arbres. 25 mêtres avant
la passe nous émergeons dans une zone découverte : nous sommes au dessus
de la fameuse “bush line” ; la forêt ne pousse pas au dessus d’une telle
limite et la progression est plus facile. Un grand sport local consiste à
passer le plus vite au dessus de la ligne et à courrir le long des
crètes, quand elles sont praticables.</p>
<p>Au niveau de la passe un petit nous attendait, coincé entre deux
montagnes. De l’autre cotée une longue vallée s’ouvrait. Longue je
présume car elle disparaissait bien vite dans la brume. Domage, je pense
qu’on aurait pu voir le fjord. Plus habitué que mes amis aux terrains
alpins je gambadais sur ces sommets découvers, mais il nous a vite fallut
rendescendre, et la descente était aussi raide que la montée. Mes
compagnons refusaient de se retourner pour passer les éboulis et les
rochers les plus raides et ils glissaient sur le dos, génés par leur
gigansteques sac à dos. Pour ma part je n’avais qu’un 70L et je me
retournais et désescaladais. Adrian a fait les frais de cette mauvaise
stratégie avec une belle chute qui a été arrétée par l’impact de sa
cuisse droite sur un rocher. Il en a été quitte pour boiter pour le reste
du voyage.</p>
<p>Arrivé au fond de la vallée nous nous sommes arrété pour manger à coté
d’une rivière. C’est aprés qu’a commencé une série de traversée de
rivière à guet : le chemin suivait les berges et passait d’un coté ou de
l’autre selon les conditions. J’ai bien vite appris qu’il n’y avait rien
à faire : il fallait se mouiller et l’accepter. La progression dans une
brousse toujours renouvellée a continuée jusqu’à ce que nous passions le
long du lac “Katharina” où, comme avec chaque lac, le chemin était dans
un état pitoyable. Un arbre était tombé trés récement en travers et la
brousse était si dense si un terrain si raide qu’il n’était même pas
envisageable de le contourner. Il a fallut passer au dessus, enfin au
dessus des branches principales, et à travers les autres. Une machette
aurait bien été utile, mais je ne sais pas si j’aurais eu le courage de
la timbraller jusque là. Un peu plus tard il nous somme passé sur un
endroit tellement raide que la végétation ne poussez même pas. Cela
aurais été un plus s’il n’avait pas fallut se pendre dans le vide
accroché à une main courante en chaine métallique. Enfin moi je trouvais
qu’on avait une plutôt belle vue sur le lac en dessous. Je dois plus ou
moins être en train de perdre mon légendaire vertige.</p>
<p>Encore une heure de marche un “pont” (trois cables), et nous voilà à
la hutte. Je dis “nous” mais Craig et moi sommes arrivés deux heures
avant les autres : nous en avions tellement marre de cette brousse
ininterrompue que nous avons poussés le pas.</p>
<p>George hut, une petite maison de bois insignifiante au fond d’un fjord
dont on ne distinguait qu’une miniscule portion. Même la plage n’était
pas praticable pendant bien longtemps ; pour explorer le fjord plus en
profondeur il faudrait une machette. Sur la plage de galets un nombre
incroyable de sandflies attendaient leur proie. Dés notre arrivé nous
nous sommes cachés dans la hutte. Aprés un thé j’ai jetté un coup d’oeil
au livre de présence de la hutte. La plus part des visiteurs venaient de
la mer en bateau, peu étaient suffisement fous pour avoir pris le même
chemin que nous. Mais certain l’étaient beaucoup plus : venus de Milford
ils étaient passés par les crêtes pour aprés couper dans la brousse la
plus profonde.</p>
<p>Un examene plus approfondi du placard de la hutte m’a révélé un livre
racontant la direction de “Scott base” pendant un été austral. “Scott
Base” est la base Néo-Zélandaise en antartique. Il y a toujours devant
l’atelier du département de physique des colis orange fluo en partance
pour l’antartique. De temps en temps on voit même revenir un physicien de
la glace, comme ils les appelent, bronzé et fatigué. Le livre en question
racontait les problèmes humains et personnels qu’a rencontré un ancien
tour du mondiste lorsqu’il s’est retrouvé à dirigé une telle base dans un
milieu hostile, remplie de scientifiques incontrolables et d’aventuriers
déracinés. Je n’ai bien sûr pas eut le temps d’aller trés loin dans cette
lecture mais je dois dire que je suis assez fasciné par ce continent. Qui
sait, peut-être vais-je me cacher dans une de ces boites avec inscrit
“preserve from frost”.</p>
<p>Les fous furieux du tramping club sont arrivés et ont commencé à
déballer leur “argonaute”, qu’ils avaient porté jusqu’ici. L’argonaute
est une grande tradition du club. C’est un bateau gonflable conçut pour
la plage que les membres de l’OUTC achètent en super marché. Ils le
baladent jusqu’au bout du monde pour le simple plaisir de naviguer sur
des lac ou des fjords desert. Danillo l’a pris pour aller jusqu’aux
chutes d’Alice, qu’on ne pouvait voir d’où nous étions. Il a confirmé que
cela ne vallait pas le coup, et que les sandflies l’avaient accompagné tout
du long.</p>
<p>Tous ont pris leurs tour sur l’argonaute, pour ma part qui à être une
poule mouillée je préférais l’être avec de l’eau douce. Je suis donc
resté à terre, me promettant que je chercherai un embarquement sur un
bateau convenable passant par là tandis que les autres m’expliquaient que
dans la mer de Tasman, à l’extérieur il y avait 3 mêtres de creux les
jours de beau temps.</p>
<p>Adrian a sorti de bas de soie et un fils de fer et a entrepri de
constituer un filet. Tous parlaient d’un air excité de “Cray-Fish”,
apparement c’est un trés bon poisson qu’on ne trouve que dans les fjords
les plus reculés et qui ne peut être peché commercialement. Adrian est
arrivé en retard pour le diner et on a plus entendu parlé de Cray-Fish.
(J’ai compris des mois plus tard que c’était tout simplement une sorte de
Homard).</p>
<p>Craig est sorti d’un air déterminé et vingt minutes plus tard Danillo
nous annonçait qu’on pouvait aller prendre le pouding autour du feu. Ce
feu là n’était pas le foyer moribond de la veille mais un superbe feu de
joie. Nous avons pu profité de la soiré sans sandflies, disparues à cause
du froid, les pieds étallés devant le feu, les chaussures ouvertes à
sécher. Les histoires du coin du feu que j’ai entendu ne sont pas celles
dont j’ai l’habitude. J’ai appris que le cerfs étant une calamité ici
(car introduits par l’homme) on les avaient longtemps chassé à
l’hélicoptère. Le pilote-chasseur est un vrai cow-boy de la brousse,
conduisant des tas de boue inommables il gagne trés bien sa vie, mais
peut la perdre trés facilement. On me raconte comment ils allait de sa
base au bar local, quelques dizaines de kilomêtres plus loin et le soir
devait rentrer totalement ivre, de nuit. Souvent la meilleure solution
était de payer suffisement pour qu’un autre ivrogne prenne sa voiture et
remonte jusqu’à la base par la route. L’hélicoptère pouvait alors suivre
la lumière des phares. Un peu plus tard Danillo raconte comment alors
qu’il randonné avec une canadienne, en visite pour un an, ils firent une
veillée bien arosée. La canadienne rentre se coucher, bien ronde mais au
moment de se coucher quelque chose la dérange : elle passe sa main sous
sont oreillé et murmure “my gun, where’s my gun”, et ce pendant une bonne
heure avant de s’écrouler de sommei. Le lendemain elle a expliqué que en
rando au canada elle garde toujours un pistolet sous son oreiller, des
fois qu’un grizzly la dérange pendant son sommeil..</p>
<p>Le lendemain matin l’agronaute est dégonflé. On prévoyé de le laisser
en cadeau à la hutte mais il a été crevé. La marche de la journée est
estimée à 10 de marche pour atteindre la hutte la plus proche. Il ne faut
pas croire que nous parcourrons de grande distance : la marche dans la
brousse se fait souvent au rythme de 1 km/h.</p>
<p>C’est le même chemin dans l’autre sens, sauf qu’il pleut des cordes. Ce
n’est en fait pas trop génant, nous somme à peine plus mouillé et le sol
est a peine plus glissant. D’ailleurs au bout de quelques heures de marche
les imperméables ont disparus. Les rivières sont un peu plus dûres à
traverser, principalement à cause du courant, mais les chutes d’eau sont
plus belle. La montée se fait dans un ruisseau qui était le chemin la
veille. A un moment nous passons sous une falaise d’où tombe un rideau de
pluie : la montagne entière ruisselle. De loin on appercoit de longue
trace blanche sur ses flancs. D’inombrables chutes zébrent les flancs.</p>
<p>Arrivé à notre campement de la veille je sens ma maléole, qui
jusqu’ici ne m’a pas du tout dérangé, se reveiller. Cela ne fait pas
encore mal mais je sais trés bien ce qui m’attends. Je presse le pas et
j’atteinds la hutte deux heures avant les autres avec juste une légère
douleur mais une grosse peur : je ne peux m’arréter ici, ou sinon cela
veux dire bivouaquer sur place.</p>
<p>Le repas me semble tout aussi suculent, d’autant plus que la pluie
tambourrine sur le toit, et la nuit qui suit est de celle qu’on ne peut
avoir qu’aprés une journée bien remplie.</p>
<hr class="docutils" />
<p>Dimanche, dernière journée, l’étape est courte mais nous sommes
fatigués. Ma maléole se met trés vite à me faire mal. A la moitié du
trajet cela devient insupportable. Heureusement je ne suis pas le seul à
trainer derrière, Adrian a aussi du mal : sa cuisse le fait souffrir. Au
bout d’un momment la douleur passe et j’avance plus vite. D’autant plus
que le chemin qui me semblait mauvais le premier jour me semble alors une
autoroute. J’ai parlé de mes problèmes de maléole avec Adrian un peu plus
tard. Il me dit avoir eut la même chose (amorcé par le même problème et
avec les même symptomes) et prétend que c’est l’os qui est lésé. D’aprés
lui il n’y a rien à faire d’autre que d’attendre et d’éviter absolument
d’appuyer dessus. Le problème est que toute chaussure de rando convenable
monte jusqu’à la maléole. Il faudra que je trouve autre chose pour le
week end suivant.</p>
<p>Nous arrivons finallement à la hutte et nos dernière provisions sont
joyeusement sacrifiées. Le restant de fuel des réchauds et mis dans le
résrvoir du moteur : nous avions consommé plus de la moitié à l’aller et
nous nous embarquons sur cette barquasse pourrie à travers le lac. Le
vent est dans notre dos ce qui économise du carburant, mais les sandflies
peuvent nous suivre. Un 360 règle le problème et le radeau de la méduse
peut continuer son chemin vers le civilisation. De l’autre coté il faut
transbahuter le moteur ; Tim et Danillo partent avec leurs sac pendant
que nous remontons le bateau. Le bout de bois que nous avions utilisé
pour transporter le moteur casse sous son poids et il faut en trouver un
autre pour Danillo et Tim. Je n’ai absolument pas l’intention de le
porter mais je ne peux m’empécher de sourire qand je vois qu’ils n’y
arriverons pas seuls. Avec l’aide de Craig le moteur se trouve vite de
l’autre coté.</p>
<p>Comme si elles avaient senti que nous sortions de leur domaine les
sandflies se sont défoulées sur nous et notre départ prend l’allure d’une
fuite. Une fois sur la vedette de Adrian les gaz sont poussé à fond pour
se débarrasser de ces petites pestes. Des que nous nous éloignons des
montagnes de Fjordland le soleil réaparait, et ont peut voir
distinctement les flancs du fond du fjord dégouliner par des centaines de
cascades alors que nous profitons d’une traversé ensoleilée du lac.
Danillo se voit confier les moteurs pendant quelque temps et il peut
s’agripper au volant, l’air crispé, tandis que le bateau tape sur les
vagues. Nous faisons un détour pour admirer un autre fjord puis cap sur
la jetée. Adrian fait une arrivée pas trés canonique, et nous laisse
aller cherchele camion. De la plage on peut le voir pousser son engin
coontre les vagues, chercher les rebonds, les accélérations, jouer avec
ses 115 chevaux ; mais quand la remorque est à l’eau il place le nez de
son bateau dessus et la bète est tirée hors de l’eau.</p>
<p>Nous nous désabillons et les habits sont étendus au soleil tandis que
l’on découvre une tablette de chocolat Cadbury. Je passe devant la
chocolaterie tous les matins et l’odeur est alléchante, mais les
chocolats étaient encore bien meilleurs une fois mérités.</p>
<p>Et c’est de nouveau la traversée en 4x4 des étendues de l’Otago, avec
bien sûr un détour par le bureau du DOC pour signaler notre retour. La
nuit tombée, nous entrons dans Gore, pour en ressortir avec des
sandwichs. Le highway que nous prenons s’appelle le “presidential
highway” : il relie Gore à une ville appellée Clinton. Tout va bien de
toute façon on a quitté la “Bush”. Au milieu de nul par une famille de
lapin surgit sur la route ; ils ne font qu’un petit bruit sur les roues
de la 4x4 alors qu’ils disparaissent sous le pare-choc, mais c’est
l’occasion d’une dissertation sur l’inconvenient posé par de tels
événements lors d’une sortie du club : “with three american chics in the
truc you’ve got the choice between going for the bunny and have the girls
bitch about your crualty, and trying to avoid it, and having the
girls bicth about your driving”. Il n’empèche que à la station de Dunedin
Adrian en constatant une nouvelle bosse dans sa carroserie ne peut
réprimer un juron “Damn, this bunny had a hard head !”.</p>
<p>Et voilà, j’ai quitté ces comiques, encore plus fous que moi, et
j’ai passé deux heures dans la douche. Puis j’ai lavé mes affaires
pendant des heures. Même aprés avoir été sérieusement rincées et
avoirsubit une machine mes chaussettes tiennent encore
debout.</p>
<hr class="docutils" />
<p>Je ne m’étendrais pas sur le week end suivant. Je suis parti en vélo
jusqu’à un lac sur la carte dont le contour me plaisait. Adrian m’avait
dit que c’était faisable, mais je ne crois pas qu’il l’ai jamais fait.
dois dire que j’ai été bien crevé (comme rarement) mais vu que je suis
borné comme pas deux je l’ai fait. Bon aujourd’hui j’ai de belles
courbutures mais j’ai le souvenir de cette nuit sur les berges du lac.
J’ai dormit seul dans mon sac de bivouac sur les quelques mêtres qui
séparent la boue du lac de la végétation si dense que l’on entend le lac
avant de le voir. La lune était pleine et le lac trés vivant. Un buisson
me séparer de l’endroit où j’avais mangé et j’ai distinctement entendu
des animaux fourrager à cet endroit. Heureusement mon picnic du lendemain
était avec moi dans le sac de bivouac.</p>
<p>Le lendemain j’ai été invité à boire le thé par un fermier, alors que
je mangais mon picnic devant sa ferme. Il y a ici un contact fabuleux
avec les gens et le pays. Je vais partir à travers ce pays un de ses
quatres, le pouce levé sur les routes pour voir où cela me mènera…</p>
Des Kiwis, des pengouins, des chinois et des colines2003-02-03T00:00:00+01:002003-02-03T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-02-03:/personnal/des-kiwis-des-pengouins-des-chinois-et-des-colines.html<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/vacuum-side.jpg" style="width: 30%;" />
<p>Ouf, l’atelier a pris du retard et je n’ai pas les pièces dont j’ai
besoin… Je ne peux donc pas travailler ! En ce moment je monte les
chambres à vides. En tout il y a de quoi s’acheter une belle Ferrari (
probablement 50 000 $, des dollars …</p><img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/vacuum-side.jpg" style="width: 30%;" />
<p>Ouf, l’atelier a pris du retard et je n’ai pas les pièces dont j’ai
besoin… Je ne peux donc pas travailler ! En ce moment je monte les
chambres à vides. En tout il y a de quoi s’acheter une belle Ferrari (
probablement 50 000 $, des dollars américains, bien sûr ), et les pièces
sont super-fragiles !</p>
<img alt="" class="align-left" src="https://gael-varoquaux.info/personnal/attachments/nz/vacuum-top.jpg" style="width: 30%;" />
<p>Le principe est simple : nous avons une série de tubes, de chambres,
de pompes, de fenêtres… Tous avec des entrées ou des sorties
circulaires où sont percés, régulièrement, des trous pour les boulons,
comme sur les hublots des vieux scaphandres. Il faut boulonner les pièces
les unes sur les autres. Mais cela n’est pas si simple. Comme il nous
faut du vide presque absolu dedans (le moins de molécules étrangères
possible) il faut de très bons joints, et éviter de contaminer
l’intérieur du système. Toute marque de doigt laisse de hydrocarbures qui
s’évaporerons lentement pendant des années dans un système sous vide,
contaminant l’expérience. Quant au joint, cela n’est pas non plus du joli
: il y a des deux cotés un rebord parfaitement lisse et régulier. Entre
les deux rebord on cale anneau de cuivre qui va servir de joint, et on le
comprime en serrant. Si l’un des rebords (les “knife edges” comme ils
disent) prend la moindre rayure (en gros s’il est touché par quelque-chose
de métallique) il est foutu !! Pour serrer, il faut serrer régulièrement,
donc on prend une clé spécial qui est réglée sur un couple donné. Au
delà de se couple elle clique. Et on serre tous les boulons, un d’un
coté, puis celui diamétralement opposé, puis celui à coté, … On fait
deux fois le tour avec la clé sur un réglage donné et on augmente le
réglage… A la fin de la journée je me sens comme Charlie Chaplin dans
les temps modernes…</p>
<hr class="docutils" />
<p>Tous les soirs de la semaine quelqu’un fait la cuisine pour les
autres. C’est très sympa car on mange tous en même temps et donc on se
voit. En plus le résultat est qu’on mange très bien (pour l’instant en
tout cas). Hier soir c’était moi qui faisait la cuisine. J’avais prévu de
faire des carottes revenu à la poelle dans de la crème. Bien sûr
éplucher des carottes pour 6 goinfres cela prend du temps, mais quand
cela a été finalement cuit j’étais bien content. Sauf que au moment où
je mets la crème, catastrophe… J’ai beau savoir que la “sour cream”
cela n’est pas de la crème je suis toujours surpris par ses propriétés.
Bon, c’était pas trop mauvais, même si les grumeaux formés par la crème
étaient un peu bizarres.</p>
<p>Dans la maison pour l’instant nous sommes neuf :</p>
<blockquote>
<ul class="simple">
<li>Grant, le proprio. Il avait les cheveux mi-longs hier matin,
mais… Il est sympa, un peu “groovy”, il me fait penser à Waynes
dans Wayne’s World . Il est plus sérieux que ce que je pensais au
début : il structure la maison et a plus ou moins un vrai travail : il
s’occupe de ce qu’il appelle des autistes, mais je me demande si ce ne
sont pas des handicapés mentaux, un peu en tout cas.</li>
<li>Andrew, un “autiste” de Grant, qui a plutôt l’air d’un anormal que
d’un autiste. On ne le voit pas souvent : il reste dans sa chambre à
regarder la télé, ou il va faire du skateboard dehors. Il paraît avoir
une vingtaine d’années, mais cela n’est pas évident de lui donner
réellement un age. Il a l’air sympa, sauf que je ne comprend rien de ce
qu’il dit car il s’exprime dans une sorte de jargon personnel, et puis
que de temps en temps il se passe des choses bizarres (comme une semaine
avant que j’arrive, où il était de mauvaise humeur, et sont skate est
passé à travers une fenêtre de la cuisine).</li>
<li>Ulrich, un allemand qui était en stage chez en fabriquant
d’éléctro-ménager. Il est maintenant en vacances et part avec son sac à
dos à travers l’ile. Il est grand, baraque, mal rasé, a les cheveux
longs, et fait du surf.</li>
<li>Matt, un autre allemand grand, baraque, mal rasé, mais sans cheveux
longs. Il fait aussi du surf, rentre à des heures absolument impossibles
les soirs, à un grand sourire, une voix super grave, et est un pur beau
gosse. D’ailleurs on voit souvent sa copine, Tony, qui est une kiwi, et
habite Portobello, sur la péninsule. Lui vient de finir son doctorat en
bio-informatique, mais en pratique moins il en fout, mieux il s’en porte
!</li>
<li>Mark, un anglais, qui viens du lake district, d’une ville où je
suis passé en septembre en randonnant. Il travail dans le bâtiment pour
l’instant, mais se fait domicilier ici pour pouvoir entrer à l’université
en payant moins cher et faire des études de sport : il aime la montagne,
et voudrait en faire son métier. Il est un peu bourru mais sympa derrière
cela.</li>
<li>Louise, la copine de Mark, qui pour l’instant se balade en Europe
(à Alban Berg en se moment), où elle fait de la compétition de descente
en luge solo (du squelette, pour les initiés).</li>
<li>Ellen, une Danoise végétarienne, qui fait des études à l’université
(mais j’ai oublié de quoi), et qu’on ne voit pas souvent en se moment car
elle fait du dériveur dans la baie.</li>
<li>Tim, son copain, qui n’habite pas officiellement ici, qui ne mange
pas avec nous, mais qui dort souvent ici ; il est californien, et fait de
l’informatique, je crois.</li>
<li>Gaël, un ahuri de français.</li>
</ul>
</blockquote>
<p>De plus nous avons une grande chambre de libre…</p>
<p>La maison est rarement entièrement pleine, il y a toujours du monde qui
se balade à droite et à gauche, mais en plus il y a les amis des
locataires, les anciens locataires qui viennent rendre visite… Bref, il
y a parfois de l’animation.</p>
<hr class="docutils" />
<p>Jeudi après-midi j’en avais vraiment marre de bosser. Je perdais mon
temps car je n’avais pas des boulons non-metriques (ces tarés
d’amerloques utilisent le système impérial, et donc les pièces qu’ils
nous ont fournis sont taraudées avec un pas non standard). Comme on avait
un visiteur de Boulder dont c’était le dernier jour, Nicola, qui avait
travaillé avec lui, lui a proposé d’aller visiter la péninsule. Et elle
m’a proposé, ainsi qu’à Stefan. Vu comme le temps était superbe je me le
suis pas fait dire deux fois, surtout que avec une voiture cela aide pas
mal.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/boulder-beach.jpg" style="width: 40%;" />
<p>Nous sommes donc partis pour la pointe de la péninsule (“point
Taiaora”, je crois) où en centre d’ “éco-tourisme” permet pour 40 dollars
(20 euros) d’admirer les rares albatros royaux. Bien sûr il n’était pas
question de claquer 40 pour cela et en marchant le long d’une plage nous
avons pu voir les superbes oiseaux :</p>
<blockquote class="epigraph">
“ A peine les ont-ils déposés sur les planches,
Que ces rois de l’azur, maladroits et honteux,
Laissent piteusement leurs grandes ailes blanches
Comme des avirons traîner à côté d’eux.”</blockquote>
<p>Je comprend maintenant les vers de Baudelaire : les albatros ne peuvent
s’envoler s’il n’y a pas suffisement de vent. Ils utilisent le vent comme
en parapente pour décoller. Heureusement ce jour là il y avait du vent.
Si non nous n’aurions rien vu.</p>
<p>Sur la plage se prélassaient aussi des “fur seals”, des phoques. Se
sont de grosse bêtes assez impressionnantes qui n’ont absolument pas peur
de l’homme. Un homme nous a même montrer un terrier de Pingouins Bleus,
mais nous n’avons pu voir que le bout de leur nez, car ils se terraient
au fond.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/sandfly-bay.jpg" style="width: 40%;" />
<p>Prochaine étape : “Sandfly beach” où une colonie de pingouins aux yeux
jaunes est établie. En sortant de la voiture la vue sur la plage était
fabuleuse : une sorte de rivière de sable descendait de la pointe en face
pour aboutir à la plage. Au milieu de ces pentes de sables des dunes
dépassaient, avec par endroit des grandes touffes d’herbe. La descente
c’est faite en courant dans une grande pente de sable et à l’arrivée nous
avions tous les chaussures pleines. Il fallait traverser la plage pour
arriver à une cache d’où on pouvait observer les pingouins sans les
effrayer, et le vent nous soufflait du sable à la figure, dans les yeux,
si violemment que nous regrettions les lunettes de protection du labo. De
long rouleaux rentraient dans la baie et defferlaient sur quelques
centaines de mètres tandis que le vent faisait s’envoler l’écume de leur
crête c’était superbe. Arrivé à l’autre bout de la plage nous l’avons
quitté pour nous cacher dans un abris aménagé par le “DOC” : Departement
Of Conservation, les Eaux et Forêts locales. Les pingouins sont très
craintifs, et ils ne se montrerons pas s’ils peuvent voir des humains. Le
soir ils rentrent de la pêche pour monter dans leur nids et nourrir leurs
petits. Il faut donc entendre, les jumelles rivées aux yeux, qu’ils
veuillent bien se montrer. Nous avons observé la bande de rochers qui
sépare la mer de leurs nids pendant une bonne heure, ce qui nous a permis
de repérer un bon nombre de phoques que nous n’avions absolument pas vu
au début et un jeune pingouin allongé sur le ventre au pied de la
colline. Nous commencions à désespérer quand le premier s’est montré. Il
est sorti de l’eau, pataud, se dandinant pour aller vite se cacher dans
les rochers, fauché de temps en temps par les vagues, ce qui entrainait
immanquablement une glissade sur le ventre. Un autre a suivi assez vite
puis ils sont restés dans les rochers pendant une demi heure (il n’y
avait qu’une centaine de mètres de largeur de rocher). De temps en temps
ils se décidaient à avancer de quelques mètres. On les voyait se courber
comme des bossus et se redresser d’un seul coup en avant, bondissant sur
un nouveau rocher. Au pied de la colline ils ont commencé leur longue
ascension quotidienne : bien 150 de dénivelé sur une pente très raide
pour arriver à leur nids. Nous les avons quitté là : il commençait à
faire froid. Sur la plage, en marchant vers la voiture, en nous
retournant, nous avons appercu une autre forme bossue et trapue se
dandinant hors de l’eau. La remontée vers le parking s’est faite en
plantant l’avant des chaussures dans le sable, comme dans de la neige.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/boulder-waves.jpg" style="width: 40%;" />
<p>Pour une soirée improvisée je dois dire qu’elle était réussie, d’autant
plus que en rentrant j’ai découvert que Matt et Ulrich avaient préparés
des crèpes.</p>
<p>Le lendemain j’ai bossé comme un malade pour terminer de boulonner tout
ce que je pouvais (ce qui n’était pas une très bonne idée car lundi
j’étais à cours de boulot). Le soir était le nouvel an chinois. Il y a
une certaine population chinoise ici et une occasion de faire la fête ne
se refuse jamais. Je suis descendu à l’ “Octogon”, la place centrale de
Dunedin en forme d’octagon, et des stands vendaient de la nourriture
chinoise tandis que sur une estrade se succédaient de danseurs, des
musiciens, et autres artistes. C’était asiatique et exotique, mais rien
comparé à Hong Kong. A minuit on attendait un feux d’artifice. Il
n’avait pas encore plut de la journée, ce qui est un peu louche ici, mais
la météo s’est vengée ; heureusement avec quelques gouttes, et cela n’a
pas compromis le feu d’artifice qui s’est révélé superbe.</p>
<hr class="docutils" />
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/cluso-silverpeaks.jpg" style="width: 40%;" />
<p>Il y a un allemand super sympa au labo, dans le groupe des théoriciens.
Il s’appelle Christophe et est ici depuis Octobre. Il fait une sorte de
stage pratique que doivent faire tous les allemands pour ce qui ressemble
à leur maîtrise.</p>
<p>Bref quoi ce marrant au grand sourire, au gros sac à dos, au pantalon
kaki de randonnée, a l’air motivé pour se balader. Il a acheté une voiture
(elles sont vraiment très bon marché ici, mais je ne crois pas que je
vais en acheter car il faut que je limite mes dépenses). Il a donc
acheté une vieille guimbarde et a déjà pas mal visité l’ile avec ses amis.
Il dit que presque tous les soirs, il les passe à la plage, et qu’il
n’est jamais chez lui quand il fait beau. Qu’à cela ne tienne, c’est
plutôt un programme qui me plait. Il m’a donc proposé d’aller marcher
quelque part dimanche, et on s’est donné rendez-vous dimanche matin dans
ma maison autour d’un petit déjeuner, avec une carte pour discuter du
programme.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/silverpeaks.jpg" style="width: 40%;" />
<p>Dimanche on a décidé d’aller faire les “Silver Peaks”, quelques
dizaines de kilomètres au nord de Dunedin, se sont les plus hauts sommets
du coin (760 m). Nous sommes donc partis dans sa Ford Cortina délabrée
sur les routes de Nouvelles Zélande. A fond (80 km/h) dans une telle
voiture sous un fabuleux soleil, avec “The Wall” en fond sonnore, à
travers des vallées où on ne croise que deux trois fermes c’était génial.
D’autant plus que dés qu’on a quitté le highway (une bonne route bien
marquée et bien goudronnée avec même de temps en temps deux files) on a
été bons pour un chemin de terre et de gravier. Il y en a énormement en
Nouvelle Zélande. Je dirais bien que la moitié de leurs routes sont
ainsi. La cortina a peiné, et les fenêtres ont étés fermés pour essayer
de garder la poussière à l’extérieur mais nous sommes arrivé, après être
passé devant plus d’une grande maison isolée, devant un panneau “DOC,
SilverPeak Reserve”. Une fois la voiture garée, et après être monté sur le
toit pour prendre une photo, nous nous mettons en route vers les pics en
question. Le terrain est très varié. Nous commençons par progresser dans
une sorte de garrigue avec de gros buissons sortant d’une couche épaisse
d’herbe, puis nous nous enfonçons dans des broussailles qui nous arrivent
jusqu’aux épaules, et par moment nous disparaissont dans une petite forêt
si dense qu’on ne peut faire trois pas en dehors du chemin. Après deux
petites heures de marche nous arrivons au sommet où une vue sur le
“central Otago” (la région à l’intérieur des terres) nous attend. Des
petites montagnes s’étendent devant nous jusqu’au “Ducan’s Range” qui
nous barre la vue et nous empêche de voir les alpes Néo-Zélandaises. De
la broussaille, essentiellement vierge à perte de vu, avec des rochers ou
des arbres dépassant par endroit. On se serait cru dans le Seigneur des
Anneaux, à la porte du Rohan. Cela ne m’étonnerait pas qu’ils aient tourné
cette partie dans le coin.</p>
<img alt="" class="align-center" src="https://gael-varoquaux.info/personnal/attachments/nz/waitati.jpg" style="width: 80%;" />
<p>Un bon casse croûte au sommet fait toujours plaisir (même si le vin
manque indéniablement). Puis c’est la descente, et en un rien de temps
nous sommes de retour à la voiture. C’est fabuleux comme on peut vite se
soustraire à la civilisation dans ce pays.</p>
<p>Comme la balade a duré moins longtemps que ce que nous avions prévu il
nous reste du temps. Chris m’enmène donc à “Long Beach”, à travers une
succession de ces routes de graviers qui traversent la montagne. Là bas
je découvre des kiwis accrochés sur une falaise : c’est l’endroit où il y
a des voies d’escalades équipées le plus proche de Dunedin. Sur la plage
Reece, Gronia, et toute le bande sont en train de faire un grand
barbecue. Le barbecue est interrompu quand Reece envoie leur Freeze-Be
dans l’eau. Cela se termine en baignade dans une eau un peu froid. Froide
peut-être, mais il y a des surfeurs qui en profitent, avec combinaisons,
certes, mais ils se débrouillent bien.</p>
<p>Chris m’enmène voir les grottes au bout de la plage. Malheureusement
nous n’avons pas de lampe torche et nous n’allons pas bien loin. Il
faudra y retourner. Le soleil commence à se coucher et nous retournons à
Dunedin. Un petit arrêt au supermarché pour refaire des provisions et
nous allons à l’appartement de Chris où nous nous cuisinons des pattes
aux légumes à la Chinoise (pas mal la recette, je retiens). Cela se
termine en écoutant des CD sur la chaîne de Ch ris. J’ai récupéré le
Requiem de Mozart et les concertos pour Orgue de Haendel, tout est sauvé.</p>
<p>Lundi matin il faut reprendre le boulot, et avec des coups de soleils,
mais j’ai des projets fous en regardant la carte. Le seul problème est
que les endroits qui m’interresent le plus sont vraiment perdus au milieu
de nul part et tout à fait inaccessibles (quand je dis inaccessibles, ici
cela veux dire des centaines de kilomètres carrés sans route ni quoi que
ce soit..</p>
<blockquote>
Affaire à suivre.</blockquote>
Ma Nouvelle maison2003-01-26T00:00:00+01:002003-01-26T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-01-26:/personnal/ma-nouvelle-maison.html<p>Je suis dans ma nouvelle maison, devant mon écran favoris. Ma chambre
est plutôt vide : Ellen et son copain Tim sont revenus de l’autre bout du
monde et ont récupéré le gigantesque lit double. A la place j’ai mis
deux matelas l’un sur l’autre : on se …</p><p>Je suis dans ma nouvelle maison, devant mon écran favoris. Ma chambre
est plutôt vide : Ellen et son copain Tim sont revenus de l’autre bout du
monde et ont récupéré le gigantesque lit double. A la place j’ai mis
deux matelas l’un sur l’autre : on se croirait dans une chambre de l’ens
(sauf que les deux matelas seraient l’un à coté de l’autre). Il ne reste
plus dans la chambre qu’une vielle table miteuse une chaise et un
fauteuil. Tout cela est bien sûr condamné à disparaître. Il va falloir
que je me trouve un moyen d’acheter des meubles bons marché.</p>
<p>Hier des tonnes de gens ont commencé à débarquer dans la maison : les
anciens habitants reviennent de vacances. Certains déménagent, d’autres
se rengagent. Quand elle est occupée comme cela la maison est plus
sympathique. Un des habitants, Ulrich (et oui :-] ) a débarqué avec trois
autres Allemands qu’il avait rencontré en visitant l’ile du nord. Et
comme il y avait une soirée pour célébrer la fin de la thèse de Matt, un
autre Allemand (moi aussi je suis perdu dans les prénoms et les gens,
alors je vous expliquerai quand je saurai plus qui est qui), ils se sont mis
en tête de faire un gigantesque repas Allemand. Ça parlait Shleux à tue
tête dans la cuisine avec un fond sonore de “???muzik” (ils m’ont
expliqué que cela voulait dire “noisy music”, la qualité en tout cas
reflète le nom. Le résultat c’est que vers 21h30 on est passé à table
avec une gigantesque salade et un plat de pattes-oignons-fromage. Il y
avait une bonne vingtaine de convives et c’était très sympa. J’ai goutté
à la Lager locale, la bière fabriquée la plus au sud du monde et aux vins
Néo-zélandais et Australiens. Je dois avouer que j’ai préféré
l’Australien, mais tous les deux étaient plutôt sirupeux.</p>
<p>Après ce somptueux repas les teutons ont mis la musique fort et sont
sortis dans le jardin (pour fumer une clopper bien sûr). Tous ont
graduellement suivit. Ils n’y avait pas que les teutons et les habitant
de la maison. Je me souviens d’une femme avec qui je devisais gaiement
qui déclare soudain qu’il ne faut pas qu’elle mange de trop car elle
avait “a very long way home”. Je m’enquière du temps que cela prendrais,
m’imaginant qu’elle habitait dans le “central Otago” et elle me répond 20
minutes. Ce n’est clairement pas la même mentallitée ici. Les gens sont
décontractés, c’est agréable.</p>
<p>Le temps est toujours assez pluvieux alors j’ai emprunté une carte de
l’ile du sud ainsi qu’un livre intitulé “Wild New Zeland” et j’ai
commencé à tripper. C’est clair je vais essayer de prendre le maximum de
longs week-ends et d’aller me balader dans l’ile.</p>
le labo2003-01-24T00:00:00+01:002003-01-24T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-01-24:/personnal/le-labo.html<p>Puisque je suis au boulôt, parlons boulôt :</p>
<p>Andrew est un type de ma taille, la quarantaine, assez sympa mais
plutôt mou. Il dirige l’équipe expérimentale qui est constituée de deux
post doc : Calum, un petit gros qui se balade toujours avec des t-shirts
où il y a écrit “Hacker …</p><p>Puisque je suis au boulôt, parlons boulôt :</p>
<p>Andrew est un type de ma taille, la quarantaine, assez sympa mais
plutôt mou. Il dirige l’équipe expérimentale qui est constituée de deux
post doc : Calum, un petit gros qui se balade toujours avec des t-shirts
où il y a écrit “Hacker” et quelque chose, et Gronia, une petite timide
absoluement charmante. Je ne vois pas beaucoup les post-doc car pour
l’instant je ne suis pas souvent au labo, et mon bureau est deux étages
au dessus du leur. Je le partage avec les thésards : Nick, Reece, Angela
et Nicola (une fille, ne nous trompons pas). Angela travaille avec Gronia
sur me condensat déjà en place, Nicola a une manipe de spectro
compliquée, Nick construit un piège magnétique (un classique, mais j’ai
oublié le nom), et Reece passe ses journées devant son ordi à traiter des
données qu’il a acqui il y a quelque temps. A coté de mon bureau il y a
le bureau des thésards théoriciens. En moyenne se sont des glaucques,
sauf un trés sympa qui s’appel Christophe et qui vient d’Allemagne. De
plus dans l’équipe il y a un technicien spécialisé dans les lasers. Il a
passé le dernier mois à construire des diodes lasers à cavitée étendue.
En se moment il était en train de faire un montage d’absorbtion saturée
mais on l’a démonté pour faire de la place pour la nouvelle table.</p>
<p>J’ai un peu fait joujou avec les ECDL (external cavity diode laser),
deux sont pour moi. C’est cool d’arriver et d’avoir tout de fait…</p>
<p>Question rythme cela n’est pas effrayant : au début je me pointais à 9
heures, mais il n’y avait personne. Ils arrivent à 9 h 30. A 11 h on a le
“tea”, je n’ai pas la moindre idée de quand est la pause déjeuné. Pour
moi la semaine dernière c’était à 12h30, car il fallait que je regagne
mon “college”. On a une deuxième pause tea vers 4 h et les bureaux
ferment officiellement à 5 h. Cela veux dire que à 5 h 30 il n’y a déjà
plus Andrew, et que les thésards, eux, partent plus tard, les bons jours.
Pour ma part je quittais à 5 h 30 car le diner était à cette heure là ;-/
.</p>
<p>Aujourd’hui on a recu la table optique : 3m de long sur un de large et
pesant 800kg (sans les pieds) c’était impréssionant : pour la monter une
grue l’a prise et l’a posé sur le balcon, ou plutôt l’a posée dans
l’embrasure de la porte qui donne sur le balcon, sur un gros chariot qui
était dans le batiment (tu passe un bout dans la porte, tu pose l’autre
parterre, tu bouge les sangles pour qu’elles ne soient plus fixée qu’au
bout qui dépasse, et en faisant rouler l’autre bout sur le chariot tu
glisse tout la table dans l’embrasure pendant que le Français de service
glisse un deuxième chariot sous le deuxième bout (oui, je suis bien le
seul Français ici, il y avait bien un Tahissien au St Margaret’s College,
mais cela ne compte pas).</p>
<p>On a amené la table dans le couloir mais comme Gronia et Angela
faisaient des mesures, et des bonnes, on l’a laissée là, sans toucher à
rien, et elle attend dans sa grosse boite en bois qu’on la déballe…</p>
<hr class="docutils" />
<p>Bon, suite de l’histoire, le lendemain (aprés que j’ai oublié ma carte
magnétique pour rentrer dans le bâtiment et que j’ai abandonné mon
portable et le mail dans mon bureau).</p>
<p>La table est toujours là, elle est partiellement sortie de sa boite de
bois, mais elle repose toujours en équilibre sur la tranche, sur le fond
de la boite. Nous la rentrerons probablement lundi matin tôt.</p>
<p>Le labo est formé de deux, bientôt trois, tables optiques, entourées
d’une grande infrastructure qui sert à porter un étage d’instruments au
dessus et à fixer par des scratchs des grands caches noirs. Sur l’une
d’entre elles il y a un laser titane-saphir, qui est le laser principal.
Il passe dans la manipe de Nicola et sur le table à coté, pour faire le
refroidisseur-piégeur pour le MOT de la machine à BEC en place.</p>
<p>Le plus amusant est que en se moment il n’y a que des filles dans le
labo : Angela et Gronia, qui passent leur temp à maniper. C’est pas
souvent de voir un labo entièrement feminin (et en plus se ne sont pas de
monstre aséxué comme c’est généralement le cas). Cela doit pour cela que
le labo est parfaitement en ordre et propre. en fait non, je crois que
c’est plutôt lié à la manipe : l’optique doit rester trés propre. On a
donc une clim qui maintient la pression interne au dessus de la pression
atmosphérique pour garder les poussière à l’exterieur. Et quand on ouvre
la porte externe du sas, tout le labo est mis au courant par un
bienveillant buzzer qui incite gentillement les gens à ne pas la laisser
trop longtemps ouverte.</p>
<p>Dés que le titane-saphir est en marche tout le monde met des lunettes
protectrice. Cela fait trop cool les superbes reflets verts. C’est bète
que Oakley ne fasse pas des lunettes protectrices…</p>
<p>Ce soir Andrew m’a aidé à déménager. Heureusement car il y avait une pluie
du tonnerre. et il s’est subitement mis à faire froid (les maisons ici
n’ont bien sûr pas de chauffage central). Je suis arrivé dans ma chambre
et j’ai frotté tout ce que j’ai pu, passé l’aspirateur, mais
malheureusement je n’ai pu terminer mon travail car je n’ai pas les
produits nécessaires, et Grant (qui sait où il se trouve) n’est pas
là.</p>
<p>Il a fait 4 celcius cette nuit : on avait du vent du sud qui venait
directement de l’Antartique. Comme je n’avais pas de chauffage j’ai
utilisé ma technique de rando : une polaire par dessus le pijama, cela
marche trés bien et ce n’est même pas inconfortable.</p>
Je me trouve un appartement2003-01-23T00:00:00+01:002003-01-23T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-01-23:/personnal/je-me-trouve-un-appartement.html<p>Ce message est sans accent : les claviers Néo-Zélandais ne pardonnent
pas…</p>
<p>Aujoud’hui j’ai trouve un appart. !! Je commencais a streeser
car je ne trouvais rien. A chaque fois c’etait la meme chose : Yes, yes,
well great, do you have a phone number where we can reach you …</p><p>Ce message est sans accent : les claviers Néo-Zélandais ne pardonnent
pas…</p>
<p>Aujoud’hui j’ai trouve un appart. !! Je commencais a streeser
car je ne trouvais rien. A chaque fois c’etait la meme chose : Yes, yes,
well great, do you have a phone number where we can reach you. All
right, we’ll get back at you.. , et puis apres plus rien :-< .</p>
<p>Alors quand le gars me dit aujourd’hui “So, are you interested ?” Bien
moi je fais yes, et puis ca c’est passe. C’est pas une super maison (
plutot pourrie meme), mais bon, je crois que j’ai un peu panique. Bon,
c’est pas trop cher. Les types sont un peu bizarres… Il y a un couple
de gais (Mecs, cela m’aurait moins gene si c’etaient des filles). Le
proprio ( ou plutot le fils des proprios, qui habite la ) a les cheveux
longs (affreux non), et quand on lui demande ce qu’il fait dans la vie,
il repond qu’il s’occuppe d’un autiste. En fait l’autiste habite dans la
maison, et je crois que le gars en question se contente de ne rien faire
de ses journees et de toucher les loyers ainsi que la prime pour
l’autiste.</p>
<a class="reference external image-reference" href="my-house.jpg"><img alt="my house" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/my-house.jpg" style="width: 50%;" /></a>
<p>Quant a la barraque, c’est une grande vieille barraque. Vieille cela ne
veut pas dire quelle a de l’age, mais qu’elle est en mauvaise etat.
Quand a l’ammeublement, bein pour l’instant j’ai un matelas, et puis un
tas de vieux trucs poussiereux, plus tot des cartons.</p>
<p>Bon, he bien je sens que je vais passer beaucoup de temps en activites
diverses et variee a l’exterieur.</p>
Voyage et arrivée en Nouvelle Zélande2003-01-19T00:00:00+01:002003-01-19T00:00:00+01:00Gaël Varoquauxtag:gael-varoquaux.info,2003-01-19:/personnal/voyage-et-arrivee-en-nouvelle-zelande.html<p>Je suis dans ma chambre et je commence donc a écrire un e-mail que je
ne pourrais envoyer (un peu débile mais j’ai des difficultés pour me
connecter à clipper)… Bon, au moins j’ai un CD de Chick Corea qui
tourne…</p>
<hr class="docutils" />
<p>Je suis donc parti avec mes quarante …</p><p>Je suis dans ma chambre et je commence donc a écrire un e-mail que je
ne pourrais envoyer (un peu débile mais j’ai des difficultés pour me
connecter à clipper)… Bon, au moins j’ai un CD de Chick Corea qui
tourne…</p>
<hr class="docutils" />
<p>Je suis donc parti avec mes quarante kilos de bagages de Choisel mardi
matin. En effet j’avais appris sur le net que j’avais peu de limitations
sur les bagages, donc j’avais 22 kilos dans mon sac jaune, une vingtaine
sur le dos plus un charmant sac rouge Millet. J’arrive à Charle de Gaule
et j’embarque sans le moindre encombre dans un 747 de la Cathay Pacific
Airways, une compagnie basée à Hong Kong comme je devais l’apprendre.
Comme dans tout avion qui se respecte la première chose que nous
faisons c’est de manger : repas chinois, naturellement. Je commence à
découvrir que la Cathay Pacifique fait les choses bien pour occuper ses
passagers : sur le dossier de chaque siège il y a un écran LCD connecté à
un fabuleux système informatique qui permet entre autre de regarder des
films que l’on choisi parmi une sélection. Résultat je vois plusieurs
film en même temps sans en regarder un seul. Cela me permet de voir mon
premier Jackie Chan (assez remarquable, inoubliable même, tellement
c’était nul), et un film chinois qui n’avais pas l’air mal (“Ma femme a
18 ans”, je crois).</p>
<p>Bien sûr l’avion est quasiment entièrement plein d’yeux bridés. Toutes
les hôtesses ont d’ailleurs le teint jaune réglementaire, mais leur
Anglais et parfait, meilleur que leur Coréen, comme l’a découvert un
malheureux passager. De plus j’ai beaucoup apprécié leur uniforme, moins
crâneur et plus joli que celui des hôtesses occidentales. Par exemple les
cols des chemises, au lieu d’être des décoltés compliqués comme sur la
pub de Chanel qui se baladait partout sur les journaux, étaient comme
celui de l’héroïne de “In the mood for love”. Bref quoi, un charmant
dépaysement, d’autant plus que je devais converser plus ou moins par
signes avec mon voisin, un vietnamien.</p>
<p>J’essaie de dormir, et j’y arrive pas trop mal. Au bout de onze
heures de vols nous atterrissons à Hong Kong International. Je
débarque de l’avion, m’attendant à rester bloqué dans l’aéroport. Au bout
d’un moment j’arrive à trouver parmi les personnages (des deux sex)
habillés de noirs avec une grosse ceinture, un talky-walky et les cheveux
courts une qui sache parler anglais. on m’informe alors que pour pouvoir
sortir de l’aéroport il me suffit de remplir une fiche d’une demi page
(au stylo bic et en lettres capitales, attention). C’est l’affaire de
quelques minutes et je suis sorti immédiatement. A moi Hong Kong !</p>
<hr class="docutils" />
<p>Il me faut d’abord prendre le train pour aller en ville. Ah, si les
métros parisiens étaient comme cela ! Encore une fois il y a des écrans
LCD derrière les fauteuils, qui informent cette fois des arrivées et des
départs.</p>
<p>Par la fenêtre je commence à découvrir ce à quoi la baie de Hong Kong
ressemble : des bouquets de grattes ciel éparses sous des montagnes
verdoyantes qui se reflètent dans l’océan. Entre les bouquets des
autoroutes portées sur des passages aériens par des piliers de béton. Au
fur et à mesure que l’on se rapproche de la ville les bouquets se
resserrent, jusqu’à ne plus former qu’un continuum. Et là le train passe
dans un tunnel. J’arrive finalement en gare de Hong Kong central. Il
est huit heures heure locale, et de nombreuses personnes bien habillées se
pressent dans les rues vers de grands grates-ciels à l’effigie de Banque
of China ou de HSBC.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/hong-kong-sky-scrapers.jpg" style="width: 30%;" />
<p>Je décide de marcher et après être passé entre quelques grattes-ciel
super modernes je décide de me diriger vers les sommets verdoyants qui
dépassent derrière. La route à suivre n’est pas trop dure : il suffit de
toujours aller vers le haut. J’emprunte donc des escaliers et des
passerelles qui se faufilent entre les ponts et les échangeurs qui
portent les routes. Je passe donc dans une sorte de dentelle de béton
recouvert de mousse verte par endroit digne d’une super cité des mangas
actuels. Plus haut j’arrive dans une banlieue résidentielle chic : les
rues passent à flanc de montagne et de part et d’autre se dressent des
immeubles portant des noms comme “Imperial court” ou “Clyde mansion”. Au
niveau de la rue un portier en uniforme monte la garde ; en dessous des
étages sont réservés aux garages, et au dessus, dans les hauteurs, des
appartements s’étalent.</p>
<p>Pour passer d’une rue à celle d’au dessus je ne tarde pas à découvrir
que la bonne solution est d’emprunter des escaliers qui prennent droit dans
la pente. On voit que les habitants de Hong Kong n’ont pas peur de monter
: des centaines de marches se suivent sans arrêt. Au bout de quelque
temps j’arrive à une rue sans immeubles au dessus. la ville s’arrête là,
brutalement, et le reste de la montagne est une jungle épaisse et
clairement infranchissable. Ma quête des hauteurs s’arrête donc net, et
je redescends bien vite.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/hong-kong-house.jpg" style="width: 30%;" />
<p>Cela me permet de découvrir une partie de la ville très interressante :
la partie pauvre, ou en tout cas moins riche que celles que j’avais déjà
vues. Ici les rues sont plus étroites et plus animées. On ne parle pas,
ou peu anglais. De petits magasins et échoppes s’ouvrent partout sur les
cotés, et des pub en Chinois dépassent dans la rue. Un ricanement vient
commenter la scène : perché sur un arbre qui dépasse de Dieu sait où un
perroquet contemple la scène. Des panneaux traduis en anglais m’informe
que des rats se baladent dans le coin, et que leur morsure est
empoisonnée ; même pas peur, NA !</p>
<p>Il me reste quelques dollars de Hong Kong et j’ai envie, juste pour
m’amuser, de prendre un vrai repas Chinois. J’avise un restaurant pas
trop miteux, et j’entre, priant pour que quelqu’un parle anglais. Je
converse dans un mélange d’anglais et de langage des signes avec un type,
mais quand le moment viens de lui dire que je n’ai plus que 50 HK$, et
que j’aimerais être sûr que cela fera l’affaire, c’est la débâcle.
Heureusement une femme vient à mon secour et je peux enfin déguster mon
riz et mon porc. Ils m’avaient même rajouté une cuillière à soupe, des
fois que je sois trop paumé pour me servir des baguettes. je peux te dire
que la nourriture de tous les jours du chinois de Hong Kong ne vaut
absolument pas les restaurants chinois du 5ème : le riz n’est absolument
pas cantonnais, et la viande vient sans sauce. Cela avait cependant la
saveur de l’autenticité.</p>
<p>Je décide alors de regagner l’aéroport : je n’avais pas vraiment
l’intention de rester bloqué à Hong Kong. Cela me faisait un peu mal de
quitter ce quartier si vivant… Bien sûr j’arrive à la porte
d’embarquement trois heures en avance : dans un petit aéroport bien
organisé, tout va tellement plus vite.</p>
<hr class="docutils" />
<p>Nouveau vol avec la Cathay Pacific ; cette fois c’est un 737, et je me
fais bien voir de l’hotesse en prenant du vin blanc avoir mon poisson et
du tea, sans sucre après tandis que tout le monde lui demande du Sprite.
Mon voisin regarde des films débiles et j’ai plus de mal à dormir.
Heureusement je suis à coté de la fenêtre, et je peux voir des iles
Indonésiennes avant que le soleil se couche. C’était le premier coucher
de soleil au quel j’ai assisté au dessus des nuages. C’était plutôt
joli, surtout quand les colonnes des cumulus qui dépassaient en dessous de
l’appareil sont restés illuminées alors que tout le reste était dans
l’ombre.</p>
<p>J’atteris à Auckland où la “Food and Drug Administation” locale passe
mes sacs aux rayons X et me fait déballer ma tente pour vérifier qu’elle
ne contient pas d’organisme étranger dangereux à l’écosystème local.
Pour aller du terminal international (qui n’a d’international que le nom
et les douanes) au terminal domestique je passe sous un fabuleux soleil.
Ma veste de quart passe vite fait dans mon gros sac jaune. Le terminal
domestique est minuscule, il n’a que deux portes. Les hôtesses de la
Quantas sont de banales blondes, qui ne valent pas les chinoises de la
Cathay question dépaysement.</p>
<p>Mon avion, un petit coucou à moitié plus petit que le 737, décolle et
survolle de fabuleux paysage avec un nombre incroyable d’iles. Hélas je
crois que c’est trop au nord, je ne naviguerai pas dans ces archipels.
Après un snack indispensable à l’éthique de l’air nous atterrissons à
Christchurch International. Comme j’ai de l’avance je me fais plaisir à
visiter l’aéroport. Le terminal du quel je pars est un hangar modifié qui
donne directment sur le tarmac. Peu de temps avant le départ je vois
atterrir un splendide coucou, bimoteur, à hélice, je le trouve tellement
charmant que je le prend en photo : cela doit être mon avion, d’autant
plus qu’il s’arrête devant mon terminal. Loupé, un autre encore plus
petit arrive et on charge mes bagages dedans. Il est tellement plus petit
qu’il tiendrait caché derrière un double-decker londonien. Je compte les
places : trente. C’est un bus local. L’équipage est constitué d’un pilote
et d’un co-pilote, ainsi que d’une hôtesse, qui ne sert vraiment qu’a
vérifier que nous attachons bien nos ceintures. L’avion décolle en
vombrissant comme c’est pas croyable et par le long de la côte, le soleil
dans le dos, vers le sud donc. J’apperçois sur ma gauche les alpes
Néo-Zélandaises, et un sommet particulièrement haut. Cela doit être le
Mont Cook. Après un thé (hé oui, même dans un tel coucou l’éthique
aérienne exige que l’on serve quelque chose), et un bonbon apporté par
une petite gamine qui avait été confié à l’hotesse à l’aéroport, l’avion
se dirige en trésautant vers une piste d’atterrissage. L’atterrissage est
moins doux que dans un 747, mais je ne crois pas qu’il y avait de guidage
radar.</p>
<hr class="docutils" />
<p>A l’aéroport Andrew m’attend, sans panneau avec marqué “Gaël”, Ouf. Après
avoir chargé mes gigantesques sacs dans sa voiture (sur le jaune je
découvre une étiquette orange qui avait été rajoutée à Auckland : “Heavy,
22kg, bend your nees”). On part donc vers la ville, il est 14h, heure
locale, et mon plus grand désir est de prendre un douche. Andrew
m’abandonne à St Margaret’s college, où il m’a réservé une chambre après
m’avoir expliqué où je pourrais trouver son bureau. Pour ma part après
une douche d’une bonne demi heure je par me promener en ville : il ne
faut pas que je dorme.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/dunedin-sunset.jpg" style="width: 50%;" />
<p>La ville est coincée entre des collines et la rade d’Otago. Elle
s’étend en fait sur une assez grande distance, mais les collines, dont
les flancs ne sont pas peuplés, donnent l’impression qu’elle est plus
petite. Elle a été peuplé il y a 150 ans par des Écossais, qui l’ont
nommé Dunedin d’après le nom gaélic de Eddinbourg.</p>
<p>Les habitants sont sympathiques, pour ce que j’ai pu voir. Je dinne
tous les soirs avec mes colocataires. Ils viennent de partout en Nouvelle
Zélande et en Australie et sont là pour des écoles d’été. Cela fait un
peu bizarre de découvrir à table que l’amie d’un tel qui habite en
Indonésie à comme “pet” trois paresseux, et que après leur douche elle
les pend au fil à linge. Cela me fait penser que je me suis trompé, il y
a bien un certain type de kangourous en Nouvelle Zélande, mais je ne
crois pas que je les verrai, car ils sont plus au nord.</p>
<p>Le groupe de Andrew à l’air sympa. Il est assez grand et formé
exclusivement de thésard et de quelques post-doc. Pour l’instant j’attend
la table optique qui doit être gruttée au troisième un de ses quatres
donc je me contente de faire de la biblio, c’est reposant. Je suis passé
à la banque pour ouvrir un compte et clairement là bas je ne ressemble
pas à quelqu’un de digne de confiance : non seulement on a mis plus de
temps à m’ouvrir que pour les autres personnes, mais en plus lorsque je
demande à ouvrir un compte la personne au guichet doit s’absenter avant
de pouvoir me donner une réponse (qui est : “repassez lundi”). Cela doit
être à cause de mes cheveux longs, et de mon ignorance totale du rasoir.</p>
<p>Aujourd’hui samedi il fait un temps superbe. Je suis allé à la plage.
Je ne me suis pas baigné car je n’avais pas pensé à prendre de maillot.
Il y avait de jolis rouleaux et pas mal de personnes avec des
skim-boards, mais pas de surf, je suis assez déçu. Demain je pars
randonner sur la péninsule d’Otago, juste à coté de la ville. Ils en font
toute une pub… je te dirais demain ce qu’elle vaut.</p>
<hr class="docutils" />
<p>Je cherche activement un appartement, mais pour l’instant le numéro de
celui qui m’interresse ne répond pas…</p>
<p>J’écris en ce moment de ma chambre au St Margaret’s College, avec le
superbe chat noir du College sur mes genoux. Il a décidé que j’étais
confortable et ronronne comme c’est pas croyable. Mais pour taper cela
n’est pas pratique alors je l’ai virer sur ma table à coté de mon
portable. il a immédiatement voulu écrire un message, mais je lui ai
interdit de tapper (j’ai jamais été un marrant tu sais), alors il est
descendu de la table et a visité la chambre (qui est dans un basard pas
croyable), puis est parti vers une autre chambre.</p>
<p>Il est 9 heures du soir et je suis crevé : je n’ai toujours pas
entièrement rattraper mon décalage horaire. Chaque jour je me force à me
coucher plus tard, et le matin à ne pas me lever avant le réveil.</p>
<p>YES je viens de remarquer que sur ma carte de téléphone il y a le Team
New Zéland : le défi Néo Zealandais à la cup of America.</p>
<p>Je disais donc que je vais me coucher….</p>
<hr class="docutils" />
<p>Bon, je n’ai pas pu envoyer le mail parce qu’il y avait
une coupure d’électricité qelquepart à l’ENS. C’est pas grave, je peux le
compléter avec ma journée d’aujourd’hui : aujourd’hui le but de la
journée était de visiter la célèbre Otago Peninsula, longue d’une grosse
trentaine de kilomètres. Je me lève tôt et je prend un bus qui doit
m’enmener à Portobello, au milieu de la peninsule. Mais le chauffeur avec
le quel j’ai tapé la discute, me dit qu’il doit aller chercher un groupe
à l’autre bout et me propose de m’enmener. Génial, cela me fait moins à
marcher, et donc je pourrais voir plus.</p>
<p>Au bout de la péninsule il y a une réserve à Albatros, mais elle est
privée et pour y accéder il faut payer. De toutes façons j’étais là trop
tôt, alors je ne pouvais même pas laisser parler ma radinerie : l’attrape
touriste était fermé. J’ai pu donc contempler de superbe falaises avec de
superbes oiseaux, que je ne connais. Puis retour vers Portobello en
marchant. Au fur et à mesure que je marche je me rend compte que sur la
plage il y a beaucoup d’oiseaux qui me sont inconnus, il faut croire que
je ne suis pas dans mon hémisphère habituel.</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/farm-1.jpg" style="width: 40%;" />
<p>Après deux heures de marches je peux enfin obliquer sur la gauche et
quitter la route principale pour une route secondaire, qui se révèle être
en terre. Elle me permet de passer de l’autre coté de la péninsule (celle
qui fait face à la mer) où une grande rade s’étale, peuplée de cygnes
noirs. Je marche (de temps en temps sous une petite pluie) et je m’élève
en petit peu au dessus de l’eau. La rade est alors en train de se vider
avec le jusant et d’en haut on voit de superbes traînées blanches et
vertes. A la sortie de la rade de grandes déferlantes naissent de la
houle du large. Je suis entouré de prés et de moutons, le voilà enfin le
fameux moutons Néo-Zélandais</p>
<img alt="" class="align-right" src="https://gael-varoquaux.info/personnal/attachments/nz/farm-2.jpg" style="width: 40%;" />
<p>Malheureusement je ne peux continuer jusqu’au phare qui me tentait :
ma maléole (que je me suis abîmé il y a trois semaines en patin à glace)
se met subitement à me faire très mal. Il faut que je rentre en boitant.
Dieu sait ce que j’ai, mais cela me casse bien les pieds (si je puis
dire). Je fais du stop (hé oui au fur et à mesure que je me rapproche de
la civilisation il commence à y avoir des voitures qui passent, une fois
toute les demis heures) et un couple d’anglais me prennent et me ramènent
à Portobello. De là un couple d’Espagnols me ramène à Dunedin.</p>
<p>Bon, je suis de retour dans ma chambre et je commence à enrager :
toujours pas d’appartement. J’en avais trouvé un bien, pas trop loin de
l’université, meublé (c’est important), avec deux filles gays (ça cela
explique peut-être le prix plutôt bas), mais leur téléphone semble ne pas
marcher. Je suis quasiment certain que cela veut dire qu’elles ont trouvé
un locataire. Zut flutte… Je m’étais fait à l’idée que celui là me
convenait. J’en essaie un autre, mais cela commence à être frustrant.</p>