Or, The Project Module and the Deadly Sins of Drupal Development

Forgive me, Drupal, for I have sinned. It's been many months since my last confession.

At the Drupal Association we've set upgrading drupal.org as our top priority this year, and much of that work depends on project.*.

The Project module and kin are some of the most important pieces of drupal.org--and some of the most difficult. Six months after Drupal 6 came out, drupal.org still is still running Drupal 5, in large part because project.* is not yet upgraded.

The project modules deliver critical functionality used every day by thousands of Drupal site administrators and developers. The module set has got a lot of great work from maintainers Derek Wright and Chad Phillips and others, notably Adam Light. But - despite significant progress - the codebase is large, uneven, and difficult to maintain.

How did we get to this place? What can we learn from the experience?

I spent a few months some time back contributing to the project module. If the module set is a challenge, I can claim a portion of the credit. Here are my confessions.

The sin of hoarding

I have committed the sin of piling up too much in one place.

Lots of us have.

It's kinda like the cardinal sin of gluttony.

Project pioneered a lot of functionality in Drupal--automated mail advisories, file attachments ... the list goes on. In my case, the functionality included a query builder. We needed the ability to browse projects by multiple criteria (date, name, category...). So I coded a query builder--great idea, no?

Except that the limited and rudimentary query builder I wrote was just for Project module. What would it have looked like if, instead, we had worked on or supported a generic query builder, one that could be used in Project but equally in any other case where there was a need to filter and present lists of items?

"You mean, like Views," you ask? Yes, like Views. And, in fact, a month before I committed the query builder patch to project module, the first version of Views was posted. So, big mistake, and years later we're still paying for it. And, Project has tons of specialized hard-coded fields (CCK, anyone?), special case handling (actions and triggers?), and so on.

The sin of being wedded to a single site

I have committed the sin of site-specific code.

Maybe this is the sin of lust? "Drupal.org, I love you above all others!"

Over and over again, we wrote Project module code for the single, very specific use case of drupal.org. Various areas of functionality are so site-specific that they're at best useless clutter on any other site running the codebase. At worst, they render the solution unusable.

To take a fairly random example: on cron, we take all project issue comments of a certain age with a status of 2 and set them to 7.

Great, if like on drupal.org you want to close old issues. And assuming 2 is 'fixed', and 7 'closed'. But if you've changed 'fixed' to 'pending' and 'closed' to 'installed', you're gonna be mighty surprised.

Piling in site-specific functionality for drupal.org has provided short-term gains--we get the tools we want. But it's limited the usefulness of the module set to other projects. It's reduced the scope for contributions and improvements. And it's made the code hard to work with and maintain.

Penance

There are lessons for all Drupal developers in the history of Project module. Early design decisions have long lasting implications.

Here's my personal penance.

  1. If it's a generic need, I will code it generically. Much better to introduce a dependency or factor out a helper module than to hack in a solution that only your module can use.
  2. I will not worship my site above all others. Just because it's something wanted on site X doesn't mean it belongs in the module--even if site X is the primary user or the funder of the module.

Of course, these lessons don't help much with the immediate problem of how to move forward with updating and expanding on the key functionality provided by project.*. But the Project module maintainers are well aware of the challenges they're working with. Productive work is underway on updating and improving the Project codebase.

The discussions and debates continue about how best to meet our very specific needs on drupal.org without increasing the problems of site specific and customized code.

We may all have a bit of penance to do, but with any luck it'll be the good kind that leaves you refreshed and ready for new challenges.

[Disclaimer: blame this reflection about sin and penance on Dante's Inferno, my current summer reading....]