The Drupal Association is a non-profit organization dedicated to helping the Drupal community with fun­ding, infra­structure, events, promotion and distribution. The current Board of Directors was elected almost a year ago, so it's election time again!

On March 1st, 2010 we will elect new Permanent Members to the General Assembly. The General Assembly wil then select our Board of Directors, who are responsible for oversight of day-to-day operations. All the details can be found in the Drupal Association's Statutes.

We are looking for people who aspire to become Directors on the Drupal Association's Board and who can provide leadership and experience to expand the reach of the Association and its activities. All current Board positions, except the President and Treasurer, are up for election, including Secretary, Legal Officer, Marketing and Communications, Infrastructure Manager, Fundraiser, and Events Manager.

The Association is also looking for new General Assembly members with skills in marketing, drupal.org webmaster coordination, project management and more. People interested in becoming a Permanent Members are invited to apply. While Permanent Members don't have the defined responsibilities of board members, membership implies a commitment to work for and promote the goals of the Association.

The Drupal Association is run by unpaid volunteers and we expect that our board members will spend a considerable amount of time working on Association responsibilities and obligations. If you think you're a good candidate, and you have the time it takes, find an existing Permanent Member to support your candidacy (i.e. find a supporter) and submit your application. Applications are due by February 22th and on March 1st, we elect the new Permanent Members and the new Board of Directors.

If you have questions, don't hesitate to ask them in the comments, or to contact the Drupal Association.

Last week we've had some "fun" with Microsoft's msnbot. They were apparently trying out their new beta and it didn't work that well. It ignored drupal.org's robots.txt and kept crawling 20-40 pages per second. You could call that a denial of sevice attack, drupal.org sure had problems.

We then resolved this by banning the whole subnet from our webservers.

Today neclimdul pointed out to me that the people running CPAN testers have had the same fun as we did and resolved this in the same manner.

If Microsoft keeps doing this it will get cut off from indexing many Open Source projects. One could think it was intentional.

Update: Kieran asked me to point out that there's no such feature as "don't index my site that fast" available from Microsoft. Google has this their webmaster tools and it can be quite convenient. Of course, why would this work if robots.txt is ignored ...?

The Oregon State University Open Source Lab (OSUOSL) has been one of the most generous organizations to the Drupal project. In mid-2005 they stepped up and offered to host drupal.org at a time when the website was crashing due to insufficient hardware. OSUOSL generously offered rackspace and bandwidth, all of it donated. Since this donation the drupal.org infrastructure has grown from a single server to more than a dozen, traffic has increased exponentially, and overall growth has exploded. OSUOSL handled all of this in stride and even provided the time of student interns to assist with hosting and infrastructure issues.

After over two years of working on their infrastructure the Association is happy to announce that the Drupal community (via the Association) has in turn supported the OSUOSL with $15,000(usd) to help with continuing operations. This money will be used to not only help Drupal.org's growing infrastructure but also support other Open source projects.

We are excited and proud to support such a great institution.

As promised, here is the recorded webinar on SEO for the Drupal.org redesign.

A couple of weeks ago we prepared to do a live webinar and take questions from the community about Drupal.org SEO. Unfortunately, I was not familiar with all the features of our webinar software. The result was that no attendees were able to attend the webinar, and the webinar was recorded in a proprietary format. Fortunately, Forest Mars was able to decode it and so I've now made it available.

If you have questions about Drupal.org SEO feel free to ask them in the comments below. The folks from Volacci will gather them up in January and respond when they are back from holiday.

If you are interested in helping to implement the Drupal.org redesign, we certainly could use some help. Don't hesitate to contact us.

Last year, the Drupal Association hired Mark Boulton Design, with financial support from the community, to work with the Drupal community and re-design Drupal.org. The final design deliverable is the Drupal.org website style guide.
During the last year, we have had 6 redesign sprints in North America and Europe. We've implemented many critical infrastructure improvements including a new search engine, a single sign-on solution, and improvements to the project issue queues. We've recently recruited a dozen Drupal services shops to help complete the redesign. You can read about our first effort. You can see the progress we are making by visiting the status section links on the assignments page. The username and password for the staging sites is: drupal, drupal

We still need more help. If your organization can help implement a section of the redesign, with theming and development skills, please contact us through the contact form or phone, (415) 894-9320). To learn more about contributing to the redesign, read the redesign implementors guide.

Webinar to educate implementers about drupal.org SEO

Next Wednesday, Dec 9th, at 12PM EST we will be hosting a webinar to help educate the re-design implementers on Drupal.org SEO techniques. All interested members of the Drupal community are welcome to register and attend the webinar.

We received word that people were unable to log into this webinar. We have recorded the webinar and will be posting it shortly for feedback

Drupal.org re-design implementers recruited

We have now recruited 12 different Drupal consulting shops to help implement the redesign. One of the big challenges in implementing the redesign, is understanding what are the implications for Search Engine Optimization.

Drupal.org lost it's Pagerank of 9

In September 2007, we launched copies of Drupal.org to help with the upgrade to Drupal 6 and our re-design efforts. The infrastructure team was contacted by members of the Drupal community who expressed concerns that these extra copies of Drupal.org were accessible by search engines. Soon after we were contacted Drupal.org's Google page rank was reduced from 9 to 8.
Drupal.org was now the only major open source content management system with a page rank of 8 or lower. Wordpress, Joomla, and Plone all kept their page rank of 9 during this period. Earlier this month, Drupal.org's page rank was increased from 8 to 9. The calculation of page rank is not fully understood outside Google, and the implications of a reduction of pagerank aren't necessarily damaging to the Drupal project. To help the implementers understand the SEO implications of the changes to Drupal.org we sought out help with Search Engine Optimization with Drupal.org.

Working with SEO experts

Over the last few months I've been meeting regularly with the Drupal search engine optimization experts from Volacci. We've done keyword research, analysis of drupal.org for SEO, SEO analysis of the redesign prototypes, and developed some social media strategies for the Drupal community to help with Search Engine Marketing. If you would like to learn what we've discovered and understand how you can help with the redesign please join us at the webinar.

For the last year the Drupal.org redesign has progressed quietly and steadily. Almost 200 people have participated in the redesign by commenting, discussing, reviewing, testing, designing, theming, or writing code to get it done. We have had 6 re-design sprints (2 in Europe and 4 in the US) and built significant infrastructure to get the redesign development and staging ready for the community to make the final push. We are now asking Drupal consulting shops to step forward and help us implement one section of the redesign.

Many of the most important features are already available such as search, or improvements to the project module. In order to get the redesign completed we've reviewed the design and identified the minimum viable product for drupal.org.

A minimum viable product for Drupal.org means deciding what are the most essential parts of drupal.org that will go live when the design is refreshed. It also means defining what will be finished off after the initial launch. The remaining features will be designed and prioritized based on feedback from people who visit the site.

To understand how we selected the minimum viable product it is important to revisit the intent of the redesign and understand the motivations of the people who use drupal.org. The most important motivations were:

1. getting access to specific information
2. monitoring issues
3. continuing conversations

The minimum viable product must have the most important user journeys re-designed. For 'new users', they consider the movement from Home to About/Why Choose Drupal and/or or Get Started the most important user journey. So we must have those sections of the site done. For ‘existing’ users the primary user journeys are to specific content via search/search results and to monitor content/issues/discussions/news etc. In the redesigned site this navigation will be done via the dashboard. The dashboard must have this basic navigation for the minimum viable product. Currently, existing drupal.org users use bookmarks, and they use the URLs which they have memorized to navigate around drupal.org.

If you want to review the motivations for the minimum viable product please read the re-design post about new users and existing users, and insights into the drupal.org design strategies.

We also wanted to take a more analytic approach and review the most common paths. We turned to Google analytics and reviewed two reports, the Top Content Detail Path report and the top content detail navigation report. This gave us 50 paths to review to ensure the new site worked before we launched the redesign.

What needs to be done:

The design deliverables from Mark Boulton Design lists approximately 25 landing pages. Many of those pages are quite complex and need both custom programming, additional theming, and content. To help get these pages done, we've evaluated the complexity of these landing pages 1(easy) through 3 (complex).

We are also asking Drupal development firms to step forward and sponsor a page. For each page, we request that a rough evaluation be started by Dec 1st, and a goal of completion by Jan 15th.If you are interested in sponsoring a page, please contact us using the "Implement part of the drupal.org redesign" category.

We have two project managers who will be helping to coordinate the efforts. They are myself, and Lisa Rex. The project managers will work with the contributing development shops and the existing contributors to the redesign to help it be completed. We also will work with Dries to coordinate how drupal.org will evolve.

We have also recruited a new redesign infrastructure team that has been managing and updating our ten development and staging sites. We are still recruiting administrators for that effort and I'll blog more about the progress we are making soon.

Redesign page Effort Score Notes 1.
Home page 3 Utility navigation which
is basically dashboard

Develop with Drupal section where do the links go: Current activity,
etc.

Dean and Kieran need to implement advertising block on the front page
of Drupal.org

Map on home page - Follow up with Dmitri connect to tweets or news
about what's going on

Druplicon stats seems to be working with dynamic number of users

[Secondary Tabs] - News, Doc Updates, Forums posts, Commits, More 2.
Dashboard 3 Check with Neil Drumm 3.
About 2 6.
News 2 Theme planet Drupal 7.Get
started 3 Conflict on getting
started page, same alias as Documentation main page

Getting started page has lots of search results. Are these implemented
as panels?

* Most popular modules

* Most popular Themes

* Translations

* Handbooks could be hard-coded

* Recent Books and resources is probably a rotating block

* Get support is just content

* Translations to other languages

* Requires dashboard, search with filters, Primary navigation, and
footer block
8.
Get involved 3 9.
Association 2 10.
Community and Support 3 * Tabs implemented in CSS

* How can add the How can we help you? is going to either be a block or
panels.

* Where is the Drupal community right column is just text

* Advertising block at the bottom

*  Recent activity block

* * Requires dashboard, search with filters, Primary navigation, and
footer block

* * mailing lists: Score 3 10.1
Community IRC 1
11.
Documentation 2 * CSS four tabs, Three column layout

* Docs home page, API do we link to other sites or is this landing page
a druplicate of those sites main pages.

* Mostly content in columns.

* Advertising block on bottom right

* Advertising books on bottom block 11.1
Documentation Index 2 * Three column layout

* Articles will be interesting

* Documentation API is just links, 2 column layout 11.2
Documentation API 1 * just text

* Recently updated, search result 12.
Download & Extend 3 * Eight tabs and lots of search blocks and it's all
project module searches 12.1
Download & Extend: Core 3 * Developers for Drupal core 12.2
Download & Extend: Modules
3 * http://drupal.org/project/modules needs to be
improved. So we have to go look at search filtering and how to improve
it. 12.2.1
Download & Extend: Module Example 3 Pretty significant reworking of a project module
interface and content

 * Groups out of questions

 * Community rating -- recommendations

Search results that are related

Activity graphing

Screenshots or carousel 12.3
Download & Extend: Themes 3 14.
Search results 3 * Search filters need to get done. Talk to Apache gurus.

* Popular searches

* Need a search again block in right hand column, removed from main page

* Order by needs to be a drop down, not a block on the right

* Implement or filter for. Which means having more content types
defined by taxonomy. 14.1
Refine your search - Themes 3 Log-in 1

One of the biggest challenges in working in a large community like the Drupal community is removing bottlenecks. All too often the community can seem to come to a grinding halt on just one issue that can only be managed by one person. On Monday Dries gave a presentation at MIT and talked about how some of the Drupal community’s biggest problems have helped create some of our best solutions. In particular, he cited how our drupal.org server melt down in 2005 lead to the creation of the Drupal association to proactively manage and plan for our infrastructure growth.
Today the community continues to be challenged with re-designing and implementing drupal.org. On the surface, it doesn’t look much has happened, but improvements in projects, and search have made big strides in making Drupal.org easier to use. The Drupal community works well in parallel and through lots of small iterations. Unfortunately, our drupal.org development and staging infrastructure has not supported dozens of developers and themers working in parallel. We’ve now taken the time to build out that parallel development and staging infrastructure.
It is important to understand why a large site like Drupal.org can be hard to do local development for. First, drupal.org serves about 30 million pages a month, and has over 600,000 nodes. It has tens of thousands of attached files including patches, screenshots, videos, and graphics. Setting up a greater than 1GB database and syncing over 5GB’s of files is a tough task for a local development environment.
Much of the new design for Drupal.org is focused on making it easier to find what you are looking for on Drupal.org. For the redesigned development sites we also need to have a working Solr server. For example, a page like the new download and extend landing page has Drupal version search filters and at least seven blocks that are search results themselves. For themers to theme pages, their Drupal.org site needs content, graphic assets, patch files which need to be attached to issue queues, and the site must have working functionality.
Also, Drupal.org content is valuable even though it is licensed under creative commons license. If a copy of the site were to fall into the hands of spammers Drupal.org could suffer abuse. For that reason, we don't casually give away copies of the drupal.org database and code to anyone who asks.
I'll review this new infrastructure and explain how our team plans to work to support at least 10 development sites in parallel.

The Servers Web server


The dedicated virtual machine at stagingvm.drupal.org contains 10 virtualhosts, and is entirely devoted to the redesign project. The webroot of this VM is /var/www.

Database Server


The database server is separate, and lives on stagingdb.drupal.org (which is a cname pointing at civicspace.drupal.org, the solr slave).

The Sites

There are 10 staging sites for the redesign, each of them are password protected with: drupal/drupal to keep out search engines and bots.
Staging 1 (staging1.drupal.org) through Staging 10 (staging10.drupal.org) are virtualhosts on the stagingvm.drupal.org server. Each site lives in it's own directory, /var/www/staging1 through /var/www/staging10, respectively. They are almost all setup with a recent copy of drupal.org.
Each of these virtualhost containers has a separate instance of drupal.org's codebase + sanitized database, and we will give selected volunteers the access to commit theming and administrative changes to an instance for testing purposes.

Backing up a site's database, restoring a site

The redesign administrators have two scripts in their stagingdb.drupal.org home dir, backup-tables and restore tables. The administrators have sudo and can run each script to backup or restore the last backup of a db. For example, if someone is going to test something on staging5 that will likely break it. You can ./backup-tables.sh staging5. Will take about 20 min and then ./restore-tables staging5 will restore the latest backup
Database copies of Drupal.org
Santized DB dumps of Drupal.org database are available on the stagingdb.drupal.org database server. The database server can hold 10 copies max. The problem there right now is that you need to take a dump of the drupal.org database server and then reload it into another database to run an SQL script on it. This continues to be a cumbersome process that can only be done by our two database administrators Narayan Newton and David Strauss.

Granting theme repository access

http://groups.drupal.org/drupalorg-redesign-implementers/guide
* Once the drupal.org re-design developers and themers have signed up for an account on infrastructure.drupal.org we need to review their request and determine if they are a likely candidate to contribute. We get a lot of requests from people who just want to run the drupal.org theme on their own site. Right now over 35 community members have write access to the SVN, and several dozen have read access so they can generate patches. Admittedly, this approval process is a significant bottleneck.
Once the contributors account has been approved they can issue the following commands:
 svn checkout https://svn.drupal.org/drupal/themes/bluecheese/
* They can also log into SVN using their username and password from http://infrastructure.drupal.org.
The documentation tab on the front page of the Drupal.org redesign implementers group for more information about how to get access and use SVN.

Pushing theme changes to development sites

There is an SVN directory called development-themes that will be checked out to each sites /themes directory. Each themer can set the default theme for anyone of the 10 instances to see their changes go live. Since Drupal can support many themes we may be able to support dozens of themers working on a site simultaneously.

Theme Deployment

We have now setup cron jobs to checkout themes from the themers sandbox every 5 minutes.

Syncing Drupal.org production with a master staging site

Code and configuration changes happen on Drupal.org. We need to push these code and configuration changes to staging sites to keep them in sync. The staging sites are managed via an SVN branch HEAD == redesign and the live site is the SVN branch DRUPAL-6--1 == drupal.org. All configuration changes should be in drupalorg.install updates.
Deployment of code, content, and configuration changes continue to be one of the big challenges in Drupal and might be the big feature of Drupal 8. Many of Drupal’s best core innovations come from drupal.org necessities.

Pushing development configuration changes to all the staging sites

When new features are being built on a Drupal.org staging site, we need to push these same configuration changes to all ten of the staging sites, or they might not work as the developers and themers need them to.

Asset management

Since themers work with so many graphic assets we need a way for them to more easily share their working assets and get them accessible to the Drupal.org redesign staging sites.

Infrastructure administrators wanted

Now that our redesign infrastructure is built, we need Drupal administrators and developers to help with the development, staging, and deployment cycles for all 10 of these re-design sites. I know we’ve asked for infrastructure administrators before and there’s been a lot of interest. If you are still interested contact me . A big thank-you to our infrastructure team for making this possible.