This week our notes are all about the Drupal.org D7 upgrade.
We are currently at 70 open issues out of 212 total. Which means we fixed 68% and have 32% to go. This matches our hours spent as well. We projected 940 hours, and are currently at 68.5% spent and 31.5% left. And we do still have about half of the budget left. Though not all invoices came in yet, so this number might be a little lower. As we enter the QA phase, we expect the number of open issues to go up, which is natural.
Melissa Anderson (eliza411) joined us this months to help with the QA phase of the project. Our BDD tests (https://drupal.org/project/doobie) will be an important part of this phase and future Drupal.org maintenance.
This is how we plan to approach QA:
At first our small team with the help of a few BDD volunteers will work through existing BDD tests, updating them to match the application behavior expected from our development site and getting as much of the test suite to pass as possible.
We’ll approach tests one section of the site at a time. Once developers feel that a particular area is done, we’ll work on the related tests, filing bug reports or updating tests.
Once most of BDD tests are passing, we’ll switch to a short period of manual testing. At this point we’ll look if there are any problems not uncovered previously by BDD testing and improve our test coverage where necessary. Oh, and we’ll fix bugs, of course.
After manual testing, we will open the site for a wider community QA. We’ll provide process details and detailed instructions on where/how to file bug reports. At this point, we will have all the data we need to begin to schedule the deployment.
We are currently at the step 1 of the QA phase. And here are our test stats.
We have 719 documented scenarios for Drupal.org, 677 of which have been automated. 69% of the automated scenarios are passing. We have 80% visibility, which means that roughly 20% of the steps are being skipped because they follow a step failure. As the failures are corrected, either by updating the test or updating the code and configuration, new issues invariably arise.
The numbers will provide the baseline by which we can gauge QA progress. The closer we get to 100% visibility, the closer we are to having the best view possible of the upgrade status as far as BDD testing is concerned, and the closer we are to the second QA phase.
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to continue the upgrade project. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.