Will >> Will's blog

purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon

Fri, 20 Dec 2013

SUMO: 2013 retrospective

It was a big year for SUMO. In 2012, we got a lot accomplished: new search, new information architecture,

One thing I didn't do was make my year-end script product better output.

Anyhow---on with stats!

Twas the year: 2013
===================


Bugzilla
========


Bugs created: 889

               a.topal : 156
              rrosario : 100
                willkg : 87
           scoobidiver : 85
                  ibai : 58
                mverdi : 54
               mcooper : 42
                feer56 : 35
          krystaiceman : 26
                rdalal : 19
            david.weir : 15
                shuhao : 13
      swarnavasengupta : 9
       andrei.hutusoru : 8
           me+bugzilla : 7
            tobbi.bugs : 6
                 mluna : 6
          joshua-smith : 6
               tdowner : 6
               leszekz : 6
        yoshi.yokotani : 5
        stephen.donner : 5
                 slurp : 5
                  mana : 5
             madperson : 4
              kbrosnan : 4
             tonnes.mb : 4
               rardila : 4
              pmcclard : 4
               dbialer : 4
             michaljev : 4
                   abc : 4
                  l10n : 4
               pcvrcek : 3
                 rdaub : 3
              fabricio : 2
             rmcguigan : 2
          sudheesh1995 : 2
          alex_mayorga : 2
          simone.lando : 2
            nishant_cs : 2
                  bram : 2
               smolejv : 2
        bob.silverberg : 2
              rtanglao : 2
        kdurant35rules : 2
            amit103065 : 2
         subedimahadev : 2
                lhenry : 2
          thomas.lendo : 2
             shawnsumo : 2
              mhammond : 1
        kdurant35rules : 1
                  djst : 1
               curtisk : 1
        chiorean.ioana : 1
                bermea : 1
               friedel : 1
        bputstudentweb : 1
     margaret.leibovic : 1
             rbillings : 1
       nikitan.dolmart : 1
       georgevidalakis : 1
            nsm.nikhil : 1
              satishb3 : 1
            bwbrowning : 1
              bugzilla : 1
                  coce : 1
              EddyCarr : 1
              gryllida : 1
        mohammed.samad : 1
                  6a68 : 1
         krupa.mozbugs : 1
           John99-bugs : 1
             wjohnston : 1
              barderne : 1
               jan0286 : 1
               fwenzel : 1
               rnewman : 1
           this4midhun : 1
              bjohnson : 1
  bugzilla-fromthedeep : 1
             iamjithin : 1
               bmo2010 : 1
    chrismore.bugzilla : 1
                 evold : 1
              jbertsch : 1
                yousef : 1
          pmjcreations : 1
               rhelmer : 1
              danishka : 1
                  mail : 1
             gphemsley : 1
              Rebeccah : 1
           ckreinbring : 1
               stephen : 1
         berker.peksag : 1
                jezdez : 1
                 nchen : 1
         iamjayakumars : 1
            netfuzzerr : 1
              benjamin : 1

Bugs resolved: 1116

              rrosario : 386 resolved, 273 fixed
                rdalal : 152 resolved, 150 fixed
               a.topal : 121 resolved, 41 fixed
               mcooper : 118 resolved, 105 fixed
                willkg : 72 resolved, 61 fixed
           scoobidiver : 31 resolved, 0 fixed
             michaljev : 20 resolved, 17 fixed
      swarnavasengupta : 18 resolved, 0 fixed
                shuhao : 16 resolved, 13 fixed
           me+bugzilla : 12 resolved, 5 fixed
         berker.peksag : 12 resolved, 12 fixed
                mverdi : 11 resolved, 5 fixed
                  erik : 9 resolved, 9 fixed
                lhenry : 8 resolved, 1 fixed
          krystaiceman : 7 resolved, 1 fixed
            tobbi.bugs : 7 resolved, 6 fixed
          joshua-smith : 6 resolved, 4 fixed
                 jfong : 5 resolved, 5 fixed
               tdowner : 5 resolved, 2 fixed
                feer56 : 5 resolved, 0 fixed
            david.weir : 5 resolved, 4 fixed
            bwbrowning : 5 resolved, 5 fixed
                  bram : 5 resolved, 4 fixed
                  ibai : 4 resolved, 1 fixed
   alastra.mariagrazia : 4 resolved, 2 fixed
                 laura : 4 resolved, 4 fixed
              williamr : 3 resolved, 1 fixed
         buchanae+bugs : 3 resolved, 3 fixed
           bharath_ves : 3 resolved, 3 fixed
              paul+moz : 3 resolved, 3 fixed
          chris.lonnen : 2 resolved, 2 fixed
               smolejv : 2 resolved, 1 fixed
             zcampbell : 2 resolved, 0 fixed
         iamjayakumars : 2 resolved, 0 fixed
               curtisk : 2 resolved, 0 fixed
               leszekz : 2 resolved, 0 fixed
                   abc : 2 resolved, 1 fixed
               pcvrcek : 2 resolved, 0 fixed
          taygunagiali : 2 resolved, 2 fixed
                  mail : 1 resolved, 0 fixed
               bmo2010 : 1 resolved, 1 fixed
               wymette : 1 resolved, 0 fixed
                beaotx : 1 resolved, 1 fixed
                nelson : 1 resolved, 1 fixed
             madperson : 1 resolved, 1 fixed
                  coce : 1 resolved, 0 fixed
              pmcclard : 1 resolved, 1 fixed
             tgavankar : 1 resolved, 1 fixed
  bugzilla-fromthedeep : 1 resolved, 0 fixed
              rtanglao : 1 resolved, 0 fixed
        stephen.donner : 1 resolved, 1 fixed
       guillermo.movia : 1 resolved, 1 fixed
              lorchard : 1 resolved, 1 fixed
              nukeador : 1 resolved, 0 fixed
             rtucker11 : 1 resolved, 0 fixed
            nishant_cs : 1 resolved, 0 fixed
                  stas : 1 resolved, 0 fixed
             mattbasta : 1 resolved, 1 fixed
              satishb3 : 1 resolved, 0 fixed
              ragsagar : 1 resolved, 1 fixed
             rmcguigan : 1 resolved, 1 fixed
                 nchen : 1 resolved, 0 fixed
                kudrom : 1 resolved, 1 fixed
       andrei.hutusoru : 1 resolved, 0 fixed
                  reed : 1 resolved, 0 fixed
           tiziana.sel : 1 resolved, 0 fixed
       chance.zibolski : 1 resolved, 1 fixed
             alice0775 : 1 resolved, 0 fixed
                  ravi : 1 resolved, 0 fixed
            nsm.nikhil : 1 resolved, 0 fixed
              gryllida : 1 resolved, 1 fixed
    deletesoftware+moz : 1 resolved, 0 fixed
          pmjcreations : 1 resolved, 0 fixed
                boerni : 1 resolved, 1 fixed
               rforbes : 1 resolved, 0 fixed
               dbialer : 1 resolved, 1 fixed
                jgross : 1 resolved, 1 fixed

            INCOMPLETE : 43
               WONTFIX : 51
             DUPLICATE : 64
               INVALID : 73
            WORKSFORME : 121
                 FIXED : 764

Research bugs: 15

761582: [research] Add feature: Articles that link to this article
788104: [research] [ux] Support multiple products in the support forum
815089: [research] Investigate telling apart Firefox for Desktop and Firefox for Mobile tweets
816970: [research] SurveyGizmo API to be used in automated exit survey
823060: [research] Use datetime instead of ints in ES mappings
823891: [research] Adding KB revisions feature
825621: [research] Store the templates, article links and images in each article
825624: [research] Investigate how to update to Twitter API v1.1
841412: [research] Bad localization strings shouldn't break the site.
845290: [research] URL bar should fade away on SUMO
854554: [research] Youtube embeds don't work with templates
889884: [research] Open Badges!
889890: [research][discuss] figure out how to improve our l10n situation with search
906992: [research] Add support for multiple ES indexes by doc type
937889: [research] Login users via a URL in email

Tracker bugs: 20

433161: [Tracker] Support for forums in other languages
625891: [tracker] HTML email
721462: [tracker] Taxonomy IA improvements
758598: [Tracker] Search UX suggestions
783262: [tracker] Add rate limiting to protect us from spammers
790785: [Tracker] L10n tools editing part
790786: [Tracker] L10n tools organization part
800962: [Tracker] Add activity history page for KB
815625: [Tracker] Segment dashboards and other contributor pages by product
817540: [tracker] AJAXify the refine+focus panel
825606: [tracker] Switch everything from Webtrends to Google Analytics
827640: [tracker] Localize Questions
838584: [Tracker] Getting ready for Firefox OS launch
845286: [tracker] Use as little bandwidth as possible on mobile version of SUMO
845773: [Tracker] move to an OS charting solution
848520: [tracker] Make all traffic HTTPS
851730: [tracker] Close threads pro-actively
871559: [tracker] update codebase to django 1.4 layout
897057: [tracker] Open Badges -- stage 1
920530: [tracker] support Webmaker on SUMO

git
===

Total commits: 1138

         Ricky Rosario : 492  (+16258, -16435, files 2972)
      Will Kahn-Greene : 178  (+8311, -3748, files 438)
           Rehan Dalal : 168  (+13016, -5554, files 680)
           Mike Cooper : 145  (+46955, -22136, files 582)
           Kadir Topal : 39  (+352, -110, files 61)
      Michał Frontczak : 19  (+229, -182, files 78)
         Berker Peksag : 15  (+570, -717, files 73)
             Shuhao Wu : 15  (+1523, -127, files 51)
       Jen Fong-Adwent : 9  (+138, -18, files 17)
                 Tobbi : 8  (+338, -204, files 13)
              browning : 5  (+140, -16, files 12)
             davd Weir : 4  (+15, -1, files 4)
          Joshua Smith : 4  (+94, -87, files 13)
         Tobias Markus : 3  (+8, -8, files 4)
                 Anush : 3  (+4, -1, files 3)
       Gaurav Dadhania : 3  (+3, -3, files 3)
  Bharath Thiruveedula : 3  (+15, -14, files 3)
                  ibai : 3  (+30, -30, files 4)
                kudrom : 2  (+9, -9, files 5)
             Nghi Tran : 2  (+2, -1, files 2)
          Tanner Filip : 2  (+4, -4, files 2)
                Börni : 2  (+30, -15, files 4)
             madperson : 2  (+5, -4, files 2)
         Taygun AGIALI : 2  (+7, -6, files 3)
           TylerDowner : 2  (+3, -3, files 2)
           James Socol : 2  (+37, -27, files 3)
               david-w : 1  (+1, -1, files 1)
              ragsagar : 1  (+16, -1, files 2)
       Guillermo Movia : 1  (+1, -0, files 1)
              rosanaar : 1  (+9, -0, files 1)
              Gryllida : 1  (+26, -6, files 3)
       Beatriz Nombela : 1  (+9, -9, files 6)

Total lines added:   88158
Total lines deleted: 49477
Total files changed: 5048
  1. Ricky does a lot of work! Holy cow!

  2. In 2011, we had 19 people who contributed code changes.

    In 2012, we had 23 people.

    In 2013, we had 32 people.

  3. Like 2011 and 2012, we resolved more bugs than we created in 2013. That's three years in a row! I've never seen that happen on a project I work on.

  4. There are a lot of people braving Bugzilla to write up bugs. Skimming the list, I see developers, non-developers, Support contributors, localizers, support team and a lot of people I don't recognize.

Here's some number comparisons:

name 2011 2012 2013
Bugs created: 1357 938 889
Bugs resolved: 1637 1025 1116
Total commits: 1137 916 1138
Code contributors: 19 23 32

I spent a good chunk of 2013 working on Input, but here's what I remember from SUMO development in 2013:

  1. We rearranged the codebase for better Django 1.4 layout. That was a project. Oy.
  2. We added support for non-English languages to the support forums!
  3. We switched email to be HTML formatted. We also reworked email to be localized.
  4. We switched to Google Analytics.
  5. We implemented Open Badges---though there's still a few important pieces to finish there.
  6. We switched to YouTube for videos.
  7. We added support for Webmaker and Firefox OS. Thunderbird support will be added to SUMO in 2014.
  8. Mike took a lantern, a crust of bread and a big sword and spelunked into the darkest dungeons filled with stinky, squelchy muck and rewrote the showfor code.
  9. We reworked our search code to handle multiple indexes, though we haven't taken advantage of that, yet.
  10. We switched deployment to use Dennis to lint all translated strings before pushing them to production. This has almost assuredly saved us from production fires. I hated those kinds of fires. Hooray for Dennis!
  11. We wrote and switched to Ernest for sprint planning and coordination.
  12. We overhauled everything to add support for Persona authentication, but had to push off deployment indefinitely because of problems with Persona which are being ironed out by the Persona team.
  13. We added an escalation system for questions that haven't received a response in x hours for some positive value of x that is still in flux.
  14. We ditched Highcharts.
  15. We wrote a command-line deployer which tells us exactly what's going out and tells New Relic, too. This gives us a much better idea of what we're deploying and how it affected the site afterwards. This command-line deployer is named chief-james in honor of James who has moved on to greener and well measured pastures.
  16. We added a bunch of new metrics, dashboards, history pages, activity pages, icons, bicons, landing pages, take-off pages, topics, subtopics, toe picks and all kinds of stuff.

That's the gist of the year: it was a lot of work, but we accomplished a ton.

w00t for 2013!

Thu, 19 Dec 2013

Input: 2013 retrospective

It was a big year for Input. In 2012, we spent the last half rewriting Input. In 2013, it went through secreview, had a bunch of things fixed and then we migrated to the new system.

Since then, we've been fixing bugs, reimplementing features that were lost and writing the scaffolding for the new set of User Advocacy dashboards and tools.

Let's look at some Bugzilla and git stats for the year:

Twas the year: 2013
===================


Bugzilla
========

Bugs created: 150

                willkg : 100
            cwwmozilla : 5
                fbraun : 4
               mgrimes : 4
               tdowner : 3
        stephen.donner : 3
           me+bugzilla : 2
        gasell+mozilla : 2
               mcooper : 2
                 glind : 2
             mozaakash : 1
        kdurant35rules : 1
            hitmanarky : 1
              kbrosnan : 1
        bob.silverberg : 1
              splewako : 1
              rrosario : 1
             mattbasta : 1
              educmale : 1
                feer56 : 1
                326374 : 1
               anthony : 1
        shopov.bogomil : 1
               peterbe : 1
                  l10n : 1
    chrismore.bugzilla : 1
                landis : 1
          dron.rathore : 1
                    rq : 1
             MattN+bmo : 1
          joshua-smith : 1
                cturra : 1
        swagat.kanungo : 1

Bugs resolved: 268

                willkg : 157
                       :    WONTFIX 50
                       :      FIXED 89
                       : WORKSFORME 8
                       :  DUPLICATE 9
                       :    INVALID 1
            cwwmozilla : 57
                       :      FIXED 1
                       :    WONTFIX 7
                       : WORKSFORME 29
                       :  DUPLICATE 1
                       :    INVALID 19
               mgrimes : 10
                       :      FIXED 1
                       :  DUPLICATE 1
                       : WORKSFORME 5
                       :    INVALID 3
        shopov.bogomil : 7
                       :    WONTFIX 1
                       : WORKSFORME 2
                       :    INVALID 1
                       :      FIXED 2
                       :  DUPLICATE 1
               mcooper : 6
                       :  DUPLICATE 1
                       :      FIXED 5
               mozilla : 5
                       :      FIXED 5
           me+bugzilla : 4
                       :    WONTFIX 1
                       :      FIXED 1
                       :  DUPLICATE 1
                       :    INVALID 1
             mozaakash : 2
                       : WORKSFORME 1
                       :    INVALID 1
        trifandreialin : 2
                       : WORKSFORME 2
              rrosario : 2
                       :      FIXED 2
          joshua-smith : 2
                       :      FIXED 1
                       :    INVALID 1
           aaron.train : 2
                       :    WONTFIX 1
                       :  DUPLICATE 1
        stephen.donner : 1
                       : INCOMPLETE 1
               emorley : 1
                       :      FIXED 1
               curtisk : 1
                       :    INVALID 1
               unghost : 1
                       : WORKSFORME 1
          rajul.iitkgp : 1
                       :      FIXED 1
             jruderman : 1
                       : INCOMPLETE 1
          chris.lonnen : 1
                       :      FIXED 1
             nigelbabu : 1
                       :      FIXED 1
              tofumatt : 1
                       :      FIXED 1
                cturra : 1
                       :      FIXED 1
               fwenzel : 1
                       :      FIXED 1
               mbrandt : 1
                       :      FIXED 1

            INCOMPLETE : 2
             DUPLICATE : 15
               INVALID : 28
            WORKSFORME : 48
               WONTFIX : 60
                 FIXED : 115


git
===

Total commits: 277

      Will Kahn-Greene : 249  (+51602, -16851, files 1130)
           Mike Cooper : 11  (+38528, -236, files 217)
        Brandon Burton : 7  (+42, -215, files 9)
         Ricky Rosario : 4  (+36, -19, files 6)
        Bob Silverberg : 3  (+19, -9, files 3)
                 Rajul : 1  (+3, -0, files 1)
          Joshua Smith : 1  (+10, -5, files 1)
               bogomil : 1  (+1, -1, files 1)

Total lines added:   90241
Total lines deleted: 17336
Total files changed: 1368

I want to highlight some interesting bits:

  1. We resolved more bugs than we created. That's partially due to us going through and closing out old bugs for the old Input that aren't relevant anymore.

  2. According to the Bugzilla and git data, there were 47 contributors to Input this year: 326374, Bob Silverberg, Brandon Burton, Joshua Smith, MattN+bmo, Mike Cooper, Rajul, Ricky Rosario, Will Kahn-Greene, aaron.train, anthony, bogomil, chris.lonnen, chrismore.bugzilla, cturra, curtisk, cwwmozilla, dron.rathore, educmale, emorley, fbraun, feer56, fwenzel, gasell+mozilla, glind, hitmanarky, jruderman, kbrosnan, kdurant35rules, l10n, landis, mattbasta, mbrandt, me+bugzilla, mgrimes, mozaakash, nigelbabu, peterbe, rajul.iitkgp, rq, splewako, stephen.donner, swagat.kanungo, tdowner, tofumatt, trifandreialin, and unghost.

    That doesn't include localizers who do a ton of work translating the strings in the Input ui.

    That includes some of the folks who work on the input-tests repository, but possibly misses some.

  3. Most of the 47 contributors are not "core developers". That's cool, but I could be doing a better job here making it easier for non-core developers.

    We maintain a Get Involved page and we hang out on #input on irc.mozilla.org. We have a input-dev mailing list. If you want to work on Input, this is where it's at!

Those are the stats.

At a high-level, we accomplished the following:

  1. stood up a new Input code base
  2. the beginnings of spam identification and removal
  3. Input API for feedback submission
  4. Firefox OS feedback form
  5. infrastructure for an Analysts group with special privileges
  6. the beginnings of an Occurrence Comparison report dashboard

One thing I discovered in 2013q4 was that it's really hard to be the mostly-solo dev on a project like this. I'm lucky that I'm part of a larger team, so peer reviews for work I've done is possible and timely. However, I find I'm switching contexts between the technical details of what I'm working on now and the high-level details of a bunch of possible future tasks/projects. That's really hard to do day-to-day and still maintain development momentum. I have some thoughts on how to serialize my work so that I'm doing less context switching and I can focus on individual things more deeply which should produce better outcomes.

My goals for Input for 2014 are these:

  1. clean up the code base: there's still a bunch of weird stuff in there from the rapid development work we did in 2012
  2. reduce barriers to entry for new contributors: better documentation, fewer steps to get up and running, more bugs marked for mentoring, more outreach, ...
  3. build infrastructure that we can use for better User Advocacy tools: watched alerts, email notifications, dashboards, ...
  4. flesh out tests: we're really light on smoketests and regression-catching tests
  5. work with Matt and Cheng to figure out where Input fits into the grand scheme of things; how can we make it a general-purpose feedback system? how can we handle non Firefox products and initiatives?

Yay for 2013!

Update 7:08pm

My script only showed top tens which misses tons of people who did work. I redid the data and that increases the number of contributors from 16 to 47. Oops!

Sat, 18 May 2013

Proposal: LDAP password resets as a unit of measure

Backstory

Every 3 months, we at Mozilla have to reset our LDAP passwords. The system helpfully sends the first reminder 2 weeks before your password expires, then the second reminder 1 week before your password expires and the last reminder 2 days before your password expires.

Sometimes time passes by faster than you know and you end up with a Locked out of LDAP account.

The 3 month LDAP password reset is such a large part of our lives that I propose it become a standard unit of measure for elapsed time.

Usage

Used in casual conversation:

Pat: Hi!

Jordan: Hi!

Pat: I haven't seen you before. How long have you been at Mozilla?

Jordan: I've been here for 6 LDAP password resets.

Pat: Oh, weird. I've been here for 7. Good to meet you! Would you like a banana?

Jordan: Would I ever!

Used in casual conversation on IRC:

<patbot> anyone use less?
<corycory> i only use sass. it's the best.
* riledupriley has quit (Quit: riledupriley)
<patbot> :(
<hugbot> (patbot)
* r1cky has joined #casualconversationexample
<r1cky> morning!
<nigelb> r1cky: hai!
<nigelb> Ah, it's nearly mfbt.
<mtjordan> sure. been using it for 3 ldap password resets.
<mtjordan> patbot: why do you ask?

Used in Bugzilla comments:

Jordan [:jordan]  1 day ago       Comment 0 [reply] [-]

Readonly mode causes the site to ISE.
Pat [:pat]  1 day ago             Comment 1 [reply] [-]

I looked into it. Turns out we haven't used readonly
mode in at least 4 LDAP password resets.

I think we just need to add a fake authentication
module. Easy peasy.

Used when joining a new group:

From: Pat
To: some-group@mozilla.org
Subject: Welcome Jordan to some-group!

Hi all!

I'd like to welcome Jordan to some-group! Jordan brings
expertise that is invaluable. I'm excited! Yay!

Jordan: Tell us about yourself!

Pat
From: Jordan
To: some-group@mozilla.org
Subject: Re: Welcome Jordan to some-group!

Hi!

I'm excited to join some-group! Hopefully I bring something
useful to the table.

I've been at Mozilla for 7 LDAP password resets, I like
top-posting and I make a mean cold brew coffee.

Looking forward to my first meeting!

Jordan


On Blah blah blah at blah blah blah, Pat wrote:
> Hi all!
>
> I'd like to welcome Jordan to some-group! Jordan brings
> expertise that is invaluable. I'm excited! Yay!
>
> Jordan: Tell us about yourself!
>
> Pat

Used in an email to everyone@ about departing:

Dear everyone!

It is with sadness that I tell you I'm leaving as of next
Friday. As you know, I've been with Mozilla for 32 LDAP
password resets and frankly, I'm totally out of usable
Sherlock Holmes story titles, so I'm off to new challenges.

I will miss you all.

Update: Potch suggested just using "LDAPs". Used in a sentence: "I've been here for 6 LDAPs." I like that.

Fri, 08 Feb 2013

SUMO: Now ... in pirate!

A while back, I wrote a post about poxx.py which talked about a script I based on Ned Batchelder's poxx.py script and overhauled to provide a faux "Swedish Chef" translation of Miro strings allowing me to test localizations of the application.

The transform from English to "Swedish Chef" had the following four impotant properties:

  1. the output is vaguely readable
  2. the output is longer than the input which helps us find ui issues
  3. the output is clearly distinguished from English which helps us find strings that aren't getting translated
  4. the output is mildly amusing which is sometimes important in dark times

Back in August, I made some changes and pulled it into fjord. This helped us suss out localization issues on a new site. However, I wasn't really happy with it. Amongst other things, I always wondered if "Swedish Chef" was kind of culturally insensitive.

A couple of weeks ago, I overhauled poxx.py again. This time, PIRATE! It continues to have the four properties I think are important for a test locale.

We're using it now for SUMO development. It's the grog to your Jolly Roger:

SUMO -- In Pirate!

SUMO -- In Pirate!

We're using this script on both SUMO and Fjord now. You can use it for your site, too! The code is at https://github.com/mozilla/kitsune/tree/master/scripts/.

If you see any problems with it, toss me a message in a bottle.

Note

This localization is only available in development environments. Unlike Miro where we shipped the Swedish Chef translation (or used to---I'm not sure if they do anymore), you cannot see this on the -dev, -stage or -prod SUMO sites.

Mon, 31 Dec 2012

SUMO and Input: 2012

This was my first full year at Mozilla and it was intense. I essentially worked on four projects: SUMO, Input, ElasticUtils and Gaia. This blog post talks about the first two which are worked on by the James' Rifles SUMINPUT Megalosaur team.

We accomplished a lot on SUMO this year. I spent a couple of hours last week throwing together a rough "year in review" script that looked at Bugzilla and git and crunched some numbers:

Twas the year: 2012
===================


Bugzilla
========

Bugs created: 938

              rrosario : 201
               a.topal : 188
                willkg : 108
           scoobidiver : 51
               igarcia : 41
                mverdi : 36
      swarnavasengupta : 30
                 james : 29
                  bram : 19
            tobbi.bugs : 17

Bugs resolved: 1025

              rrosario : 335
                       : WORKSFORME 18
                       :    INVALID 16
                       :  DUPLICATE 23
                       :    WONTFIX 7
                       :      FIXED 263
                       : INCOMPLETE 8
               a.topal : 182
                       : WORKSFORME 36
                       :    INVALID 41
                       :  DUPLICATE 11
                       :    WONTFIX 70
                       :      FIXED 21
                       : INCOMPLETE 3
                willkg : 131
                       :  DUPLICATE 6
                       :      FIXED 110
                       : WORKSFORME 2
                       :    WONTFIX 11
                       :    INVALID 2
                rdalal : 84
                       :      FIXED 84
                 james : 51
                       : WORKSFORME 6
                       :    INVALID 5
                       :  DUPLICATE 3
                       :    WONTFIX 15
                       :      FIXED 14
                       : INCOMPLETE 8
               mcooper : 37
                       :      FIXED 36
                       :    INVALID 1
            tobbi.bugs : 29
                       :      FIXED 29
             tgavankar : 28
                       :    WONTFIX 1
                       : WORKSFORME 1
                       :      FIXED 26
           scoobidiver : 28
                       :      FIXED 4
                       :  DUPLICATE 4
                       : WORKSFORME 11
                       :    WONTFIX 3
                       :    INVALID 6
               bmo2010 : 13
                       :      FIXED 1
                       :  DUPLICATE 3
                       : WORKSFORME 3
                       :    INVALID 6

            INCOMPLETE : 21
             DUPLICATE : 61
            WORKSFORME : 82
               INVALID : 91
               WONTFIX : 117
                 FIXED : 653

git
===

Total commits: 916

         Ricky Rosario : 430
      Will Kahn-Greene : 192
           Rehan Dalal : 98
           Mike Cooper : 44
             Erik Rose : 34
                 Tobbi : 29
        Tanay Gavankar : 23
           Kadir Topal : 11
             Tim Watts : 10
         Berker Peksag : 9
           James Socol : 7
            Victor Neo : 6
      Cesar Carruitero : 5
           David Lilly : 4
                  Ibai : 3
        Isac Lagerblad : 2
                 icaaq : 1
           TylerDowner : 1
              browning : 1
         ricky rosario : 1
    Anatoli Papirovski : 1
     Clauber Stipkovic : 1
          Jason Thomas : 1
                atopal : 1
      Florin Strugariu : 1

There are some interesting bits in there:

  1. Ricky does a lot of work! Holy cow!

  2. There were 23 people who contributed code to Kitsune (the SUMO codebase) this year. Of those, about half are volunteer contributors.

    Compare with 2011, we had 19 people who contributed to the code base and less than half were volunteer contributors.

  3. We resolved more bugs than we created in 2012. We did that in 2011 as well, so that's two years in a row. I've never seen that happen before on a project I work on.

The codebase is pretty different now than it was at the beginning of the year. I helped with the following semi-massive overhauls:

  1. The push for more metrics and dashboards to view the numbers.
  2. The switch from Sphinx to ElasticSearch.
  3. The new Information Architecture which affected browsing and searching across the site.
  4. The site redesign which covered both the desktop and mobile versions of the site.
  5. The upgrade to Django 1.4.
  6. The switch from arecibo to sentry.
  7. The push to switch from fixtures to model makers for all our tests.
  8. The switch from weekly deployments on Tuesdays to deploying whenever we want. Continuous deployment is fantastic.
  9. Started switching the whole site from Webtrends to Google Analytics. I saw Ricky write up a bunch of bugs to finish up that work, so I'll say it's in progress.
  10. During the redesign, Rehan redid all the CSS and switched us to use LESS.
  11. I spun off some code I wrote for richard, then ported to Fjord, then improved into a project called django-eadred. That makes it a lot easier to generate sample data for a variety of purposes like new contributors, bootstrapping, and large random data sets.

On top of that, we did a lot of work on the documentation and making it easier to get to a working Kitsune development environment. We switched to a sprint-based work flow using Scrumbugz. We also nixed our daily checkin conference call for an IRC-based checkin system that we wrote called Standup.

It's been a big year.

For Input, it was a bigger year. We decided to abandon the old Input codebase (omfg yay) in favor of rewriting it from the ground up. The rewrite took a couple of months and then has sort of been sitting around waiting for a security review. In the meantime, we (actually, Mike did) fixed a bunch of issues with the old site code because that's what's currently in production.

Rewriting Input wouldn't have taken so long except that we did a lot of work fixing bugs in external libraries and updating Playdoh. That work definitely cut into our schedule, but it benefitted a bunch of other groups/people/sites, so that's good.

That's the gist of the year: it was a lot of work, but we accomplished a ton.

w00t for 2012!

Wed, 02 May 2012

webdev workweek thoughts

I'm at the webdev work week in Santa Cruz, CA, USA this week. It's great to meet people I've been talking to for the last 6 months. It's also kind of nice to take a break from the SUMO sprints. I've been spending the time lifting my head and seeing what's been happening while I wasn't paying attention.

List of three things on my mind:

  1. I wish I had more time. There are so many people doing exciting things that I'd love to work on, too, but I just don't have enough time. It's hard to suppress the, "Wow---that looks awesome! I'll help you on that project!" reflex. I just don't have time. It's kind of a bummer. On the flip side, if I do discover myself with free time, I have a long (long) list of things I'm interested in.
  2. It's nice to talk about technical things. It's nice to make a (bad) joke about mime types and get a response. (An HTTP response! ha ha!)
  3. Everyone is really super and friendly. I don't think I've been in a group this big that's lacking a token jerk. This has caused me to idly wonder if maybe I'm the token jerk. If so, I hope someone leaves me a note so I can adjust accordingly.

Things I'm taking away (so far):

  1. I should dig up my compilers book and re-read it because I really liked studying compilers way back when.
  2. I screwed up with spreading elasticsearch knowledge. I keep having conversations about problems we've seen and solved in SUMO that other projects are having. Plus it seems like everyone has their own branch of elasticutils with differing features. I'm going to try to spend some time working on making the situation better both by documenting things and also improving elasticutils so we're not all maintaining our own versions.
  3. Need to spend more time learning things.

Sun, 15 Apr 2012

mozilla status: April 15th, 2012

I haven't had time to blog much in the last few months. At work, I've been spending all my time with elasticsearch, elasticutils, and SUMO bug fixing. I've been working on the conversion from Sphinx search to elasticsearch for SUMO since I started at Mozilla, but I've only recently felt like I'm really getting the hang of it. There are a bunch of elasticutils-related things I want to blog about, but those will come in fugure entries.

In my spare time, I've been working on richard. This project has nothing to do with Richard of air mozilla fame, but rather is a video indexing web application. It's the software that runs pyvideo.org.

pyvideo.org has the distinction of being the first Django application I've built from the ground up. That distinction is both a virtue (yay for first apps!) and a vice (boo for silly things I did when doing it!).

The one thing I did that I'm really proud of is that when building the software, I knew I needed help if it was to succeed and thus I worked to make it easy and inviting for contributors to get involved:

The end result of that is that there are 4 contributors to richard including myself and one of them is very active.

Asheesh did a talk at LibrePlanet 2012 that mentioned Mako's power law of contributions to open source projects. The gist of it is that most open source projects only ever have one contributor. [3]

Well, I've got 5 on my video index web application software that I "launched" a month ago. I'm feeling good about that.

[1]Several of my friends point out that GitHub kind of takes the D out of DVCS.
[2]Though didn't have any tests when I "launched".
[3]I may fix this paragraph after Asheesh corrects me.