Will >> Will's blog

purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon
Page 1 of 4  >> (less recent)

Wed, 18 Mar 2015

Input status: March 18th, 2015

Development

High-level summary:

  • new Alerts API
  • Heartbeat fixes
  • bunch of other minor fixes and updates

Thank you to contributors!:

  • L Guruprasad: 6
  • Ricky Rosario: 2

Landed and deployed:

  • 73eaaf2 bug 1103045 Add create survey form (L. Guruprasad)
  • e712384 bug 1130765 Implement Alerts API
  • 6bc619e bug 1130765 Docs fixes for the alerts api
  • 1e1ca9a bug 1130765 Tweak error msg in case where auth header is missing
  • 067d6e8 bug 1130765 Add support for Fjord-Authorization header
  • 1f3bde0 bug 909011 Handle amqp-specific indexing errors
  • 3da2b2d Fix alerts_api requests examples
  • 601551d Cosmetic: Rename heartbeat/views.py to heartbeat/api_views.py
  • 8f3b8e8 bug 1136810 Fix UnboundLocalError for "showdata"
  • 1721758 Update help_text in api_auth initial migration
  • 473e900 Fix migration for fixing AlertFlavor.allowed_tokens
  • 2d3d05a bug 1136809 Fix (person, survey, flow) uniqueness issues
  • 3ce45ec Update schema migration docs regarding module-level docstrings
  • 2a91627 bug 1137430 Validate sortby values
  • 6e3961c Update setup docs for django 1.7. (Ricky Rosario)
  • 6739af7 bug 1136814 Update to Django 1.7.5
  • 334eed7 Tweak commit msg linter
  • ac35deb bug 1048462 Update some requirements to pinned versions. (Ricky Rosario)
  • 8284cfa Clarify that one token can GET/POST to multiple alert flavors
  • 7a60497 bug 1137839 Add start_time/end_time to alerts api
  • 7a21735 Fix flavor.slug tests and eliminate needless comparisons
  • 89dbb49 bug 1048462 Switch some github url reqs to pypi
  • e1b62b5 bug 1137839 Add start_time/end_time to AlertAdmin
  • 3668585 bug 1103045 Add update survey form (L. Guruprasad)
  • ab706c6 bug 1139510 Update selenium to 2.45
  • 6df753d Cosmetic: Minor cleanup of server error testing
  • 1dcaf62 Make throw_error csrf exempt
  • ceb53eb bug 1136840 Fix error handling for better debugging
  • 92ce3b6 bug 1139545 Handle all exceptions
  • e33cf9f bug 1048462 Upgrade gengo-python from 0.1.14 to 0.1.19
  • 4a8de81 bug 1048462 Remove nuggets
  • ff9f01c bug 1139713 Add received_ts field to hb Answer model
  • d853fa9 bug 1139713 Fix received_ts migration
  • ae5cb13 bug 1048462 Upgrade django-multidb-router to 0.6
  • 649b136 bug 1048462 Nix django-compressor
  • 1547073 Cosmetic: alphabetize requirements
  • e165f49 Add note to compiled reqs about py-bcrypt
  • ecdd00f bug 1136840 Back out new WSGIHandler
  • cc75bef bug 1141153 Upgrade Django to 1.7.6
  • d518731 bug 1136840 Back out rest of WSGIHandler mixin
  • 12940b0 bug 1139545 Wrap hb integrity error with logging
  • 8b61f14 bug 1139545 Fix "get or create" section of HB post view
  • d44faf3 bug 1129102 ditch ditchchart flag (L. Guruprasad)
  • 7fa256a bug 1141410 Fix unicode exception when feedback has invalid unicode URL (L. Guruprasad)
  • c1fe25a bug 1134475 Cleanup all references to input-dev environment (L. Guruprasad)

Landed, but not deployed:

  • 1cac166 bug 1081177 Rename feedback api and update docs
  • 026d9ae bug 1144476 stop logging update_ts errors (L. Guruprasad)

Current head: 9b3e263

Rough plan for the next two weeks

  1. removing settings we don't need and implementing environment-based configuration for instance settings
  2. prepare for 2015q2

End of OPW and thank you to Adam!

March 9th was the last day of OPW. Adam did some really great work on Input which is greatly appreciated. We hope he sticks around with us. Thank you, Adam!

Thu, 19 Feb 2015

Me: 2014 retrospective

This is belated because I've been busy.

2014 was crazy. It had a lot of epically lousy parts, but also had some good parts. Like 2013, I don't know what I did with my 2014 goals.

2013 involved a lot of projects. 2014 involved fewer.

I also spent time on a bunch of other projects:

At work, I spend most of my time on fjord which powers Mozilla Input.

2014 was interesting:

Challenges for the year:

In 2015, I want to:

That's 2014 in a nutshell!

Wed, 18 Feb 2015

Input status: February 18th, 2015

Development

High-level summary:

  • Some minor fixes
  • Upgraded to Django 1.7

Thank you to contributors!:

  • Adam Okoye: 1
  • L Guruprasad: 5 (now over 25 commits!)
  • Ricky Rosario: 8

Landed and deployed:

  • 57a540f Rename test_browser.py to something more appropriate
  • 6c360d9 bug 1129579 Fix user agent parser for firefox for android
  • 0fa7c28 bug 1093341 Fix gengo warning emails
  • 39f3d25 bug 1053384 Make the filters visible even when there are no results (L. Guruprasad)
  • 9d009d7 bug 1130009 Add pyflakes and mccabe to requirements/dev.txt with hashes (L. Guruprasad)
  • 5b5f9b9 bug 1129085 Infer version for Firefox Dev
  • b0e0447 bug 1130474 Add sample data for heartbeat
  • 91de653 Update django-celery to master tip. (Ricky Rosario)
  • 6eda058 Update django-nose to v1.3 (Ricky Rosario)
  • f2ba0d0 Fix docs: remove stale note about test_utils. (Ricky Rosario)
  • 3b7811f bug 1116848 Change thank you page view (Adam Okoye)
  • 8d8ee31 bug 1053384 Fix selected sad/happy filters not showing up on 0 results (L. Guruprasad)
  • fea60dc bug 1118765 Upgrade django to 1.7.4 (Ricky Rosario)
  • 7aa9750 bug 1118765 Replace south with the new django 1.7 migrations. (Ricky Rosario)
  • dcd6acb bug 1118765 Update db docs for django 1.7 (new migration system) (Ricky Rosario)
  • c55ae2c bug 1118765 Fake the migrations for the first deploy of 1.7 (Ricky Rosario)
  • 1288d5b bug 1118765 Fix wsgi file
  • c9a326d bug 1118765 Run migrations for real during deploy. (Ricky Rosario)
  • f2398c2 Add "migrate --list" to let us know migration status
  • bf8bf4c Split up peep line into multiple commands
  • 0710080 Add a "version" to the jingo requirement so it updates
  • 0d1ca43 bug 1131664 Quell Django 1.6 warning
  • 7545259 bug 1131391 update to pep8 1.6.1 (L. Guruprasad)
  • 0fa0aab bug 1130762 Alerts app, models and modelfactories
  • be95d8e bug 1130469 Add filter for hb test rows and distinguish them by color (L. Guruprasad)
  • f3abd8e Add help_text for Survey model fields
  • f6ba2a2 Migration for help_text fields in Survey
  • f8cd339 bug 1133734 Fix waffle cookie thing
  • c8a6805 bug 1133895 Upgrade grappelli to 2.6.3

Current head: 11aa7a4

Rough plan for the next two weeks

  1. Adam is working on the new Thank You page
  2. I'm working on the Alerts API
  3. I'm working on the implementation work for the Gradient Sentiment project

That's it!

Wed, 04 Feb 2015

Input status: February 4th, 2015

Preface

Since the last status report, we did a year-in-review and probably some other emails, so I figured I'd hold off on a status report until mid-January. I missed that by a couple of weeks.

Development

High-level summary:

  1. Lots of fixes and improvements to documentation, git hooks, test infrastructure and setting up development environments
  2. Heartbeat v2
  3. Browser data capture
  4. Upgraded libraries in preparation for upgrading to Django 1.7
  5. Scaffolding for the new Thank You page
  6. Switched from in-house Jenkins to Travis-CI

Thank you to contributors!:

  • Adam Okoye: 3
  • Adrian Gaudebert: 1
  • L Guruprasad: 9
  • Ricky Rosario: 14
  • deshraj: 3

Landed and deployed:

  • a88a25a Ignore the commented lines in commit message before linting it (L. Guruprasad)
  • 1fc67a3 bug 1110543 Add noscript section (Adam Okoye)
  • 1d608c4 bug 1116543 Peg pip at 1.5.6
  • 5256b31 Add ihavepower command
  • f9f789a bug 1116498 Fix date filter API tests
  • c33c398 Improve data migration docs
  • 6d69baf Fix qunit tests; spruce up qunit docs
  • 43ec5da Handle whitespace URLs
  • 0aa0245 Fix commit message lint to gracefully handle empty commit message (L. Guruprasad)
  • fff1101 bug 1091716 Allow about: and chrome:// URLs in client-side validation (L. Guruprasad)
  • 575fceb Fix jshint errors in generic_feedback.js (L. Guruprasad)
  • 74c3ed8 Remove analytics_flagged view. (Ricky Rosario)
  • 2b58388 bug 1091716 Update smoketests regarding url validation changes
  • 2e006e9 Remove fjord/analytics/templates/analytics/analyzer/flags.html (Ricky Rosario)
  • 6a1ad6d bug 1091716 Add smoketest for whitespace in url data
  • 7cf5b3c bug 1091716 Carry changes over to generic_feedback_dev.js
  • 18e5c89 Nix last vestiges of schematic
  • 6661094 bug 1114335 Record what happened when purge_data script ran, in journal (L. Guruprasad)
  • 58fcfcf Update purge_data test to cover RemoteTroubleshootingInfo objects
  • 342eac3 bug 1108156 Upgrade peep and handle pip 6.0
  • de4621e bug 1119015 Fix API with ES 1.2.4
  • 4c5072b bug 900992 Update south to 1.0.2
  • 0fd833c bug 900992 Update south in vendor/ to 1.0.2
  • 565e05d bug 1093222 Stop using django-cache-machine. (Ricky Rosario)
  • 53d8043 bug 1093222 Removing django-cache-machine from vendor. (Ricky Rosario)
  • 0a47fdc bug 1093222 Remove django-cache-machine from peep requirements. (Ricky Rosario)
  • 28ad328 bug 1116523 Add response ID to session (Adam Okoye)
  • ae85859 bug 1119972 Fix remote-troubleshooting checkbox
  • 19fd092 back button hit area increased in generic_feedback.html (deshraj)
  • 39a9099 Document how to configure the timezone of the VM to match the host (L. Guruprasad)
  • 55aa382 bug 1105435 Increasing Hit Target of Back Button (deshraj)
  • 9921759 bug 1116838 Create waffle flag for thank you page (Adam Okoye)
  • 06e7eb6 Add l10n comment to clarify "Submit" string
  • 33e9f9e No bug - Fixed l10n documentation and added l10n dependencies to vagrant. (Adrian Gaudebert)
  • 1c6eda9 bug 1122060 add description for heartbeat survey model (L. Guruprasad)
  • 8396274 bug 1071044 Initial travis setup. (Ricky Rosario)
  • 08baebb bug 1119305 Upgrade django to 1.6.10
  • 92b275e bug 1104863 Upgrade to celery v3.1.17 (Ricky Rosario)
  • 1e3bb5c bug 1123744 Nix jenkins
  • 60553bc bug 1120952 Fix remotetroubleshooting metrics
  • 16250bf Tweak reverse to allow you to override the locale
  • 7398332 bug 1124497 Add ResponsePI table and migration
  • 61a57e2 bug 1124497 Move ResponseTroubleshootingInfo to ResponsePI
  • 899c7a0 bug 1124497 Delete ResponseTroubleshootingInfo
  • 36eb60b bug 990777 Add high resolution images for favicon for retina displays (L. Guruprasad)
  • ba498e3 bug 1126030 Enable CORS for Heartbeat API. (Ricky Rosario)
  • 74dfc6d Handle Windows 10 user agents in sniffer
  • 301e9e5 bug 1054001 Upgrade to django-browserid v0.11.1 (Ricky Rosario)
  • 2c76974 bug 1127312 Document how to keep up with changes to fjord (L. Guruprasad)
  • 6fe9630 bug 934979 Upgrade django rest framework to v2.4.4 (Ricky Rosario)
  • a6b396b bug 1111641 Enable browser data capture for all locales
  • 94b5292 bug 1127431 Add survey filtering to hbdata view
  • 15b19b2 bug 1113171 Remove requests 1.1.0
  • 83079bf bug 1113171 Upgrade to requests 2.5.1
  • 5f5b93d bug 1118765 Upgrade jingo to master tip. (Ricky Rosario)
  • 0bb2e56 bug 1124379 Upgrade peep and un-goofify it
  • b5676cf bug 1118765 Upgrade waffle to v0.10.1 for django 1.7 (Ricky Rosario)
  • 997702f bug 1076879 Fix Response indexing retry
  • deaa5eb bug 1111265 Fix browser data re: products and browsers
  • 540b404 bug 1111265 Add browser_data_browser to admin and forms
  • 16f0a78 bug 1119813 Merge browser data ask branch
  • 2ef8099 Update peep to 2.2

Current head: d3fe0fc

Rough plan for the next two weeks

  1. Adam is working on the new Thank You page
  2. Ricky is working on updating all the things so that we can upgrade to Django 1.7
  3. I'm working on fleshing out the Alerts project plan
  4. I'm working on the implementation work for the Gradient Sentiment project

That's it!

Mon, 05 Jan 2015

ElasticUtils: I'm stepping down and deprecating the project

What is^Wwas it?

ElasticUtils is^Wwas a Python library for building and executing Elasticsearch searches.

See the Quickstart for more details.

I'm taking my ball and going home

For the last few years, I've maintained ElasticUtils. It's a project I picked up after Jeff, Dave and Erik left.

Since I picked it up, we've had 13 or so releases. We switched the underlying library from pyes to pyelasticsearch and then to elasticsearch-py. We added support for a variety of Elasticsearch versions up to about 1.0. We overhauled the documentation. We overhauled the test suite. We generalized it from just a Django library into a library with a Django extension. We added MappingTypes. We worked really hard to make it backwards compatible from version to version. It's been a lot of work, but mostly in spurts.

Since the beginning of the project in May 2011 (a year before I picked up maintenance), we've had 33 people contribute patches. We've had many more point out issues, ask questions on the #elasticutils IRC channel, send me email about this and that.

That's a good run for an open source project. There are a bunch of things I wish I had done differently when I was at the helm, but it's a good run nonetheless.

The current state of things, however, is not great. ElasticUtils has a ton of issues. Lots of technical debt accrued over the years, several architectural decisions that turned out to suck and have obnoxious consequences, lack of support for new features in Elasticsearch > 1.0, etc. It'll take a lot of work to clean that up. Plus it's got a CamelCase name and that's so passé.

At PyCon 2014, Rob, Jannis and I worked with Honza on the API for the library that is now elasticsearch-dsl-py. This library has several of the things I like about ElasticUtils. It's a project that's being supported by the Elasticsearch folks. It's got momentum. It supports many of the Elasticsearch > 1.0 features. There are several libraries that sit on top of elasticsearch-dsl-py as Django shims now.

ElasticUtils is at a point where it's got a lot of problems and there are good alternatives that are better supported.

Thus, this is an excellent time for me to step down as maintainer. Going forward from today, I won't be doing any more development or maintenance.

Further, I'm deprecating the project because no one should be using an unmaintained broken project when there are better supported alternatives. That way lies madness!

So, if you're using ElasticUtils, what do you do now?

ElasticUtils 0.10.2 has the things you need to bridge from Elasticsearch 0.90 to 1.x. You could upgrade to that version and then switch to elasticsearch-dsl-py or one of the Django shims.

On that note, so long ElasticUtils and thank you to everyone who's helped on and used ElasticUtils over the years!

Wed, 31 Dec 2014

Input: 2014 retrospective

2014 was a big year for Input. This is the year we hit a point where the project is mature and stable. This is the year we worked on making it easier for new people to start working on Input. This is the year we fixed all the issues that made it difficult for Input to support multiple products. This is the year we created an API that allowed anyone to access the data to let them write their own dashboards. This is the year we implemented a translation infrastructure so that non-English feedback can flow through our tools (previously, it was handled manually). It was a big year.

Bugzilla and git stats

Let's look at some Bugzilla and git stats for the year:

Year 2014 (2014-01-01 -> 2014-12-31)
====================================


Bugzilla
========

Bugs created: 277
Creators:

         Will Kahn-Greene [:willkg] : 240
                             [:Cww] : 8
     Ricky Rosario [:rrosario, :r1c : 4
      Swarnava Sengupta (:Swarnava) : 3
     Nicolas Perriault (:NiKo`) — n : 1
                           vivek :) : 1
           Trif Andrei-Alin[:AlinT] : 1
             Kohei Yoshino [:kohei] : 1
             t.schmittlauch+persona : 1
     Mike "Pomax" Kamermans [:pomax : 1
                       Mark Filipak : 1
              Matt Grimes [:Matt_G] : 1
        Padraic Harley [:thelodger] : 1
                         deshrajdry : 1
           Matt Brubeck (:mbrubeck) : 1
                  Christian Corrodi : 1
              phaneendra.chiruvella : 1
                        vivek.kiran : 1
                          lgp171188 : 1
                                 me : 1
                Matthew N. [:MattN] : 1
                David Weir (satdav) : 1
     Hal Wine [:hwine] (use needinf : 1
          Ian Kronquist (:muricula) : 1
                       EventHorizon : 1
               Laura Thomson :laura : 1

Bugs resolved: 245

                                    : 1
                            WONTFIX : 17
                         WORKSFORME : 8
                          DUPLICATE : 13
                              FIXED : 206

                         Tracebacks : 9
                           Research : 5
                            Tracker : 11

Research bugs: 5

    788597: [research] Should we use a stacked graph on the dashboard?
    792976: [research] swap jingo-minify for django-compressor and
        django-jingo-offline-compressor
    889370: [research] morelikethis for feedback responses
    990774: [research] Investigate database schema changes for easier
        extensibility
    1048459: [research] check if updating from a tarball in a virtual
        environment with peep removes files

Tracker bugs: 11

    963275: [tracker] support Metro
    964260: [tracker] overhaul mobile and desktop feedback forms
    965791: [tracker] implement product table
    966425: [tracker] create analyzer search view
    985645: [tracker] Upgrade to Django 1.6
    988612: [tracker] product dashboards
    1040773: [tracker] dashboards for everyone
    1042669: [tracker] reduce contributor pain
    1052459: [tracker] heartbeat v1
    1062429: [tracker] integrate spicedham classifier
    1081412: [tracker] add tests for fjord_utils.js functions

Resolvers: 14

         Will Kahn-Greene [:willkg] : 209
                          lgp171188 : 7
                             aokoye : 7
          Ian Kronquist (:muricula) : 5
     Ricky Rosario [:rrosario, :r1c : 4
           Joshua Smith [:joshua-s] : 3
      Swarnava Sengupta (:Swarnava) : 2
      Rehan Dalal [:rehan, :rdalal] : 2
                    Ruben Vereecken : 1
                                Rob : 1
                                cww : 1
     Schalk Neethling [:espressive] : 1
                    bhargav.kowshik : 1
                               Anna : 1

Commenters: 65

                             willkg : 1261
                           rrosario : 19
                            mgrimes : 17
                                cww : 14
                          MattN+bmo : 11
                            mcooper : 11
                          thewanuki : 7
                             aokoye : 7
                   swarnavasengupta : 7
                            padraic : 6
                       iankronquist : 6
                MarkFilipak.mozilla : 5
                 nicolas.barbulesco : 5
              schalk.neethling.bugs : 5
                              hwine : 4
                             rdalal : 4
                          lgp171188 : 4
                     stephen.donner : 4
                           educmale : 4
                            anthony : 4
                           mbrubeck : 4
                     ms.annaphilips : 4
                          nigelbabu : 3
                              rdaub : 3
                        me+bugzilla : 3
                              pomax : 3
                         deshrajdry : 3
                    bhargav.kowshik : 3
                     rubenvereecken : 3
                           rrayborn : 3
                       rajul.iitkgp : 2
                           kbrosnan : 2
                          christian : 2
                              glind : 2
                            peterbe : 2
                            hcondei : 2
                         david.weir : 2
                       dron.rathore : 1
                       pradeeppaddy : 1
                            tdowner : 1
                  margaret.leibovic : 1
                        aaron.train : 1
                     scottstensland : 1
                            senicar : 1
              phaneendra.chiruvella : 1
                     trifandreialin : 1
                              jesse : 1
             t.schmittlauch+persona : 1
                     viveknjadhav19 : 1
                          mozaakash : 1
                                 me : 1
                               glob : 1
                             cturra : 1
                         nperriault : 1
                          rmcguigan : 1
                        vivek.kiran : 1
                     kdurant35rules : 1
                         vega.james : 1
                             fbraun : 1
                             326374 : 1
                vivekb.balakrishnan : 1
                              mhoye : 1
                              mluna : 1
                             feer56 : 1
                         lorenzo567 : 1

git
===

Total commits: 651

      Will Kahn-Greene :   561  (+243270, -205399, files 2746)
         L. Guruprasad :    23  (+406, -78, files 33)
            Adam Okoye :    22  (+213, -67, files 66)
         Ian Kronquist :    12  (+402, -106, files 21)
         Ricky Rosario :    10  (+165, -350, files 30)
        ossreleasefeed :     3  (+197, -42, files 9)
       Bhargav Kowshik :     3  (+114, -2, files 11)
          Joshua Smith :     3  (+91, -36, files 10)
          Anna Philips :     2  (+734, -6, files 24)
       Ruben Vereecken :     2  (+69, -29, files 12)
            Gregg Lind :     2  (+18, -16, files 6)
         Deshraj Yadav :     2  (+2, -2, files 2)
     Swarnava Sengupta :     2  (+2, -2, files 2)
           Rehan Dalal :     2  (+315, -169, files 13)
                aokoye :     2  (+42, -8, files 4)

Total lines added: 246040
Total lines deleted: 206312
Total files changed: 2989

Everyone
========

    326374
    aaron.train
    Adam Okoye
    Anna
    Anna Philips
    anthony
    Bhargav Kowshik
    christian
    Christian Corrodi
    cturra
    cww
    David Weir
    Deshraj Yadav
    deshrajdry
    dron.rathore
    educmale
    EventHorizon
    fbraun
    feer56
    glob
    Gregg Lind
    Hal Wine
    hcondei
    Ian Kronquist
    jesse
    Joshua Smith
    kbrosnan
    kdurant35rules
    Kohei Yoshino [:kohei]
    L. Guruprasad
    Laura Thomson :laura
    lorenzo567
    margaret.leibovic
    Mark Filipak
    Matt Brubeck (:mbrubeck)
    Matt Grimes
    Matthew N. [:MattN]
    Mike Cooper
    me
    me+bugzilla
    mhoye
    Mike "Pomax" Kamermans [:pomax]
    mluna
    mozaakash
    ms.annaphilips
    Nicolas Perriault
    nicolas.barbulesco
    nigelbabu
    Padraic Harley [:thelodger]
    peterbe
    phaneendra.chiruvella
    pomax
    pradeeppaddy
    rajul.iitkgp
    Ralph Daub
    Rehan Dalal
    Ricky Rosario
    rmcguigan
    Rob
    Robert Rayborn
    Ruben Vereecken
    Schalk Neethling
    scottstensland
    senicar
    Stephen Donner
    Swarnava Sengupta
    t.schmittlauch+persona
    Tyler Downer
    thewanuki
    Trif Andrei-Alin
    vega.james
    vivek.kiran
    vivekb.balakrishnan
    viveknjadhav19
    Will Kahn-Greene

Some observations:

  1. In 2013, we resolved more bugs than we created partially because we closed a bunch of bugs related to the old Input that weren't relevant anymore.

    In 2014, we created more bugs than we closed by 10%. I think that's about what we want.

  2. 15 people had git commits. 26 people created bugs. 14 people resolved bugs. 65 people commented on bugs.

    One thing I don't have is counts for who helped translate strings which is a really important part of development. My apologies.

  3. Of all those people, only 1 is a "core developer"--that's me. Everyone else contributed their time and energies towards making Input better. I really appreciate that. Thank you!

That's the stats!

Accomplishments

Things we did in 2014:

Site health dashboard: I wrote a site health dashboard that helps me understand how the site is performing before and after deployments as well as after releases and other events.

Client side smoke tests: I wrote smoke tests for the client side. I based it on the defunct input-tests code that QA was maintaining up until we rewrote Input. The smoke tests have been invaluable for reducing/eliminating data-loss bugs.

Vagrant: I took some inspiration from Erik Rose and DXR and wrote a Vagrant provisioning shell script. This includes a docs overhaul as well.

Automated translation system (human and machine): I wrote an automated translation system. It's generalized so that it isn't model/field specific. It's also generalized so that we can add plugins for other translation systems. It's currently got plugins for Dennis, Gengo machine translation and Gengo human translation. We're machine translating most incoming feedback. We're human translating Firefox OS feedback. This was a HUGE project, but it's been immensely valuable.

Better query syntax: We were upgraded to Elasticsearch 0.90.10. I switched the query syntax for the dashboard search field to use Elasticsearch simple_query_string. That allows users to express search queries they weren't previously able to express.

utm_source and utm_campaign handling: I finished the support for handling utm_source and utm_campaign querystring parameters. This allows us to differentiate between organic feedback and non-organic feedback.

More like this: I added a "more like this" section to the response view. This makes it possible for UA analyzers to look at a response and see other responses that are similar.

Dashboards for everyone: We wrote an API and some compelling examples of dashboards you can build using the API. It's being used in a few places now. We'll grow it going forward as needs arise. I'm pretty psyched about this since it makes it possible for people with needs to help themselves and not have to wait for me to get around to their work.

Dashboards for everyone project plan.

Vagrant: We took the work I did last quarter and improved upon it, rewrote the docs and have a decent Vagrant setup now. L. Guruprasad improved on this and the documentation and setting up a Vagrant-based vm for Input development is much easier.

Reduce contributor pain project plan.

Abuse detection: Ian spent his internship working on an abuse classifier so that we can more proactively detect and prevent abusive feedback from littering Input. We gathered some interesting data and the next step is probably to change the approach we used and apply some more complex ML things to the problem. The key here is that we want to detect abuse with confidence and not accidentally catch swaths of non-abuse. Input feedback has some peculiar properties that make this difficult.

Reduce the abuse project plan.

Loop support: Loop is now using Input for user sentiment feedback.

Heartbeat support: User Advocacy is working on a project to give us a better baseline for user sentiment. This project was titled Heartbeat, but I'm not sure whether that'll change or not. Regardless, we added support for the initial prototype.

Heartbeat v1 project plan.

Data retention policy: We've been talking about a data retention policy for some time. We decided on one, finalized it and codified it in code.

Shed the last vestiges of Playdoh and funfactory: We shed the last bits of Playdoh and funfactory. Input uses the same protections and security decisions those two projects enforced, but without being tied to some of the infrastructure decisions. This made it easier to switch to peep-based requirements management.

Switched to FactoryBoy and overhauled tests: Tests run pretty fast in Fjord now. We switched to FactoryBoy, so writing model-based tests is a lot easier than the stuff we had before.

Python 2.7: Input is now running on Python 2.7. Thank you, Jake!

Remote troubleshooting data capture: The generic feedback form which is hosted on Input now has a section allowing users to opt-in to sending data about their browser along with their feedback. This data is crucial to helping us suss out problems with video playback, graphics cards/drivers and malicious addons.

This code is still "alpha". We'll be finishing it up in 2015q1.

Remote troubleshooting data capture project plan.

Heartbeat v1 and v2: People leave feedback on Input primarily when they're frustrated with something. Because of this, the sentiment numbers we get on Input tilt heavily negative and only represent people who are frustrated and were able to find the feedback form. Heartbeat will give us sentiment data that's more representative of our entire user base.

As a stop-gap to get the project going, Input is the backend collecting all the Heartbeat data. We rewrote the Heartbeat-related code for Heartbeat v2 in 2014q4.

Heartbeat v2 project plan.

Feedback form overhaul: We rewrote the feedback form to clean up the text, reduce confusion about what data is made public and what data is kept private, reduce the number of steps to leave feedback and improve the form for both desktop and mobile devices.

Feedback form overhaul project plan.

We also fixed the form so it supports multiple products because we're collecting feedback for multiple products on Input now.

Multiple products project plan.

Thoughts

Looking at the stats above, it's pretty clear that this is predominantly a one-person project. Ricky, Mike and Rehan do all my code review. Without them, things would be a lot worse.

Having said that, the "baby-factor" stinks on this project. That's something I'm going to work on over 2015:

  • improving the documentation and making sure all the important things are covered or easily figured out
  • improving the project planning and making sure most/all project planning is done in public or at least has a publicly available record
  • reducing the complexity of the application, reducing dependencies and making the parts less tangled
  • reducing the number of things that make Input a "special snowflake" in regards to other sites my peers maintain

Want to be a part of the team?

Input is a great little Django application that collects feedback for Mozilla products. It's an important part of the Mozilla product ecosystem. Working on it helps millions of people. If you're interested in being a part of the team that develops it, here are some helpful links:

Input: 2014q4 quarter in review

2014q4 was interesting. Between the faux all-hands, holidays and production freezes, it was essentially a two-month quarter. I literally raced through the quarter getting things done as quickly as I could do them opting for the minimal viable implementation wherever possible. That's not a great thing, but so it goes.

Things to know:

Bugzilla and git stats

I rewrote my script and the data is richer now.

Quarter 2014q4 (2014-10-01 -> 2014-12-31)
=========================================


Bugzilla
========

Bugs created: 95
Creators:

         Will Kahn-Greene [:willkg] : 85
                                 me : 1
     Nicolas Perriault (:NiKo`) — n : 1
                David Weir (satdav) : 1
                       Mark Filipak : 1
                           vivek :) : 1
               Laura Thomson :laura : 1
      Swarnava Sengupta (:Swarnava) : 1
                             [:Cww] : 1
                         deshrajdry : 1
              phaneendra.chiruvella : 1

Bugs resolved: 89

                            WONTFIX : 13
                         WORKSFORME : 4
                          DUPLICATE : 5
                              FIXED : 67

                         Tracebacks : 1
                           Research : 4
                            Tracker : 7

Research bugs: 4

    788597: [research] Should we use a stacked graph on the dashboard?
    792976: [research] swap jingo-minify for django-compressor and
        django-jingo-offline-compressor
    889370: [research] morelikethis for feedback responses
    1048459: [research] check if updating from a tarball in a virtual
        environment with peep removes files

Tracker bugs: 7

    964260: [tracker] overhaul mobile and desktop feedback forms
    985645: [tracker] Upgrade to Django 1.6
    988612: [tracker] product dashboards
    1040773: [tracker] dashboards for everyone
    1042669: [tracker] reduce contributor pain
    1052459: [tracker] heartbeat v1
    1081412: [tracker] add tests for fjord_utils.js functions

Resolvers: 7

         Will Kahn-Greene [:willkg] : 75
                             aokoye : 7
                          lgp171188 : 4
                                cww : 1
                    bhargav.kowshik : 1
      Rehan Dalal [:rehan, :rdalal] : 1

Commenters: 27

                             willkg : 420
                             aokoye : 7
                            mcooper : 6
                            padraic : 6
                MarkFilipak.mozilla : 5
                              hwine : 4
                             rdalal : 4
                     stephen.donner : 4
                          nigelbabu : 3
                         deshrajdry : 3
                    bhargav.kowshik : 3
                         david.weir : 2
                            mgrimes : 2
                       dron.rathore : 1
                         lorenzo567 : 1
                            senicar : 1
                     viveknjadhav19 : 1
                         nperriault : 1
                                cww : 1
                   swarnavasengupta : 1
                          mozaakash : 1
                        me+bugzilla : 1
              phaneendra.chiruvella : 1
                                 me : 1
                             fbraun : 1
                vivekb.balakrishnan : 1
                           rrosario : 1

git
===

Total commits: 239

      Will Kahn-Greene :   191  (+27879, -10550, files 822)
            Adam Okoye :    24  (+255, -75, files 70)
         L. Guruprasad :    17  (+390, -42, files 27)
       Bhargav Kowshik :     3  (+114, -2, files 11)
            Gregg Lind :     2  (+18, -16, files 6)
         Deshraj Yadav :     2  (+2, -2, files 2)

Total lines added: 28658
Total lines deleted: 10687
Total files changed: 938

Everyone
========

    Adam Okoye
    Bhargav Kowshik
    cww
    David Weir
    Deshraj Yadav
    deshrajdry
    dron.rathore
    fbraun
    Gregg Lind
    hwine
    L. Guruprasad
    Laura Thomson :laura
    lorenzo567
    Mark Filipak
    mcooper
    me
    me+bugzilla
    mgrimes
    mozaakash
    Nicolas Perriault (:NiKo`) — needinfo me if you need my attention
    nigelbabu
    nperriault
    padraic
    phaneendra.chiruvella
    Rehan Dalal
    rrosario
    senicar
    stephen.donner
    Swarnava Sengupta
    vivekb.balakrishnan
    viveknjadhav19
    Will Kahn-Greene

Code line counts:

2014q1: April 1st, 2014:        15195 total  6953 Python
2014q2: July 1st, 2014:         20456 total  9247 Python
2014q3: October 7th. 2014:      23466 total  11614 Python
2014q4: December 31st, 2014:    30158 total  13615 Python

Nothing wildly interesting there other than noting that the codebase for Input continues to grow.

Contributor stats

Adam Okoye started as an intern through OPW on December 9th. He's contributed to Input in the past through the Ascend project. Over the course of the OPW internship, he'll be working on Input bugs and the Thank you page project.

L. Guruprasad spent a lot of time working on pre-commit linters, Vagrant provisioning and generally improving the experience contributors will have.

We had a few commits from other people, too.

Thank you everyone who contributed!

Accomplishments

Python 2.7: Input is now running on Python 2.7. Thank you, Jake!

Remote troubleshooting data capture: The generic feedback form which is hosted on Input now has a section allowing users to opt-in to sending data about their browser along with their feedback. This data is crucial to helping us suss out problems with video playback, graphics cards/drivers and malicious addons.

This code is still "alpha". We'll be finishing it up in 2015q1.

Remote troubleshooting data capture project plan.

Heartbeat v1 and v2: People leave feedback on Input primarily when they're frustrated with something. Because of this, the sentiment numbers we get on Input tilt heavily negative and only represent people who are frustrated and were able to find the feedback form. Heartbeat will give us sentiment data that's more representative of our entire user base.

As a stop-gap to get the project going, Input is the backend collecting all the Heartbeat data. We rewrote the Heartbeat-related code for Heartbeat v2 in 2014q4.

Heartbeat v2 project plan.

Feedback form overhaul: We rewrote the feedback form to clean up the text, reduce confusion about what data is made public and what data is kept private, reduce the number of steps to leave feedback and improve the form for both desktop and mobile devices.

Feedback form overhaul project plan.

We also fixed the form so it supports multiple products because we're collecting feedback for multiple products on Input now.

Multiple products project plan.

Summary

2014q4 was tough because of the limited time, but it was a good quarter and we got a lot done.

The faux all-hands involved a bunch of discussions related to Input development. 2015q1 is going to be busy busy busy.

Fri, 19 Dec 2014

Input status: December 18th, 2014

Preface

It's been 3 months since the last status report. Crimey! That's not great, but it's even worse because it makes this report crazy long.

First off, lots of great work done by Adam Okoye, L. Guruprasad, Bhargav Kowshik, and Deshraj Yadav! w00t!

Second, we did a ton of stuff and broke 1,000 commits! w00t!

Third, I've promised to do more frequent status reports. I'll go back to one every two weeks.

Onward!

Development

High-level summary:

  • Lots of code quality work.
  • Updated ElasticUtils to 0.10.1 so we can upgrade our Elasticsearch cluster.
  • Heartbeat v2.
  • Overhauled the generic feedback form.
  • remote-troubleshooting data capture.
  • contribute.json file.
  • Upgrade to Django 1.6.
  • Upgrade to Python 2.7!!!!!
  • Improved and added to pre-commit and commit-msg linters.

Landed and deployed:

  • ce95161 Clarify source and campaign parameters in API
  • 286869e bug 788281 Add update_product_details to deploy
  • bbedc86 bug 788281 Implement basic events API
  • 1c0ff9f bug 1071567 Update ElasticUtils to 0.10.1
  • 7fd52cd bug 1072575 Rework smart_timedelta
  • ce80c56 bug 1074276 Remove abuse classification prototype
  • 7540394 bug 1075563 Fix missing flake8 issue
  • 11e4855 bug 1025925 Change file names (Adam Okoye)
  • 23af92a bug 1025925 Change all instances of fjord.analytics.tools to fjord.analytics.utils (Adam Okoye)
  • ae28c60 bug 1025925 Change instances of of util relating to fjord/base/util.py (Adam Okoye)
  • bc77280 Add Adam Okoye to CONTRIBUTORS
  • 545dc52 bug 1025925 Change test module file names
  • fc24371 bug 1041703 Drop prodchan column
  • 9097b8f bug 1079376 Add error-email -> response admin view
  • d3cfdfe bug 1066618 Tweak Gengo account balance warning
  • a49f1eb bug 1020303 Add rating column
  • 55fede0 bug 1061798 Reset page number Resets page number when filter checkbox is checked (Adam Okoye)
  • 1d4fd00 bug 854479 Fix ui-lightness 404 problems
  • a9bf3b1 bug 940361 Change size on facet calls
  • c2b2c2b bug 1081413 Move url validation code into fjord_utils.js Rewrote url validation code that was in generic_feedback.js and added it to fjord_utils.js (Adam Okoye)
  • 4181b5e bug 1081413 Change code for url validation (Adam Okoye)
  • 2cd62ad bug 1081413 Add test for url validation (Adam Okoye)
  • f72652a bug 1081413 Correct operator in test_fjord_utils.js (aokoye)
  • c9b83df bug 1081997 Fix unicode in smoketest
  • cba9a2d bug 1086643 bug 1086650 Redo infrastructure for product picker version
  • e8a9cc7 bug 1084387 Add on_picker field to Product
  • 2af4fca bug 1084387 Add on_picker to management forms
  • 1ced64a bug 1081411 Create format test (Adam Okoye)
  • 00f8a72 Add template for mentored bugs
  • e95d0f1 Cosmetic: Move footnote
  • d0cb705 Tweak triaging docs
  • d5b35a2 bug 1080816 Add A/B for ditching chart
  • fa1a47f Add notes about running tests with additional warnings
  • ddde83c Fix mimetype -> content_type and int division issue
  • 2edb3b3 bug 1089650 Add a contribute.json file (Bhargav Kowshik)
  • d341977 bug 1089650 Add test to verify that the JSON is valid (Bhargav Kowshik)
  • dcb9380 Add Bhargav Kowshik to CONTRIBUTORS
  • 7442513 Fix throttle test
  • f27e31c bug 1072285 Update Django, django-nose and django-cache-machine
  • dd74a3c bug 1072285 Update django-adminplus
  • ececdf7 bug 1072285 Update requirements.txt file
  • 6669479 bug 1093341 Tweak Gengo account balance warning
  • f233aab bug 1094197 Fix JSONObjectField default default
  • 11193d7 Tweak chart display
  • 9d311ca Make journal.Record not derive from ModelBase
  • f778c9d Remove all Heartbeat v1 stuff
  • e5f6f4d Switch test__utils.py to test_utils.py
  • cab7050 bug 1092296 Implement heartbeat v2 data model
  • 5480c42 bug 1097352 Response view is viewable by all
  • 46b5897 bug 1077423 Overhaul generic feedback form dev
  • da31b47 bug 1077423 Update smoke tests for generic feedback form dev
  • e84094b Fix l10n email template
  • d6c8ea9 Remove gettext calls for product dashboards
  • e1a0f74 bug 1098486 Remove under construction page
  • 032a660 Fix l10n_status.py script history table
  • 19cec37 Fix JSONObjectField
  • 430c462 Improve display_history for l10n_status
  • d6c18c6 Windows NT 6.4 -> Windows 10
  • 73a4225 bug 1097847 Update django-grappelli to 2.5.6
  • 4f3b9c7 bug 1097847 Fix custom views in admin
  • 3218ea3 Fix JSONObjectfield.value_to_string
  • 67c6bf9 Fix RecordManager.log regarding instances
  • a5e8610 bug 1092299 Implement Heartbeat v2 API
  • 17226db bug 1092300 Add Heartbeat v2 debugging views
  • 11681c4 Rework env view to show python and django version
  • 9153802 bug 1087387 Add feedback_response.browser_platform
  • f5d8b56 bug 1087387 bug 1096541 Clean up feedback view code
  • c9c7a81 bug 1087391 Fix POST API user-agent inference code
  • 4e93fc7 bug 1103024 Gengo kill switch
  • de9d1c7 Capture the user-agent
  • 4796e4e bug 1096541 Backfill browser_platform for Firefox dev
  • f5fe5cf bug 1103141 Add experiment_version to HB db and api
  • 98c40f6 bug 1103141 Add experiment_version to views
  • 0996148 bug 1103045 Create a menial hb survey view
  • 965b3ee bug 1097204 Rework product picker
  • 6907e6f bug 1097203 Add link to SUMO
  • e8f3075 bug 1093843 Increase length of product fields
  • 2c6d24b bug 1103167 Raise GET API throttle
  • d527071 bug 1093832 Move feedback url documentation
  • 6f4eb86 Abstract out python2.6 in deploy script
  • f843286 Fix compile-linted-mo.sh to take pythonbin as arg
  • 966da77 Add celery health check
  • 1422263 Add space before subject of celery health email
  • 5e07dbd [heartbeat] Add experiment1 static page placeholders
  • 615ccf1 [heartbeat] Add experiment1 static files
  • d8822df [heartbeat] Add SUMO links to sad page
  • 3ee924c [heartbeat] Add twitter thing to happy page
  • d87a815 [heartbeat] Change thank you text
  • 06e73e6 [heartbeat] Remove cruft, fix links
  • 8208a72 [heartbeat] Fix "addons"
  • 2eca74c [heartbeat] Show profile_age always because 0 is valid
  • 4c4598b bug 1099138 Fix "back to picker" url
  • b2e9445 Add note about "Commit to VCS" in l10n docs
  • 9c22705 Heartbeat: instrument email signup, feedback, NOT Twitter (Gregg Lind)
  • 340adf9 [heartbeat] Fix DOCTYPE and ispell pass
  • 486bf65 [heartbeat] Change Thank you text
  • d52c739 [heartbeat] Switch to use Input GA account
  • f07716b [heartbeat] Fix favicons
  • eff9d0b [heartbeat] Fixed page titles
  • 969c4a0 [heartbeat] Nix newsletter form for a link
  • dce6f86 [heartbeat] Reindent code to make it legible
  • dad6d82 bug 1099138 Remove [DEV] from title
  • 4204b43 fixed typo in getting_started.rst (Deshraj Yadav)
  • 7042ead bug 1107161 Fix hb answers view
  • a024758 bug 1107809 Fix Gengo language guesser
  • 808fa83 bug 1107803 Rewrite Response inference code
  • d9e8ffd bug 1108604 Tweak paging links in hb data view
  • 00e8628 bug 1108604 Add sort and display ts better in hb data view
  • 17b908a bug 1108604 Change paging to 100 in hb data view
  • 39dc943 bug 1107083 Backfill versions
  • fee0653 bug 1105512 Rip out old generic form
  • b5bb54c Update grappelli in requirements.txt file
  • f984935 bug 1104934 Add ResponseTroubleshootingInfo model
  • c2e7fd3 bug 950913 Move 'TRUNCATE_LENGTH' and make accessable to other files (Adam Okoye)
  • b6f30e1 bug 1074315 Ignore deleted files for linting in pre-commit hook (L. Guruprasad)
  • 4009a59 Get list of .py files to lint using just git diff (L. Guruprasad)
  • c81da0b bug 950913 Access TRUNCATE_LENGTH from generic_feedback template (Adam Okoye)
  • 9e3cec6 bug 1111026 Fix hb error page paging
  • b89daa6 Dennis update to master tip
  • 61e3e18 Add django-sslserver
  • 93d317b bug 1104935 Add remote.js
  • ad3a5cb bug 1104935 Add browser data section to generic form
  • cc54daf bug 1104935 Add browserdata metrics
  • 31c2f74 Add jshint to pre-commit hook (L. Guruprasad)
  • 68eae85 Pretty-print JSON blobs in hb errorlog view
  • 8588b42 bug 1111265 Restrict remote-troubleshooting to Firefox
  • b0af9f5 Fix sorby error in hb data view
  • 8f622cf bug 1087394 Add browser, browser_version, and browser_platform to response view (Adam Okoye)
  • c4b6f85 bug 1087394 Change Browser Platform label (Adam Okoye)
  • eb1d5c2 Disable expansion of $PATH in the provisioning script (L. Guruprasad)
  • 59eebda Cosmetic test changes
  • aac733b bug 1112210 Tweak remote-troubleshooting capture
  • 6f24ce7 bug 1112210 Hide browser-ask by default
  • 278095d bug 1112210 Note if we have browser data in response view
  • 869a37c bug 1087395 Add fields to CSV output (Adam Okoye)

Landed, but not deployed:

  • 4ee7fd6 Update the name of the pre-commit hook script in docs (L. Guruprasad)
  • d4c5a09 bug 1112084 create requirements/dev.txt (L. Guruprasad)
  • 4f03c48 bug 1112084 Update provisioning script to install dev requirements (L. Guruprasad)
  • 03c5710 Remove instructions for manual installation of flake8 (L. Guruprasad)
  • a36a231 bug 1108755 Add a git commit message linter (L. Guruprasad)

Current head: f0ec99d

Rough plan for the next two weeks

  1. PTO. It's been a really intense quarter (as you can see) and I need some rest. Maybe a nap. Plus we have a deploy freeze through to January, so we can't push anything out anyhow. I hope everyone else gets some rest, too.

That's it!

Wed, 26 Nov 2014

Input: New feedback form

Since the beginning of 2014, I've been laying the groundwork to rewrite the feedback form that we use on Input.

Today, after a lot of work, we pushed out the new form! Just in time for Firefox 34 release.

This blog post covers the circumstances of the rewrite.

Why?

In 2011, James, Mike and I rewrote Input from the ground up. In order to reduce the amount of time it took to do that rewrite, we copied a lot of the existing forms and styles including the feedback forms. At that time, there were two: one for desktop and one for mobile. In order to avoid a translation round, we kept all the original strings of the two forms. The word "Firefox" was hardcoded in the strings, but that was fine since at the time Input only collected feedback for Firefox.

In 2013, in order to reduce complexity on the site because there's only one developer (me), I merged the desktop and mobile forms into one form. In order to avoid a translation round, I continued to keep the original strings. The wording became awkward and the flow through the form wasn't very smooth. Further, the form wasn't responsive at all, so it worked ok on desktop machines, but mediocre on other viewport sizes.

2014 rolled around and it was clear Input was going to need to branch out into capturing feedback for multiple products---some of which were not Firefox. The form made this difficult.

Related, the smoketest framework I wrote in 2014 struggled with testing the form accurately. I spent some time tweaking it, but a simpler form would make smoketesting a lot easier and less flakey.

Thus over the course of 3 years, we had accumulated the following problems:

  1. The flow through the form felt awkward, instructions weren't clear and information about what data would be public and what data would be private wasn't clear.
  2. Strings had "Firefox" hardcoded and wouldn't support multiple products.
  3. The form wasn't responsive and looked/behaved poorly in a variety of situations.
  4. The form never worked in right-to-left languages and possibly had other accessibility issues.
  5. The architecture didn't let us experiment with the form---tweaking the wording, switching to a more granular gradient of sentiment, capturing other data, etc.

Further, we were seeing many instances of people putting contact information in the description field and there was a significant amount of dropoff.

I had accrued the following theories:

  1. Since the email address is on the third card, users would put their email address in the description field because they didn't know they could leave their contact information later.
  2. Having two cards would reduce the amount of drop-off and unfinished forms than three cards.
  3. Having simpler instruction text would reduce the amount of drop-off.

Anyhow, it was due for an overhaul.

So what's changed?

I've been working on the overhaul for most of 2014, but did the bulk of the work in October and November. It has the following changes:

  1. The new form is shorter and clearer text-wise and design-wise.
  2. It consists of two cards: one for capturing sentiment and one for capturing details about that sentiment.
  3. It clearly delineates data that will be public from data that will be kept private.
  4. It works with LTR and RTL languages (If that's not true, please open a bug.)
  5. It fixes some accessibility issues. (If you find any, please open a bug.)
  6. It uses responsive design, mobile first. Thus it was designed for mobile devices and then scaled to desktop-sized viewports.
  7. It's smaller in kb size and requires fewer HTTP requests.
  8. It's got a better architecture for future development.
  9. It doesn't have "Firefox" hardcoded anymore.
  10. It's simpler so the smoketests work reliably now.
The old Input feedback form.

The old Input feedback form.

The new Input feedback form.

The new Input feedback form.

Note: Showing before and after isn't particularly exciting since this is only the first card of the form in both cases.

Going forward

The old and new forms were instrumented in various ways, so we'll be able to analyze differences between the two. Particularly, we'll be able to see if the new form performs worse.

Further, I'll be checking the data to see if my theories hold true especially the one regarding why people put contact data in the description.

There are a few changes in the queue that we want to make over the course of the next 6 months. Now that the new form has landed, we can start working on those.

Even if there are problems with the new form, we're in a much better position to fix them than we were before. Progress has been made!

Take a moment---try out the form and tell us YOUR feedback

Have you ever submitted feedback? Have you ever told Mozilla what you like and don't like about Firefox?

Take a moment and fill out the feedback form and tell us how you feel about Firefox.

Thanks, etc

I've been doing web development since 1997 or so. I did a lot of frontend work back then, but I haven't done anything serious frontend-wise in the last 5 years. Thus this was a big project for me.

I had a lot of help: Ricky, Mike and Rehan from the SUMO Engineering team were invaluable reviewing code, helping me fix issues and giving me a huge corpus of examples to learn from; Matt, Gregg, Tyler, Ilana, Robert and Cheng from the User Advocacy team who spent a lot of time smoothing out the rough edges of the new form so it captures the data we need; Schalk who wrote the product picker which I later tweaked; Matej who spent time proof-reading the strings to make sure they were consistent and felt good; the QA team which wrote the code that I copied and absorbed into the current Input smoketests; and the people who translated the user interface strings (and found a bunch of issues) making it possible for people to see this form in their language.

Tue, 28 Oct 2014

Input: Removing the frontpage chart

I've been working on Input for a while now. One of the things I've actively disliked was the chart on the front page. This blog post talks about why I loathe it and then what's happening this week.

First, here's the front page dashboard as it is today:

Input front page dashboard

Input front page dashboard (October 2014)

When I started, Input gathered feedback solely on the Firefox desktop web browser. It was a one-product feedback gathering site. Because it was gathering feedback for a single product, the front page dashboard was entirely about that single product. All the feedback talked about that product. The happy/sad chart was about that product. Today, Input gathers feedback for a variety of products.

When I started, it was nice to have a general happy/sad chart on the front page because no one really looked at it and the people who did look at it understood why the chart slants so negatively. So the people who did look at it understood the heavy negative bias and could view the chart as such. Today, Input is viewed by a variety of people who have no idea how feedback on Input works or why it's so negatively biased.

When I started, Input didn't expose the data in helpful ways allowing people to build their own charts and dashboards to answer their specific questions. Thus there was a need for a dashboard to expose information from the data Input was gathering. I contend that the front page dashboard did this exceedingly poorly--what does the happy/sad lines actually mean? If it dips, what does that mean? If they spike, what does that mean? There's not enough information in the chart to make any helpful conclusions. Today, Input has an API allowing anyone to fetch data from Input in JSON format and generate their own dashboards of which there are several out there.

When I started, Input received some spam/abuse feedback, but the noise was far outweighed by the signal. Today, we get a ton of spam/abuse feedback. We still have no good way of categorizing spam/abuse as such and removing it from the system. That's something I want to work on more, but haven't had time to address. In the meantime, the front page dashboard chart has a lot of spammy noise in it. Thus the happy/sad lines aren't accurate.

Thus I argue we've far outlived the usefulness of the chart on the front page and it's time for it to go away.

So, what happens now? Bug 1080816 covers removing the front page dashboard chart. It covers some other changes to the front page, but I think I'm going to push those off until later since they're all pretty "up in the air".

If you depend on the front page dashboard chart, toss me an email. Depending on how many people depend on the front page chart and what the precise needs are, maybe we'll look into writing a better one.