Will >> Will's blog

purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon
Page 1 of 2  >> (less recent)

Wed, 31 Dec 2014

Input: 2014 retrospective

2014 was a big year for Input. This is the year we hit a point where the project is mature and stable. This is the year we worked on making it easier for new people to start working on Input. This is the year we fixed all the issues that made it difficult for Input to support multiple products. This is the year we created an API that allowed anyone to access the data to let them write their own dashboards. This is the year we implemented a translation infrastructure so that non-English feedback can flow through our tools (previously, it was handled manually). It was a big year.

Bugzilla and git stats

Let's look at some Bugzilla and git stats for the year:

Year 2014 (2014-01-01 -> 2014-12-31)
====================================


Bugzilla
========

Bugs created: 277
Creators:

         Will Kahn-Greene [:willkg] : 240
                             [:Cww] : 8
     Ricky Rosario [:rrosario, :r1c : 4
      Swarnava Sengupta (:Swarnava) : 3
     Nicolas Perriault (:NiKo`) — n : 1
                           vivek :) : 1
           Trif Andrei-Alin[:AlinT] : 1
             Kohei Yoshino [:kohei] : 1
             t.schmittlauch+persona : 1
     Mike "Pomax" Kamermans [:pomax : 1
                       Mark Filipak : 1
              Matt Grimes [:Matt_G] : 1
        Padraic Harley [:thelodger] : 1
                         deshrajdry : 1
           Matt Brubeck (:mbrubeck) : 1
                  Christian Corrodi : 1
              phaneendra.chiruvella : 1
                        vivek.kiran : 1
                          lgp171188 : 1
                                 me : 1
                Matthew N. [:MattN] : 1
                David Weir (satdav) : 1
     Hal Wine [:hwine] (use needinf : 1
          Ian Kronquist (:muricula) : 1
                       EventHorizon : 1
               Laura Thomson :laura : 1

Bugs resolved: 245

                                    : 1
                            WONTFIX : 17
                         WORKSFORME : 8
                          DUPLICATE : 13
                              FIXED : 206

                         Tracebacks : 9
                           Research : 5
                            Tracker : 11

Research bugs: 5

    788597: [research] Should we use a stacked graph on the dashboard?
    792976: [research] swap jingo-minify for django-compressor and
        django-jingo-offline-compressor
    889370: [research] morelikethis for feedback responses
    990774: [research] Investigate database schema changes for easier
        extensibility
    1048459: [research] check if updating from a tarball in a virtual
        environment with peep removes files

Tracker bugs: 11

    963275: [tracker] support Metro
    964260: [tracker] overhaul mobile and desktop feedback forms
    965791: [tracker] implement product table
    966425: [tracker] create analyzer search view
    985645: [tracker] Upgrade to Django 1.6
    988612: [tracker] product dashboards
    1040773: [tracker] dashboards for everyone
    1042669: [tracker] reduce contributor pain
    1052459: [tracker] heartbeat v1
    1062429: [tracker] integrate spicedham classifier
    1081412: [tracker] add tests for fjord_utils.js functions

Resolvers: 14

         Will Kahn-Greene [:willkg] : 209
                          lgp171188 : 7
                             aokoye : 7
          Ian Kronquist (:muricula) : 5
     Ricky Rosario [:rrosario, :r1c : 4
           Joshua Smith [:joshua-s] : 3
      Swarnava Sengupta (:Swarnava) : 2
      Rehan Dalal [:rehan, :rdalal] : 2
                    Ruben Vereecken : 1
                                Rob : 1
                                cww : 1
     Schalk Neethling [:espressive] : 1
                    bhargav.kowshik : 1
                               Anna : 1

Commenters: 65

                             willkg : 1261
                           rrosario : 19
                            mgrimes : 17
                                cww : 14
                          MattN+bmo : 11
                            mcooper : 11
                          thewanuki : 7
                             aokoye : 7
                   swarnavasengupta : 7
                            padraic : 6
                       iankronquist : 6
                MarkFilipak.mozilla : 5
                 nicolas.barbulesco : 5
              schalk.neethling.bugs : 5
                              hwine : 4
                             rdalal : 4
                          lgp171188 : 4
                     stephen.donner : 4
                           educmale : 4
                            anthony : 4
                           mbrubeck : 4
                     ms.annaphilips : 4
                          nigelbabu : 3
                              rdaub : 3
                        me+bugzilla : 3
                              pomax : 3
                         deshrajdry : 3
                    bhargav.kowshik : 3
                     rubenvereecken : 3
                           rrayborn : 3
                       rajul.iitkgp : 2
                           kbrosnan : 2
                          christian : 2
                              glind : 2
                            peterbe : 2
                            hcondei : 2
                         david.weir : 2
                       dron.rathore : 1
                       pradeeppaddy : 1
                            tdowner : 1
                  margaret.leibovic : 1
                        aaron.train : 1
                     scottstensland : 1
                            senicar : 1
              phaneendra.chiruvella : 1
                     trifandreialin : 1
                              jesse : 1
             t.schmittlauch+persona : 1
                     viveknjadhav19 : 1
                          mozaakash : 1
                                 me : 1
                               glob : 1
                             cturra : 1
                         nperriault : 1
                          rmcguigan : 1
                        vivek.kiran : 1
                     kdurant35rules : 1
                         vega.james : 1
                             fbraun : 1
                             326374 : 1
                vivekb.balakrishnan : 1
                              mhoye : 1
                              mluna : 1
                             feer56 : 1
                         lorenzo567 : 1

git
===

Total commits: 651

      Will Kahn-Greene :   561  (+243270, -205399, files 2746)
         L. Guruprasad :    23  (+406, -78, files 33)
            Adam Okoye :    22  (+213, -67, files 66)
         Ian Kronquist :    12  (+402, -106, files 21)
         Ricky Rosario :    10  (+165, -350, files 30)
        ossreleasefeed :     3  (+197, -42, files 9)
       Bhargav Kowshik :     3  (+114, -2, files 11)
          Joshua Smith :     3  (+91, -36, files 10)
          Anna Philips :     2  (+734, -6, files 24)
       Ruben Vereecken :     2  (+69, -29, files 12)
            Gregg Lind :     2  (+18, -16, files 6)
         Deshraj Yadav :     2  (+2, -2, files 2)
     Swarnava Sengupta :     2  (+2, -2, files 2)
           Rehan Dalal :     2  (+315, -169, files 13)
                aokoye :     2  (+42, -8, files 4)

Total lines added: 246040
Total lines deleted: 206312
Total files changed: 2989

Everyone
========

    326374
    aaron.train
    Adam Okoye
    Anna
    Anna Philips
    anthony
    Bhargav Kowshik
    christian
    Christian Corrodi
    cturra
    cww
    David Weir
    Deshraj Yadav
    deshrajdry
    dron.rathore
    educmale
    EventHorizon
    fbraun
    feer56
    glob
    Gregg Lind
    Hal Wine
    hcondei
    Ian Kronquist
    jesse
    Joshua Smith
    kbrosnan
    kdurant35rules
    Kohei Yoshino [:kohei]
    L. Guruprasad
    Laura Thomson :laura
    lorenzo567
    margaret.leibovic
    Mark Filipak
    Matt Brubeck (:mbrubeck)
    Matt Grimes
    Matthew N. [:MattN]
    Mike Cooper
    me
    me+bugzilla
    mhoye
    Mike "Pomax" Kamermans [:pomax]
    mluna
    mozaakash
    ms.annaphilips
    Nicolas Perriault
    nicolas.barbulesco
    nigelbabu
    Padraic Harley [:thelodger]
    peterbe
    phaneendra.chiruvella
    pomax
    pradeeppaddy
    rajul.iitkgp
    Ralph Daub
    Rehan Dalal
    Ricky Rosario
    rmcguigan
    Rob
    Robert Rayborn
    Ruben Vereecken
    Schalk Neethling
    scottstensland
    senicar
    Stephen Donner
    Swarnava Sengupta
    t.schmittlauch+persona
    Tyler Downer
    thewanuki
    Trif Andrei-Alin
    vega.james
    vivek.kiran
    vivekb.balakrishnan
    viveknjadhav19
    Will Kahn-Greene

Some observations:

  1. In 2013, we resolved more bugs than we created partially because we closed a bunch of bugs related to the old Input that weren't relevant anymore.

    In 2014, we created more bugs than we closed by 10%. I think that's about what we want.

  2. 15 people had git commits. 26 people created bugs. 14 people resolved bugs. 65 people commented on bugs.

    One thing I don't have is counts for who helped translate strings which is a really important part of development. My apologies.

  3. Of all those people, only 1 is a "core developer"--that's me. Everyone else contributed their time and energies towards making Input better. I really appreciate that. Thank you!

That's the stats!

Accomplishments

Things we did in 2014:

Site health dashboard: I wrote a site health dashboard that helps me understand how the site is performing before and after deployments as well as after releases and other events.

Client side smoke tests: I wrote smoke tests for the client side. I based it on the defunct input-tests code that QA was maintaining up until we rewrote Input. The smoke tests have been invaluable for reducing/eliminating data-loss bugs.

Vagrant: I took some inspiration from Erik Rose and DXR and wrote a Vagrant provisioning shell script. This includes a docs overhaul as well.

Automated translation system (human and machine): I wrote an automated translation system. It's generalized so that it isn't model/field specific. It's also generalized so that we can add plugins for other translation systems. It's currently got plugins for Dennis, Gengo machine translation and Gengo human translation. We're machine translating most incoming feedback. We're human translating Firefox OS feedback. This was a HUGE project, but it's been immensely valuable.

Better query syntax: We were upgraded to Elasticsearch 0.90.10. I switched the query syntax for the dashboard search field to use Elasticsearch simple_query_string. That allows users to express search queries they weren't previously able to express.

utm_source and utm_campaign handling: I finished the support for handling utm_source and utm_campaign querystring parameters. This allows us to differentiate between organic feedback and non-organic feedback.

More like this: I added a "more like this" section to the response view. This makes it possible for UA analyzers to look at a response and see other responses that are similar.

Dashboards for everyone: We wrote an API and some compelling examples of dashboards you can build using the API. It's being used in a few places now. We'll grow it going forward as needs arise. I'm pretty psyched about this since it makes it possible for people with needs to help themselves and not have to wait for me to get around to their work.

Dashboards for everyone project plan.

Vagrant: We took the work I did last quarter and improved upon it, rewrote the docs and have a decent Vagrant setup now. L. Guruprasad improved on this and the documentation and setting up a Vagrant-based vm for Input development is much easier.

Reduce contributor pain project plan.

Abuse detection: Ian spent his internship working on an abuse classifier so that we can more proactively detect and prevent abusive feedback from littering Input. We gathered some interesting data and the next step is probably to change the approach we used and apply some more complex ML things to the problem. The key here is that we want to detect abuse with confidence and not accidentally catch swaths of non-abuse. Input feedback has some peculiar properties that make this difficult.

Reduce the abuse project plan.

Loop support: Loop is now using Input for user sentiment feedback.

Heartbeat support: User Advocacy is working on a project to give us a better baseline for user sentiment. This project was titled Heartbeat, but I'm not sure whether that'll change or not. Regardless, we added support for the initial prototype.

Heartbeat v1 project plan.

Data retention policy: We've been talking about a data retention policy for some time. We decided on one, finalized it and codified it in code.

Shed the last vestiges of Playdoh and funfactory: We shed the last bits of Playdoh and funfactory. Input uses the same protections and security decisions those two projects enforced, but without being tied to some of the infrastructure decisions. This made it easier to switch to peep-based requirements management.

Switched to FactoryBoy and overhauled tests: Tests run pretty fast in Fjord now. We switched to FactoryBoy, so writing model-based tests is a lot easier than the stuff we had before.

Python 2.7: Input is now running on Python 2.7. Thank you, Jake!

Remote troubleshooting data capture: The generic feedback form which is hosted on Input now has a section allowing users to opt-in to sending data about their browser along with their feedback. This data is crucial to helping us suss out problems with video playback, graphics cards/drivers and malicious addons.

This code is still "alpha". We'll be finishing it up in 2015q1.

Remote troubleshooting data capture project plan.

Heartbeat v1 and v2: People leave feedback on Input primarily when they're frustrated with something. Because of this, the sentiment numbers we get on Input tilt heavily negative and only represent people who are frustrated and were able to find the feedback form. Heartbeat will give us sentiment data that's more representative of our entire user base.

As a stop-gap to get the project going, Input is the backend collecting all the Heartbeat data. We rewrote the Heartbeat-related code for Heartbeat v2 in 2014q4.

Heartbeat v2 project plan.

Feedback form overhaul: We rewrote the feedback form to clean up the text, reduce confusion about what data is made public and what data is kept private, reduce the number of steps to leave feedback and improve the form for both desktop and mobile devices.

Feedback form overhaul project plan.

We also fixed the form so it supports multiple products because we're collecting feedback for multiple products on Input now.

Multiple products project plan.

Thoughts

Looking at the stats above, it's pretty clear that this is predominantly a one-person project. Ricky, Mike and Rehan do all my code review. Without them, things would be a lot worse.

Having said that, the "baby-factor" stinks on this project. That's something I'm going to work on over 2015:

  • improving the documentation and making sure all the important things are covered or easily figured out
  • improving the project planning and making sure most/all project planning is done in public or at least has a publicly available record
  • reducing the complexity of the application, reducing dependencies and making the parts less tangled
  • reducing the number of things that make Input a "special snowflake" in regards to other sites my peers maintain

Want to be a part of the team?

Input is a great little Django application that collects feedback for Mozilla products. It's an important part of the Mozilla product ecosystem. Working on it helps millions of people. If you're interested in being a part of the team that develops it, here are some helpful links:

Input: 2014q4 quarter in review

2014q4 was interesting. Between the faux all-hands, holidays and production freezes, it was essentially a two-month quarter. I literally raced through the quarter getting things done as quickly as I could do them opting for the minimal viable implementation wherever possible. That's not a great thing, but so it goes.

Things to know:

Bugzilla and git stats

I rewrote my script and the data is richer now.

Quarter 2014q4 (2014-10-01 -> 2014-12-31)
=========================================


Bugzilla
========

Bugs created: 95
Creators:

         Will Kahn-Greene [:willkg] : 85
                                 me : 1
     Nicolas Perriault (:NiKo`) — n : 1
                David Weir (satdav) : 1
                       Mark Filipak : 1
                           vivek :) : 1
               Laura Thomson :laura : 1
      Swarnava Sengupta (:Swarnava) : 1
                             [:Cww] : 1
                         deshrajdry : 1
              phaneendra.chiruvella : 1

Bugs resolved: 89

                            WONTFIX : 13
                         WORKSFORME : 4
                          DUPLICATE : 5
                              FIXED : 67

                         Tracebacks : 1
                           Research : 4
                            Tracker : 7

Research bugs: 4

    788597: [research] Should we use a stacked graph on the dashboard?
    792976: [research] swap jingo-minify for django-compressor and
        django-jingo-offline-compressor
    889370: [research] morelikethis for feedback responses
    1048459: [research] check if updating from a tarball in a virtual
        environment with peep removes files

Tracker bugs: 7

    964260: [tracker] overhaul mobile and desktop feedback forms
    985645: [tracker] Upgrade to Django 1.6
    988612: [tracker] product dashboards
    1040773: [tracker] dashboards for everyone
    1042669: [tracker] reduce contributor pain
    1052459: [tracker] heartbeat v1
    1081412: [tracker] add tests for fjord_utils.js functions

Resolvers: 7

         Will Kahn-Greene [:willkg] : 75
                             aokoye : 7
                          lgp171188 : 4
                                cww : 1
                    bhargav.kowshik : 1
      Rehan Dalal [:rehan, :rdalal] : 1

Commenters: 27

                             willkg : 420
                             aokoye : 7
                            mcooper : 6
                            padraic : 6
                MarkFilipak.mozilla : 5
                              hwine : 4
                             rdalal : 4
                     stephen.donner : 4
                          nigelbabu : 3
                         deshrajdry : 3
                    bhargav.kowshik : 3
                         david.weir : 2
                            mgrimes : 2
                       dron.rathore : 1
                         lorenzo567 : 1
                            senicar : 1
                     viveknjadhav19 : 1
                         nperriault : 1
                                cww : 1
                   swarnavasengupta : 1
                          mozaakash : 1
                        me+bugzilla : 1
              phaneendra.chiruvella : 1
                                 me : 1
                             fbraun : 1
                vivekb.balakrishnan : 1
                           rrosario : 1

git
===

Total commits: 239

      Will Kahn-Greene :   191  (+27879, -10550, files 822)
            Adam Okoye :    24  (+255, -75, files 70)
         L. Guruprasad :    17  (+390, -42, files 27)
       Bhargav Kowshik :     3  (+114, -2, files 11)
            Gregg Lind :     2  (+18, -16, files 6)
         Deshraj Yadav :     2  (+2, -2, files 2)

Total lines added: 28658
Total lines deleted: 10687
Total files changed: 938

Everyone
========

    Adam Okoye
    Bhargav Kowshik
    cww
    David Weir
    Deshraj Yadav
    deshrajdry
    dron.rathore
    fbraun
    Gregg Lind
    hwine
    L. Guruprasad
    Laura Thomson :laura
    lorenzo567
    Mark Filipak
    mcooper
    me
    me+bugzilla
    mgrimes
    mozaakash
    Nicolas Perriault (:NiKo`) — needinfo me if you need my attention
    nigelbabu
    nperriault
    padraic
    phaneendra.chiruvella
    Rehan Dalal
    rrosario
    senicar
    stephen.donner
    Swarnava Sengupta
    vivekb.balakrishnan
    viveknjadhav19
    Will Kahn-Greene

Code line counts:

2014q1: April 1st, 2014:        15195 total  6953 Python
2014q2: July 1st, 2014:         20456 total  9247 Python
2014q3: October 7th. 2014:      23466 total  11614 Python
2014q4: December 31st, 2014:    30158 total  13615 Python

Nothing wildly interesting there other than noting that the codebase for Input continues to grow.

Contributor stats

Adam Okoye started as an intern through OPW on December 9th. He's contributed to Input in the past through the Ascend project. Over the course of the OPW internship, he'll be working on Input bugs and the Thank you page project.

L. Guruprasad spent a lot of time working on pre-commit linters, Vagrant provisioning and generally improving the experience contributors will have.

We had a few commits from other people, too.

Thank you everyone who contributed!

Accomplishments

Python 2.7: Input is now running on Python 2.7. Thank you, Jake!

Remote troubleshooting data capture: The generic feedback form which is hosted on Input now has a section allowing users to opt-in to sending data about their browser along with their feedback. This data is crucial to helping us suss out problems with video playback, graphics cards/drivers and malicious addons.

This code is still "alpha". We'll be finishing it up in 2015q1.

Remote troubleshooting data capture project plan.

Heartbeat v1 and v2: People leave feedback on Input primarily when they're frustrated with something. Because of this, the sentiment numbers we get on Input tilt heavily negative and only represent people who are frustrated and were able to find the feedback form. Heartbeat will give us sentiment data that's more representative of our entire user base.

As a stop-gap to get the project going, Input is the backend collecting all the Heartbeat data. We rewrote the Heartbeat-related code for Heartbeat v2 in 2014q4.

Heartbeat v2 project plan.

Feedback form overhaul: We rewrote the feedback form to clean up the text, reduce confusion about what data is made public and what data is kept private, reduce the number of steps to leave feedback and improve the form for both desktop and mobile devices.

Feedback form overhaul project plan.

We also fixed the form so it supports multiple products because we're collecting feedback for multiple products on Input now.

Multiple products project plan.

Summary

2014q4 was tough because of the limited time, but it was a good quarter and we got a lot done.

The faux all-hands involved a bunch of discussions related to Input development. 2015q1 is going to be busy busy busy.

Fri, 19 Dec 2014

Input status: December 18th, 2014

Preface

It's been 3 months since the last status report. Crimey! That's not great, but it's even worse because it makes this report crazy long.

First off, lots of great work done by Adam Okoye, L. Guruprasad, Bhargav Kowshik, and Deshraj Yadav! w00t!

Second, we did a ton of stuff and broke 1,000 commits! w00t!

Third, I've promised to do more frequent status reports. I'll go back to one every two weeks.

Onward!

Development

High-level summary:

  • Lots of code quality work.
  • Updated ElasticUtils to 0.10.1 so we can upgrade our Elasticsearch cluster.
  • Heartbeat v2.
  • Overhauled the generic feedback form.
  • remote-troubleshooting data capture.
  • contribute.json file.
  • Upgrade to Django 1.6.
  • Upgrade to Python 2.7!!!!!
  • Improved and added to pre-commit and commit-msg linters.

Landed and deployed:

  • ce95161 Clarify source and campaign parameters in API
  • 286869e bug 788281 Add update_product_details to deploy
  • bbedc86 bug 788281 Implement basic events API
  • 1c0ff9f bug 1071567 Update ElasticUtils to 0.10.1
  • 7fd52cd bug 1072575 Rework smart_timedelta
  • ce80c56 bug 1074276 Remove abuse classification prototype
  • 7540394 bug 1075563 Fix missing flake8 issue
  • 11e4855 bug 1025925 Change file names (Adam Okoye)
  • 23af92a bug 1025925 Change all instances of fjord.analytics.tools to fjord.analytics.utils (Adam Okoye)
  • ae28c60 bug 1025925 Change instances of of util relating to fjord/base/util.py (Adam Okoye)
  • bc77280 Add Adam Okoye to CONTRIBUTORS
  • 545dc52 bug 1025925 Change test module file names
  • fc24371 bug 1041703 Drop prodchan column
  • 9097b8f bug 1079376 Add error-email -> response admin view
  • d3cfdfe bug 1066618 Tweak Gengo account balance warning
  • a49f1eb bug 1020303 Add rating column
  • 55fede0 bug 1061798 Reset page number Resets page number when filter checkbox is checked (Adam Okoye)
  • 1d4fd00 bug 854479 Fix ui-lightness 404 problems
  • a9bf3b1 bug 940361 Change size on facet calls
  • c2b2c2b bug 1081413 Move url validation code into fjord_utils.js Rewrote url validation code that was in generic_feedback.js and added it to fjord_utils.js (Adam Okoye)
  • 4181b5e bug 1081413 Change code for url validation (Adam Okoye)
  • 2cd62ad bug 1081413 Add test for url validation (Adam Okoye)
  • f72652a bug 1081413 Correct operator in test_fjord_utils.js (aokoye)
  • c9b83df bug 1081997 Fix unicode in smoketest
  • cba9a2d bug 1086643 bug 1086650 Redo infrastructure for product picker version
  • e8a9cc7 bug 1084387 Add on_picker field to Product
  • 2af4fca bug 1084387 Add on_picker to management forms
  • 1ced64a bug 1081411 Create format test (Adam Okoye)
  • 00f8a72 Add template for mentored bugs
  • e95d0f1 Cosmetic: Move footnote
  • d0cb705 Tweak triaging docs
  • d5b35a2 bug 1080816 Add A/B for ditching chart
  • fa1a47f Add notes about running tests with additional warnings
  • ddde83c Fix mimetype -> content_type and int division issue
  • 2edb3b3 bug 1089650 Add a contribute.json file (Bhargav Kowshik)
  • d341977 bug 1089650 Add test to verify that the JSON is valid (Bhargav Kowshik)
  • dcb9380 Add Bhargav Kowshik to CONTRIBUTORS
  • 7442513 Fix throttle test
  • f27e31c bug 1072285 Update Django, django-nose and django-cache-machine
  • dd74a3c bug 1072285 Update django-adminplus
  • ececdf7 bug 1072285 Update requirements.txt file
  • 6669479 bug 1093341 Tweak Gengo account balance warning
  • f233aab bug 1094197 Fix JSONObjectField default default
  • 11193d7 Tweak chart display
  • 9d311ca Make journal.Record not derive from ModelBase
  • f778c9d Remove all Heartbeat v1 stuff
  • e5f6f4d Switch test__utils.py to test_utils.py
  • cab7050 bug 1092296 Implement heartbeat v2 data model
  • 5480c42 bug 1097352 Response view is viewable by all
  • 46b5897 bug 1077423 Overhaul generic feedback form dev
  • da31b47 bug 1077423 Update smoke tests for generic feedback form dev
  • e84094b Fix l10n email template
  • d6c8ea9 Remove gettext calls for product dashboards
  • e1a0f74 bug 1098486 Remove under construction page
  • 032a660 Fix l10n_status.py script history table
  • 19cec37 Fix JSONObjectField
  • 430c462 Improve display_history for l10n_status
  • d6c18c6 Windows NT 6.4 -> Windows 10
  • 73a4225 bug 1097847 Update django-grappelli to 2.5.6
  • 4f3b9c7 bug 1097847 Fix custom views in admin
  • 3218ea3 Fix JSONObjectfield.value_to_string
  • 67c6bf9 Fix RecordManager.log regarding instances
  • a5e8610 bug 1092299 Implement Heartbeat v2 API
  • 17226db bug 1092300 Add Heartbeat v2 debugging views
  • 11681c4 Rework env view to show python and django version
  • 9153802 bug 1087387 Add feedback_response.browser_platform
  • f5d8b56 bug 1087387 bug 1096541 Clean up feedback view code
  • c9c7a81 bug 1087391 Fix POST API user-agent inference code
  • 4e93fc7 bug 1103024 Gengo kill switch
  • de9d1c7 Capture the user-agent
  • 4796e4e bug 1096541 Backfill browser_platform for Firefox dev
  • f5fe5cf bug 1103141 Add experiment_version to HB db and api
  • 98c40f6 bug 1103141 Add experiment_version to views
  • 0996148 bug 1103045 Create a menial hb survey view
  • 965b3ee bug 1097204 Rework product picker
  • 6907e6f bug 1097203 Add link to SUMO
  • e8f3075 bug 1093843 Increase length of product fields
  • 2c6d24b bug 1103167 Raise GET API throttle
  • d527071 bug 1093832 Move feedback url documentation
  • 6f4eb86 Abstract out python2.6 in deploy script
  • f843286 Fix compile-linted-mo.sh to take pythonbin as arg
  • 966da77 Add celery health check
  • 1422263 Add space before subject of celery health email
  • 5e07dbd [heartbeat] Add experiment1 static page placeholders
  • 615ccf1 [heartbeat] Add experiment1 static files
  • d8822df [heartbeat] Add SUMO links to sad page
  • 3ee924c [heartbeat] Add twitter thing to happy page
  • d87a815 [heartbeat] Change thank you text
  • 06e73e6 [heartbeat] Remove cruft, fix links
  • 8208a72 [heartbeat] Fix "addons"
  • 2eca74c [heartbeat] Show profile_age always because 0 is valid
  • 4c4598b bug 1099138 Fix "back to picker" url
  • b2e9445 Add note about "Commit to VCS" in l10n docs
  • 9c22705 Heartbeat: instrument email signup, feedback, NOT Twitter (Gregg Lind)
  • 340adf9 [heartbeat] Fix DOCTYPE and ispell pass
  • 486bf65 [heartbeat] Change Thank you text
  • d52c739 [heartbeat] Switch to use Input GA account
  • f07716b [heartbeat] Fix favicons
  • eff9d0b [heartbeat] Fixed page titles
  • 969c4a0 [heartbeat] Nix newsletter form for a link
  • dce6f86 [heartbeat] Reindent code to make it legible
  • dad6d82 bug 1099138 Remove [DEV] from title
  • 4204b43 fixed typo in getting_started.rst (Deshraj Yadav)
  • 7042ead bug 1107161 Fix hb answers view
  • a024758 bug 1107809 Fix Gengo language guesser
  • 808fa83 bug 1107803 Rewrite Response inference code
  • d9e8ffd bug 1108604 Tweak paging links in hb data view
  • 00e8628 bug 1108604 Add sort and display ts better in hb data view
  • 17b908a bug 1108604 Change paging to 100 in hb data view
  • 39dc943 bug 1107083 Backfill versions
  • fee0653 bug 1105512 Rip out old generic form
  • b5bb54c Update grappelli in requirements.txt file
  • f984935 bug 1104934 Add ResponseTroubleshootingInfo model
  • c2e7fd3 bug 950913 Move 'TRUNCATE_LENGTH' and make accessable to other files (Adam Okoye)
  • b6f30e1 bug 1074315 Ignore deleted files for linting in pre-commit hook (L. Guruprasad)
  • 4009a59 Get list of .py files to lint using just git diff (L. Guruprasad)
  • c81da0b bug 950913 Access TRUNCATE_LENGTH from generic_feedback template (Adam Okoye)
  • 9e3cec6 bug 1111026 Fix hb error page paging
  • b89daa6 Dennis update to master tip
  • 61e3e18 Add django-sslserver
  • 93d317b bug 1104935 Add remote.js
  • ad3a5cb bug 1104935 Add browser data section to generic form
  • cc54daf bug 1104935 Add browserdata metrics
  • 31c2f74 Add jshint to pre-commit hook (L. Guruprasad)
  • 68eae85 Pretty-print JSON blobs in hb errorlog view
  • 8588b42 bug 1111265 Restrict remote-troubleshooting to Firefox
  • b0af9f5 Fix sorby error in hb data view
  • 8f622cf bug 1087394 Add browser, browser_version, and browser_platform to response view (Adam Okoye)
  • c4b6f85 bug 1087394 Change Browser Platform label (Adam Okoye)
  • eb1d5c2 Disable expansion of $PATH in the provisioning script (L. Guruprasad)
  • 59eebda Cosmetic test changes
  • aac733b bug 1112210 Tweak remote-troubleshooting capture
  • 6f24ce7 bug 1112210 Hide browser-ask by default
  • 278095d bug 1112210 Note if we have browser data in response view
  • 869a37c bug 1087395 Add fields to CSV output (Adam Okoye)

Landed, but not deployed:

  • 4ee7fd6 Update the name of the pre-commit hook script in docs (L. Guruprasad)
  • d4c5a09 bug 1112084 create requirements/dev.txt (L. Guruprasad)
  • 4f03c48 bug 1112084 Update provisioning script to install dev requirements (L. Guruprasad)
  • 03c5710 Remove instructions for manual installation of flake8 (L. Guruprasad)
  • a36a231 bug 1108755 Add a git commit message linter (L. Guruprasad)

Current head: f0ec99d

Rough plan for the next two weeks

  1. PTO. It's been a really intense quarter (as you can see) and I need some rest. Maybe a nap. Plus we have a deploy freeze through to January, so we can't push anything out anyhow. I hope everyone else gets some rest, too.

That's it!

Wed, 26 Nov 2014

Input: New feedback form

Since the beginning of 2014, I've been laying the groundwork to rewrite the feedback form that we use on Input.

Today, after a lot of work, we pushed out the new form! Just in time for Firefox 34 release.

This blog post covers the circumstances of the rewrite.

Why?

In 2011, James, Mike and I rewrote Input from the ground up. In order to reduce the amount of time it took to do that rewrite, we copied a lot of the existing forms and styles including the feedback forms. At that time, there were two: one for desktop and one for mobile. In order to avoid a translation round, we kept all the original strings of the two forms. The word "Firefox" was hardcoded in the strings, but that was fine since at the time Input only collected feedback for Firefox.

In 2013, in order to reduce complexity on the site because there's only one developer (me), I merged the desktop and mobile forms into one form. In order to avoid a translation round, I continued to keep the original strings. The wording became awkward and the flow through the form wasn't very smooth. Further, the form wasn't responsive at all, so it worked ok on desktop machines, but mediocre on other viewport sizes.

2014 rolled around and it was clear Input was going to need to branch out into capturing feedback for multiple products---some of which were not Firefox. The form made this difficult.

Related, the smoketest framework I wrote in 2014 struggled with testing the form accurately. I spent some time tweaking it, but a simpler form would make smoketesting a lot easier and less flakey.

Thus over the course of 3 years, we had accumulated the following problems:

  1. The flow through the form felt awkward, instructions weren't clear and information about what data would be public and what data would be private wasn't clear.
  2. Strings had "Firefox" hardcoded and wouldn't support multiple products.
  3. The form wasn't responsive and looked/behaved poorly in a variety of situations.
  4. The form never worked in right-to-left languages and possibly had other accessibility issues.
  5. The architecture didn't let us experiment with the form---tweaking the wording, switching to a more granular gradient of sentiment, capturing other data, etc.

Further, we were seeing many instances of people putting contact information in the description field and there was a significant amount of dropoff.

I had accrued the following theories:

  1. Since the email address is on the third card, users would put their email address in the description field because they didn't know they could leave their contact information later.
  2. Having two cards would reduce the amount of drop-off and unfinished forms than three cards.
  3. Having simpler instruction text would reduce the amount of drop-off.

Anyhow, it was due for an overhaul.

So what's changed?

I've been working on the overhaul for most of 2014, but did the bulk of the work in October and November. It has the following changes:

  1. The new form is shorter and clearer text-wise and design-wise.
  2. It consists of two cards: one for capturing sentiment and one for capturing details about that sentiment.
  3. It clearly delineates data that will be public from data that will be kept private.
  4. It works with LTR and RTL languages (If that's not true, please open a bug.)
  5. It fixes some accessibility issues. (If you find any, please open a bug.)
  6. It uses responsive design, mobile first. Thus it was designed for mobile devices and then scaled to desktop-sized viewports.
  7. It's smaller in kb size and requires fewer HTTP requests.
  8. It's got a better architecture for future development.
  9. It doesn't have "Firefox" hardcoded anymore.
  10. It's simpler so the smoketests work reliably now.
The old Input feedback form.

The old Input feedback form.

The new Input feedback form.

The new Input feedback form.

Note: Showing before and after isn't particularly exciting since this is only the first card of the form in both cases.

Going forward

The old and new forms were instrumented in various ways, so we'll be able to analyze differences between the two. Particularly, we'll be able to see if the new form performs worse.

Further, I'll be checking the data to see if my theories hold true especially the one regarding why people put contact data in the description.

There are a few changes in the queue that we want to make over the course of the next 6 months. Now that the new form has landed, we can start working on those.

Even if there are problems with the new form, we're in a much better position to fix them than we were before. Progress has been made!

Take a moment---try out the form and tell us YOUR feedback

Have you ever submitted feedback? Have you ever told Mozilla what you like and don't like about Firefox?

Take a moment and fill out the feedback form and tell us how you feel about Firefox.

Thanks, etc

I've been doing web development since 1997 or so. I did a lot of frontend work back then, but I haven't done anything serious frontend-wise in the last 5 years. Thus this was a big project for me.

I had a lot of help: Ricky, Mike and Rehan from the SUMO Engineering team were invaluable reviewing code, helping me fix issues and giving me a huge corpus of examples to learn from; Matt, Gregg, Tyler, Ilana, Robert and Cheng from the User Advocacy team who spent a lot of time smoothing out the rough edges of the new form so it captures the data we need; Schalk who wrote the product picker which I later tweaked; Matej who spent time proof-reading the strings to make sure they were consistent and felt good; the QA team which wrote the code that I copied and absorbed into the current Input smoketests; and the people who translated the user interface strings (and found a bunch of issues) making it possible for people to see this form in their language.

Tue, 28 Oct 2014

Input: Removing the frontpage chart

I've been working on Input for a while now. One of the things I've actively disliked was the chart on the front page. This blog post talks about why I loathe it and then what's happening this week.

First, here's the front page dashboard as it is today:

Input front page dashboard

Input front page dashboard (October 2014)

When I started, Input gathered feedback solely on the Firefox desktop web browser. It was a one-product feedback gathering site. Because it was gathering feedback for a single product, the front page dashboard was entirely about that single product. All the feedback talked about that product. The happy/sad chart was about that product. Today, Input gathers feedback for a variety of products.

When I started, it was nice to have a general happy/sad chart on the front page because no one really looked at it and the people who did look at it understood why the chart slants so negatively. So the people who did look at it understood the heavy negative bias and could view the chart as such. Today, Input is viewed by a variety of people who have no idea how feedback on Input works or why it's so negatively biased.

When I started, Input didn't expose the data in helpful ways allowing people to build their own charts and dashboards to answer their specific questions. Thus there was a need for a dashboard to expose information from the data Input was gathering. I contend that the front page dashboard did this exceedingly poorly--what does the happy/sad lines actually mean? If it dips, what does that mean? If they spike, what does that mean? There's not enough information in the chart to make any helpful conclusions. Today, Input has an API allowing anyone to fetch data from Input in JSON format and generate their own dashboards of which there are several out there.

When I started, Input received some spam/abuse feedback, but the noise was far outweighed by the signal. Today, we get a ton of spam/abuse feedback. We still have no good way of categorizing spam/abuse as such and removing it from the system. That's something I want to work on more, but haven't had time to address. In the meantime, the front page dashboard chart has a lot of spammy noise in it. Thus the happy/sad lines aren't accurate.

Thus I argue we've far outlived the usefulness of the chart on the front page and it's time for it to go away.

So, what happens now? Bug 1080816 covers removing the front page dashboard chart. It covers some other changes to the front page, but I think I'm going to push those off until later since they're all pretty "up in the air".

If you depend on the front page dashboard chart, toss me an email. Depending on how many people depend on the front page chart and what the precise needs are, maybe we'll look into writing a better one.

Tue, 07 Oct 2014

Input: 2014q3 post-mortem

This is the 2014q3 Input post-mortem. It was a better quarter than 2014q2--that one kind of sucked. Instead 2014q2 started out well and then got kind of busy and then I was pretty overwhelmed by the end.

Things to know:

Bug and git stats

Bugzilla
========

Bugs created:        58
Bugs fixed:          54

git
===

Total commits: 168

      Will Kahn-Greene : 144  (+195460, -188019, files 658)
         Ian Kronquist : 12  (+402, -106, files 21)
         L. Guruprasad : 6  (+16, -36, files 6)
            Adam Okoye : 4  (+34, -34, files 34)
       Ruben Vereecken : 2  (+69, -29, files 12)

Total lines added: 195981
Total lines deleted: 188224
Total files changed: 731

We added a bunch of code this quarter:

  • October 7th. 2014: 23466 total, 11614 Python

Compare to previous quarters:

  • 2014q1: April 1st, 2014: 15195 total, 6953 Python
  • 2014q2: July 1st, 2014: 20456 total, 9247 Python

Nothing wildly interesting there other than noting that the codebase for Input continues to grow.

Contributor stats

Ian Kronquist was the Input intern for Summer 2014. He contributed several fixes to Input. Yay!

We spent a bunch of time making our docs and Vagrant provisioning script less buggy so as to reduce the problems new contributors have when working on Input. I talked with several people about things they're interested in working on. Plus several people did some really great work on Input.

Generally, I think Input is at a point where it's not too hard to get up and running, we've got several lists of bugs that are good ones to start with and the documentation is good-ish. I think the thing that's hampering us right now is that I'm not spending enough time and energy answering questions, managing the work and keeping things going.

Anyhow, welcome L. Guruprasad, Adam Okoye and Ruben Vereecken! Additionally, many special thanks to L. Guruprasad who fixed a lot of issues with the Vagrant provisioning scripts. That work is long and tedious, but it helps everyone.

Accomplishments

Dashboards for everyone: We wrote an API and some compelling examples of dashboards you can build using the API. It's being used in a few places now. We'll grow it going forward as needs arise. I'm pretty psyched about this since it makes it possible for people with needs to help themselves and not have to wait for me to get around to their work. Dashboards for everyone project plan.

Vagrant: We took the work I did last quarter and improved upon it, rewrote the docs and have a decent Vagrant setup now. Reduce contributor pain project plan.

Abuse detection: Ian spent his internship working on an abuse classifier so that we can more proactively detect and prevent abusive feedback from littering Input. We gathered some interesting data and the next step is probably to change the approach we used and apply some more complex ML things to the problem. The key here is that we want to detect abuse with confidence and not accidentally catch swaths of non-abuse. Input feedback has some peculiar properties that make this difficult. Reduce the abuse project plan.

Loop support: Loop is now using Input for user sentiment feedback.

Heartbeat support: User Advocacy is working on a project to give us a better baseline for user sentiment. This project was titled Heartbeat, but I'm not sure whether that'll change or not. Regardless, we added support for the initial prototype. Heartbeat project plan.

Data retention policy: We've been talking about a data retention policy for some time. We decided on one, finalized it and codified it in code.

Shed the last vestiges of Playdoh and funfactory: We shed the last bits of Playdoh and funfactory. Input uses the same protections and security decisions those two projects enforced, but without being tied to some of the infrastructure decisions. This made it easier to switch to peep-based requirements management.

Switched to FactoryBoy and overhauled tests: Tests run pretty fast in Fjord now. We switched to FactoryBoy, so writing model-based tests is a lot easier than the stuff we had before.

Summary

Better than 2014q2 and we fixed some more technical debt further making it easier to develop for and maintain Input. Still, there's lots of work to do.

Mon, 06 Oct 2014

Input: Dashboards for Everyone v1

Summary

In 2014q3, I created the Dashboards for Everyone project. This blog post covers what the project is, what's out there now, some examples of usage and the future.

Input collects sentiment data on Mozilla products. Currently, this data can be seen on the front page dashboard. This dashboard sucks for a variety of reasons amongst which is that it's a slog of data that isn't particularly informative.

Further, it's pretty clear that the greater Mozillaverse has different informational needs. Some people are interested in issues with products in specific locales. Some people are interested in today's hot topic. Some people are interested in comparing the first week after release of one version of a product with another. Etc.

Given that, we have two paths:

  1. Create dashboards that live on Input catering to all these needs.
  2. Create an API that allows people to create their own dashboards and host them wherever.

These two paths aren't mutually exclusive. However, I want to put effort into the second one first for the following reasons:

  1. It enables people to individually and collectively solve their own informational problems.
  2. It enables people to solve informational problems as problems crop up rather than wait for Input developers to have time to write up a dashboard.
  3. As people are solving their informational problems, we'll learn a lot more about what informational problems exist which will guide how we build dashboards that live on Input.

The Dashboards for Everyone project aims to create the infrastructure for enabling people to write their own dashboards using Input data.

Version 1

Over the course of the quarter we put enough of the bits in place that building your own dashboard is a viable thing now.

First, we wrote and honed an API for getting feedback data.

Then we wrote a bunch of proof-of-concept dashboards that use this data.

I threw together these two dashboards which are defined in Github gists and "hosted" on bl.ocks.org:

Ian Kronquist (Input intern for summer 2014) wrote this one:

Those are examples of fetching and manipulating the data into a chart. They're not very informative. We used them to help flesh out the API and their current purpose is as examples for using the API to do things.

However, we've built some real things that use the data and are "in production" now:

  • I use the Input API for the Response Breakdown graphs on the Input Health Dashboard. That helps me figure out whether we've just pushed out bugs preventing people from leaving feedback. That uses a d3-based charting library I've been toying with to help me flesh out possible data transform needs. It also uses a library that lets me defined the charts in HTML attributes and they get "auto-charted" without me having to write JavaScript.
  • Cheng used the Input API to build the Hello dashboard for tracking Hello feedback.
  • I then reused most of his ideas and some of his code to produce a Firefox for Android trends dashboard. That's still got some issues namely that the tokenizing isn't very good and thus there's enough noise in the words lists that it doesn't bring real trending issues into starker relief.

That's where we're at. Now I need to tell people about it so that you know these possibilities exist.

Do you have informational needs for the data on Input?

Can the API help you solve your specific needs?

Are there important things missing that you need to have implemented in order to solve your needs?

If you use the Input API to build your own dashboards and you run into problems, write up a bug.

Version 2 and future

Bugs, conversations and seeing other peoples' dashboards will inform us as to how we need to grow the API going forward. That will set the priorities for the next version.

Also, I have a few ideas I've been mulling over:

  1. Build an index of charts: If there are a sufficient number of interesting charts out there, then we should build an index of them on Input.

  2. Build tokenizing into the Input API: If you look at the JavaScript code for my Firefox for Android Trends chart, a good portion of it deals with tokenizing. If tokenizing is a common thing people are doing to build charts, then we should pull that tokenizing in-house.

  3. Build an auto-charter or chart widget library: Right now you have to do all the charting by hand. It'd be really nice to be able to throw dashboards together using chart "widgets" and possibly define the dashboard entirely in HTML. I've been toying with this with the Input Health Dashboard. I'm curious to find out if there's a need for throwing dashboards together quickly to follow certain issues (e.g. e10s, Hello, ...).

    The Metrics team produces some really great looking dashboards that use MetricsGraphics.js. Maybe we could build a shim on top of that letting people more easily use that library with Input data?

Thanks

I really want to thank Matt Grimes for coordinating d3 training, Cheng Wang for giving me some compelling code to reuse and Mike Cooper for helping me debug code whose mystery was only exceeded by its brokenness.

Thoughts, comments, etc

I'm using the fruits of this labor now, so at a bare minimum, it solved some of my problems.

Can it help you solve yours?

Fri, 12 Sep 2014

Input status: September 12th, 2014

Development

High-level summary:

  • Updated to ElasticUtils v0.10 which will allow us to upgrade our cluster to Elasticsearch 1.1. I'm working on a fix that'll let us to go to Elasticsearch 1.2, but that hasn't been released, yet.
  • Integrated the spicedham library prototype and set it up to classify abusive Input feedback. It's not working great, but that's entirely to be expected. I'm hoping to spend more time on spicedham and classification in Input in 2014q4. Ian did a great job with laying the foundation! Thank you, Ian!
  • Implemented a data retention policy and automated data purging.
  • Made some changes to the Input feedback GET and POST APIs to clarify things in the docs, fix some edge cases and make it work better for Firefox for Android and Loop.
  • Fixed the date picker in Chrome. Thank you, Ruben!

Landed and deployed:

  • c4e8e34 [bug 1055520] Update to ElasticUtils v0.10
  • e023fa4 [bug 1055520] Fix two reshape issues post EU 0.10 update
  • f9ba829 [bug 1055785] Codify data retention policy
  • 91396a8 Generalize About page text so it works for all products
  • 6fc03bf [bug 1053863] Update django to 1.5.9
  • 85709b2 [bug 1055788] Implement data purging
  • c0677a1 [bug 965796] Add a products update page
  • 121588d [bug 1057353] Update django-statsd and pystatsd
  • fe1c740 Add PII-related notes to the API fields
  • f77ecfa [bug 799562] Clarify API field documentation
  • c5eec03 [bug 1055789] Restrict front page dashboard and api to 6 months
  • 0892546 [bug 1059826] Add max_length to url field in API
  • f192f84 [bug 1057617] Fix url data validation
  • aad961d [bug 1030901] Document Input GET API
  • 2f212c5 [bug 1015788] Add flake8 linting
  • d673947 Update coding conventions
  • 27a1b6b Add "maximum" arg to GET API
  • 4f671e4 [bug 1062436] Add flags app and Flag model
  • 9d03d4b Fix flake8_lint issues
  • 0411d91 [bug 1062453] Add flagged view
  • 56f7e24 [bug 1062439] Celery task for classification
  • 7aa2930 [bug 1062455] Add spicedham to vendor (Ian Kronquist)
  • 0d90df3 We don't need spicedham under vendor/packages (Ian Kronquist)
  • a2a491d fix bug 1012965 - Date picker looks broken in chrome (Ruben Vereecken)
  • 0c42213 [bug 1063825] Integrate spicedham into fjord
  • 78a2d63 [bug 1062444] Initial training data
  • 5ca816e [bug 1020307] Prepare for adding gradient to generic form

Current head: 5ca816e

Rough plan for the next two weeks

  1. Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  2. Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)

What I need help with

  1. (django) Update to django-rest-framework 2.3.14 (bug #934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
  2. (django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug #910691) -- I have no idea what's going on here because I haven't looked into it much.

For details, see our GetInolved page:

https://wiki.mozilla.org/Webdev/GetInvolved/input.mozilla.org

If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

Additional thoughts

I've been codifying project plan details on the wiki:

https://wiki.mozilla.org/Firefox/Input

I have no idea who's going to use that information or whether it helps. If you see things that are missing, let me know. It'll help me hone the project management templates I'm using and know which information is important to keep up to date and which information I can let slide until rainy days.

That's it!

Tue, 19 Aug 2014

Input status: August 19th, 2014

Development

High-level summary:

It's been a slower two weeks than normal, but we still accomplished some interesting things:

  • L Guruprasad finished cleaning up the Getting Started guide--that work helps all future contributors. He did a really great job with it. Thank you!
  • Landed a minor rewrite to rate-limiting/throttling.
  • Redid the Elasticsearch indexing admin page.
  • Fixed some Heartbeat-related things.

Landed and deployed:

  • cf2e0e2 [bug 948954] Redo index admin
  • f917d41 Update Getting Started guide to remove submodule init (L. Guruprasad)
  • 5eb6d6d Merge pull request #329 from lgp171188/peepify_submodule_not_required_docs
  • c168a5b Update peep from v1.2 to v1.3
  • adf7361 [bug 1045623] Overhaul rate limiting and update limits
  • 7647053 Fix response view
  • f867a2d Fix rulename
  • 8f0c36e [bug 1051214] Clean up DRF rate limiting code
  • 0f0b738 [bug 987209] Add django-waffle (v0.10)
  • b52362a Make peep script executable
  • 461c503 Improvie Heartbeat API docs
  • 8f0ccd3 [bug 1052460] Add heartbeat view
  • d1604f0 [bug 1052460] Add missing template

Landed, but not deployed:

  • ed2923f [bug 1015788] Cosmetic: flake8 fixes (analytics)
  • afdfc6a [bug 1015788] Cosmetic: flake8 fixes (base)
  • 05e0a33 [bug 1015788] Cosmetic: flake8 fixes (feedback)
  • 2d9bc26 [bug 1015788] Cosmetic: flake8 fixes (heartbeat)
  • dc6e990 Add anonymize script

Current head: dc6e990

Rough plan for the next two weeks

  1. Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  2. Update Input to ElasticUtils v0.10 (bug 1055520)
  3. Land all the data retention policy work (bug 946456)
  4. Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)
  5. Product administration views (bug 965796)

Most of that is in some state of half-done, so we're going to spend the next couple of weeks focusing on finishing things.

What I need help with

  1. (django) Update to django-rest-framework 2.3.14 (bug 934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
  2. (django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug 910691) -- I have no idea what's going on here because I haven't looked into it much.
  3. (html) Fixing the date picker in Chrome (bug 1012965) -- The issue is identified. Someone just needs to do the fixing.

For details, see our GetInolved page:

https://wiki.mozilla.org/Webdev/GetInvolved/input.mozilla.org

If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

Additional thoughts

We're in the process of doing a Personally Identifiable Information audit on Input, the systems it's running on and the processes that touch and move data around. This covers things like "what data are we storing?", "where is the data stored?", "who/what has access to that data?", "does that data get copied/moved anywhere?", "who/what has access to where the data gets copied/moved to?", etc.

I think we're doing pretty well. However, during the course of the audit, we identified a few things we should be doing better. Some of them already have bugs, one of them is being worked on already and the otehrs need to be written up.

Some time this week, I'll turn that into a project and write up missing bugs.

That's about it!

Mon, 04 Aug 2014

Input status: August 4th, 2014

Summary

This is the status report for development on Input. I publish a status report to the input-dev mailing list every couple of weeks or so covering what was accomplished and by whom and also what I'm focusing on over the next couple of weeks. I sometimes ruminate on some of my concerns. I think one time I told a joke.

Work summary

  • Continued work on the development environment with awesome help from L. Guruprasad
  • Implement the bits required to support Loop
  • Add smoketest coverage for Firefox OS feedback form
  • Fix some technical debt issues

Development

Landed and deployed:

  • 6097571 [bug 1040919] Change version to only have one dot
  • dcf8f91 [bug 1040919] Fix .0.0 versions to .0
  • 4012933 Remove CEF-related things
  • 4887c49 [bug 1030905] Add product/version tests
  • 1bdd3a8 [bug 1042222] update Vagrantfile to use 14.04 LTS (L. Guruprasad)
  • c1bdfbb Add L. Guruprasad to CONTRIBUTORS
  • 3e607a5 [bug 1042560] Remove unused packages in dev VM (L. Guruprasad)
  • c0759d1 Update django-mozilla-product-details
  • bcf8ec8 Update product details
  • 58579ab [bug 987801] peep-ify requirements
  • 248b0ab [bug 987801] Exit if requirements are mismatched
  • 5db57bc [bug 987801] Redo how we figure out whether to use vendor/
  • df8400d Fix sampledata generation to use bulk create
  • fa0caac [bug 1041622] Capture querystring slop
  • 98745b5 [bug 1021155] Add basic FirefoxOS smoketest
  • 6cf9e39 [bug 1041664] Capture slop in Input API
  • ff4b711 Show context in response view
  • 976ebbf [bug 1045942] Add category to response table
  • 34622ad Add docs for category and extra context for Input API
  • d1a8dd5 Add an additional note about keys
  • fec1319 [bug 998726] Quell Django warnings
  • 288afe4 Rename fjord/manage.py to fjord/manage_utils.py
  • d40e8a9 Update dennis to v0.4.3

Landed, but not deployed:

  • 28cd90f [bug 1047681] Fix mlt call to be more resilient

Current head: 28cd90f

Over the next two weeks

  1. Keep an eye out for any Loop or Heartbeat related work--that's top priority.
  2. Work on gradient support and product picker support

What I need help with

  1. (Google chrome, JavaScript, CSS, HTML) Investigate what's wrong with the date picker in Chrome and fix it [bug 1012965]
  2. Test out the Getting Started instructions. We've had a few people go through these already, but it's definitely worth having more eyes. http://fjord.readthedocs.org/en/latest/getting_started.html

If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

That's it!