Talk:Task:Getting Nokia involved in bugs.maemo.org

Contents

[edit] Best practices for everybody

There is one assumption that not always becomes true: Bugzilla users know how to deal with bug tracking. This might apply both to users (poor reports, wrong severity/priority, reopening of wontfixes/invalids just because, bad behavior...) and maintainers (harsh resolving, lack of explanations, little knowledge of the possibilities of the tool, bad behavior too...). We see this in most free software projects and bugs.maemo.org is not exempt of this risk.

In fact, having @nokia guys around makes things a bit more difficult sometimes: certain users take the chance to establish a personal war against "Nokia", others might simply ignore the complexity behind a simple bug and a simple Nokia developer commenting on that bug. On the other hand, Nokia employees are busy enough with an internal bug tracker that counts in their internal process, and communicating to the outside makes things more complex: because of wearing the Nokia shirt you can be quoted anywhere and because some (actually many) bug resolutions are related to confidential information.

Also, Maemo gets many newcomers not familiar with free software development practices, both in the user and developer side, which makes the own concept of a public bug tracker strange for some.

For all these reasons it would be good to have a concise 'for dummies' guide for users and maintainers that everybody could refer to when learning how to get involved.--qgil 11:54, 24 July 2008 (UTC)

[edit] Success (and really wrong) stories

Putting the pressure on single developers doesn't help actually. The reality of the average Maemo SW developer is: busy with Nokia internal processes (including automated and human testing) and reporting to a project manager (directly) and a release manager (indirectly) that most probably don't follow the activity in bugs.maemo.org. Considering this setting, the public Bugzilla brings more work in exchange of an unclear outcome. This is why good triaging by the maemo.org bugmasters and the Bugsquad is good, because they help triage what really matters.

However, the real step is to prove that good usage of bugs.maemo.org results in less work and better quality, not more work resulting in worse performance/quality. I'm not saying that this is *the* position of Nokia. What I'm saying is that Nokia processes have been evolving during years and just recently they had to consider something like a public Bugzilla. Coming from an open source background the importance of public bug trackers is clear, and doesn't need further explanation. But on the other hand, device programs and non-free software development programs can also show good success rates with test and error management process done internally, getting user feedback from surveys, controlled tests, etc.. The Maemo SW is in between, affected by device & closed source software development and also by pure open source development.

There are many things that are being changed internally to adapt to this situation, but there is one task where the Maemo community can help: bring the success stories happening in Maemo and elsewhere with a similar setting (e.g. public bugtrackers efficiently handled by companies shipping software preinstalled in devices). Equally useful would be clear stories of things that went wrong or could have gone clearly better if Nokia would have made a better use of bugs.maemo.org.

Being specific and objective, showing statistics and trends, helps a lot convincing not one or two developers but whole teams and management structures.--qgil 12:08, 24 July 2008 (UTC)

[edit] Status of projects

Something that would help in the Fremantle timeline is to define what products (aka which teams) are doing better and worse. Now there is an overall complaint, which is unfair to actually those individuals and teams that are doing a good work in bugs.maemo.org. The general complaint is also more difficult for us to chase reasons why things are how they are. If instead there would be a wiki page or something showing "red - yellow - green - star" status of each product matching a team, then we all would be able to focus on the right spots.--qgil 07:56, 19 November 2008 (UTC)

I really hope the people who are putting forth that extra effort realize the complaints don't include them (even if they're not specifically excluded). I certainly appreciate the hard work those people put in, and I know most everybody else does, too. :) —GeneralAntilles 08:25, 19 November 2008 (UTC)

[edit] What metrics?

Should be pretty easy to implement, but what are our metrics? Initial response time? Total fix turnaround time? Probably too complicated (the more complicated it is, the easier it is to be unfair) A simple two-part response rate and fix rate combination? Should the metrics be based entirely on hard-data, or would it be appropriate to factor in perception from, say, the Bugsquad ("Yes, this team's numbers look good, but the responses were largely unhelpful and slow to arrive, and the fixes sub-par."). If the goal is simply to pin-point teams that need to improve their participation, than a perception-based grade with some statistical number crunching sounds best. —GeneralAntilles 08:25, 19 November 2008 (UTC)
Perception is more than enough, I think. Red = too bad. Yellow = Well, ok. Green = Good!. Star = You're amazing, thanks!. The guys are getting plenty of robotic metrics in their jobs and at the end the objective is "perceived quality". If you want to add numbers I'm sure they will be useful, but I think those 4 states (or even 2: good/bad) are good enough to chase the specific problems of the affected teams and congratulate publicly those doing a good work. --qgil 11:56, 20 November 2008 (UTC)

[edit] How often and where?

How often should the report be updated? Weekly? Monthly? Quarterly? Obviously a wiki page should be maintained for reference, but where should the reports go to? maemo-developers? A blog for this purpose?
I'd say monthly. I was tempted to say quarterly since all teams might have some months specially stressful, but the longer the period the lesser your memory is reliable and the more you need to sustain perceptions with actual numbers.--qgil 12:10, 20 November 2008 (UTC)
Can we start now? :D —GeneralAntilles 08:25, 19 November 2008 (UTC)
OK, here's a really rough draft outline Task:Bugzilla report card. —GeneralAntilles 09:07, 19 November 2008 (UTC)
Hum, I'd start with something more minimalistic: x=months / y=products / content of cell: bad (red background) / meh (yellow) / good (green) / star .--qgil 12:10, 20 November 2008 (UTC)

[edit] Improvements in the weekly reports

The weekly Bug Jar is just fantastic. It is also a very visual proof about the community capacity to get organized and help. Even if unrelated, it came more or less at the same time than the maemo.org bugmasters joined the team. Both things have cause a very good impact in the internal bug management people and process. Some ideas for improvement in the weekly reports:

  • Website bugs out of the Jars. The Website bugs are reasonably well tracked and managed mostly at a community/public level. There are many and they add too much data to the Jars. Data like that is almost pure noise for the average Nokia guy working in Maemo Software.
  • Visualizing the progress: now we get a picture every week, but we rely on our own memory to judge whether there is progress or not, small progress or big. Ideally there would be statistics and graphs showing the progress. A very good discovery and proofpoint would be to find out that before bugmasters and jars the lines were pointing trend A, and after that the lines point to much cooler trend B. This might sound stupid but those lines might convince more people inside and outside Nokia, more users and developers and managers, than the most rationalized arguments.
  • Bug Jars in this wiki? Forget about this if it's too much extra work, but documenting the reports in the wiki would be useful. It could even be the default interface and then send updates to maemo.org lists and ITt.

--qgil 07:57, 25 July 2008 (UTC)

Karsten is also working on porting some weekly summary/overview to Maemo Bugzilla, see [1]. This is on the map for 100Days/Sprint4. Stephen is CC'ed and I naively hope that some code (or at least ideas) can be shared. --andre 18:08, 12 August 2008 (UTC)

[edit] Better organization of products and components

If you look inside components like System Software, System Management or Multimedia you will see that there is a bit of inconsistency aka mess in the components. This is not only poor usability for the bug reporters. It also makes life harder for Nokia developers to track where are the reports that affect to their work. Specially if you consider that developers work on teams, and many times teams are mapped to certain products and components. It is easier for Team A to know that they have to follow Product A and forget safely about the rest. This is now clear for e.g. Connectivity or Development Platform, which happen to be products that are reasonably well maintained. System Software would be the opposite case.--qgil 10:09, 30 July 2008 (UTC)

I'm going to work on improving this in 100Days/Sprint4, see [2]. --andre 18:14, 12 August 2008 (UTC)
The organization is mostly my fault*, I didn't particularly like the organization used in the internal bug tracker and was trying to aim for something which more closely mapped to what users would see when they used their device. In the n800, that's basically: 1. Browser, 2. Communications, and 3. Desktop. After those come distinct blobs: A. Connectivity, B. Multimedia and C. Games. Then comes a section which is harder to divide. I chose to map things that appeared in/near control panel as "System management" and everything else into "System software". Note that mozilla.org is currently experiencing a similar growing pain, the current bugzilla.mozilla.org organization is built around functionality that only the engineers who work on an area understand. This means that if you're an end user, or even an engineer from a different area you have a terrible time trying to find the right place to file a bug. * Except for Internet Tablet Video Converter which someone added on behalf of a different Nokia group. --timeless 18:51, 3 September 2008 (UTC)
Hi, I have been thinking and my conclusion is that we need a structure mapping platform areas and components down to the packages level. This would serve a much better purpose for generating reports and finding out the bugs in a way the Maemo teams at Nokia can process and handle better.--qgil 06:50, 22 October 2008 (UTC)
Package level? That'd be (for example) having components for dsme, bme and mce? Or breaking down "Browser" into microb-eal, microb-engine, tablet-browser-ui, tablet-browser-dialogs, tablet-browser-widgets, tablet-browser-controls, etc, etc? That sounds (to me, and assuming I'm actually reading "packages level" correctly) like hell for triagers and a good way to scare off reporters. —GeneralAntilles 07:01, 22 October 2008 (UTC)
Ok, ok, don't take the package level literally always. I agree they make less sense at the application level, where most end users will probably land. My main concern is not the package level but the fact that bugzilla products reflect better platform structure and (as a happy coincidence) Maemo SW team structure. For instance, now "Applications" list in fact "Rest of applications", since Browser or Media Player are elsewhere. At the same time, "Multimedia" is handling both application and platorm components. The Maemo SW team has a clear divide between applications and platform. Users probably will be happier too if they would have an easy way to access the whole application level without having to mess with more obscure platform components. By doing this we can go to deeper levels in the platform since now there is a single component for e.g. "Multimedia Framework" when actually it would be good to know whether a bug is in GStreamer, PulseAudio or some Nokia closed component (for instance, upstream developers could track better what is going on there). No need to go down to the full list of GStreamer related packages, agreed. Let me start drafting a structure... Don't look at the components so much, I have concentrated on the products in this first round. --qgil 06:08, 23 October 2008 (UTC)
Overall I like the reorg. Renaming System management to Settings & Maintenance (which is mostly what you did) makes a lot of sense, I'm sorry I hadn't come up w/ that name in my reorg. X-Discontinued is an interesting hack. I'd sooner move it out of Official Applications. At mozilla.org, we have a Graveyard classification for this. That python is listed under discontinued is odd.... splitting some of System software into Desktop platform seems reasonable. DUN=(bt) Dial up networking; ICD=Internet Connectivity Daemon. Splitting Core out may make the Maemo Software Core team happy, but I'm not sure that Internationalization belongs in Desktop platform. I guess from an ownership perspective it does. From someone who usually doesn't pay attention to how Maemo Software (this is an organization in Nokia) works it doesn't. --timeless 19:14, 10 November 2008 (UTC)

[edit] Draft structure (implemented on Dec 06th, 2008)

The 3 levels respond to Classification, Product and Component. What is today "Maemo Software" would be split in Official Applications and Official Platform.

  • Official Applications (respond to a structure evident from a user point of view)
    • Browser
      • Bookmarks
    • Chat & Internet Call (was: Communication)
      • Chat
      • General (was: Communication/General (9 bugs))
      • Internet Call (was: Communication/Internet Call)
      • Presence (was: Communication/Presence)
      • SIP (was: Communication/SIP)
      • XMPP (was: Communication/XMPP)
    • Contacts (was: Communication/Contacts)
    • E-mail (was: Communication/Email) (Note: Webmail notifier: Under Home applets)
    • Games
      • Blocks
      • Chess
      • Mahjong
      • Marbles
    • Home applets
      • Battery
      • Clock
      • Connection
      • Contacts
      • Display
      • Getting started
      • Internet radio
      • Internet search
      • RSS feed reader
      • Webmail Notifier (was: Applications/Webmail Notifier, 8 bugs)
    • Images & Camera (fits better to also have Cam bugs somewhere)
      • Camera (was: Multimedia/Camera)
      • Image viewer (was: Applications/Image viewer)
    • Map
    • Media Player (was: Multimedia/Media Player)
    • RSS feed reader (was: Applications/RSS feed reader)
    • Settings & Maintenance
      • Application manager (was: System management/Application manager)
      • Backup/Restore (was: System management/Backup/Restore)
      • Connection manager UI (was: System management/Connection manager)
      • Control panel (was: System management/Control panel)
      • Startup wizard (was: System management/Startup Wizard)
      • Software updater
    • Utilities
      • File manager (was: System management/File manager)
      • Calculator
      • Clock
      • Help
      • Internet Tablet Video Converter (TODO: Not yet moved due to subcomponents)
      • Notes
      • PDF reader
      • Sketch
      • Search (TODO: What is this and who added it here? Not yet created)
      • X Terminal (was: System software/X Terminal)
  • Official Platform (respond to an architectural structure)
    • Desktop platform
      • cairo
      • clutter
      • File System UI (was: System software/File System UI)
      • Finger keyboard (was: System software/Finger keyboard)
      • Fonts (was: System software/Fonts)
      • gconf (was: System software/gconf)
      • general (was: System software/general) (TODO: Clean up and split up into Desktop platform and system software)
      • glib (was: System software/glib)
      • gnome-vfs (was: System software/gnome-vfs)
      • gtk (was: System software/gtk)
      • gvfs
      • hildon-libs (was: System software/hildon-libs)
      • hildon-thumbnail (was: System software/hildon-thumbnail)
      • Home (was: Desktop/Home)
      • Icons (was: Desktop/Icons)
      • Input method framework (was: System software/Input method framework)
      • Internationalization
      • libosso (was: System software/libosso)
      • pango
      • sapwood (was: System software/sapwood)
      • startup-shutdown (was: System software/startup-shutdown)
      • Task navigator (was: Desktop/Task navigator)
      • Themes (was: Desktop/Themes and System software/hildon-theme) TODO: Merge with System software/hildon-theme
      • Virtual keyboard (was: System software/Virtual keyboard)
      • window-manager (was: System software/window-manager)
    • Connectivity
      • Bluetooth
      • DUN
      • ICD
      • Networking
      • Operator Setup Wizard (was: System management/Operator Setup Wizard)
      • WiFi
      • WiMAX
    • Core
      • Busybox (was: System software/Busybox)
      • Kernel (was: System software/Kernel)
      • X Server (was: System software/X Server)
    • Data
      • Meta Tracker
      • metalayer-crawler (TODO: Move to X-Graveyard as Fremantle does NOT have initfs)
      • SQLite
    • Development platform
      • Documentation
      • general
      • installer
      • rootstrap
      • SDK
      • Tools
    • Flasher
    • Location (was: Connectivity/Location Framework)
    • Multimedia
      • DSP (TODO: Create and clean up; Though DSP gateway = Kernel) SETNEWQA: dsp-bugs@maemo.bugs
      • Gstreamer
      • Media streamer (TODO: better in applications? --qgil 19:32, 5 November 2008 (UTC) )
      • Multimedia framework (TODO: Vague)
      • Pulseaudio
      • Real
    • Real Time Communication
      • Telepathy
    • System software
      • application-killer
      • D-Bus (was: dbus)
      • Device lock (was: Desktop/Device lock) TODO: Triage - some bugs belong here, but some are in fact hildon widget bugs)
      • dsme (was: System Software/dsme AND System Software/Watchdog, TODO: merge?)
      • File system
      • general
      • glibc
      • HAL
      • libxml
      • maemo-launcher
      • mmc-and-usb
      • OHM
      • upstart (TODO: Add this for Fremantle?)
    • Translations
    • UI Specification
    • User guide & Help content (should get rid off vague "Docu" term)
      • Help content
      • User guide
  • X-Graveyard
    • X-Discontinued
      • Audio player (770) (was: Multimedia/Audio player)
      • core-initfs (TODO: Create, move 2021, 1855, 3852, 3589 and maybe 3745. Fremantle does NOT have initfs)
      • FM Radio (N800) (was: Multimedia/FM Radio)
      • "Misdirected" product bugs
      • Browser: Opera engine (was: Browser/Opera engine (770/N800))
      • osso-email (was: Communication/osso-email)
      • python (was: System software/python)
      • Video player (770) (was: Multimedia/Video player)

Andre: Getting rid of the "Applications" product is very good. It was often confused with Desktop (e.g. mixing up Applets and Apps with the same name). But having Desktop now in Platform might also be confusing if you're used to the old structure so I changed this to "Desktop Platform". What I have already done: I've moved Developer Docu from Website to Dev Platform (because it's out of community scope, see bug 3562). Removed ancient Games/Startscreen subcomponent (only 2 bugs), Removed Desktop/PC connection (1 wrong bug), Desktop/Tableteer Info (0 bugs), Desktop/Desktop Info applet (0 bugs). Removed Communication/Accounts (3 bugs, 2 wrong). Removed System software/libcommon-error (0 bugs). Removed System management/Teach handwriting (0 bugs). Removed Web shortcut (0 bugs, development stalled since Diablo). Several additions done on Dec 06th 2008 (metalayer-crawler, HAL, OHM, Software Updater, Maps, gvfs, pango, Internationalization, meta tracker, SQLite, Pulseaudio, GStreamer, Real, etc). Removed iconf (moved into Internationalization), removed User Interaction (cleaned up and moved into other components).

[edit] Draft implementation

Original plan:

  1. Create new default QA virtual accounts for new products/components in advance (done)
  2. Define a good day/time (Sunday?)
Friday night's or holiday weekends are better timeless 17:37, 19 November 2008 (UTC)
  1. Display info message about reorg one week in advance in /template/en/default/global/banner.html.tmpl
  2. Create dummy products (X-DoNotUse1 etc) in advance with their TMs and versions to save time (as number of planned products > current number of products)
Create products with correct names but which have a group access restriction to the admin group. They won't show up to anyone else. timeless 17:37, 19 November 2008 (UTC)
Use the "shutdownhtml" param to keep everyone else out (you need a feature which we need to upstream, contact moznet#bmo for it if we haven't posted a bug+patch) timeless 17:37, 19 November 2008 (UTC)
  1. Disable mail notification
  2. Implement changes
  3. Correct some other inconsistent QA Contact settings in some bugs that I have on a list
TODO: Check whether adding a disabled virtual QA Contact to watchlist really provides receiving mail - I doubt with the current settings --Andre)
disabling an account doesn't prevent it from getting mail, you need to disable mail in editprefs, there's a cheap url to doing this, it's something like https://bugzilla/userprefs.cgi?tab=email&dosave=1 you can use impersonation to flop in and out of each user fairly quickly. timeless 17:37, 19 November 2008 (UTC)
  1. Enable mail notification
  2. Remove banner

timeless prefers to keep mail notification enabled while doing this, see discussion. So far, I disagree. :) --Andre 12:06, 11 November 2008 (UTC)

the other thing i messed up in the bmo reorg basically destroyed date state about a set of bugs, the only way i had of finding them was searching through my mail timeless 17:37, 19 November 2008 (UTC)

Retrieved from "https://wiki.maemo.org/Talk:Task:Getting_Nokia_involved_in_bugs.maemo.org"