Editing Extras-testing/QA Checklist/QA Improvements

Warning: You are not logged in. Your IP address will be recorded in this page's edit history.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision Your text
Line 1: Line 1:
'''Note: Work in progress, none of the action points below are definitive.'''
'''Note: Work in progress, none of the action points below are definitive.'''
-
 
-
== Roles ==
 
-
* Maintainer - the owner of the package under testing
 
-
* Tester - Any community member
 
-
* Master/Admin - selected testing squad members.
 
== Automatic checks/Autobuilder ==
== Automatic checks/Autobuilder ==
-
* <s>Bugtracker field</s> - '''Done'''
 
-
* <s>That the description field is not empty</s> - '''Done'''
 
-
* Require description field content check only if description has changed
 
-
* Optified and dependencies are optified too
 
-
* License files included and headers have copyright/license.
 
-
== Tool for easy on device testing ==
+
* Bugtracker field - TBD
-
* Some easy way to capture a log of power usage, used files and open ports.
+
* Optified - TBD
== Package Interface ==
== Package Interface ==
-
* <s>Changelog should be displayed</s> - '''Done'''
+
 
-
* A list of application specific test cases should be displayed (if available. if not available testers should be able to create one.)
+
* Votes should be changeable.
-
* If the package is a library there should be shown packages of application using this library. So everybody can test libraries indirect on application level.
+
-
* <s>Votes should be changeable</s> - '''Done'''
+
* Each package that enters or leaves testing triggers a e-mail for the testing squad list
* Each package that enters or leaves testing triggers a e-mail for the testing squad list
-
* Link to Wiki so that details of test criteria are always accessible to new testers
 
=== Thumbs Up ===
=== Thumbs Up ===
 +
* Collapsable [[#Check_List | check list]] will appear, testers should mark the fields tested.  
* Collapsable [[#Check_List | check list]] will appear, testers should mark the fields tested.  
* Promotion should occur at +10 karma (if there's at least one completed check list ???)
* Promotion should occur at +10 karma (if there's at least one completed check list ???)
-
** or maybe it would be better to have one check per testing task expect for  "Working provided features." which should have for example 5 votes. This would be better to make sure that there is no functionality blockers than just one cheking the functionality. (see detailed description in http://talk.maemo.org/showpost.php?p=481536&postcount=55 )
 
=== Thumbs Down ===
=== Thumbs Down ===
 +
* Testers must comment on thumbs down.
* Testers must comment on thumbs down.
* Maintainers thumb down will demote the application from the testing queue.
* Maintainers thumb down will demote the application from the testing queue.
Line 35: Line 23:
=== Demotion ===
=== Demotion ===
-
* Packages can be demoted at any time by their maintainers ('''Implemented''') or by a member of the testing squad (demotions should be advertised in the testing squad list).
 
-
* When demoting a package there's an option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.
 
-
=== Speed Promotion ===
+
* Packages can be demoted at any time by his maintainers or by a member of the testing squad (demotions should be advertised in the testing squad list).
-
* Maintainer can request speed promotion through interface in well defined cases: critical bug in "Extras"-version needs urgent fix; only cosmetic changes (new translations, icons, package description,...);
+
* When demoting a package there's a option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.
-
* Changes must be easily visible/documented (run diff agains version in Extras)
+
-
* Speed promotion is done by selected members of the testing sqad. No extra requirements like "10 days" or minimum package karma. [[User:ossi1967|ossi1967]]
+
-
=== Speed Promotion (alternative) ===
+
== Check List ==
-
Alternate suggestion: given that the smallest change by a developer can cause a serious regression, and there's no way round that - is that once a package reaches the "tipping point" (say, 5 days and 8 votes) another version of the package is let into Extras-testing.
+
-
However, people can still rate the earlier version (although not install it) and get it through; whilst the newer version starts its QA process. Obviously if there's a bug, the developer can demote their earlier version and prevent it going through to Extras. --[[User:jaffa|Jaffa]] 13:12, 12 May 2010 (UTC)
 
-
 
-
== Check List ==
 
<pre>
<pre>
1. [ ] Licensing ok.
1. [ ] Licensing ok.
-
2. [ ] Description field ok.
+
2. [ ] No illegal/dubious content.
-
--
+
3. [ ] Working provided features.
-
3. [ ] Announced features available.
+
4. [ ] No missing announced features.
-
4. [ ] Working provided features.
+
5. [ ] Optification ok.
-
--
+
6. [ ] No performance problems.
-
5. [ ] No performance problems.
+
7. [ ] No power management issues.
-
6. [ ] No power management issues.
+
8. [ ] No known security risks.
-
7. [ ] No known security risks.
+
-
8. [ ] No illegal/dubious content.
+
Additional comments:
Additional comments:
Line 72: Line 50:
<pre>
<pre>
1. [x] Licensing ok.
1. [x] Licensing ok.
-
2. [x] Description field ok.
+
2. [ ] No illegal/dubious content.
-
FAIL: the package doesn't have a description.
+
3. [x] Working provided features.
-
--
+
-
3. [ ] Announced features available.
+
-
4. [x] Working provided features.
+
  FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123).  
  FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123).  
  FAIL: When exporting file the program crashes (see bug: http://url/456)
  FAIL: When exporting file the program crashes (see bug: http://url/456)
-
--
+
4. [ ] No missing announced features.
-
5. [ ] No performance problems.
+
5. [x] Optification ok.
-
6. [ ] No power management issues.
+
FAIL: the package uses 1512kb from root.
-
7. [ ] No known security risks.
+
6. [ ] No performance problems.
-
8. [ ] No illegal/dubious content.
+
7. [ ] No power management issues.
 +
8. [ ] No known security risks.
Additional comments:  
Additional comments:  
Line 90: Line 66:
== Testing Squad ==
== Testing Squad ==
-
* Can demote packages when there are known blockers.
+
 
 +
* Can demote packages when there's blockers.
* Can promote packages when they are stuck in the testing queue for a while without any known blocker.
* Can promote packages when they are stuck in the testing queue for a while without any known blocker.
-
* Can promote packages in speed promotion process [[User:ossi1967|ossi1967]]
 
=== Testing Squad mailing list ===
=== Testing Squad mailing list ===
-
* Public mailing list where are discussed any situation/issue concerning the applications in the maemo.org repositories.
+
 
-
* Receives an automatic notification for each package that enters testing, is demoted or is promoted.
+
* Public mailing list where are discussed concerns/improvements to the QA process.
-
[[Category:Quality Assurance]]
+
* Receives a notification for each packages that enters testing, is demoted or is promoted.

Learn more about Contributing to the wiki.


Please note that all contributions to maemo.org wiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see maemo.org wiki:Copyrights for details). Do not submit copyrighted work without permission!


Cancel | Editing help (opens in new window)
Retrieved from "https://wiki.maemo.org/Extras-testing/QA_Checklist/QA_Improvements"