Editing Extras-testing/QA Checklist/QA Improvements

Warning: You are not logged in. Your IP address will be recorded in this page's edit history.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.

Learn more about Contributing to the wiki.

'''Note: Work in progress, none of the action points below are definitive.'''

== Roles ==
* Maintainer - the owner of the package under testing
* Tester - Any community member
* Master - selected persons, for example with high testing karma or testing squad members.

== Automatic checks/Autobuilder ==
* Bugtracker field
* Optified and dependencies too
* License files included and headers have copyright/license.

== Tool for easy on device testing ==
* Some easy way to capture a log of power usage, used files and open ports.

== Package Interface ==
* Changelog should be displayed.
* Votes should be changeable.
* Each package that enters or leaves testing triggers a e-mail for the testing squad list
* Link to Wiki so that details of test criteria are always accessible to new testers

=== Thumbs Up ===
* Collapsable [[#Check_List | check list]] will appear, testers should mark the fields tested. 
* Promotion should occur at +10 karma (if there's at least one completed check list ???)
** or maybe it would be better to have one check per testing task expect for  "Working provided features." which should have for example 5 votes. This would be better to make sure that there is no functionality blockers than just one cheking the functionality. (see detailed description in http://talk.maemo.org/showpost.php?p=481536&postcount=55 )

=== Thumbs Down ===
* Testers must comment on thumbs down.
* Maintainers thumb down will demote the application from the testing queue.
* X (fix me) thumbs down will demote the app.

=== Demotion ===
* Packages can be demoted at any time by his maintainers or by a member of the testing squad (demotions should be advertised in the testing squad list).
* When demoting a package there's a option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.

== Check List ==
<pre>
1. [ ] Licensing ok.
2. [ ] Description field ok.
--
3. [ ] Announced features available.
4. [ ] Working provided features.
--
5. [ ] No performance problems.
6. [ ] No power management issues.
7. [ ] No known security risks.
8. [ ] No illegal/dubious content.

Additional comments:
</pre>

* Put [x] for those tests you have done, elaborate on separate row if the test is FAIL.
* Vote up if there were no FAILs. If there was even one FAIL, vote down.
* UI usability issues cannot be used as a reason for vote down.
* Always test functionality - that is, run the program and try if it works as it should.

imaginary example:
<pre>
1. [x] Licensing ok.
2. [x] Description field ok.
 FAIL: the package doesn't have a description.
--
3. [ ] Announced features available.
4. [x] Working provided features.
 FAIL: There is choice between tabs and spaces as separators but spaces are always used (see bug: http://url/123). 
 FAIL: When exporting file the program crashes (see bug: http://url/456)
--
5. [ ] No performance problems.
6. [ ] No power management issues.
7. [ ] No known security risks.
8. [ ] No illegal/dubious content.

Additional comments: 
I liked the program, even though, as is, I have to vote it down due to the bugs. I added few usability enhancements to bugzilla, see http://url/567 and http://url/678
</pre>

== Testing Squad ==
* Can demote packages when there's blockers.
* Can promote packages when they are stuck in the testing queue for a while without any known blocker.

=== Testing Squad mailing list ===
* Public mailing list where are discussed concerns/improvements to the QA process.
* Receives a notification for each packages that enters testing, is demoted or is promoted.

Please note that all contributions to maemo.org wiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see maemo.org wiki:Copyrights for details). Do not submit copyrighted work without permission!

Summary:

Cancel | Editing help (opens in new window)

Retrieved from "https://wiki.maemo.org/Extras-testing/QA_Checklist/QA_Improvements"

Personal tools

Navigation

Views

Editing Extras-testing/QA Checklist/QA Improvements

@@ Line 4: / Line 4: @@
 * Maintainer - the owner of the package under testing
 * Tester - Any community member
-* Master/Admin - selected testing squad members.
+* Master - selected persons, for example with high testing karma or testing squad members.
 == Automatic checks/Autobuilder ==
-* <s>Bugtracker field</s> - '''Done'''
+* Bugtracker field
-* <s>That the description field is not empty</s> - '''Done'''
+* Optified and dependencies too
-* Require description field content check only if description has changed
-* Optified and dependencies are optified too
 * License files included and headers have copyright/license.
@@ Line 17: / Line 15: @@
 == Package Interface ==
-* <s>Changelog should be displayed</s> - '''Done'''
+* Changelog should be displayed.
-* A list of application specific test cases should be displayed (if available. if not available testers should be able to create one.)
+* Votes should be changeable.
-* If the package is a library there should be shown packages of application using this library. So everybody can test libraries indirect on application level.
-* <s>Votes should be changeable</s> - '''Done'''
 * Each package that enters or leaves testing triggers a e-mail for the testing squad list
 * Link to Wiki so that details of test criteria are always accessible to new testers
@@ Line 35: / Line 31: @@
 === Demotion ===
-* Packages can be demoted at any time by their maintainers ('''Implemented''') or by a member of the testing squad (demotions should be advertised in the testing squad list).
+* Packages can be demoted at any time by his maintainers or by a member of the testing squad (demotions should be advertised in the testing squad list).
-* When demoting a package there's an option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.
+* When demoting a package there's a option to keep the current app karma (minor issues) or reset it (big blockers), and a text field where should be added the reason for demotion.
-=== Speed Promotion ===
-* Maintainer can request speed promotion through interface in well defined cases: critical bug in "Extras"-version needs urgent fix; only cosmetic changes (new translations, icons, package description,...);
-* Changes must be easily visible/documented (run diff agains version in Extras)
-* Speed promotion is done by selected members of the testing sqad. No extra requirements like "10 days" or minimum package karma. [[User:ossi1967|ossi1967]]
-=== Speed Promotion (alternative) ===
-Alternate suggestion: given that the smallest change by a developer can cause a serious regression, and there's no way round that - is that once a package reaches the "tipping point" (say, 5 days and 8 votes) another version of the package is let into Extras-testing.
-However, people can still rate the earlier version (although not install it) and get it through; whilst the newer version starts its QA process. Obviously if there's a bug, the developer can demote their earlier version and prevent it going through to Extras. --[[User:jaffa|Jaffa]] 13:12, 12 May 2010 (UTC)
 == Check List ==
@@ Line 90: / Line 76: @@
 == Testing Squad ==
-* Can demote packages when there are known blockers.
+* Can demote packages when there's blockers.
 * Can promote packages when they are stuck in the testing queue for a while without any known blocker.
-* Can promote packages in speed promotion process [[User:ossi1967|ossi1967]]
 === Testing Squad mailing list ===
-* Public mailing list where are discussed any situation/issue concerning the applications in the maemo.org repositories.
+* Public mailing list where are discussed concerns/improvements to the QA process.
-* Receives an automatic notification for each package that enters testing, is demoted or is promoted.
+* Receives a notification for each packages that enters testing, is demoted or is promoted.
-[[Category:Quality Assurance]]