OpenOffice.org L10n Test Collaboration Plan

v0.6 03/22/2005

1. Overview

   We want to improve th quality of l10n versions of OO.o/SO with extensive QA corporating with OpenOffice.org Native-Lang communities and sharing test resources and strategy.

   The aims of this plan are

   • cover as many local platform/language as possible

   • provide common test environment

   • share test cases between Sun and community

   • ease participation

   • ease the tracking of test coverage

   • ease the tracking of test results

   • share information who is doing what

2. TCM

   TCM is the Test Case Management system with Web interface which Sun is using internally. We'd like to utilize this tool for sharing information with community and providing collaborative test environment.

   TCM will cover the following.

   • maintain test cases (including extension)

   • assign tester (sharing information of who is doing what)

   • enter test result with common UI (by assigned tester)

   • track test result (for all of test result which all tester entered)

   • track test coverage for each language/platform

   TCM entry page.

3. Test cases

   Current set of test cases which is managed by TCM would be a start for our collaborative l10n testing. We would like to add/extend these cases so that more features we will cover in the future. Test case can be extended with TCM Edit Test Cases

4. Test case coverage table

   The following is the coverage table template.

   Platform / Test Area	Writer-1	Writer-2	Calc-1	Calc-2	Impress	Draw	Database	Overall-1	Overall-2
   Linux 1
   ...
   Linux n
   Windows 1
   ....
   Windows n
   Solaris 1
   ...
   Solaris n
   Other 1
   ...
   Other n

   This table will exist per language because which platforms (ex: Linux distributions) are major depends on language/country. Anyone will be able to see what platform/languages are tested by whom in TCM.

   Test cases are divided into reasonable amount of units (items in the first row) so that individual tester can execute it in a few days. Any tester will be assigned for this matrix basis.

   The below is the Japanese sample coverage table. ('Other' will be major OpenOffice.org platform (not supported by Sun) in that language, but Sun will not provide l10n build for that. So tester might have to prepare/find it by themselves

   ## This table should be replaced with link to TCM page which shows actual table.
   ## Low/Column will be different.

   	Writer-1	Writer-2	Calc-1	Calc-2	Impress	Draw	Database	Overall-1	Overall-2
   TurboLinux
   Fedora Core
   JDS
   Windows98(SE)
   Windows ME
   Windows 2K
   Windows XP
   Solaris 8
   Solaris 9
   Solaris 10
   FreeBSD
   MacOS X

5. Administration

   Sun will maintain TCM and assign contributors to cover test efforts. The assignments will show in the TCM assignment table which has one cell for each test unit. The table will be created for each build (one table per build). Anyone can offer to contribute for any test unit. This means more than one tester can be assigned to a test unit (one cell in the table), also the same tester can sign up for several test units. Obviously, it's more valuable to sign up first for test units which are not covered by anyone. The process of signing up for test coverage will be conducted via email using an open mailing list <tcmrequest@l10n.openoffice.org> so that anyone can see assignment process and participate efficiently.

6. Execute test case

   Most of the execution of the test cases will be done manually. (StarOffice QA team in Hamburg is doing automated testing extensively. We l10n testing will focus on manual/visual testing for now)

   Entering test result can be done with TCM common UI.

   When a test case item fails, a bug will be filed (test case must have IssueZilla ID). Before filing a bug check/search IssueZilla first, to verify if the problem has alreasy been reported.

7. Test cycle (l10n build)

   Sun will provide Linux/Windows/Solaris(SPARC/x86) OpenOffice.org l10n builds for community testing at each of testing cycle. But stability of milestone build is hard to predict. So here is rough schedule(once per month from Nov/2004 – Mar/2005). Sun will announce the l10n build available date at one week before its actual downloadable binaries will be ready. Test period would be 2 weeks.

   	Previous	Current	Status
   Pre-Beta testing	Nov/2004 (m58)	m65: 12/16/2004 - 12/29/2004	Done
   Beta testing	Dec/2004	m79: 02/25/2005 - 03/17/2005	Done
   pre-RC testing	Jan/2005	TBD : 04/14/2005 - 04/28/2005	-
   RC testing	Feb/2005	N/A	-
   Final sanity checking	Mar/2005	May/2005	-

   ## This is tentative and subject to change

8. General process (who will do what)

   Here is the proposed procedure we should follow in each testing cycle.

   a) announce l10n build for QA available date by Sun (per language)

   b) Sun / Community members sign up by sending a request to maling list (tcmrequest@l10n.openoffice.org) with following information. Testing platform and test case unit(s) are chosen considering interests and the current coverage:

   OO.o Login Name:
   Language of the product you're testing:
   Platform (i.e. W2K, WXP, Fedora, Solaris10/x86, etc.):
   
   Test Areas of your interest (i.e. Calc-2, Overall-1, etc): 
   

   c) Sun TCM administrator assigns test units to tester

   b) and c) will be done in parallel

   d) Sun TCM administrator announce test start and current l10n build available

   e) each tester do test and update TCM (including file issue in IssueZilla)

   This period will be one month.

   f) planned test coverage/issue status/summary become available in TCM as soon as each test report is updated

9. Participation steps

   1) see current assignments assignment page

   2) Signup by sending email to the TCM administrator at tcmrequest@l10n.openoffice.org

   3) In 24 hours check the TCM assignment page for your language to see your assignment is confirmed

   4) Wait for the Sun TCM administorator announcement of test cycle start and current l10n build available

   5) Test and update report result

   6) File bugs