James Laska さんは書きました:

On Tue, 2010-01-05 at 15:43 +1000, Noriko Mizumoto wrote:

...

Hi translators and testers

Regarding the following task entries of F13, recently I have contacted QA team (f-test-l) for live image creation and heard positive replies. So I like to reopen different thread to discuss entire picture with QA team and L10N team together. Please throw your idea and improve this event better. If any doubt/question, pls do ask.

My first thought when seeing the previous thread where to ask whether your testing instructions were documented already. We have test cases for the Fedora 12 i18n test day [1], so if this is a common sequence of tasks that you'd like to encourage others to follow, this might help. If you have more than just a series of test cases (perhaps a higher strategy, organization and pass/fail criteria), I might suggest a lightweight test plan [2].

We only had very tiny window to organize this review event for F12, and there was no instruction documented. I joined F12 i18n test day and learned all well documented. i18n test day is good example and l10n should have similar documentation. For F13, something like "https://fedoraproject.org/wiki/Category:L10n_Test_Cases" to be written. L10n test day (this is our review process) is different from i18n.

My original mail has followed " Template:QA/Test Plan" [1].

[1]:https://www.redhat.com/archives/fedora-test-list/2009-December/msg00352.html

...

As you can notice, it is my bad expressing this in several way such as 'QA test for L10N', 'Review task', 'QA event' etc. What is the best to describe?

We hosted i18n focused Test Days in previous releases and I believe they worked rather well to organize folks around a common goal. I don't see any problems using that vehicle for Fedora 13 if there is a sufficient amount of work that can be distributed. If you'd like more information, take a look at https://fedoraproject.org/wiki/QA/SOP_Test_Day_management. We'll be happy to assist in setting up an event.

Thanks for the pointer! Here is L10n SOP.

1 Get people on board => Translators and testers 2 Set a date => 2010-03-08 to 2010-03-16 3 Create the Wiki page o 3.1 Create test cases => I do have some draft not published yet (how-to-access-gui) o 3.2 Set up the results table => it was bug tracker for F12 4 Build a live CD => We need "nightly compose 2010-03-05" somewhere accessible between 2010-03-08 to 2010-03-16 5 Promote the Test Day => reminder of task schedule to be sent by John 6 Pre-event reminder => same the above 7 Update Test_Day:Current => same with 3.2 bug tracker 8 Do a post-event review => any idea?

noriko

...

For newer translators and testers, this event has just started since Fedora 12. The background is that translators are usually working on PO file only and unable to check the quality of translation how it appears in GUI. Thus to allow translators to review translation in GUI for high quality translation, this event has been added with FESCo approval.

[f13-trans-tasks]

• Build F-13 collection packages for all language translators (Wed 2010-03-03 Fri 2010-03-05)

• Review and correct software translation in built UI (Mon 2010-03-08 Tue 2010-03-16)

[Steps]

1. Remind devel-announce for building F13 packages with latest

translation by 2010-03-04. 2. QA team places nightly-compose of 2010-03-05 at specific location, and make it available during review period. 3. Translators and testers start QA test (review) during 2010-03-08 and 2010-03-16. 4. Packagers fix bugs to include them into final build for GA.

I am thinking to create simple packages list with how-to-access-gui. Any finding which needs packager's action should be reported as a bug [1].

Would this be useful information to document on the wiki? Perhaps as a QA [3] or L10N SOP?

Thanks, James

[1] https://fedoraproject.org/wiki/Category:I18n_Test_Cases [2] http://fedoraproject.org/wiki/Category:Test_Plans [3] https://fedoraproject.org/wiki/Category:QA_SOPs

Em Qua, 2010-01-06 às 12:24 +1000, Noriko Mizumoto escreveu:

For F13, something like

...

"https://fedoraproject.org/wiki/Category:L10n_Test_Cases" to be written.

Done :-) https://fedoraproject.org/wiki/Category:L10n_Test_Cases

This is still incomplete, and I wish everyone to have a look and complete this list and to add any idea. I meant 'built status' is if package was built by specified date.

Noriko, I suggest the inclusion of a test case for Anaconda using the Installation Media in addition to the Live Media. Of course, we would have to use an additional build.

The Installation Media has a different set of strings and IMHO they are worth to be tested. For instance, on Fedora 12, Anaconda was 100% translated for Brazilian Portuguese but some strings remained in English on the Installation Media, although everything was OK on the Live Media.

Regards, Igor Pires Soares

Domingo Becker さんは書きました:

2010/1/6 Igor Pires Soares igorsoares@gmail.com:

...

The Installation Media has a different set of strings and IMHO they are worth to be tested. For instance, on Fedora 12, Anaconda was 100% translated for Brazilian Portuguese but some strings remained in English on the Installation Media, although everything was OK on the Live Media.

The same happened for Spanish, but next time I will also test the the dvd installation media during alpha phase. Last time I only tested the live cd.

I am not sure if installation media is available for us in this review period. But it is good to know the cause why this happens.

Em Qui, 2010-01-07 às 09:43 +1000, Noriko Mizumoto escreveu:

Noriko Mizumoto さんは書きました:

...

Domingo Becker さんは書きました:

...

2010/1/6 Igor Pires Soares igorsoares@gmail.com:

...

The Installation Media has a different set of strings and IMHO they are worth to be tested. For instance, on Fedora 12, Anaconda was 100% translated for Brazilian Portuguese but some strings remained in English on the Installation Media, although everything was OK on the Live Media.

The same happened for Spanish, but next time I will also test the the dvd installation media during alpha phase. Last time I only tested the live cd.

I am not sure if installation media is available for us in this review period. But it is good to know the cause why this happens.

Sorry replied myself. I will be holiday traveling between 8th to 27th January. Due to the place to visit, no internet no mobile available. It would be nice that many of you keep this ball rolling.

Since Noriko is on holiday traveling, I updated the Fedora 13 l10n Results Template wiki page [1] in order to provide some more information for translators, although the page isn't complete yet.

We are using a command-line driven approach to write the testing steps. This is a good way to go because commands are language independent, so we can standardize the process. Translators, if you feel uncomfortable with this approach please let me know.

I guess when we have more documentation prepared we can split it in more pages, so far it looks ok for me.

[1] https://fedoraproject.org/wiki/QA:Fedora_13_l10n_Results_Template

Em Ter, 2010-01-19 às 14:41 +0530, Ankit Patel escreveu:

Igor Pires Soares wrote:

...

I guess when we have more documentation prepared we can split it in more pages, so far it looks ok for me.

[1] https://fedoraproject.org/wiki/QA:Fedora_13_l10n_Results_Template

Hi,

Thanks for the heads up Igor.

some inputs from my side.

• We need to review the list of packages, as some of the packages do not

really comes as default or being obsoleted by some other packages.

IMHO it is important to include not only those packages that come as default, at least I think we should include all system-config-* apps since some of them don't come as default but are also important. I realize some packages as switchdesk and hwbrowser seems obsoleted, but they are still being packed for F13, anyway I'm not sure if they are worth testing at first. I just removed rhpxl and system-config-cluster from the list since both are really obsoleted.

• Steps section needs to be updated. (e.g. for Anaconda testing - 1.

Issue command "liveinst" 2. Check buttons' translations 3. Check whether every message on each window is translated correctly 4. Check the consistency of translation across the application and desktop items. [5. Checking the menu items, dialog boxes' translation if any] If any of these fails file a bug)

If we want more complete steps I think we should write them in other page or write separated test cases like i18n does and link them using a wiki category, as suggested earlier. So we can clean up the results template page by moving the "steps" column to an individual installer test case, for instance.

Regards, Igor Pires Soares

Em Qua, 2010-02-17 às 15:26 +1000, Noriko Mizumoto escreveu:

Now only those packages have no step for review, 'smoon', 'readahead', 'python-meh', 'pykickstart', 'libuser' and 'kexec-tools'. Anyone, any idea?

Some of these packages are most commonly used by other packages that rely on them. I would appreciate if some developers tell us the best way for testing those strings.

Thanks, Igor Pires Soares

James Laska さんは書きました:

On Wed, 2010-01-06 at 12:24 +1000, Noriko Mizumoto wrote:

...

For F13, something like

...

"https://fedoraproject.org/wiki/Category:L10n_Test_Cases" to be written.

Done :-) https://fedoraproject.org/wiki/Category:L10n_Test_Cases

This is still incomplete, and I wish everyone to have a look and complete this list and to add any idea. I meant 'built status' is if package was built by specified date.

Greetings Noriko,

Typically, the category wiki pages are mostly a method for organizing related content on the wiki. So for the [[Category:L10n_Test_Cases]], you might use that to organize different wiki test case pages that have instructions for the l10n testing. For other examples, take a look at https://fedoraproject.org/wiki/Category:I18n_Test_Cases or https://fedoraproject.org/wiki/Category:Installer_Test_Cases. These pages aren't used for posting test results, just for grouping related wiki pages together for easier future navigation.

Hi James, I doubt if translators need further instructions. The one created worked so far for F12 internally. But this is just incomplete and need a help to improve. Once translators know how to get gui, they know what they want to see or check. The idea I draw in my mind is that one wiki page to be created per release, as packages to test may increase/decrease. This is just idea and can be improved in anyway.

It seems like you've already started outlining how you wish to gather test results. Way to get a head start! For that, I might recommend creating a test results wiki page (or pages) similar to https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template. Good news is that you already have the content for this page ... it's just in the wrong spot. I've moved the content you created into a new page ... https://fedoraproject.org/wiki/QA:Fedora_13_l10n_Results_Template. Hope that helps.

From here, one way I could see you proceeding is ...

 1. Start defining some basic test cases to guide testers.  Be sure
    to link any created test case pages into [[Category:L10n Test
    Cases]]
 2. When testing starts, create a wiki page for each test run (e.g.
    Test_Results:Fedora 13 L10n Alpha Test Results]].  Results will
    be posted to this page, not your template.

Some of this is touched on by two of our existing SOP's [1][2], but if this is a process you'd like to repeat throughout the release (and future releases), I'd recommend taking your instructions above and creating a new [[SOP L10n Test Management]].

There are two goals for this review process. One is to have a chance to review own work of translation in GUI and check the quality. Second is to check if latest translation done reflected in the package. In short frame of one week, we filed loads of bugs last time for second goal. Those bugs push the thing forwards. Honestly I like to avoid to create too much wiki pages specially in that period. But I am happy to create any page required.

What I am really looking for is "nightly compose of 2010-03-05" available during 2010-03-08 to 2010-03-16 from QA team. With separate thread, yet installation media request comes up for reviewing.

Thanks a lot noriko

2010/1/7 James Laska jlaska@redhat.com:

The QA team doesn't provide/build the compose images, that is a service provided by release engineering. Looking at the dates you mention, the current F-13 QA schedule [2] shows that F-13-Alpha will be available for translation testing at that time.

Don't ask translators to build a live image from rawhide. Based on my own experience, it will go wrong. I was never able to make a 'Live CD from rawhide' work. Something always goes wrong when booting. It seems that development people starts from an early installer and uses the update system since then.

But if we use the Alpha compose to test, and our corrections and improvements will be present in Alpha 2 or Beta compose to check once more, that's ok and may work well. We will need that packagers pay attention to our commits to their repositories.

Additionally, there are nightly rawhide live images available [3] if testing against more recent updates is desired.

If these nightly rawhide live images work, it would be good. But packagers would have to pay attention to our commits in order to rebuild their packages with newer translations as soon as possible.

My two cents.

kind regards

Domingo Becker (es)

On Wed, 2010-02-17 at 14:28 +1000, Noriko Mizumoto wrote:

Domingo Becker さんは書きました:

...

2010/1/7 James Laska jlaska@redhat.com:

...

The QA team doesn't provide/build the compose images, that is a service provided by release engineering. Looking at the dates you mention, the current F-13 QA schedule [2] shows that F-13-Alpha will be available for translation testing at that time.

Don't ask translators to build a live image from rawhide. Based on my own experience, it will go wrong. I was never able to make a 'Live CD from rawhide' work. Something always goes wrong when booting. It seems that development people starts from an early installer and uses the update system since then.

But if we use the Alpha compose to test, and our corrections and improvements will be present in Alpha 2 or Beta compose to check once more, that's ok and may work well. We will need that packagers pay attention to our commits to their repositories.

'Compose alpha candidates' date is 2010-2-18, while 'String freeze' date is 2010-02-16. Alpha composed this timing may not contain most of new/updated translation strings for review and correction, no?

As far as I understand, if the compose happens after the string freeze, the alpha should contain all frozen translation content.

...

...

Additionally, there are nightly rawhide live images available [3] if testing against more recent updates is desired.

If these nightly rawhide live images work, it would be good. But packagers would have to pay attention to our commits in order to rebuild their packages with newer translations as soon as possible.

It is said that having a 'nightly-composes of 5-Mar-2010' image available for certain period is possible [1]. Someone can give a hand on this?

If there is a specific image you need for l10n testing, I'd recommend submitting a request to release engineering for guidance on hosting the image. Several folks could host the image on their fedorapeople.org pages, but I'd suggest starting with something more formal and rel-eng can advise alternatives.

https://fedorahosted.org/rel-eng/newticket

Thanks, James

...

...

If there is a specific image you need for l10n testing, I'd recommend submitting a request to release engineering for guidance on hosting the image. Several folks could host the image on their fedorapeople.org pages, but I'd suggest starting with something more formal and rel-eng can advise alternatives.

https://fedorahosted.org/rel-eng/newticket

New ticket has been created! Thank you! https://fedorahosted.org/rel-eng/ticket/3420

FYI During today's F13 Alpha Readiness Meeting, this has been mentioned and marked to go forward. After meeting, had the chat with Jesse, so I can say that the compose will be provided on time, stay tuned :-)

noriko