[Proposal] .xot bundles, for translations

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Proposal] .xot bundles, for translations

Sayamindu Dasgupta-2
Hello,
One of the things in my TODO for 9.1 is to have a better mechanism for
language packs[1] in the XO. The primary goal of language packs is to
decouple the process of translations from the process of OS release as
much as possible, since as our software gets larger and more
complicated, it will become more and more difficult for translators to
keep up with the pace of development.
Our current language pack mechanism handles the decoupling part, but
two of its significant shortcomings include

a) Overwriting of existing translation files (it may overwrite the
original .mo file in certain cases)
b) Difficulty for deployments (deployments have to manual start each
XO, and run the pack installer script from a console)
c) No auto update mechanism

I have been thinking of having a separate place in the filesystem for
_new_ translations, and using RPM to manage the installation and
upgradation of the new translations.
However, Scott suggested in a recent email conversation that deploying
new translations through a bundle like format (used for activities and
content right now) may make more sense as users themselves can use the
Sugar control panel to download updated translations (as currently
done with activities). I think this may be a better option than RPMs
as

a) It makes the new translations user modifiable (we can have a
translate activity later on which would let users modify the
translations)
b) It would be pretty trivial to add support for a new .xot format in
the customization key mechanism (just unzip them in /home/olpc)

However, this would need XO specific changes in glibc, python, etoys
and scratch (I think). I already have patches for glibc and python
(based on patches from Ubuntu, which already uses a similar system,
where they generate language packs out of their launchpad/rosetta
based translations)

Am I missing something out here ? If there are no problems with this
proposal, I would like to start testing such a system in Joyride (with
at least glibc and python patched) by the end of the month.

Thanks,
Sayamindu


[1]http://wiki.laptop.org/go/Localization/LanguagePacks
--
Sayamindu Dasgupta
[http://sayamindu.randomink.org/ramblings]
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Localization] [Proposal] .xot bundles, for translations

Walter Bender-4
It doesn't sound too invasive of a change and yet it is in keeping
with the basic update system and may evolve into a simpler framework
for end users to contribute translations as well.

-walter

On Thu, Nov 13, 2008 at 3:58 PM, Sayamindu Dasgupta <[hidden email]> wrote:

> Hello,
> One of the things in my TODO for 9.1 is to have a better mechanism for
> language packs[1] in the XO. The primary goal of language packs is to
> decouple the process of translations from the process of OS release as
> much as possible, since as our software gets larger and more
> complicated, it will become more and more difficult for translators to
> keep up with the pace of development.
> Our current language pack mechanism handles the decoupling part, but
> two of its significant shortcomings include
>
> a) Overwriting of existing translation files (it may overwrite the
> original .mo file in certain cases)
> b) Difficulty for deployments (deployments have to manual start each
> XO, and run the pack installer script from a console)
> c) No auto update mechanism
>
> I have been thinking of having a separate place in the filesystem for
> _new_ translations, and using RPM to manage the installation and
> upgradation of the new translations.
> However, Scott suggested in a recent email conversation that deploying
> new translations through a bundle like format (used for activities and
> content right now) may make more sense as users themselves can use the
> Sugar control panel to download updated translations (as currently
> done with activities). I think this may be a better option than RPMs
> as
>
> a) It makes the new translations user modifiable (we can have a
> translate activity later on which would let users modify the
> translations)
> b) It would be pretty trivial to add support for a new .xot format in
> the customization key mechanism (just unzip them in /home/olpc)
>
> However, this would need XO specific changes in glibc, python, etoys
> and scratch (I think). I already have patches for glibc and python
> (based on patches from Ubuntu, which already uses a similar system,
> where they generate language packs out of their launchpad/rosetta
> based translations)
>
> Am I missing something out here ? If there are no problems with this
> proposal, I would like to start testing such a system in Joyride (with
> at least glibc and python patched) by the end of the month.
>
> Thanks,
> Sayamindu
>
>
> [1]http://wiki.laptop.org/go/Localization/LanguagePacks
> --
> Sayamindu Dasgupta
> [http://sayamindu.randomink.org/ramblings]
> _______________________________________________
> Localization mailing list
> [hidden email]
> http://lists.laptop.org/listinfo/localization
>



--
Walter Bender
Sugar Labs
http://www.sugarlabs.org
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Localization] [Proposal] .xot bundles, for translations

Korakurider-2
In reply to this post by Sayamindu Dasgupta-2
Hello.

On Fri, Nov 14, 2008 at 5:58 AM, Sayamindu Dasgupta <[hidden email]> wrote:

> Hello,
> One of the things in my TODO for 9.1 is to have a better mechanism for
> language packs[1] in the XO. The primary goal of language packs is to
> decouple the process of translations from the process of OS release as
> much as possible, since as our software gets larger and more
> complicated, it will become more and more difficult for translators to
> keep up with the pace of development.
> Our current language pack mechanism handles the decoupling part, but
> two of its significant shortcomings include
>
> a) Overwriting of existing translation files (it may overwrite the
> original .mo file in certain cases)
> b) Difficulty for deployments (deployments have to manual start each
> XO, and run the pack installer script from a console)
> c) No auto update mechanism
>
> I have been thinking of having a separate place in the filesystem for
> _new_ translations, and using RPM to manage the installation and
> upgradation of the new translations.
> However, Scott suggested in a recent email conversation that deploying
> new translations through a bundle like format (used for activities and
> content right now) may make more sense as users themselves can use the
> Sugar control panel to download updated translations (as currently
> done with activities). I think this may be a better option than RPMs
> as
>
> a) It makes the new translations user modifiable (we can have a
> translate activity later on which would let users modify the
> translations)
> b) It would be pretty trivial to add support for a new .xot format in
> the customization key mechanism (just unzip them in /home/olpc)
>
> However, this would need XO specific changes in glibc, python, etoys
> and scratch (I think). I already have patches for glibc and python
> (based on patches from Ubuntu, which already uses a similar system,
> where they generate language packs out of their launchpad/rosetta
> based translations)
>
> Am I missing something out here ? If there are no problems with this
> proposal, I would like to start testing such a system in Joyride (with
> at least glibc and python patched) by the end of the month.

Syamindu, sorry for late response.

+ I think people using non-XO platform could make use of lang pack.
   Will the proposed technology work on non-XO platforms like "sugar
on stick" or Ubuntu?

+ Could you explain proposed directory structure and file layout that
the lang pack will install, so that we could validate the design with
Etoys?

/Korakurider

>
> Thanks,
> Sayamindu
>
>
> [1]http://wiki.laptop.org/go/Localization/LanguagePacks
> --
> Sayamindu Dasgupta
> [http://sayamindu.randomink.org/ramblings]
> _______________________________________________
> Localization mailing list
> [hidden email]
> http://lists.laptop.org/listinfo/localization
>
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Localization] [Proposal] .xot bundles, for translations

C. Scott Ananian
Re: Scratch & etoys:  the problem with updating translations "in
place" is that it doesn't support distributed work on translations:
OLPC might do basic translations; they might be further developed in a
country or region, etc.  Each might be updated individually.

Further, you want to be able to back out changes, in order to protect
against getting malicious "translations" from a friend.  Uninstalling
a language bundle gives you that, un-merging changes to a shared
in-place translation file is... more difficult.

 --scott

--
                         ( http://cscott.net/ )
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

Martin Langhoff
In reply to this post by Sayamindu Dasgupta-2
On Thu, Nov 13, 2008 at 6:58 PM, Sayamindu Dasgupta <[hidden email]> wrote:
> I have been thinking of having a separate place in the filesystem for
> _new_ translations, and using RPM to manage the installation and
> upgradation of the new translations.

What is the downside of RPMs? If users edit the localisation locally,
that is _fine_ and we can provide a mechanism to make an rpm easily
out of it.

rpm has limited support for "user installable" packages that are meant
to be installed in your homedir. Maybe it can serve this purpose, even
within its limitations?

If that doesn't work properly, maybe we install the rpm as root, but
invoking rpm with --noscripts, and perhaps auditing the pkg manifest
to check for anything with suid flags, etc. We could even build a dumb
rpm unpacker/installer but I doubt it is needed.

A new bundle format makes us more incompatible with the world.
Example:  someone builds a localisation for us, it won't work for
Fedora, and viceversa.

Building bespoke sofware has a huge long term cost so when we do it,
we better get a ton of value, something radically better and hopefully
with immediate payoff.

Installing a bunch of files in /home is not leading edge enough to
justify this, IMHO.

cheers,


martin langhoff
--
 [hidden email]
 [hidden email] -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

C. Scott Ananian
On Tue, Dec 2, 2008 at 2:26 PM, Martin Langhoff
<[hidden email]> wrote:

> On Thu, Nov 13, 2008 at 6:58 PM, Sayamindu Dasgupta <[hidden email]> wrote:
>> I have been thinking of having a separate place in the filesystem for
>> _new_ translations, and using RPM to manage the installation and
>> upgradation of the new translations.
>
> What is the downside of RPMs? If users edit the localisation locally,
> that is _fine_ and we can provide a mechanism to make an rpm easily
> out of it.
>
> rpm has limited support for "user installable" packages that are meant
> to be installed in your homedir. Maybe it can serve this purpose, even
> within its limitations?
>
> If that doesn't work properly, maybe we install the rpm as root, but
> invoking rpm with --noscripts, and perhaps auditing the pkg manifest
> to check for anything with suid flags, etc. We could even build a dumb
> rpm unpacker/installer but I doubt it is needed.
>
> A new bundle format makes us more incompatible with the world.
> Example:  someone builds a localisation for us, it won't work for
> Fedora, and viceversa.

Fedora does not have a standard solution either,  so I'm not sure
where you're going with this.  We have to invent something.  RPM is
not obviously the right solution.
 --scott


--
                         ( http://cscott.net/ )
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

Martin Langhoff
On Tue, Dec 2, 2008 at 6:49 PM, C. Scott Ananian <[hidden email]> wrote:
> Fedora does not have a standard solution either,  so I'm not sure
> where you're going with this.  We have to invent something.  RPM is
> not obviously the right solution.

So Fedora doesn't use rpm files for localization packages? What does
it use then?

If I say 'yum search catalan' it returns a bunch of rpms -
kde-l10n-Catalan for example. What else could this mean?

Debian does the same, AFAICS...

cheers,


m
--
 [hidden email]
 [hidden email] -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

C. Scott Ananian
On Tue, Dec 2, 2008 at 4:26 PM, Martin Langhoff
<[hidden email]> wrote:

> On Tue, Dec 2, 2008 at 6:49 PM, C. Scott Ananian <[hidden email]> wrote:
>> Fedora does not have a standard solution either,  so I'm not sure
>> where you're going with this.  We have to invent something.  RPM is
>> not obviously the right solution.
>
> So Fedora doesn't use rpm files for localization packages? What does
> it use then?
>
> If I say 'yum search catalan' it returns a bunch of rpms -
> kde-l10n-Catalan for example. What else could this mean?
>
> Debian does the same, AFAICS...

Please re-read Sayamindu's original message.  Thanks.
 --scott

--
                         ( http://cscott.net/ )
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

Martin Langhoff
On Tue, Dec 2, 2008 at 7:34 PM, C. Scott Ananian <[hidden email]> wrote:
> Please re-read Sayamindu's original message.  Thanks.

I don't find anything too special there. Perhaps I wasn't clear earlier.

What I meant to say is that all the good things we get from a bespoke
packaging format, we can get from rpm with a few conventions as to the
directories where things land.

You can still do "local user edits" by either editing the files
in-place, or editing a copy of the files, which is kept in a "local"
directory that is in the 'path' for localization files. The pros and
cons of both options can be argued separately, but both can work, and
many additional tricks can be put into action too.

So - I can't see anything that rpm can't handle, and I can't see an
interesting upside to building a bespoke mechanism to deploy files.
Perhaps it exists, and is really an overriding advantage that explains
why we'd want to carry the significant additional long term burden (on
us, and on everyone else) of a bespoke format.

cheers,



m
--
 [hidden email]
 [hidden email] -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Localization] [Proposal] .xot bundles, for translations

Korakurider-2
In reply to this post by C. Scott Ananian
On Wed, Dec 3, 2008 at 3:47 AM, C. Scott Ananian <[hidden email]> wrote:
> Re: Scratch & etoys:  the problem with updating translations "in
> place" is that it doesn't support distributed work on translations:
> OLPC might do basic translations; they might be further developed in a
> country or region, etc.  Each might be updated individually.
   Yes, current lang pack is being generated from translations on Pootle
   and can be used to replace older translations (that was from Pootle
also) in the box.
   Essential use case for the facility seems different from that what
you imagine (distributed translation work).

   I think that, even if some deployment want to do custom translation
in distributed fassion,
   they will need to build and deploy the authoritive version of
translation pack that they merge translations from their team and do
extensive human review.  So I believe lang pack will still work for
them.

>
> Further, you want to be able to back out changes, in order to protect
> against getting malicious "translations" from a friend.  Uninstalling
> a language bundle gives you that, un-merging changes to a shared
> in-place translation file is... more difficult.
    Lang pack just replaces entire contents of each MO, not merge
contents from multiple MOs.  Sayamindu 's proposal is about how to
replace MO.  So uninstalling seems OK.

Sayamindu, could you confirm my understanding?

/Korakurider
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Localization] [Proposal] .xot bundles, for translations

Jameson Quinn
I must admit, I don't really understand this proposal; and I want to, because it raises important issues for activity signing. Sayamindu, can you explain the options as you see them:

-in very general terms, what is the ui ("control panel" is enough on this point)
-what code is activated by this ui do the actual "installing" of language packs
-where do those files go

Thanks,
Jameson

On Tue, Dec 2, 2008 at 8:14 PM, Korakurider <[hidden email]> wrote:
On Wed, Dec 3, 2008 at 3:47 AM, C. Scott Ananian <[hidden email]> wrote:
> Re: Scratch & etoys:  the problem with updating translations "in
> place" is that it doesn't support distributed work on translations:
> OLPC might do basic translations; they might be further developed in a
> country or region, etc.  Each might be updated individually.
  Yes, current lang pack is being generated from translations on Pootle
  and can be used to replace older translations (that was from Pootle
also) in the box.
  Essential use case for the facility seems different from that what
you imagine (distributed translation work).

  I think that, even if some deployment want to do custom translation
in distributed fassion,
  they will need to build and deploy the authoritive version of
translation pack that they merge translations from their team and do
extensive human review.  So I believe lang pack will still work for
them.

>
> Further, you want to be able to back out changes, in order to protect
> against getting malicious "translations" from a friend.  Uninstalling
> a language bundle gives you that, un-merging changes to a shared
> in-place translation file is... more difficult.
   Lang pack just replaces entire contents of each MO, not merge
contents from multiple MOs.  Sayamindu 's proposal is about how to
replace MO.  So uninstalling seems OK.

Sayamindu, could you confirm my understanding?

/Korakurider
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar


_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Proposal] .xot bundles, for translations

Martin Langhoff
In reply to this post by Martin Langhoff
On Tue, Dec 2, 2008 at 8:33 PM, Martin Langhoff
<[hidden email]> wrote:
> What I meant to say is that all the good things we get from a bespoke
> packaging format, we can get from rpm with a few conventions as to the
> directories where things land.

A couple of additional notes from a private subthread...

...there are a few ways to use rpm/yum for unprivileged users
(alternative DB, fakeroot, relocatable pkgs...), and I think we can
use them for this. In fact, we could even build a simplistic rpm
installer in python that handles a subset of what rpm does (hopefully
this is not needed, it'd detract from the idea quite a bit)

One valid criticism to using rpm - from a Sugar perspective - is that Sugar
won't want to become tied to Fedora/RH. There's a case for thinking
through if we can actually use rpm the way we want on Debian and/or
apt on Fedora. Both rpm and apt are available in old/buggy versions in
the "other" family of distros.

Using rpm or apt Sugar would getting a bit further away from Windows
(does cygwin carry either?) - a bit less so on OSX (where the fink
toolchain will probably work alright, specially with translation pkgs,
which are by definition "noarch").

cheers,




m
--
 [hidden email]
 [hidden email] -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff
_______________________________________________
Sugar mailing list
[hidden email]
http://lists.laptop.org/listinfo/sugar
Loading...