Apache OpenOffice (AOO) Bugzilla – Issue 20958
Encoding and escaping problems with Evolution addressbook
Last modified: 2006-05-31 14:29:06 UTC
Two problems with the evolution addressbook. 1) The column headers are always displayed as US ASCII. It should use the system encoding. e.g. German Geschäftsadresse is currently displayed as Gesch?ftsadresse. 2) The evolution-address-book exporter escapes any non-ASCII character to \ooo where ooo is the octal representation of the character. OOo's csv data source (base of the evolution data source) doesn't unescape this. I have sent a patch to evolution-patches@lists.ximian.com to remove the \ooo escaping. But then the evolution datasource must read the csv file in utf-8 -> other part of the patch.
Created attachment 10143 [details] Patch. Also available here: http://ooo.ximian.com/patches/OOO_1_1/evo-i18n.diff
mkretzschmar (*), thanks for this contribution! There already is an issue which deals with this bug, but since you already provided an intial patch here, we can use this one here for discussing the solution. CC'ing the developers for the Evolution Address Book Integration, and confirming the issue. (*) do you mind telling us your real name? I don't like talking to nicknames, and I don't like the fact that no real names are possible in OOo's infrastructure ...
Okay, now for the patch (: One minor thing: The change in LTable.cxx can be done slightly better: The unpatched line is nonsense, anyway: What is does is converting an Unicode-string (aColumnFinalName, of type "::rtl::OUString") to ASCII (with OUStringToOString), and creating another Unicode-string (sColumnFinalName, of type "String") out of this ASCII representation. In other words, the line is buggy, anyway :). Since OUStrings and Strings are interoperable nowadays, you could simply replace the line in question with sColumnFinalName = aColumnFinalName; , and everything should be fine. (Indeed this change is on a TODO list which resulted from reviewing the evoab code). The first part (in LConnection.cxx) looks good to me. I could imagine a more extended solution, which, for compatibility reasons, also cares for encoded characters. This would not require to use a patched Evolution, but also work with the current Evolution versions. Would you be interested in enhancing your patch this way? If so, you might want to look at the OEvoabString class, implemented in LNoException.cxx, which manages the conversion/split of lines from the exported files. From your nickname, I somehow suppose that you won't have a problem with the german comments in this class :).
Hi Frank; meet Martin - who's been working as a Ximian intern for a while, but is an able, longer term OO.o hacker :-) I guess the problem with making this more 'intelligent' is that should someone in fact want to pass \032 inside their utf-8 string; then it would proceed to get mangled by OO.o - which seems unlikely, but ... I think it's best to fix this in 1 place, once. Ximian does fairly regular evo-1.4.N updates anyway - due to the occasional security fix, so I anticipate that it's prolly best for everyone to fix it once in the right place; that is unless we can do some version checking etc. but it's nice to have cleaner code going foward I think. Thanks for commenting so quickly.
Michael (and: Hi Martin :) > I guess the problem with making this more 'intelligent' is that should > someone in fact want to pass \032 inside their utf-8 string; then it > would proceed to get mangled by OO.o - which seems unlikely, but ... Well, I would have expected that in such a case the exporter would produce "\\032" or something like this. If it doesn't, then we of course can always construct cases where our reading on SDBC fails .... I can check in the patch here, but speaking strictly, I cannot set the bug to FIXED then, since it requires a new Evolution version, which perhaps doesn't even exist at the moment. So it seems that whatever we can do requires close sync between Ximian and OOo. Is there a public bug tracking system for Evolution, where an issue for the UTF-8 change exists? I'd like to link it here then. If there *is*, then I'd check in the fix here, state that people need an Evolution update for this, too, and link to the issue.
Hi Frank I created an evolution bug http://bugzilla.ximian.com/show_bug.cgi?id=49519 It does in fact escape \ to \\ because it still escapes line breaks to \n. Hmm, I didn't check if the csv importer handles \n correctly.
checked this in on the CWS dba01pp1 (branch cws_srx645_dba01pp1). This for the moment helps us to get rid of the "umlaut problem", and once the Evolution exporter uses UTF-8 encoding, this should also give use the proper texts for characters >127. Thanks Martin!
Created attachment 10329 [details] for the records: patch as checked in
fs->msc: please verify in CWS dba01pp1. Thanks :)
*** Issue 19252 has been marked as a duplicate of this issue. ***
fixed in cws dba01pp1
verified in cws dba01pp1
Hi, this will be fixed in OOo 1.1.1 which will be available soon. I close this issue now as fixed. bye Marc
change subcomponent to 'none'
mkretzschmar, it seems that version of 1.4.6 of Evolution/exporter does not contain the patch you mentioned - special characters are still exported as \ooo (with the octal representation). Do you know more about this?
fs: indeed. I expected the patch to go in faster. In the ximian bugzilla this is the closing comment: ------- Additional Comments From Chris Toshok 2004-07-22 17:36 ------- hm, this appears to already be in CVS. no ChangeLog entry for it explicitly. I guess I merged it back on 05/24 when I revived the exporter code. *shrug*. closing. ------- So the patch is included in evolution 2.0. Which will be released on Wednesday. I wouldn't expect a 1.4.7.
okay, thanks
Hi all, There is something that my little brain doesn't understand here. The issue is stated as being fixed because the patch will be integrated into Evolution 2.0. Has this actually occurred ? The trouble with bugs like this one that are dependent on an outside program, no matter how good the collaboration between two projects, is that visibility gets lost for the average Joe Bloggs user like myself. I don't even know if I can get a package for Evo 2.0 for my Mandrake 10 distrib, it doesn't appear to be available, only a 1.4.6 something or other. I won't be downloading from Cooker, it's bad enough testing the dev snapshots of OOo ;-) alex
Well, on OOo's side, everything has been done (to our best knowledge). The bug isn't really fixable for an older Evolution, so the only chance is to tell people "go get Evo 2.0". I agree that this is not really ... the definition of "transparency", however, I don't really see another way ....