Issue 20958 - Encoding and escaping problems with Evolution addressbook
Summary: Encoding and escaping problems with Evolution addressbook
Status: CLOSED FIXED
Alias: None
Product: Base
Classification: Application
Component: code (show other issues)
Version: OOo 1.1
Hardware: PC Linux, all
: P3 Trivial (vote)
Target Milestone: OOo 1.1.1
Assignee: marc.neumann
QA Contact: issues@dba
URL:
Keywords:
: 19252 (view as issue list)
Depends on:
Blocks: 17159
  Show dependency tree
 
Reported: 2003-10-09 15:36 UTC by mkretzschmar
Modified: 2006-05-31 14:29 UTC (History)
5 users (show)

See Also:
Issue Type: PATCH
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Patch. Also available here: http://ooo.ximian.com/patches/OOO_1_1/evo-i18n.diff (2.40 KB, patch)
2003-10-09 15:42 UTC, mkretzschmar
no flags Details | Diff
for the records: patch as checked in (2.35 KB, patch)
2003-10-15 10:52 UTC, Frank Schönheit
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description mkretzschmar 2003-10-09 15:36:33 UTC
Two problems with the evolution addressbook.

1) The column headers are always displayed as US ASCII. It should use the system
encoding. e.g. German Geschäftsadresse is currently displayed as Gesch?ftsadresse.

2) The evolution-address-book exporter escapes any non-ASCII character to \ooo
where ooo is the octal representation of the character. OOo's csv data source
(base of the evolution data source) doesn't unescape this. I have sent a patch
to evolution-patches@lists.ximian.com to remove the \ooo escaping. But then the
evolution datasource must read the csv file in utf-8 -> other part of the patch.
Comment 1 mkretzschmar 2003-10-09 15:42:57 UTC
Created attachment 10143 [details]
Patch. Also available here: http://ooo.ximian.com/patches/OOO_1_1/evo-i18n.diff
Comment 2 Frank Schönheit 2003-10-10 08:42:59 UTC
mkretzschmar (*), thanks for this contribution! There already is an
issue which deals with this bug, but since you already provided an
intial patch here, we can use this one here for discussing the solution.

CC'ing the developers for the Evolution Address Book Integration, and
confirming the issue.


(*) do you mind telling us your real name? I don't like talking to
nicknames, and I don't like the fact that no real names are possible
in OOo's infrastructure ...
Comment 3 Frank Schönheit 2003-10-10 09:08:58 UTC
Okay, now for the patch (:

One minor thing: The change in LTable.cxx can be done slightly better:
The unpatched line is nonsense, anyway: What is does is converting an
Unicode-string (aColumnFinalName, of type "::rtl::OUString") to ASCII
(with OUStringToOString), and creating another Unicode-string
(sColumnFinalName, of type "String") out of this ASCII representation.
In other words, the line is buggy, anyway :).
Since OUStrings and Strings are interoperable nowadays, you could
simply replace the line in question with
  sColumnFinalName = aColumnFinalName;
, and everything should be fine. (Indeed this change is on a TODO list
which resulted from reviewing the evoab code).


The first part (in LConnection.cxx) looks good to me.

I could imagine a more extended solution, which, for compatibility
reasons, also cares for encoded characters. This would not require to
use a patched Evolution, but also work with the current Evolution
versions.
Would you be interested in enhancing your patch this way? If so, you
might want to look at the OEvoabString class, implemented in
LNoException.cxx, which manages the conversion/split of lines from the
exported files. From your nickname, I somehow suppose that you won't
have a problem with the german comments in this class :).
Comment 4 mmeeks 2003-10-10 11:56:52 UTC
Hi Frank; meet Martin - who's been working as a Ximian intern for a
while, but is an able, longer term OO.o hacker :-)

I guess the problem with making this more 'intelligent' is that should
someone in fact want to pass \032 inside their utf-8 string; then it
would proceed to get mangled by OO.o - which seems unlikely, but ...

I think it's best to fix this in 1 place, once. Ximian does fairly
regular evo-1.4.N updates anyway - due to the occasional security fix,
so I anticipate that it's prolly best for everyone to fix it once in
the right place; that is unless we can do some version checking etc.
but it's nice to have cleaner code going foward I think.

Thanks for commenting so quickly.
Comment 5 Frank Schönheit 2003-10-10 14:13:22 UTC
Michael (and: Hi Martin :)
> I guess the problem with making this more 'intelligent' is that should
> someone in fact want to pass \032 inside their utf-8 string; then it
> would proceed to get mangled by OO.o - which seems unlikely, but ...

Well, I would have expected that in such a case the exporter would
produce "\\032" or something like this. If it doesn't, then we of
course can always construct cases where our reading on SDBC fails ....


I can check in the patch here, but speaking strictly, I cannot set the
bug to FIXED then, since it requires a new Evolution version, which
perhaps doesn't even exist at the moment.
So it seems that whatever we can do requires close sync between Ximian
and OOo.

Is there a public bug tracking system for Evolution, where an issue
for the UTF-8 change exists? I'd like to link it here then. If there
*is*, then I'd check in the fix here, state that people need an
Evolution update for this, too, and link to the issue.
Comment 6 mkretzschmar 2003-10-10 17:21:40 UTC
Hi Frank

I created an evolution bug
http://bugzilla.ximian.com/show_bug.cgi?id=49519

It does in fact escape \ to \\ because it still escapes line breaks to
\n. Hmm, I didn't check if the csv importer handles \n correctly.
Comment 7 Frank Schönheit 2003-10-13 10:01:12 UTC
checked this in on the CWS dba01pp1 (branch cws_srx645_dba01pp1). This
for the moment helps us to get rid of the "umlaut problem", and once
the Evolution exporter uses UTF-8 encoding, this should also give use
the proper texts for characters >127.

Thanks Martin!
Comment 8 Frank Schönheit 2003-10-15 10:52:24 UTC
Created attachment 10329 [details]
for the records: patch as checked in
Comment 9 Frank Schönheit 2003-10-30 10:29:42 UTC
fs->msc: please verify in CWS dba01pp1. Thanks :)
Comment 10 marc.neumann 2003-11-04 12:20:28 UTC
*** Issue 19252 has been marked as a duplicate of this issue. ***
Comment 11 marc.neumann 2003-11-26 09:23:33 UTC
fixed in cws dba01pp1
Comment 12 marc.neumann 2003-11-26 09:23:47 UTC
verified in cws dba01pp1
Comment 13 marc.neumann 2004-01-27 16:11:37 UTC
Hi,

this will be fixed in OOo 1.1.1 which will be available soon. I close this issue
now as fixed.

bye Marc
Comment 14 hans_werner67 2004-02-02 13:05:03 UTC
change subcomponent to 'none'
Comment 15 Frank Schönheit 2004-09-06 10:19:26 UTC
mkretzschmar, it seems that version of 1.4.6 of Evolution/exporter does not
contain the patch you mentioned - special characters are still exported as \ooo
(with the octal representation). Do you know more about this?
Comment 16 mkretzschmar 2004-09-13 18:26:06 UTC
fs: indeed. I expected the patch to go in faster. In the ximian bugzilla this is
the closing comment:

------- Additional Comments From Chris Toshok 2004-07-22 17:36 -------

hm, this appears to already be in CVS.  no ChangeLog entry for it
explicitly.  I guess I merged it back on 05/24 when I revived the
exporter code.  *shrug*.

closing.

-------

So the patch is included in evolution 2.0. Which will be released on Wednesday.
I wouldn't expect a 1.4.7.
Comment 17 Frank Schönheit 2004-09-14 08:05:07 UTC
okay, thanks
Comment 18 alex.thurgood 2004-10-21 07:40:57 UTC
Hi all,

There is something that my little brain doesn't understand here. The issue is
stated as being fixed because the patch will be integrated into Evolution 2.0.
Has this actually occurred ? The trouble with bugs like this one that are
dependent on an outside program, no matter how good the collaboration between
two projects, is that visibility gets lost for the average Joe Bloggs user like
myself. I don't even know if I can get a package for Evo 2.0 for my Mandrake 10
distrib, it doesn't appear to be available, only a 1.4.6 something or other. I
won't be downloading from Cooker, it's bad enough testing the dev snapshots of
OOo ;-)

alex
Comment 19 Frank Schönheit 2004-10-21 08:19:09 UTC
Well, on OOo's side, everything has been done (to our best knowledge). The bug
isn't really fixable for an older Evolution, so the only chance is to tell
people "go get Evo 2.0".

I agree that this is not really ... the definition of "transparency", however, I
don't really see another way ....