Apache OpenOffice (AOO) Bugzilla – Full Text Issue Listing |
Summary: | OLE/DDE link from Calc to Writer trashes non-ASCII characters (special characters, umlauts) | ||
---|---|---|---|
Product: | Calc | Reporter: | slothman <slothman> |
Component: | code | Assignee: | oc |
Status: | CLOSED FIXED | QA Contact: | issues@sw <issues> |
Severity: | Trivial | ||
Priority: | P3 | CC: | issues, kamataki, kami911, kpalagin, lohmaier, Mathias_Bauer, ooo, os_ooo, rail_ooo, srt2006, tora3, y-catch |
Version: | OOo 1.1 | Keywords: | oooqa |
Target Milestone: | --- | ||
Hardware: | PC | ||
OS: | Linux, all | ||
Issue Type: | PATCH | Latest Confirmation in: | --- |
Developer Difficulty: | --- | ||
Attachments: |
Description
slothman
2002-11-30 23:12:40 UTC
Created attachment 3825 [details]
Spreadsheet source for OLE link
Created attachment 3826 [details]
Writer document demonstrating OLE link
Reassigned to MRU This problem went away at one point when I reinstalled Red Hat 8.0 from scratch. It has since returned after installing Red Hat 9 from scratch. Could this be caused by some sort of odd configuration problem? In your sample Writer file you didn't use a DDE link to your Calc table. What you did there was just inserting an embedded OLE object of the Calc table. A DDE link is something quite different and AFAIK not supported by Linux platforms. OK, I evidently became confused about the nature of the link. I'm still mystified by why it is that under some installations, the same document will wind up with extra garbage whenever I use non-ASCII characters, and under some installations it won't. If I want to examine the issues of the infrastructure behind locales and non-ASCII characters under OpenOffice, where should I start? Thanks, Max MRU->OC: There 's a DDE-link inside the Calc-OLE, which seems to be "causus cnaxus" ;-) When inserting DDE link in a Calc document and using the attached .sxw as source, you can see some formats and characters garbled. So no Writer problem Hi Niklas, please have a look Oliver, I don't see how this should happen. I think I figured it out-- problems using CPAN with Perl on Red Hat Linux and the relevant fix (switching from LANG = en_US.utf-8 to LANG = en_US) made me try it for OpenOffice.org. Sure enough, if I open my files using LANG=en_US.utf-8, I get the weird symbols. If I use LANG=en_US, the problem evaporates. The problem persists under openoffice.org-1.1.0-16 under Fedora Core 1. LANG=en_US is fine, LANG=en_US.UTF-8 garbles non-ASCII characters. Hi Niklas, as discussed this one back to you. It seems that DDE Links are sent encoded but would not be decoded as I've understand our discussion. You've said we need a clear specification of the DDE Process. Frank *** Issue 51603 has been marked as a duplicate of this issue. *** *** Issue 51740 has been marked as a duplicate of this issue. *** Just for the record, the DDE server side code is located in sc/source/ui/docshell/servobj.cxx ScServerObject::GetData( ... ) while the DDE client side code is located in sc/source/core/tool/ddelink.cxx ScDdeLink::DataChanged( ... ) Furthermore, there is a "fix" at least for Linux. In the DDE client code referenced above, there is this line: ScByteSequenceToString::GetString( aLinkStr, rValue, DDE_TXT_ENCODING ); which probably gets the string from the DDE server. In this line, changing the encoding from "DDE_TXT_ENCODING" to "gsl_getSystemTextEncoding()" solved this problem at least for the Japanese encoding. Tora also reported that doing the same also solved it on the Japanese version of Windows. Just as a note, here is the link to MS's DDE documentation (thanks to Tora) http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/winui/windowsuserinterface/dataexchange/dynamicdataexchangemanagementlibrary.asp This page specifies that for a DDE transaction, a CP_WINANSI code page should be used for non-multibyte character set, and CP_WINUNICODE for unicode character set. *** Issue 66342 has been marked as a duplicate of this issue. *** This problem can be observed with OOo_2.0.4. This issue was firstly reported at the time of OOo 1.1. At least, within the integrated, productive office suite, e.g. Calc to Calc, we should try to solve this issue. There are two different ways to link a cell or range across files. Scenario A: This works. 1. Create two empty spreadsheet. (1st and 2nd one) 2. Fill a cell with a non-Western script in the 1st one. 3. Move to the 2nd one by clicking the 2nd one's window. 4. Type an equal mark to enter to edit mode. 5. Click on the title bar of the window of the 1st one. 6. Click on the cell in the 1st one prepared with the step 2. (*1) 7. Click on the title bar of the window of the 2nd one. 8. Type an Enter key to go out from the edit mode. You would see a non-Western text was correctly displayed. *1: If you select some cells, i.e. a range, type a Ctrl-Shift-Enter in the step 8 to treat them as a matrix. Scenario B: This does not works. 1. Create two empty spreadsheet. (1st and 2nd one) 2. Fill a cell with a non-Western script in the 1st one. 3. Copy the cell. 4. Move to the 2nd one by clicking the 2nd one's window. 5. Edit - Paste Special with a check mark on the option "Link" You would find a non-Western text was displayed with garbage characters unexpectedly. Thanks. *** Issue 72695 has been marked as a duplicate of this issue. *** *** Issue 73246 has been marked as a duplicate of this issue. *** Ok, to confirm this problem. I can confirm as well that the problem appears when UTF-8 encoding is used as the system default. I used to use es_MX as the locale and this problem never occured (I have a series of documents linked together with DDE Links) and I converted to UTF a couple months ago and today I revisited those documents to see them all garbled! The most ironic thing is that I converted to UTF-8 because of openoffice! See bug #61732, when the locale is not UTF-8 enabled, Openoffice doesn't handle international characters in filenames! So I ran from one problem right into another. And this one is quite disastrous. I'm trying kohei's suggestion (patch in sc/source/core/tool/ddelink.cxx) but it will be a couple of hours more of compiling before I know if it works or not (it's easy to compile the patched sources, I'm running gentoo) Anyway, i think this is a serious problem and it's hard for me to believe that it hasn't at least received a target milestone or even changed the status to confirmed (taking into account it's been active since 2003!! and with a lot of dupes!!) What can I help with?? Is there any way we can get this process moving? Thanks for the great work... really... this is a great application (and as such it's supposed to have its glitches ;) Oh my gosh!! It works!!!! It actually makes sense to get the system wide encoding (through gsl_getSystemTextEncoding), instead of using text encoding (through DDE_TXT_ENCODING) I definitely am not an openoffice developer or anything for saying that, but what I know is that it works!! Ok, any chance this could be incorporated into openoffice so I don't have to be patching the sources everytime??? (I mean, I'm happy to do it if it's what's required to keep things working, but changing one line in the main trunk makes sense when it solves a problem like this one!) Thanks, tora, kohei, and everyone for the great work!! please attach the patch extended the summary to make it easier to find.. *** Issue 65867 has been marked as a duplicate of this issue. *** *** Issue 77749 has been marked as a duplicate of this issue. *** *** Issue 27009 has been marked as a duplicate of this issue. *** Note that as said in issue 65867, this is a Framework bug, not limited to Spreadsheet. Confirmed on 2.2 en-US version with French language pack (official RPMs) under Ubuntu Dapper, locale is fr_FR.UTF-8 : When special characters are pasted as DDE link (either coming from Calc or Writer) into Writer (or Calc), they're not correctly displayed, here is a sample of the result : € -> € £ -> £ ¤ -> ¤ µ -> µ $ -> $ (OK) @ -> @ (OK) Also reported here : http://www.oooforum.org/forum/viewtopic.phtml?t=39424 (under 2.0.2 but OS has not been given). Also confirmed under Mac OS X 10.4.9 here (in French) : http://www.forum-openoffice.org/forum/ftopic5440-0-0-asc-.html To cloph: Sorry!! I actually don't have a patch by itself. What I did was that I edited the file sc/source/core/tool/ddelink.cxx itself by replacing the word DDE_TEXT_ENCODING with gsl_getSystemTextEncoding() but I did all that manually. Don't have a diff to apply with patch.. Would be more than willing to do that the only thing is I don't have any idea of how to do it :( (I think I would type it in an editor but then It would be extremely probable that I would type it with errors on it) Mede Created attachment 45821 [details]
Patch to file sc/source/core/tool/ddelink.cxx
Well.. as with anything in this great linux world. Put your money where your mouth is :) (no matter how small this may be) I've attached a patch that is FAAAAR from perfect. It says "missing header for unified diff at line 3 of patch" but corrects the file with the command: patch -p1 ddelink.cxx patch.ddelink It has to be run from the directory where the file ddelink.cxx is located Comments appreciated ;) Mede Created attachment 46045 [details]
Corrected Patch to file sc/source/core/tool/ddelink.cxx
hehe... Just as expected.. The previous patch didn't work at all.. I had spaces instead of tabs so the patch didn't get applied correctly.. I've just compiled openoffice.org 2.2.1 with the above patch which was applied, being in the directory of ddelink.cxx with patch ddelink.cxx patch.ddelink -b (The -b just for precaution, creating a backup) and everything worked alright.. Openoffice compiled without a hitch and dde links of special non-ascii characters work as expected.. If only someone with the right priviledges would erase the previous patch (which is wrong) Thanks! Mede Thanks for the patch, I change the issuetype accordingly, so it appears in the developer's radar more prominently. Unfortunately, IssueZilla doesn't allow deletin of attachments by non-site-admins, and doesn't offer a way to flag attachments as obsolete or similar. *** Issue 81049 has been marked as a duplicate of this issue. *** Hi Niklas, what's about 2.4 ? Frank For completeness see also issue 15540 in Writer and for possible DDE scenarios see http://framework.openoffice.org/servlets/ReadMsg?listName=dev&msgNo=1108 DDE encoding may need to be addressed OOo wide. As this is something that at touches Writer at least remotely I would be interested to hear something about the status of this patch. Is it rejected? Will be take it? Does it need some rework? Depending on the answers to these questions either the target or the type of this issue might need a change. I am suffering from this issue, are there any chance to get in the game? Attached is another patch which fixes (in the same way) DDE encoding in SC and SW. Of course we assume that source and target applications use the same encoding (not hard coded cp1252 as now). This is the easiest way but it works in most of cases (tested :) ). BTW, can we mark issue 15540 as duplicate of this issue? Created attachment 51643 [details]
Proposed patch
Using gsl_getSystemTextEncoding() should indeed fix the most common DDE usage, linking between applications on the same machine and user account. It may break though reading from remote hosts and the other scenarios mentioned in http://framework.openoffice.org/servlets/ReadMsg?listName=dev&msgNo=1108 However, I think that having the common scenario fixed is more important than corner cases, with one exception: DDE to services on remote hosts that deliver "real time" data in Windows-1252 or cp-850 encoding. @mba, @nn: do we know of any remote host scenarios actively used by customers that rely on encoding? I only remember one case on OS/2 that was about 12 years ago.. I'll mark issue 15540 as a duplicate as you can't fix them independently and this issue here has more details. *** Issue 15540 has been marked as a duplicate of this issue. *** Any chance to retarget to 3.0? @er: I don't know any customer relying on such remote scenarios. But I'm not sure if that tells us anything. ;-) So what is the decision - solve the problem or not? I still see this issue, which was reported six years ago, in version 2.4.1 on Ubuntu Hardy, LANG=en_US.UTF-8. This is a serious issue, which makes OOo unsuitable for use in non-English speaking environments where users must link spreadsheets containing text (which one would assume is pretty common use case)! I can confirm that in ubuntu Hardy with locale "es_ES.UTF-8" (OOo.org 2.4.1), doing a "Paste special" and link, the result is that I get bad encoding **using different files**, but inside the same file, the encoding is ok. Sorry if this last detail has already been supplied (I didn't fully read all past comments). I wonder: do OOo.org documents store the used encoding anywhere? I think that relaying on the environment locale is not a good idea (if this the internal behaviour). Please fix this issue. It happens also in: - OSX (2.4.1 and 3.0rc1) - Languaje: spanish when doing a paste special->link from diferent file. CP_WINANSI stands for "default codepage", so using gsl_getSystemTextEncoding is correct. On Windows systems in western Europe and America, this is MS_1252, which we used before, so we don't break anything there. In other regions, it seems unlikely that someone's relying on MS_1252 characters from some DDE server that doesn't follow the rules. If mba or os doesn't object to the Writer part, we can use this patch (except that swdtflvr.cxx could use a temporary variable instead of the repeated gsl_getSystemTextEncoding calls). I aggree to the changes in sw (with temporary variable) I added the last patch (plus the temporary variable) to CWS "calc47". I would like to say: "Thank you!" all those who worked on this issue Reassigning to QA for verification verified in internal build cws_calc47 I tested DDE link in DEV300_m40 built (Windows XP SP2) and the problem is still there. Instead of āāāā is see ????. As far as Issue 9709 concerned in version OOo-dev 3.1 .0 (OOO310m7 Build:9393) for Windows XP the problem seems to solve for writer, but for calc the problem still remains. See attachments. Created attachment 61342 [details]
Writer OOo-dev 3.1 .0 (OOO310m7 Build:9393) for Windows XP
Created attachment 61343 [details]
Calc OOo-dev 3.1 .0 (OOO310m7 Build:9393) for Windows XP
Vladimir, please provide sample .ods, as I can't repro your scenario using either m44 or m8. Thanks. This issue is closed automatically and wasn't rechecked in a current version of OOo. This fixed issue should be integrated in OOo since more than half a year. If you think this issue isn't fixed in a current version (OOo 3.1), please reopen it and change the field 'Target Milestone' accordingly. If you want to download a current version of OOo => http://download.openoffice.org/index.html If you want to know more about the handling of fixed/verified issues => http://wiki.services.openoffice.org/wiki/Handle_fixed_verified_issues Sorry this issue was wrongly closed. This issue will be reopened automatically. And will be set after that back to fixed/verified. Set to state 'fixed'. Set back to state 'verified/fixed'. Again. Sorry for the mass of mails. This issue is closed automatically. It should be fixed in a version with is available for longer than half a year (OOo 3.1). If you think this issue isn't fixed in the current version (OOo 3.2) please reopen it. But then please pay attention about the field 'target milestone'. The closure was approved by the Release Status Meeting at 22nd of February 2010 and it is based on the issue handling guideline for fixed/verified issues : http://wiki.services.openoffice.org/wiki/Handle_fixed_verified_issues |