Apache OpenOffice (AOO) Bugzilla – Issue 26466
dead-key diacriticals don't work in OOWriter
Last modified: 2006-10-09 16:17:55 UTC
I can't find these in existing issues, and I queried the OOWriter forum and got no responses. I use several fonts that insert diacritical marks by means of 'dead' (non-advancing) keys, or the equivalent in four-digit codes on the num pad. These work fine in MS Word and in my music writing program (NoteWorthy Composer), but when I try to use them in OO Writer whatever is assigned to the key in some other font, alphanumeric or symbol, is inserted after the letter, instead of the diacritical appearing on the letter. I would attach examples, as I read that attachments are permitted with these issue submissions, but hanged if I see anything on this page that would let me do it. Stephen_R
Created attachment 13790 [details] How they look in MS Word
Created attachment 13791 [details] How they look in OOWriter
MRU->US: Fonts, pls have a look.
Fonts, have you looked at this yet? Stephen
@submitter: could you pls. attach a screenshot how it should look like. Thx.
Hello, fonts, Good to hear from you. I can't get the "print screen" button to work, but the sample from MS Word shows how it should look. This problem does not exist in Word, and I have encountered it so far only in OO Writer; so to see how it should look, just refer to the attachment with the text in MS Word. Stephen
As far as I have seen (from a similar issue with linux), OOo will not correct problems with environment variables which define a locale. In linux, you have to edit a particular file (/etc/I18N) to include LANG=en_US. Once you've done that you can access any keyboard you want. I suppose you could use any other supported LANG variable. But the developers have replied that they do not intend to have OOo use any of the workarounds that almost all other applications use so that deadkeys will work. I don't know if that is truly policy or was something peculiar to the developer who looked at my issue. If you cannot find a way to introduce the correct setting into the Windows ME enviroment you will be unable to use the deadkeys in OOo.
That's beyond my competence; I would need step-by-step instructions to get the correct setting into ME. It is disappointing; I will now have to use fonts that do not use dead keys, but all of these have horrible keyboard layouts and I will have to entire each letter with the four-digit code. They have to have so many symbols to get all those diacriticals in that Janko doesn't have room and can't show the layout, so I can't use the keyboard generator to make the keyboard useful. So I won't be able to do texts of any length; if I need to do long texts, I will have to forget OO and go back to MS Word. I really like OO, and I can't understand why they don't want to accommodate the dead-key fonts. Stephen
It's not as hopeless as all that. Join the users' list and ask your question there. Someone is likely to know how to set the environment on Windows ME so that the marks will work. It's not a guarantee, but there's a lot of knowledge represented on that list and that's how I discovered how to fix my own system.
OK, Blackeagle, thanks for your help. I did take this to the users' forum first. The query sat there for two weeks; some people looked at it, no one replied--no one at all. That's when I came here. But it certainly won't do any harm to go back to forum with what you have told me; there is more to go on, and perhaps I will have better luck this time. There was a Black Eagle Hotel in Uzhhorod before the Second World War (then in Czechoslovakia, now in Ukraine); I don't know if it is still there, but it was the place at one time, I thing founded in the 19th century. Stephen
The problem here is not the insertion of the letters, but the display of the fonts. Both documents contain the same "letters". The locale thing is on linux. When the locale is set to C or POSIX dead keys won't work since these are ASCII-Only locales and thus OOo only allows ASCII. But this is not the problem here, so you don't have to look after locales. OOo on linux displays the file the same way as MS Word viewer, so this is a windows-specifix problem. Only tested with SPIonic (from <http://www.monachos.net/other/fonts/get_greek_font.shtml>), since I didn't find Staroslav-font with google, but I assume the problem is the same. just open both the doc as well as the sxw in OOo... The font SPIonic uses the latin-area for the glyphs. (ttmkfontdir needs the -c switch, otherwise it will ignore it) since this is a font-issue I'll reassign this to gsl. I strongly suggest to use unicode fonts though (makes the text independent of the font, now if you want to share your document, the one who gets the doc has to install the font to view the contents). You may consider setting up linux if you cannot configure a decent keyboard layout on windows. Linux is very flexible in this regard.
Created attachment 14462 [details] Word Viewer and OOo both showing the greek text..
Switching to linux (as I have) is a major decision for a Windows user. At first there is a lot of concern about compatibility with applications. It does not matter that one can have the same functionality eventually, the interface is different enough to leave a new user confused so, while I think linux is both more flexible and has more functionality than Windows, it is not something I recommend when we are trying for strong multi-platform compatibility. A point on asking on a list: ask more than once if you don't get an answer within three days as sometimes you will miss people on vacation, who expect someone else will reply or are heavily focused on another problem. I understand the issues in getting this to work on linux. In my case, I had to edit the /etc/I18N file -- not something I would ever have thought of had it not been for another user who told me where to put the environment variable. Two things seem to be going on simultaneously: one is to have some sort of "workaround" for both Windows and linux, acknowledging that it is not necessary for most Windows and linux implementations. The other is to provide the information for the geekier of us to modify our systems to provide support for multiple keyboards right now. Another change is that version 2.0 won't have that problem and cloph's suggestion is a valid one. Unicode fonts are far freer from errors, but it still doesn't really resolve the multiple keyboard issue. For example, I use Times New Roman a lot. It has the umlauted vowels and they are accessible from an English w/deadkeys OR a German keyboard in every application except OOo. I find that a bit strange, but I also note that it is peculiar to those few of us who use multiple keyboard configurations AND to certain implementations of our OS. I have asked our distro "manufacturer" to put the fix in the next version.
@robert: Don't get confused by the "dead-keys" here. In this case it is not the problem that entering dead-key-accented letters like one can see with wrong locales on linux. Here the letters can be entered, but for some reason OOo on windows uses a wrong font to display the characters. As you can see from my screenshot, the word-doc and the sxw both show the same text (on linux). The problem here is that the fonts used here don't display an "s" when you enter an "s", but a sigma in the SPIonic (greek-font case). So when the "s" is taken form another font, you will get the "s" instead of the sigma. And I didn't recommend linux to circumvent the current problem, but to ease the use of "real" fonts - or more precise: You can modify your keyboard layout in such a way that it matches the SPIonic-mapping: Pressing the key labled "s" on your keyboard will result in the sigma (only an example) On windows I don't know an easy way to remap keys, that's why I mentioned linux. The locale problem will probably be solved with OOo 2.0 when OOo automatically replaces the C or POSIX locale by a fallback (but again, this is not the problem here)
*** Issue 28990 has been marked as a duplicate of this issue. ***
*** Issue 25050 has been marked as a duplicate of this issue. ***
But hey, this is interesting. OOWriter 1.0.1 could handle dead-key insertions, but 1.1 cannot? I would FAR rather go back to the old version than lose the ability to use SPIonic and Staroslav! Stephen
I looked for a site with 1.0.1 for download; they are all gone or converted to 1.1.1. Since 1.0.2 was a bugfix, if dead-key entry worked in 1.0.1 it should work in 1.0.2. But of course I can't find a download site for that either. So how can I revert to 1.0.2? Assuming that I really can use the fonts with it, of course. Stephen
Instead of reverting back to OOo 1.0.x you may consider switching your OS, linux doesn't show the problem ;-> (not meant as a honest advice, we had that already, so please forget immediately :-) If you want to stick with OOo 1.0.x, then you should look for version 1.0.3.1 which should be availabe on the mirrors (in stable/1.0.3 for the english version and in localized/<language>/1.0.3/)
I downloaded OpenOffice 1.0.3.1 and found that it does indeed accept the fonts I need. So I have removed 1.1 from my computer and will stay with 1.0.3 until such time as OpenOffice decides to restore the use of dead-key entry. Since 1.0 had that capacity, it should not be a major thing to restore it, but there is always the attitude that since few people use such fonts, to hell with them. I hope OO will not go that way. Meanwhile, those who do want to use these fonts have a solution in older versions of OO. Stephen
cp->hdu: glyphs with zero advance in windows 9x/me. probably duplicate, please have a look
I note that this is indeed a regression as it also happened on my linux system. 1.0.3 works just fine but not 1.1.1 or later. I suggest you try to set your keyboard options so you can select a "w/deadkeys" or foreign language keyboard format. To make sure you can map keys correctly, type out a sample document with the keys on the keyboard and the remapped keys on a second line for a quickie reference. See if this works for you. The settings should be under one of the "My Computer" options.
Robert, most probably this has nothing to do with keyboard settings. This is a problem on non Unicode Windows operating systems. The feature that makes languages like Arabic, Hebrew, or Hindi work caused exactly this regression. There is no easy workaround or fix since the Windows 9x/ME API is broken in this regard.
Now I *am* confused. If the problem exists in some versions of Windows only, they why does Mr Blackeagle find it with Linux? A hypothesis: in OOo 1.0, "languages like Arabic, Hebrew, Hindi etc." could not be displayed; the steps needed to fix this in OOo 1.1 involved buggering up the dead-key fonts. And because of bad programming in Windows there is no way (?) to get both to work. Is that it? And what do right-to-left Arabic and Hebrew have in common with left-to-right Hindi except non-Roman alphabets? Would someone please explain "Vote for this issue"? I would like to vote for a *solution*. Anyhow, I have now gone back to 1.0.3.1; does that mean I can't do "languages like Arabic, Hebrew, Hindi etc." in OOo and will have to rely on MSWord for them? Will I need a Greek-and-Slavonic word processor and a different Arabic-and-Hebrew-and-Hinde word processor?
There is a lot confusion here... I hope this explanaition makes the situation understandable: The problem with zero-width glyphs was introduced by the "glyph fallback" feature. Glyph fallback means, that if the selected font does not support a unicode required to display a text an alternative font gets used. The glyph fallback feature has not much to do with the feature to support complex text (Hindi, Arabic, Hebrew) except that the more international use OOo gets the more desirable the glyph fallback feature becomes. Now the problem is just to find out whether a font supports a special unicode. On UNX platforms this can be easily determined because there we have tight control over the fonts. On WIN systems we use the system API of the last supported version W98, which does not directly have the required APIs. Since the question whether a unicode is supported needs to be answered a gazillion times per document layout/refresh/whatever getting a good answer fast is quite essential. The heuristic we currently use treats zero width glyphs as candidates for glyph fallback. Except for very few fonts it works quite well. Unfortunately not for all fonts. The effort to get this right for more fonts we will have to directly read the relevant parts of the font files. Getting this right under all circumstances without impacting the performance for the >99% of the customers who don't use the fonts is what makes the fix expensive. BTW the root cause is the same as in issue 26434.
hdu, this is a quite clear explanation. I take it that the creator of the font can arrange it so that the zero-width glyphs work right, and that the designers of the dead-key entry fonts I need have not done so. Well, I shall simply rely on 1.0.3.1 until this problem is fixed, if ever it is. Now I can use the fonts in OO Writer, which is what I was mostly concerned about. (I also have problems getting some fonts into PDF format, but that is not Open Office's doing, and those fonts I can replace easily.) I have been disgusted for years and the paleolithic state of affairs in this respect, esp. with the inability to send anything other than straight Latin alphabet by e-mail with any confidence that it will come out right at the other end. A single universal standars is long overdue. Stephen
let me just add to the explanation that the Linux problem rblackeagle is referring to, has nothing to do with this issue. On Linux you might have problems *entering* certain characters, depending on the selected locale. On Windows 98/ME you might have problems *displaying* certain characters, depending on the selected font. The technical background of both issues is quite different.
I don't want to undully add to this, but I believe two separate issues are being discussed here: the display of characters entered and the entering of characters. The display problem is being handled under more than one already reported issue. The entering of the proper character in a given font is another problem. One user reports that numeric codes can be used (a better solution in many cases) -- this is an OS feature, as I recall. KDE permits easy addition of multiple keyboard layouts that can be selected pretty much on the fly. I believe that most versions of Windows do as well, but do not know where they are located (likely under "My Computer"). The problem that I labelled a regression is that 1.0.1 and 1.0.3.1 all supported alternate deadkeay substitutions -- that is, I could enter the English International keyboard (English w/deadkeys) or German (my most commonly used foreign language) and enter the characters directly form the keyboard. They display correctly in Windows when the file is saved to Windows. However, since 1.1.0 was first introduced, that functionality simply vanished. Now I had to know an obscure file, edit it in a non-intuitive way, and then the alternative keyboard layouts would work to get diacritical marks and special characters. Apparently no one knows just what the workaround is for Windows. This is a keyboard entry issue rather than one of display, so far as I can tell. The keyboard (let us say, Greek) has the characters and fixing the weird file in linux enables it to input the proper code, but nothing can be done to get the proper code into Windows applications as yet. I was hoping for at least a workaround to get this fixed for Windows applications so it would be as usable as 1.0.3.1. I appreciate the clarificaiton of the display problem, though, as it makes it clear just what the issues are that the developers are facing for that one. It still does nothing about the entry problem. I cannot imagine going to Insert > Special Character for an entire text of Greek or Hindi quotation, let alone for German or Spanish or Hebrew -- that is simply asking too much of any user.
Numeric codes can be used, but not with the same font. Fonts that use dead-key insertion do not also have a collection of single-glyph letters-+-diacriticals. A font that uses the latter method can theoretically be employed, with numeric code entry, but in practice I cannot do this, for a reason I find absolutely ridiculous: these fonts have a completely irrational keyboard layout. This is not just a non-qwerty layout such as Dvorak or the standard Russian keyboard; if you just run through the keys in order, what would be `1234 to bnm,./ on qwerty, you see a horrid mishmash of upper and lower case with and without diacriticals. In other words, the rational method of typing the text until you come to a letter with diacritical, using the numerical code for this, and then returning to the keyboard until the next diacritical turns up, cannot be used. Every maternally fornicating letter must be put in by numeric code, and this is *worse* than having to use Insert Special Character for every letter. I know that the keyboard layout can be revised. The programs that I have cannot be used to rationalize the fonts in question. These programs display the entire font and the keyboard to permit particular letters to be associated with particular keys. However, there are so many letters that must be included in such a font (in Greek, for example, seven vowels, each with all possible combinations of two breathing marks and three accents and most three with all these combinations plus subscript iota) that the programs *cannot* display them all. I will try other such programs as time goes by, but so far I have not found one that can accommodate such a font. So numeric-code entry is not an option at this point. I expect that I am missing some improvements in going "back" from 1.1.1 (which I have entirely deleted) to 1.0.3.1, but I can once again use OO Writer for my purposes and do not have to rely on MSWord, and for that I am grateful. I hope OOo developers will address this problem, although this thread makes it clear that it will be a source of rectal discomfort for them. What we really need is beyond the scope even of OOo: a single universal encoding, so that one can send e-mail in any known writing system and any combination of known writing system—say, Meroitic, Tengwar, Russian Old Orthography, and Hangul in the same message—and be as confident that it will come out right at the other end as we now are in plain English. Then font designers will have a stable set of presuppositions and Slavonic and Greek can be transmitted as easily as English is now. Of course, if they are still so irresponsible as to foist off a bloody mess of a keyboard layout on us, we will continue to have headaches. Stephen
> What we really need is beyond the scope even of OOo: a single universal encodin The encoding aready exists: Unicode. OOo can use unicode, your fonts don't. If you really were concerened about exchanging your documents (the contents), you should use a unicode font and drop your currently used fonts. These fonts were a workaraound for the times where unicode-awareness was rare. These are no longer necessary. You should not enter "sou" if you really mean <sigma><omicron><nu>. You should enter "σον" instead (don't know if issueZilla can handle this).
The fix for the zero width glyph problem in issue 21821 also fixes this one. Need to verify once the CWS fontlists02 for target OOo2.0 is integrated into the master. The other problems mentioned in this issue with input methods, encodings, etc. need to be addressed in other/new issues.
OK, if I understand correctly issue 21820 has a fix that will enable me to use fonts with dead-key insertion in OOo 1.1. But what I find there is a little collection of initialisms--CSW and so on--that are as clear to me as if the thing had been written in Yupik (sorry, Yupikophones, but I don't understand the language at all). Meanwhile I have gone back to 1.0 and have no trouble with the fonts, so I don't want to disturb my present arrangement without very good cause. I recommend beefing up the discussion with a step-by-step rundown on how to accomplish the fix in 1.1. If I know what to do, I may tackle it, but if all I have is something only a tech could understand, I will stick with what is now working for me. I did have some further font problems, but they turned out to have nothing to do with OO. The problem was with pdf995, and Ghostscript fixed it. Stephen
> OK, if I understand correctly issue 21820 has a fix that will enable me to use > fonts with dead-key insertion in OOo 1.1 Again: Your problem is NOT the insertion of the dead-key diacritics. Please see the screenshot I attached. Both the doc as well as the sxw contain exactly the same characters. Your problem here is the *display* of the file that is different on windows compared to linux (and to the one in OOo 1.0). Your problem has nothing to do with locales. Your problem has nothing to do with dead-keys. >I recommend beefing up the discussion with a step-by-step rundown on how to >accomplish the fix in 1.1. You have a couple of solutions: 1st: Dump your currenlty used fonts that misuse the character codes (e.g. your fonts have greek characters where there should be latin characters). Use fonts that contain the greek characters at the appropriate character code. (you're apparently not willing to do so) 2nd: Use OOo on linux. Your file will look as expected. (you're not willing to do so) 3rd: Wait for OOo 2.0 to be released and use OOo 1.0 in the meantime.
Cloph, I do understand that the problem is zero-width glyphs and that if I had fonts that put everything where it should go it would work fine. But MY problem damn well IS deadkey entry, because the fonts I have are the fonts I have and with them I can't do deadkey entry in 1.1 and get the appropriate displat--yes, the reason is the zero-width thing, but what I want to do and can't is to get the right display with deadkey entry. You assume that I can get fonts that put greek resp. slavonic letters where they ought to go. I require polytonic Greek, and Slavonic with all the bells and whistles, although for the latter I can *sometimes* use a grazhdanska font. As I explained in an earlier message, I cannot do this because the Greek fonts with all the stuff are too big to fit into my present font-diddler program. Slavonic fonts are all, every last one of them, nutty in some way or other, and there are not many of them out there--*Cyrillic* fonts are thick on the ground, but 98% cover only the official new orthographies of the national languages--hell, I can't even do Carpatho-Rusyn right with them, let alone Slavonic. My willingness has little to do with it. If I were 20 years younger I would learn the craft and generate my own fonts. I doubt that you have faced this problem. And my willingness to adopt Linux, similarly, depends on my acquiring a degree of expertise that would require a deal of time. I may do this--believe me, I am not greatly enamored of Bill G's stuff. But that will have to wait. The third solution will have to do, and since OOo 1.0.3.1 is not at all a bad word processor. I look forward to 1.2, but I can wait as long as necessary as long as I have 1.0.3.1. So thanks for your help, and I did learn in this thread the solution to my problem. Stephen
No, this problem doesn't have to do with zero-width-glyphs. The reason why the other issue fixes this one as well is because it changes the way how OOo looks for fonts on windows. > But MY problem damn well IS deadkey entry, If the problem was /entry/ of the characters, how do you explain why the file looks as expected on linux? Do you think OpenOffice.org can guess what you did want to type? You successfully /entered/ the characters. They were saved in the file. The problem is that OOo did not use the selected font to display the characters *on windows*. On linux, it takes the right font and everything is fine. > I cannot do this because the Greek fonts with all the stuff are too big to > fit into my present font-diddler program. ? What fonts did you try? Gentium is <350K Courier is ~300K so not that much of a difference. You don't need to use fonts that cover the whole unicode-area like Arial Unicode (that is ~20MB). It is sufficient if the encoding is unicode. If you ned Archaic greek as well, see http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=SILgrkuni (font is ~150K) for cyrillic and roman scripts see http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=DoulosSILfont (font is <800K)
OK, I did download the Unicode fonts. It looks as though I can do polytonic Greek this way. Slavonic is another matter. I have SIL's Doulos, and what I find is this: In the Unicode Cyrillic table, columns 040 and 045 contain the letters specific to Serbian, Bielarusian, and Macedonian. Columns 041 through 044 contain the letters of the Russian New Orthography. Columns 048-04F cover the various Central Asian languages that employ Cyrillic. (Ukrainian letters are scattered here and there among 040, 045 and 049.) Komi is covered in 052F. *All* of these are included in the Doulos font. Slavonic, and old-orthography vernaculars, require columns 046 and 047. Doulos skips over these. It is consequently of limited use to me. Granted, the number of us who need to do Slavonic is not large; is it smaller than those who need Komi? In any case, for Slavonic I still need to relay on Staroslav, with its zero-width glyphs, so I must continue to use OO 1.0 and hope that OO 2, when it appears, will have resolved the zero-width problem, or else that someone will come out with a Unicode font for Slavonic. Stephen
You could use a version of OOo newer than SRC680m48, which has the zero width glyph problem from issue 21821 fixed.
*** Issue 35243 has been marked as a duplicate of this issue. ***
*** Issue 38859 has been marked as a duplicate of this issue. ***
marking this as fixed for 2.0