Apache OpenOffice (AOO) Bugzilla – Issue 12889
Optimize psprint's font collection / use of pspfontcache
Last modified: 2003-04-29 12:49:52 UTC
Hi Phillip, During startup 'SalGraphics::GetDevFontList()' on unix systems does call 'psp::PrintFontManager::getFontList()' to collect the list of fonts installed on the system. Currently 'getFontList()' scans all known font directories and does a 'stat()' system call on every font file, e.g.: open64("/usr/j2se/jre/lib/fonts", O_RDONLY|O_NDELAY) = 14 27450: getdents64(14, 0x000EE4C0, 1048) = 456 27450: stat("/usr/j2se/jre/lib/fonts/.", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/..", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaBrightDemiBold.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaBrightDemiItalic.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaBrightItalic.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaBrightRegular.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaSansDemiBold.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaSansRegular.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaTypewriterBold.ttf", 0xFFBEDAE8) = 0 27450: stat("/usr/j2se/jre/lib/fonts/LucidaTypewriterRegular.ttf", 0xFFBEDAE8) = 0 This strategy is applied to possibly hundreds of font files reachable through the file system. Even worse, at times it looks like this: open64("/usr/openwin/lib/X11/fonts/F3", O_RDONLY|O_NDELAY) = 14 27450: getdents64(14, 0x000EE4C0, 1048) = 1032 27450: stat("/usr/openwin/lib/X11/fonts/F3/.", 0xFFBEDAE8) = 0 27450: stat("/usr/openwin/lib/X11/fonts/F3/..", 0xFFBEDAE8) = 0 27450: stat("/usr/openwin/lib/X11/fonts/F3/afm", 0xFFBEDAE8) = 0 27450: stat("/usr/openwin/lib/X11/fonts/F3/map", 0xFFBEDAE8) = 0 27450: stat("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Book.f3b", 0xFFBEDAE8) = 0 27450: lstat("/usr/openwin/lib/X11/fonts/F3/pspfontcache", 0xFFBED394) Err#2 ENOENT 27450: open("/usr/openwin/lib/X11/fonts/F3/pspfontcache", O_RDONLY) Err#2 ENOENT 27450: stat("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Demi.f3b", 0xFFBEDAE8) = 0 27450: lstat("/usr/openwin/lib/X11/fonts/F3/pspfontcache", 0xFFBED394) Err#2 ENOENT 27450: open("/usr/openwin/lib/X11/fonts/F3/pspfontcache", O_RDONLY) Err#2 ENOENT 27450: stat("/usr/openwin/lib/X11/fonts/F3/Courier-Bold.f3b", 0xFFBEDAE8) = 0 27450: lstat("/usr/openwin/lib/X11/fonts/F3/pspfontcache", 0xFFBED394) Err#2 ENOENT 27450: open("/usr/openwin/lib/X11/fonts/F3/pspfontcache", O_RDONLY) Err#2 ENOENT 27450: stat("/usr/openwin/lib/X11/fonts/F3/Courier-BoldOblique.f3b", 0xFFBEDAE8) = 0 27450: lstat("/usr/openwin/lib/X11/fonts/F3/pspfontcache", 0xFFBED394) Err#2 ENOENT 27450: open("/usr/openwin/lib/X11/fonts/F3/pspfontcache", O_RDONLY) Err#2 ENOENT 27450: stat("/usr/openwin/lib/X11/fonts/F3/Courier-Oblique.f3b", 0xFFBEDAE8) = 0 Here, for every font file, a 'pspfontcache' file is accessed twice via 'lstat()' and 'open()'. There are 3 things wrong with this approach: 1) The ENOENT result of 'lstat()' is ignored, leading to the unnecessary 'open()' call, failing with ENOENT as well. 2) This sequence is re-tried before accessing each and every font, instead of once per directory. 3) The probability that a 'pspfontcache' exists under a system fontpath like '/usr/openwin/lib/X11/fonts/...' seems to be equal to zero, anyhow. These possibly hundreds (if not more) of (file-) system calls during each and every startup do significantly contribute to the total (cold) startup time. As the font installation on any particular system is most probably only slowly varying, I'd like to propose a different strategy to be followed until we can make widespread use of 'fontconfig' or better. The current strategy of collecting the fonts should be used once, only, preferably during setup, but the first startup after setup will do as well. The collected font information is then saved in a 'pspfontcache' file, either in the 'share' or a 'user' directory of an office installation. During normal office startup, the information in 'pspfontcache' should be regarded as 'trusted' and valid, and at a maximum re-verified against the 'fonts.dir' contents of the individual font directories. Only when actually needing a particular font, that font should be re-verified by accessing the respective font file. I'm leaving it up to you to come up with a better strategy than mine. The only thing that counts is to avoid the currently enormous number a file system accesses during startup. Thanks, Matthias
Added target milestone 'OOo 1.1 Beta2'.
The stat and subsequent open is not due to psprint but to SvFileStream; anyhow the cache's contents will now be accepted until the mtime of a font directory changes.
fixed in vcl07
*** Issue 12619 has been marked as a duplicate of this issue. ***
> ------- Additional Comments From pl@openoffice.org 2003-04-01 07:34 PST ------- > The problem is fixed 2 ways: > 1) the behaviour was intended since the font cache (that is the C++ > object behind the file pspfontcache) is supposed to check its > consistency; therefore it checks for files inside the cache and > removes them from the cache if either they are not available anymore > or the have changed modification times This won't clear the fonts from the list if the files are still there because they were only linked with spadmin the first place. They are all still there, I just asked to have there reference removed with spadmin. > 2) the above method of updating was not deemed fast enough and > therefore the cache code was rewritten (see issue 12889) so that the > font cache invalidates itself when a cached directory changes. No, that won't work either because the directory with my fonts won't change any more than the files inside of it. Say, do you have a 2 Mega Byte font cache file that needs clearing? Do you have >1000 fonts on your system? Try linking several hundred fonts, then removing before approving any system for resetting the pspfontcache file. One would have hoped that my use of spadmin where I *explicitly* instruct the system the fonts as no longer used would give the necessary small hint that the fonts are no longer to be used. My issue is not about how often pspfontcache is read nor what it does when reading it, it's about why spadmin does not clear the list. I think 12889 is different from 12619. I can see that if spadmin *removes* a font then subsequently it will get tidied but this isn't the case.
If yor font directory doesn't change then you haven't removed anything, so the cache contents are still valid. If you removed any link inside any directory then the directory WILL have changed, regardless whether you do this with spadmin or by hand, so the cache file will be invalidated. What more do you want ?
<watch_my_lips> When I use spadmin to notify OOo that a font is no longer used I don't want OOo to used it. </watch_my_lips> I used spadmin to "remove" my fonts but they are still in the font cache file and still read on start up. The fonts themselves are *NOT* removed when the linked option of spadmin was used to install. The fonts stay where they where before, during and after the use of spadmin and Ooo.
Don't you say. Since i wrote the thing i actually knew that. But since the links are the only thing that matters to OOo (unless you have the fonts in your regular fontpath anyway in which case it was not necessary to "install" them in the first place) this will remove the fonts out of OOo. The one thing you do not seem to understand is that in OOo 1.0.* the removed fonts are stat'ed, the cache notes that they are invalid and removes them. Since this costs time on every startup it is not done anymore, instead the cache for a directory will be invalidated when said directory changes (e.g. if you add or remove a font or a link to a font). Why would you care whether this is done at the end of spadmin or the start of either spadmin or soffice ? It will cost you the same time anyway.
> The one thing you do not seem to understand is that in OOo 1.0.* the > removed fonts are stat'ed, the cache notes that they are invalid and > removes them. I now understand this is your intention. Please understand it's not happening. > Why would you care whether this is done at the end of spadmin > or the start of either spadmin or soffice ? But evidently it's never being done, not by spadmin nor by soffice. My duty is to report this, not to understand why.
That explains it. I apologize, i did not understand from your description that it does not work; i thought you simply wanted the font cache file to update in spadmin. But i think the new implementation should fix a not working cache, too. I'll check, if it does.
Yes, I'm sorry but sometimes it is very difficult when the dumb user (me) does not understand how the program works in order to explain a problem. I have done another test and in fact the problem is slightly different than I thought - and slightly worse, maybe the new implementation won't fix the problem but I think we are getting closer to understanding it. $ cp ~/OpenOffice.org1.0.2/user/psprint/pspfontcache ~/pspfontcache.before $ spadmin and add a new font (1 new font were added) ^ ^^^^ ;-) Now: $ diff ~/OpenOffice.org1.0.2/user/psprint/pspfontcache ~/pspfontcache.before and wow! 105 new fonts have been added to this file. I seems that *all* the fonts that were in directories use by spadmin to add were added. Sure enough: $ truss OpenOffice.org1.0.2/soffice 2>&1 | grep myfontdir shows the extra 105 xstat()s even though none of these fonts were actually added; this time I only added one, not a full directory. Removal with spadmin does not alter this file and the names remain valid because they are vaild paths to my font storage so the font cache never twigs that they should be removed from the list. Obviously your new way of checking the directory for changes will speed things but it won't get rid of the underlying problem: spadmin added these fonts to pspfontcache and they don't get removed. ....and while on this subject, you do realise there are many repeated entries in the pspfontcache file? For example the line: OpenSymbol;0;5;5;2;65535;917;313;230;0;1262;1230;1262;1230;8 is repeated 574 times. Bloatware alert.
You found another bug with my implementation, so you cannot be a dumb user :-) The pspfontcache file did not actually get truncated when it was refreshed, so extra lines at the end tended to occur. But i got this one nailed while fixing this issue, too. But i'll check this issue with the new implementation, too.
I checked the new implementation and font import as well as removal works as discussed now. Please have a look at beta 2 when it comes out.
> I checked the new implementation and font import as well as removal > works as discussed now. Great! Thank you for bearing with me on this. > Please have a look at beta 2 when it comes out. I wish! Please ask you chums to compile one for me. What is the problem with Solaris-x86? The latest version is 1.0.2 and no SDK either. Why is it any more complex than typing "make"? (It is more complex to me because the makefile is broken with gcc.)
I don't know about Solaris X86, i think that's a question for dev@porting.openoffice.org
changes are in vcl07
Hi Phillip, so far, so good, i.e. looks much better. But one issue still remains: There are some font directories (probably those where you were confused with looking for a 'pspfontcache' file) still show not understandable behaviour, e.g.: open64("/usr/openwin/lib/X11/fonts/F3", O_RDONLY|O_NDELAY) = 18 4935: fcntl(18, F_SETFD, 0x00000001) = 0 4935: fstat64(18, 0xFFBECEF0) = 0 4935: getdents64(18, 0x000EFF80, 1048) = 1032 4935: stat("/usr/openwin/lib/X11/fonts/F3/.", 0xFFBED238) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/..", 0xFFBED238) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/afm", 0xFFBED238) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/map", 0xFFBED238) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Book.f3b", 0xFFBED238) = 0 4935: access("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Book.f3b", 4) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Demi.f3b", 0xFFBED238) = 0 4935: access("/usr/openwin/lib/X11/fonts/F3/AvantGarde-Demi.f3b", 4) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/Courier-Bold.f3b", 0xFFBED238) = 0 4935: access("/usr/openwin/lib/X11/fonts/F3/Courier-Bold.f3b", 4) = 0 4935: stat("/usr/openwin/lib/X11/fonts/F3/Courier-BoldOblique.f3b", 0xFFBED238) = 0 4935: access("/usr/openwin/lib/X11/fonts/F3/Courier-BoldOblique.f3b", 4) = 0 Please have another look at it. Thanks, Matthias
FontCache did not contain directories in which no fonts were found, so they were searched through every time. fixed now in vcl07.
new install sets on vcl07 available
Fix verified on 'cws_srx644_vcl07'. Looks good now!
Verified...
Verified integration into SRX644/m12s1 -> closing.