Carsten, was anything important changed wrt. menu and toolbar merging between
2.3 and 2.4?

add keyword : regression.

how about other performance comparisson between 2.3 and 2.4 m 239? 

Thanks

cd->mba: Nothing at all has been changed between 2.3 and 2.4 for merging or in
the menu implementation. From my point of view it's clear that Java extensions
can hit menu performance to a wide degree. The Java VM must be started to get
the dispatch objects when you try to open the menu. So for the first time
opening a menu can be very slow. That's why I currently don't like Java
extensions which integrate into the menu.
I already described possible performance problems in the merging documentation.
If you increase the number of extensions the problem can get worser. Every new
extension costs a little performance.

So there would be a regression only if the *same* extension created a
performance problem in 2.4 that didn't exist in 2.3 (and even then the bug could
be in the extension). Are you really using the same instance of your extensions?  

A recommendation for Java based extensions: if they have a toolbar, try to use
stateless buttons. Otherwise OOo will need to instantiate the extension at
startup time and so starts a JVM. If you have menu entries, try to put them into
a sub menu so that the extension will not be loaded before the user
intentionally accesses one of these entries.

cd->brinzing: Please describe a reproducible scenario where you can see the
performance problem only on SRC680m239. I need to know which extensions (they
must be public available) you installed to reproduce your problem. The best way
would be add these extensions to this issue. I already used a SRC680m239 with
some Java extensions and wasn't able to see any obvious performance problem.

Hi,

> cd->brinzing: Please describe a reproducible scenario where you can see the
> performance problem only on SRC680m239.

i just reinstalled oo 2.4 m239 and deployed the two responsible extensions

- 6 addons.xcu files with images and calc/writerwindowstate.xcu
  (merged into the Addons.xcu, filesize about 80 kb)

- c++ services (Protocolhandler, about 10 c++ dll's)

so i don't think, java is responsible ...
is it possible, that some debugging code is in the m239 build ?

if i copy the complete .\share\uno_packages directory
to my oo 2.3.1 installation everything works fine ...

i will install an older oo 2.4 build over the weekend and try to
give you more information ...

thanks for your help ...

Oliv€er

I have noticed a very similar, sever, performance degradation with m239.

In trying to investigate I attempted to disable the "Sun Template Pack " and
this caused OOo to stop responding under Windows XP.

Once the extensions were disabled, after a restart, the menus are still slow to
refresh.

Then, I removed all extensions and performance returned to the level I was
seeing previously ( i.e. m239 performs at least as well as OOo 2.3.1 )

I then added "LanguageTool 0.9.1" extension and performance immediately degraded
again.

Further investigation showed that the Sun template pack did not suffer from this
effect.

Language tool integrates a Java based menu item and this would seem to be the
underlying cause of the problem.

>I then added "LanguageTool 0.9.1" extension 
> and performance immediately degraded again

i can confirm this, after installing "LanguageTool-0.9.1.zip" 
the performance is even *much* worser than with my mentioned 
extension's ...

Oliver

Hi,

just installed "OOo_2.4.0rc1_20080211_Win32Intel_install_en-US"
and can confirm again. After installing the "LanguageTool-0.9."
it will take about a second before a menu/context menu will open.

it's impossible to work ...

Oliver

I confirm performance problem after installing the 
extension "LanguageTool-0.91" in OOo 2.4.0 RC1 under Kubuntu 6.06 LTS. 
But I do not have any problem with Bookmarks or Sun Presentation Minimizer 
extensions.
Removing extension LanguageTool solves the performance problem but disabling is 
not sufficient.

My Java version : Sun 1.6.0

@brinzing: 
Does that happen for every menu? 
Does it happpen each time you open the menu or only the first time?

Hi,

>Does that happen for every menu? 

as soon as the extension is installed (no restart required),
the menu's "Edit", "View", "Insert", "Data" and "Window" and Context menu's
are very slow.

The menu's "File", "Format" and "Help" seems to behave normal.

>Does it happpen each time you open the menu or only the first time?

it happens each time if one opens a menu, until i remove the extension
((no restart required) - disabling does not help ...

as mentioned before, the same binaries run with oo 2.3.1 without any problems

Oliver

I can also reproduce this with LanguageTool 0.9.1 and OOo 2.4.0 RC1, but not 
with the development version of LanguageTool. This development version is now 
an *.oxt file, could that make a difference? 0.9.1 is deployed as a ZIP.

Indeed, LanguageTool-0.9.2-dev.oxt doesn't show the bug, the same file (!) 
named LanguageTool-0.9.2-dev.zip makes everything slow. It seems you cannot 
test this with with 0.9.1 as renaming it to *.oxt won't work, so you need the 
LanguageTool version from CVS if you want to reproduce this.

This is very strange indeed.

So the good news is that (as extensions should be oxt files anyway) it seems as
this shouldn't be a persistent problem and that it can be fixed easily be
repackaging the extension. 

OTOH I think we should find out the reason for the performance problem. I hope
that I can find someone with some time on his hands...

> OTOH I think we should find out the reason for the performance problem.

I agree, but i can not confirm that repacking will solve the problem,
my extensions are all *.oxt (including description.xml, manifest.xml),
and i still have this performance problem.

Is it possible that the protocolhandler implementation causes the problem ?
Or is there a special packing required for *.dll/*.jar files ?

Of course protocol handlers can be a problem - but that doesn't explain why
*all* menus are slow, even if they don't contain any "alien" commands. And it
doesn't explain that the packaging seems to make a difference at least in case
of language tool.

You can now test this with our new LanguageTool release at 
www.languagetool.org. Just rename *.oxt to *.zip before installation to see 
the effect.

Can confirm this. Deploying as zip makes everything slow.

Is it possible that someone can debug/compare the *.rdb and *.db files ?

I don't know how to deploy 2 identical packages, due to the 
"uno_packages/CBF5.tmp" path ...

I noticed the "Windows_x86.rdb","Windows_x86_.rdb" and "Windows_x86rc" are
identical. 

The "common.rdb", "common_.rdb", "unorc", "registered_packages.db" and
"Addons.xcu" differ ...

for example:

the "unorc" (zip):

UNO_SERVICES=?$ORIGIN/common.rdb ${$ORIGIN/${_OS}_${_ARCH}rc:UNO_SERVICES}

the "unorc" (oxt):

UNO_SERVICES=?$ORIGIN/common_.rdb ${$ORIGIN/${_OS}_${_ARCH}rc:UNO_SERVICES}

What about the "_" in common_.rdb ?

brinzing -> Does any other extension deployed as zip file create a problem for
you? For example:

http://extensions.services.openoffice.org/project/templatechanger

(I cannot test it myself right now)

my StarOffice installation had 4 extensions installed:
- Sun Google SearchBar (or however this beast is called)
- Sun Report Builder
- Sun Weblog Publisher
- native PostgreSQL driver

With those extensions installed, SO became incredibly slow, after having been
updated to the OOH680m7 build. While I seldom use menus, it's noticeable that
even simplest operations in a spreadsheet document take ages: Move the cursor to
another cell, copy a cell to the clipboard, paste the clipboard content to a
cell - everything is so slow that effectively, working with my spreadsheets is a
pain. In numbers: Between clicking onto a cell and the cursor being painted in
that cell, there is at least 1 second.

Removing all extensions but the (already disabled) search toolbar improved the
situation. Also removing the last extension returned the performance to the
normal situation - my spreadsheets are as responsive as before.

I will try to reproduce this with a cleaner installation (i.e. a fresh 2.4 RC1),
and give more precise steps and numbers. So far, it looks to me as if installed
extensions impose a severe performance penalty to the complete OOo.

Hmm, simply installing all those extensions in my 2.4 RC1 does not yield any
performance problems.

Re-installed the 4 extensions in my StarOffice, which is patched to OOH680m7.
Opened one of the problematic spreadsheet documents. Started Filemon, to monitor
file accesses of soffice.bin. Now behold: Putting the focus to the Calc window,
and back to filemon, gave me the stunning number of *8630* file accesses. Nearly
all of those point to the user/uno_packages resp. share/uno_packages folder, and
their sub folders. (I'm going to attach an anonymized version of the log file.)

Doing the same - simply focus and unfocus the Calc window, with the same
document - after removing all 4 extensions gave a list with exactly *1* entry in
filemon.

Created attachment 51581 [details]
log file showing SO going crazy with thousands of file accesses to the installed extensions

Another test:
- removed all 4 extensions
- closed SO
- remove my user data tree
- start SO
- click through the welcome wizard
- install all 4 extensions, again
- re-start SO
- open one of the spreadsheets in question
=> everything works fine. No performance problems at all.

Conclusion: There's some setting(s) in my user data which cause SO to go crazy,
the more crazy the more extensions are installed.

I really think this is a framework issue (our "extensions" project is way to
understaffed, see for example the "Target milestone" list, which is a joke), and
should be fixed even before the 2.4 release.

Created attachment 51586 [details]
stack of the file operations

can other people who encounter the problem please check the state of the
following setting in their installation:
Tools / Options / OpenOffice.org / General:
  Help ------------------------------------
  [ ] Tips
      [ ] Extended Tips
Are those *check* or *unchecked*? If they're checked: does unchecking them solve
the performance problem?

Do I understand correctly: there are only "Open Directory Access" logs but no
file access? Sounds strange.

Is it possible that something triggers a "Live update" search so that
configuration and UNO services are rescanning the folders? What could cause
sucha a behavior? Maybe we should start our search at the code that triggers
these "scanning" processes. I assume that Stephan and/or Jochen know something
about that.

Why I ask this: In my SO installation, unchecking those options makes the
performance problem vanish. In my 2.4 RC1 installation, *checking* them does not
make the performance problems appear. So, it sounds to me as if those settings
are required for the problem, but not sufficient.

The attached stack has been created by placing a breakpoint in some osl_openFile
function. What it shows is that with every mouse movement, OOo tries to find an
active help text for the content under the mouse. During this, our help
framework seems to check all installed extensions whether they can contribute to
the help. This happens again and again and again ...

Note 1: The to-be-opened file is the manifest.xml of an installed extension,
plus the script.xlb and dialog.xlb, if they exist for the given extension.

Note 2: Attaching with a debugger to the process reveals that with every mouse
move, a high number of IllegalArgumentExceptions is thrown. They originate from
desktop/source/deployment/registry/*/dp_*.cxx. In all cases, the code tries to
determine some media type, fails, and throws an exception. In case of the
dp_executable.cxx, this is also asserted when compiled with debug.

That would match the observations with menus where IIRC also help texts are
searched for. But I have no idea why "zip" and "oxt" may behave differently
here. If "extensible help" could be a part of the problem - maybe we should ask
Andreas Bregas?

To find out about the media types of the content of an extension the
manifest.xml is investigated. Then manifest only contains entries for files
which need to be processed by the Extension Manager. In the absence of a
manifest.xml, which is the case with .zip extension, the Extension Manager need
to scan all the contents of the extensions.This may cost some more time.
I am putting Andreas on cc as well. Maybe he can shed some light on how the
"help" works.

Heureka!

I mentioned I could not reproduce the problem on a fresh installation - I was
wrong. There was a difference in how I started both OOo/SO versions: One from
within an environment where the environment variable "help_debug" was set to
something non-empty, and one where it wasn't. With the variable being empty, the
performance problem happens, with it being non-empty, it does not happen.

Analyzed this with AB and JL and came to the following conclusions:

Moving the mouse requests a quick help text for the Window under the mouse
cursor, if extended tips are ON. This results in a help URL being built, from
the Window's help ID, and requesting a help text for this URL, at the help
provider. The help provider scans all extensions for help content.
Unfortunately, scanning a particular extension triggers a lot of code -
effectively, a complete parsing of the package, which in sum is expensive.

Note that for every Window instance, the quick help text is cached, if it has
been requested before and was not empty. That's where "help_debug" makes a
difference: This variable enables additional debug information in the quick help
texts. As a consequence, *every* Window instance has a non-empty quick help
text, which is retrieved only once, and then cached.


There are different possibilities how to fix this. Since this could be
considered a stopper for 2.4, we agreed to aim for a minimally invasive solution
for the moment. That is, Andreas will introduce a caching mechanism in the help
provider, which will prevent extensions being scanned too often. This will
prevent the most obvious performance problems. Still, some more seldom actions
will take longer than needed, but that will have to wait for a later fix.

So, I assign this issue to AB for an intermediate fix, and target it to 2.4.
I'll nominate it as show stopper for the 2.4 release (which does not necessarily
mean it will be accepted as such).

So could others who reported the problem if it goes away for them also if they
assign "something" to help_debug (lowercase?!)? Or is it necessary to set it to
something useful to get rid of the problem?

thanks a lot for debugging :-)

Is there a change to get back to the OO 2.3.1 behaviour ?

I just disabled the "Extended Tips" and noticed a little better performance
for my *.oxt extensions, but it's not the same speed as in OO 2.3.1.

fs@: IMO this be related to the Extensible Help Project.
For every extension command URL, it will try to find out is there is a tooltip.

This feature is introduced in 2.4, and included in these releases (I can tell
because I'm testing the Extensible Help on 2.4 RC1)

brinzing@:  I agree with you: I also disabled the "Extended Tips", but there is
still a very serious performance issue with menus (NOT related to Java extension
menu items - which require to load the extensions jar  for the first time the
menu is activated-, but also OOo Basic extensions).

The performance only comes back to the one in 2.3.1 when I remove *all* the
extensions (and not only the Java ones); so the new Extensible Help feature is
to blame here.

To all: wouldn't it be more serious to remove this new feature (if this is the
one to blame), and introduce it ONLY once you are sure that it works as expected?

Which new feature? Until now we don't know what creates the problem with the
menu performance. If switching off extended help still doesn't give the same
performance as before it is obviously something different. So it seem we need
some more investigations.

mba@: which new feature? the new Extensible Help
http://specs.openoffice.org/appwide/help/ExtensibleHelp.odt 

This is introduced in 2.4 builds, and is "working" (I was already able to add
some help files to test it in 2.4 RC1).
I think the problem may be directly related to the new tooltips for extensions
feature: if there is an extension installed, now, in 2.4, OOo will look for
tooltips in every extension:
"Context sensitive help is based on Help Ids that are assigned to toolbar items,
menu items and UNO Dialogs / Dialog controls. Toolbar and menu items do specify
a Command URL to bind the item to an action to be taken when the item is
executed. This Command URL is also used as Help Id, both for extended tool tips
as for assigning a help page to F1."



Concerning disabling the extended help, I did some other tests, and it improves
the performance, but in two times I could only feel it after restarting OOo.
Could this be related to the caching mechanisms?:
" Due to caching mechanisms used by the HelpProvider implementation changes in
the extension state may not be visible before the next start of OpenOffice.org."

I'm not convinced that the Extended Help is what caused the problem for the
Language Tool extension, at least the comments that switching it off doesn't fix
the performance loss completely lets me think that their might be something else
we still have to look for.

Besides that I have a lot of confidence that now where a problem has been
identified, Andreas will be able to provide a sufficient fix without disabling
the feature completely.

just tested with language tool: setting help_debug didn't change anything (and
extended help also was disabled).

Looking into the same problem scope by investigating issue 85648.
I can not confirm that a disabled extensible help has any positive effect on the
problem. Also set the debug_help variable does not change anything.

What I noticed however is that the time needed to display the context menu is
rather randomly. It can be from 1 or 2 secs to more than 20 secs. 
This randomness is kinda puzzling me.

BTW in my setting LanguageTool-0.8.4.zip is the only installed extension.

help_debug only helps in contexts where a quick/help text is cached - as it is
the case with Window instances. For places where it is retrieved repeatedly
without caching, it cannot help (The effect of help_debug is merely to ensure
non-empty quick/help texts, this giving a simple cache a chance to cache them.).

For disabling the extension not having any effect: This is also the case in my
scenario. However, debugging revealed that the extended help implementation
first checked whether an extension was actually disabled, and this already gave
a huge performance penalty. So, I don't see a contradiction here.

Also, since Andreas' fix is to cache the information whether a certain extension
contains help content, I am pretty confident that this fix will address all
related problems, since then the expensive query is made once per session,
instead of permanently.

Thomas was talking about disabling the extended help (and so was I). And
following your words help_debug also should have changed something as indeed we
have been testing on Windows.

I hope that the problem described by Thomas indeed also is related to quickhelp.
So let's see if Andreas' fix will also work for Thomas and me.

Ah, the amazing unclarity of words ....

When I said "Windows", I mean "instances of the class Window", not "the
operating system Windows" :)

help_debug helps insofar as it ensures non-empty quick/help texts for all
instances of the class Window, which allows those instances to cache them. (The
"cache" in the class Window is implemented as "when I ever had a non-empty text,
then remember and re-use it".)

help_debug does *not* help where no such caching is involved, so I assume the
performance problem of menus won't be affected by this "workaround".

STARTED

*** Issue 86249 has been marked as a duplicate of this issue. ***

I too found UI laziness (see Issue 86249, now set as duplicate of this one).
Config :
 Windows XP Home. 
 No tips checked in Tools / Options / OpenOffice.org / General : Help

I only had two add-ons (zip packages). Right-click on a Writer text takes one second.

Uninstall both add-ons : Right-click is immediate now.

Install add-on MultiPages.zip (see next attachment, this add-on appears in Calc , 
Draw, Impress) : right-click in Writer becomes slow again.

Uninstall MultiPages.zip
I re-created exactly the same add-on, but as an extension package : MultiPages.oxt 
(see next attachment).
Installation of MultiPages.oxt : right-click is still immediate !

Conclusion : the handler of legacy add-ons is responsible for these performance 
problems.
The default is easily reproducible. 

Created attachment 51598 [details]
Add-on MultiPages (works in Calc, Draw, Impress)

Created attachment 51599 [details]
Same MultiPages but packaged as an extension

ab->bmarcelly: I doubt that we have only have one reason for all mentioned 
performance problems and I don't think it makes sense to set all issues
that look similar or related at the fist glance as duplicate to this one. The
problem resulting from "Extended tips" cannot be blaimed on the legacy
add-on handler as I also can reproduce it without any old zip extension.

So lets be careful with putting too much different stuff into this issue. For now
I will use this issue to fix the Extended Tool Tips related Extensible help pro-
blem. Then let's see which problems still remain.

Andreas, indeed currently we don't know whether the problem you are working on
is *the* problem or only a part of it. But as long as we don't know enough IMHO
it makes a lot of sense to collect more data and information. Splitting this up
in several issues only makes sense if you have identified a particular problem.
As long as we just collect data to help us identifying the root cause(s), I
prefer to do that in *one* issue only. And apparently the issue from bmarcelly
looks pretty similar to this one that was reported by brinzing.

I'm not sure that the "help tips" problem you are working on is the root cause
for this issue at all or if it is just something different accidently found by
Frank when testing with Calc documents. But we won't know that before we have a
fixed version just to test it.

In case in the meantime you feel more comfortable with having an "own" issue for
the help tips performance issue, please create one that is related only and
definitely to that problem.

I think it makes sense to keep this one as the "mother issue" for performance
problems with extensions in 2.4 as this is how we started. 

As 2.4 is due I'm very happy that we get so much help with this problem. A big
"thank you" to all!

ab->mba: Ok, we can handle this as a mother issue and I don't really
need an issue for my own to work on. :-) I just want to avoid the situation
that my fix isn't accepted for this issue because it may not solve all the
problems collected here.

To make my above-mentioned confidence somewhat more concrete, I played a little
bit ...

- fresh OOo 2.4 RC1
=> menus (from the menu bar) open fine, context menus in Writer open fine, no
   performance problems
- installed LanguageTool 0.9.1 (the extension originally mentioned in
  this issue, a ZIP package), restarted OOo
=> menus are slooooow to open, a context menu in Writer takes a lot of seconds
   to open, application is hung meanwhile
- de-installed the LanguageTool extension, restarted OOo
=> performance is back to normal level
- re-installed the extension
- changed the implementation of DataBaseIterator::nextDb to not look for
  help in the shared and user extensions
=> context/menu performance is absolutely fine

Debugged this a little. What happens when a context/menu is opened is a call to
GetHelpAnchor_Impl in sfx2/source/appl/sfxhelp.cxx. There, a UCB content is
created for the given help URL, and then asked for its AnchorName property. And
somewhere within this call getPropertyValue( "AnchorName" "), the help backend
decides to scan all extension packages.

So, I am *really* confident that a caching mechanism, which caches the
information whether a given package actually contains help content or not, will
resolve all performance issues: main menus, context menus, Calc behavior. And
more, probably.

(Note that I think more optimization can and probably should be done later. For
instance, it is not really arguable why asking a given UCB content, which refers
to an existing help topic in the base installation, needs to scan the
extension's help when being asked for a AnchorName property. However, this
should be a different fix, to be on the safe side for 2.4.)

@ab: understood. :-)
But from Frank's great news we can deduce that probably your fix will be
sufficient for all the problems found so far. Great stuff. :-)

cd: Added bh on CC.

*** Issue 85648 has been marked as a duplicate of this issue. ***

It turns out that the fix from Andreas does fix all problems in my specific
issue 85648. (Tested with Linux and Windows)
Thus I have set the later one as duplicate to this one.

Thanks to Andreas for providing the fix! :-)

*** Issue 85648 has been marked as a duplicate of this issue. ***

FIXED

ab->tl: Please verify

TL: Code for fix reviewed and it solves the tested problems.

May be this issue can be closed ?

Sure. Usually I'm not the last one in the lie-time of issues. Thus I missed it.
Closing.

Sure. Usually I'm not the last one in the life-time of issues. Thus I missed it.
Closing.

*** Issue 86105 has been marked as a duplicate of this issue. ***