Issue 4121 - form letter creation is slow and crashes after ~7 docs
Summary: form letter creation is slow and crashes after ~7 docs
Status: CLOSED FIXED
Alias: None
Product: Writer
Classification: Application
Component: code (show other issues)
Version: OOo 1.0.0
Hardware: PC Windows 2000
: P3 Trivial (vote)
Target Milestone: ---
Assignee: h.ilter
QA Contact: issues@sw
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-04-18 16:51 UTC by helmerj
Modified: 2013-08-07 14:39 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Attachment from Juergen (23.22 KB, patch)
2002-07-03 08:46 UTC, h.ilter
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description helmerj 2002-04-18 16:51:38 UTC
When creating a form letter with approx. 30 fields OpenOffice 641.d on windows  
2000 the creation process is extramly slow (output into a new file creates one  
doc every 5 minutes). It seems the app is blanked out white and seems to  
resize from the middle of the screen to maximize several time during the  
creation process for each doc. The app finally crashed due to lack of memory  
after creating 7 output docs. The created docs have two pages of text 
containing ~30 fields.  
each.
Comment 1 stefan.baltzer 2002-05-21 09:37:04 UTC
Reassigned to Hasan.
Comment 2 h.ilter 2002-05-21 10:46:31 UTC
Hello Juergen,

I need more details for your problem.
- Which database and format you use *.dbf?
- What are the properties and options for the print?
- Which printer?
and it would be nice, if you could attach your doc with the 30 fields.
Comment 3 Unknown 2002-06-07 19:41:49 UTC
Something similar is happening with v1.0 on linux. Mail-merging a 14k
form-letter using 4 records from a 14k spreadsheet takes a couple of
minutes and memory usage climbs from 43MB (form letter loaded) through
61MB (printing starts) to 119MB (four pages printed).
This was using the "generic printer" to a postscript file.
Printing a 15-page document (also to a file) used only 44MB -
essentially unchanged from before printing.

The problem seems similar if I print using the PDF converter (running
gs) rather than the generic printer. In addition, there seems to be an
openoffice process hanging after the mail-merge (have not tested this
for reproducibility).

Mergeing the form-letter to new sxw files seems to operate OK without
pushing the memory up (max 61MB).

System: oo1.0, clean Redhat 7.3 with KDE3

If it would be useful, I'll reduce to a test-case.
Comment 4 helmerj 2002-06-24 14:15:56 UTC
  I have tested this on the same platform same computer but this 
time  
I did use thelatest openoffice 1.0. I did use the same documents  
(the main documetn containing the fields and the spreadsheet that I  
impotrted as a data source).  
The outcome is unreliable.   
  
I did this three times:  
  
1. Load the main docu press F4 to connect to the data source, Choose  
output as file for mail merge. Choose create filname from field.  
Choose output dir.  
  
the output dir is ignored (defaults always to the working dir  
specified in the paths setup).   
the first file is created (field name is ignored the first file is  
always called "_0".  
The first file is created empty is stays empty after 20 minutes when  
I shot down the app. In the task manager the spooler is taking up  
all the processor time with out generating any output as the file  
stays at o bytes.  
  
2. same as above but I did use the manual file name option which i  
did set to test. The file are finally created at a rate of one file  
every 6-7 minutes while the app does not respond and use all  
processor time. While this might be fine for a few letters in the  
case of 24 this takes for ever. (MS Office does this in seconds, and  
the OpenOffice under Linux does this at a rate of a file every 30  
seconds. Still slow but bearable...)  
  
3. In order to be able to provide the files for further testing I  
change all delicate or personal) data in them to fake values.  
  
I tested the automated filename generation from field name again and  
like in the original there is no output after 20 minutes when I  
stopped.  
Comment 5 h.ilter 2002-06-27 16:03:28 UTC
Hi Juergen,
I used your attached files and did all what you described with
TurboLinux7.0 and OpenOfice1.0.
The Formletter works well and the first file with the Mr0.sxw isn't empty.
Due to the field name is always the same, Office will number the files
and beginns with 0.

The File target problem in Windows is a known bug and occurs only when
you use the system dialog. 
Choose under Tools - Options - General the own dialog for the workaround.

Please let me know if there is somethink that I forget.
Comment 6 Unknown 2002-06-27 16:43:27 UTC
Is there an issue number for the Windows File target bug?
I couldn't find one...

Thanks (sorry for the spam).
Comment 7 h.ilter 2002-07-03 08:46:15 UTC
Created attachment 2135 [details]
Attachment from Juergen
Comment 8 h.ilter 2002-07-03 08:51:01 UTC
HI->PL: I could not reproduce the formletter problem and have no idea
whats going on there. Hope there is something you know.
I pasted a further mail from Juergen below:

Hi!

I did as you said and it's the same thing. As I mentioned before this
is a 
Windows bug which is why I filed it as such. I have this working under 
Madnrake Linux OO 1.0 as well. I get an output file every 30 seconds
which is 
still considerably slower than mail megering under MS Office but who
cares if 
it works in the end. So in oder to check on this you will have to move
to a 
windows machine if you have one. Even with the settings changes the mail 
merge takes for ever and outputs a file every five minutes.

Juergen

Comment 9 philipp.lohmann 2002-07-04 08:58:05 UTC
pl->ama: I'm not sure why Hasan sent me this, the problem seems to be
on windows only and then i don't know about form controls. I also
don't know whether this lies in writer, but i think you know more
about form controls than i.
Comment 10 andreas.martens 2002-10-14 09:07:23 UTC
Ok, we will have a look at the time consumer.
Comment 11 helmerj 2002-11-20 21:24:26 UTC
After coming back to this problem, I noticed, that the status bar on the 
bottom on the page tells me what OO is actually doing during the mail 
merge that takes forever. It is mostly busy doing "reformating cells". 
 
Taking into account that some of the fields are located within tables, 
this kinda makes sense. What doesn't make sense is, that this happens 
after every single insertion of a value for a field. Why not insert all the 
fields and then in the end do some reformating if really needed.  
 
Well just another 2 cents from me. 
 
Thanks for taking care of this! 
 
Juergen 
Comment 12 Unknown 2002-12-24 22:53:33 UTC
I have a test case for this which can reproduce two, and sometimes
three different problems with the form-letter capability.

It involves making up a bunch of sticky-labels my wife wanted, to
label the photographic prints from our recent vacation.  I started
with a text file, containing one line per label, three fields (roll
number, photo number on roll, caption) separated by "@", and set this
up as a flat-file text database.

I then did a "New labels", specified Avery 5167 labels (eighty per
sheet), and added the three fields from the table, with some text
separators between them.  A sample label for photo 6 from roll 3 might
read 

  R3-6 Sunset on Maui

I then used "Form letter" and selected all of the records from the
flat file.

Three problems were noted:

[1] I couldn't print more than one page of labels at a time.  Trying
to do two or more pages of labels in a single job (either direct to
printer, to a single PostScript file, or to separate PostScript files)
caused OpenOffice 1.0 to crash.  Typically it would put up a dialog
reporting an error of some sort why trying to save the .sxw file.

I worked around this problem by printing pages individually, selecting
record ranges from the file by number.

[2] If I tried to print a single page of labels, and if the amount of
text from the flat-file database record was enough to cause the
information printed on the label to exceed the total amount of space
available (including line-wrapping), StarOffice 1.0 would crash with
no warning and no attempt to save the file.

I worked around this problem by editing the flat-file and simplifying
the text on the labels.

[3] Occasionally, there would be an "off by one" error in the record
range printed... e.g. if I said to print from records 81 through 160,
the page printed would actually be for records 82 through 161. 
Difficult to reproduce - not always consistent.

Steps to reproduce problem #1, at least, on a Linux system (Debian
Woody) running OpenOffice 1.0:

[a] Download http://www.radagast.org/~dplatt/misc/southwestlabels.txt
[b] Create a directory and drop this file into it.
[c] Launch OpenOffice
[d] Tools, Data Sources.  Create a new "text" data source, and select
the directory.  Hit the Text tab, and set the field separator to '@'.
Approve the creation of the new data source.
[e] View Data Sources, select the new data source, open the
"southwestlabels" table, and confirm that the fields show up OK.
[f] New... Labels.  Choose Avery Letter labels.  Select the Avery 5167
return address labels.  Clear out the "Label text" field.  Select the
new data source and the "southwestlabels" field.  By typing, and
selecting fields, set the label text to "R", the Roll field, "-", the
Photo field, " ", and the Label field.  Click "New Document".
[g] When the new document has been created, do a "Select All" and set
the font size to 6 points.
[h] Start up "top" running in another window.
[i] Choose "Form letter".  Select "All Records" and "Printer" (you
might want to try printing to a file) and start the printing process.
[j] Watch the program thrash itself to death.

According to "top", the RSS of OpenOffice climbs from around 60 MB to
upwards of 80 MB while formatting just the first page of the labels.
It rises up to about 94 MB while working on page 2, and the app
crashes before it finishes this page.

You can probably reproduce problem [2] by opening up the
southwestlabels.txt file, and appending a few hundred characters of
text to the end of several of the lines in the first page's worth
of labels.  Try to print this page, and the app will crash - I'd
guess that it's being confused by a "paragraph" which is longer than a
"page", perhaps?

I don't know what it's doing [right or wrong] but I can say that
trying to generate mailing labels using 1.0 on a Pentium 233MMX,
128 MB of RAM, is an exercise in masochism :-(.  Maybe it's just
the wrong tool for the job... but that's a shame, as OO is so very
useful in numerous other ways.
Comment 13 michael.brauer 2003-03-11 18:08:43 UTC
HI, can you please have a look at this issue again. It seems to me
that it now covers three issues. Can you please also set an apropriate
target. Thank you
Comment 14 helmerj 2003-03-12 02:52:24 UTC
Hi! 
 
as far as I am concerned this issue can be considered closed with the 
release of OO 1.0.2. I didn't have the time to test this on a windoze 
machine, since I don't have one anymore ;-) 
 
Anyway. I used the same template with an updated but essentially the 
same data source. It runs fine using the following: 
 
Linux: OO 1.0.2 
win 98: OO 1.0.2 
 
So as long one can exclude the faint possibility that this is due to some 
bug in w2k for which I initially filed the bug for the mailmerge works 
fine. 
 
Juergen 
Comment 15 h.ilter 2003-03-13 11:41:36 UTC
Verified with given target.
Comment 16 h.ilter 2003-03-13 11:41:55 UTC
.