Issue 28717 - Build failed in scp2/util: par2script.pl/guw.pl issue?
Summary: Build failed in scp2/util: par2script.pl/guw.pl issue?
Status: CLOSED FIXED
Alias: None
Product: Build Tools
Classification: Code
Component: code (show other issues)
Version: current
Hardware: PC Windows Server 2003
: P2 Trivial (vote)
Target Milestone: OOo 2.0
Assignee: quetschke
QA Contact: issues@tools
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-05-04 13:02 UTC by pavel
Modified: 2005-01-07 03:59 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Patch for guw.pl (847 bytes, patch)
2004-06-07 23:25 UTC, quetschke
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description pavel 2004-05-04 13:02:05 UTC
Hi,

I can't build cws_src680_ooo20040509 because of:

pavel@drogo:~/BuildDir/ooo_cws_src680_ooo20040509_src/scp2/util> dmake
------------------------------
Making: ../wntmsci8.pro/bin/osl/setup_osl.inf
guw.pl /usr/bin/perl
/cygdrive/e/home/pavel/BuildDir/ooo_cws_src680_ooo20040509_src/solenv/bin/par2script.pl
-i
../wntmsci8.pro/par,/cygdrive/e/home/pavel/BuildDir/ooo_cws_src680_ooo20040509_src/solver/680/wntmsci8.pro/par
@@/tmp/mkb01248 -o ../wntmsci8.pro/bin/osl/setup_osl.inf
Command: /usr/bin/perl
readline() on closed filehandle CMD at
/cygdrive/e/home/pavel/BuildDir/ooo_cws_src680_ooo20040509_src/solenv/bin/guw.pl
line 185.
print() on closed filehandle CMD at
/cygdrive/e/home/pavel/BuildDir/ooo_cws_src680_ooo20040509_src/solenv/bin/guw.pl
line 207.

***************************************************************
ERROR: Cannot find file \tmp\mkb01248
in function: check_file
Saved logfile: logfile.log
***************************************************************
dmake:  Error code 255, while making '../wntmsci8.pro/bin/osl/setup_osl.inf'
echo: No match.
pavel@drogo:~/BuildDir/ooo_cws_src680_ooo20040509_src/scp2/util> 

Is this an issue in guw.pl (see \tmp\...)?

I use tcsh build. I can provide any other data either here or via IRC instantly.
Comment 1 Olaf Felka 2004-05-04 13:04:59 UTC
of @ mh: Please have  a look.
Comment 2 Martin Hollmichel 2004-05-04 14:34:37 UTC
reassign and cc
Comment 3 quetschke 2004-05-04 17:37:13 UTC
This is related to issue 27173 and issue 25475. CCing is

I cannot investigate, my build is not there yet, it breaks in vcl/source/helper.

Volker
Comment 4 quetschke 2004-05-04 23:34:12 UTC
Sorry for pointing at others first, guw.pl is complaining, that's new ;)

It doesn't like the two @ in @@/tmp/mkb01444. I'm not sure why two @@ make it
into the commandline, there might be a problem, but I'll implement a workaround
in guw.pl tonight and let it condense two @ to one.

Wait a second ...

What is par2script.pl doing with @@<filename> when no wrapper is used?

Is the wrapper right here? That might be the real problem, the script running
in cygwin tcsh might want to keep the POSIX paths? What is par2script doing?

Secondly, I'm pretty sure that guw.pl doesn't convert something like:
 -i ../wntmsci8.pro/par,/cygdrive/d/v1/(snip)/680/wntmsci8.pro/par

Comma separated paths?! Why not $PATH_SEPERATOR?
Comment 5 quetschke 2004-05-05 03:10:19 UTC
I just committed a patch for guw.pl in cws_src680_ooo20040509.

I checked in my old build logs for cws_src680_ooo20040329 that the commandline
was exactly the same, besides the double @.
Comment 6 quetschke 2004-05-05 03:27:11 UTC
Argh! Can we please be a bit consistent! With the patch collapsing multiple @
to one @ I get:

Making: ../wntmsci8.pro/bin/osl/setup_osl.inf
guw.pl /bin/perl /cygdrive/d/v1/cws_src680_ooo20040509/solenv/bin/par2script.pl
-i
../wntmsci8.pro/par,/cygdrive/d/v1/cws_src680_ooo20040509/solver/680/wntmsci8.pro/par
@@/tmp/mkb01648 -o ../wntmsci8.pro/bin/osl/setup_osl.inf
Command: /bin/perl

************************************************
Error: List of par files not set!
************************************************

--------------------------------------------------------------
par2script V1.0 (c) Ingo Schmidt 2003
The following parameter are needed:
-i: include pathes, comma separated list
-o: setup script file name
-v: writing logfile.txt (optional)
@@list: list of all par files
 
Example:
 perl par2script.pl -i ..\wntmsci8\par,o:\SRX645\wntmsci8\par.m24 
 		@@C:\DOCUMEN~1\is\LOCALS~1\Temp\mk6pd
 		-o ..\wntmsci8.pro\bin\osl\setup_osl.inf [-v]

For the record: cl and friends evaluate @<parafilename>. I'll tweak guw
now to keep the number of @ constant, but I don't like this.
Comment 7 quetschke 2004-05-05 03:47:05 UTC
Ok, the current guw.pl can handle @@, and the project compiles completely,
but does it compile correct? par2script reports the following:

"par2script -i ..\wntmsci8.pro\par,\cygdrive\d\v1\cws_src680_ooo20040509\
solver\680\wntmsci8.pro\par @@C:\cygwin\tmp\mkb01768 -o
..\wntmsci8.pro\bin\osl\setup_osl.inf" (one long line)

The \cygdrive\d\v1\cws... path is not converted, do we need "," as a directory
seperator?
Comment 8 ingo.schmidt-rosbiegal 2004-05-05 17:03:12 UTC
The string "par2script -i ..\wntmsci8.pro\par,\cygdrive\d\v1\cws_src680_ooo20040509\
solver\680\wntmsci8.pro\par @@C:\cygwin\tmp\mkb01768 -o
..\wntmsci8.pro\bin\osl\setup_osl.inf"  (in one line) is absolutely correct. Did
you have problems with the installation set?
I discussed with Ause that we should no longer use te "@" for the temporary
file, but include an additional parameter. Besides it seems to be better, no to
use one comma separated list but one parameter for each path. Would this be
better for you? Unfortunately we cannot start these changes in May, but earliest
in June. 

  Ingo
Comment 9 quetschke 2004-05-05 19:34:49 UTC
Hi Ingo,

> The string "par2script -i ..\wntmsci8.pro\par,\cygdrive\d\
> v1\cws_src680_ooo20040509\solver\680\wntmsci8.pro\par
> @@C:\cygwin\tmp\mkb01768 -o ..\wntmsci8.pro\bin\osl\setup_osl.inf"
> (in one line) is absolutely correct.
> Did you have problems with the installation set?
I didn't try, but the build finished without problems.

> I discussed with Ause that we should no longer use te "@" for the temporary
> file, but include an additional parameter.
No problem with that, now guw would even work with @@@@@@@@@@@filename ;)

> Besides it seems to be better, no to use one comma separated list but one
> parameter for each path. Would this be better for you?
Actually it doesn't matter, the only problem that I see is, that at the moment
filenames seperated by "," are not correctly transformed to the posix format,
only the "/" to "\" translation is done, which leaves a very strangely
formatted pathname: \cygdrive\d\v1\cws_src680_ooo20040509\(snip)\par
is just wrong, either: /cygdrive/d/v1/cws_src680_ooo20040509/(snip)/par
or: d:\v1\cws_src680_ooo20040509\(snip)\par

I didn't look at par2script at all, but something is a bit strange here. For
the 4NT shell you would have DOS paths everywhere, but you need POSIX paths
if you use cygwin-tcsh. Is this supported?

For tcsh and activestate perl you probably need DOS paths too, I'm not sure
here, but this is realized by guw.pl (Beside, that it isn't for the -i para
at the moment.)

And for tcsh and cygwin perl (The generic community environment) you feed
DOS paths into a cygwin perl, (This should work, cygwin can work with
DOS paths most of the time) but the usage of the POSIX paths would be more
natural.

> Unfortunately we cannot start these changes in May, but earliest in June.
Seperation by "," or space is IMHO not that important, but it is important
that par2script does its job. Can it really work with incorectly converted
POSIX paths?

  Grüße

          Volker
Comment 10 quetschke 2004-05-05 19:52:37 UTC
Brrr, I contradict myself.

> I didn't look at par2script at all, but something is a bit strange here. For
> the 4NT shell you would have DOS paths everywhere, but you need POSIX paths
> if you use cygwin-tcsh. Is this supported?
As written below in that message, yes, it is.
Comment 11 quetschke 2004-05-17 18:53:17 UTC
vq_>is:

Hi Ingo, can you please comment on this problem, otherwise I do not know if
a fix for guw.pl is needed:

>> Besides it seems to be better, no to use one comma separated list but one
>> parameter for each path. Would this be better for you?
> Actually it doesn't matter, the only problem that I see is, that at the moment
> filenames seperated by "," are not correctly transformed to the posix format,
> only the "/" to "\" translation is done, which leaves a very strangely
> formatted pathname: \cygdrive\d\v1\cws_src680_ooo20040509\(snip)\par
> is just wrong, either: /cygdrive/d/v1/cws_src680_ooo20040509/(snip)/par
> or: d:\v1\cws_src680_ooo20040509\(snip)\par
Comment 12 ingo.schmidt-rosbiegal 2004-05-18 17:22:35 UTC
IS -> VQ: 

Hi Volker,
if you can fix the problem in the guw.pl, please do so. I do not know yet,
whether I can solve the problem in pre2par.pl or in par2script.pl. Both scripts
use in globals.pm the following if statement to define, whether "/" or "\" shall
be used:

if (( $plat =~ /MSWin/i ) || (( $plat =~ /cygwin/i ) && ( $ENV{'USE_SHELL'} eq
"4nt" )))
{ $separator = "\\"; }
else { $separator = "/"; }

Does this lead to "\" in your environment? Of course the pathes have to be
correct in the  temporary file, that is given to the par2script perl script (the
@@ parameter).
Comment 13 quetschke 2004-05-18 19:23:04 UTC
Hi Ingo,

I still didn't look into pre2par.pl or in par2script.pl, but as long as we use
the POSIX->DOS wrapper we should do the transformation correct. I'll look
into the extension to guw.pl to use "," as a path separator.

> Does this lead to "\" in your environment? Of course the pathes have to be
> correct in the  temporary file, that is given to the par2script perl script
> (the @@ parameter).

I'd like to keep OOo buildable and the community likes to build with tcsh and
cygwin's perl. But check the options:

PERL            SHELL              Wrapper          Wrapper needed
====            =====              =======          ==============
ActiveState     4nt                no               N/A
Cygwin          4nt                no               N/A
Cygwin          tcsh               yes              don't know (1)
ActiceState     tcsh               yes              yes        (2)

I still have no clue what pre2par.pl or par2script.pl are used for, do they
write files that contain paths? 
(1) If yes, then all parameters should be convertet to dos format, that means
the wrapper is needed. Cygwin perl can handle that and this makes sure
(hopefully) that valid DOS paths are written.
If no, the wrapper is not needed and the paths can stay in the POSIX format.

(2) Do we support ActiceState perl and cygwin tcsh? (Not in the build
instructions) If yes, we need the wrapper, if no ->

(1)+(2): No wrapper needed under the assumptions made above, but shouldn't
hurt when it is used correctly.

Still, once you start parsing something and use $separator everything
can go terribly wrong.

Is there a short description what pre2par.pl and par2script.pl are supposed
to do? Yes I could read the source, but I was thinking about a one line
description ;)

I'm a bit worried that (1) uses the wrapper ATM (all "/" -> "\") but you define
$separator = "/". That looks suspicious to me.

(That cygwin perl can handle "/" and "\" might only hide the real problem.)

P.S.: This
-  while ( $variable =~ /(?:;|\A)[\'\"]?((?:\/[\w\.\- ~]+)+)[\'\"]?(?:;|\Z)/ ) {
# Normal paths
+  while ( $variable =~ /(?:[;,]|\A)[\'\"]?((?:\/[\w\.\-
~]+)+)[\'\"]?(?:[;,]|\Z)/ ) { # Normal paths
is the change to guw.pl to accept comma separeted path, but there is no
680er branch open ATM.
Comment 14 ingo.schmidt-rosbiegal 2004-05-19 10:29:14 UTC
Hi Volker,

I introduced pre2par.pl and par2script.pl to enhance the functionalities in the
scp2 project and to decrease the scp2 build time drastically. They substitute
the old scp tooling, in which the scp files in the scp2 project are converted to
the setup script, that you can find in the solver. The process is the following:
1. The product is described in the script particles (scp files in scp2 project)
2. A preprocessor converts the scp to pre files (in the misc directory in the
scp2 output tree). All #ifdefs are solved.
3. pre2par.pl makes some changes in the structure of the pre files. The only
parameter for pre2ar are the input name of the pre file and the output name of
the par file. The par files are written into the par directory of the scp2
output tree. They have the correct structure for the linker, which is nowadays
par2script.pl. 
4. par2script.pl links together all the par files that are needed for a special
product and creates the setup script, which describes the product and is used by
the packaging process to create the installation set. The setup script is the
result of the scp2 project. par2script.pl needs therefore as parameter a list of
include pathes,  the name of the setup script, that shall be created and a
temporary file, that contains a list of all par files that are needed by the
product. 

For all operations in which I need to create a path or read a file, I introduced
in globals.pm the $separator, which tells me, which separator is used on this
platform. Obviously my first assumption, that MSWin is the only case in which I
use "\" was too simple, because of the usage of the Cygwin. But of course the
programs must be able to handle Cygwin, too.
Comment 15 quetschke 2004-05-25 15:55:03 UTC
Hi Ingo,

I think that the guw.pl patch is all that is needed. I'll check with the next
680er branch.
Comment 16 ingo.schmidt-rosbiegal 2004-05-27 11:37:42 UTC
Hi Volker,

good news. Thanks.
Comment 17 quetschke 2004-06-07 23:24:47 UTC
vq->is:
I tested the following patch and diffed an backuped copy of scp2/wntmsci8.pro
against the version with the patch applied. No differences, but the commandline
is correct translated:

Making: ../wntmsci8.pro/bin/osl/setup_osl.inf
guw.pl /bin/perl /v1/cws_src680_ooo20040620/solenv/bin/par2script.pl -i
../wntmsci8.pro/par,/v1/cws_src680_ooo20040620/solver/680/wntmsci8.pro/par
@@/tmp/mkb01788 -o ../wntmsci8.pro/bin/osl/setup_osl.inf

par2script -i
..\wntmsci8.pro\par,d:\v1\cws_src680_ooo20040620\solver\680\wntmsci8.pro\par
@@C:\cygwin\tmp\mkb01788 -o ..\wntmsci8.pro\bin\osl\setup_osl.inf

I guess we never got a problem because "solver\680\wntmsci8.pro\par" is only
filled by scp2 (here) and therefore all par files are in ..\wntmsci8.pro\par.

I'll commit this to cws_src680_ooo20040620 once the current resync is finished.

PS. the @@ files also use ",", is this necessary?
Comment 18 quetschke 2004-06-07 23:25:30 UTC
Created attachment 15726 [details]
Patch for guw.pl
Comment 19 ingo.schmidt-rosbiegal 2004-06-08 13:24:27 UTC
IS -> VQ: 
Hi Volker,

you are right. For building OpenOffice.org only the path
"solver\680\<platform>\par" is needed. It has to  contain all par files. They
all are created in scp2 module. par files are nothing else then platform
resolved, macro resolved scp files. 
I do not know why the temporary file containing the list of par files, contains
the files in a comma separated list. This seems to have historically reasons.
When I exchanged the old par file linker scplink.exe with the new par2script.pl,
I did not change the syntax. If you want to change this, of course the
par2script.pl has to be adapted, too.
  

Comment 20 quetschke 2004-06-08 16:12:20 UTC
vq->is:

Hallo Ingo,

Nah, no need to change the @@file syntax, unless we want to remove the "," as
a path separator from guw again. Then it would be nice to choose " " or
$(PATH_SEPERATOR).
Comment 21 quetschke 2004-06-09 02:18:46 UTC
Committed to cws_src680_ooo20040620.
Comment 22 quetschke 2005-01-07 03:59:39 UTC
Works
Comment 23 quetschke 2005-01-07 03:59:55 UTC
close