Issue 9818 - Parallel build failure: $(shell...) macros not executing properly
Summary: Parallel build failure: $(shell...) macros not executing properly
Status: CLOSED FIXED
Alias: None
Product: Build Tools
Classification: Code
Component: code (show other issues)
Version: 643C
Hardware: PC Linux, all
: P3 Trivial (vote)
Target Milestone: OOo 1.0.3
Assignee: hjs
QA Contact: issues@tools
URL:
Keywords:
Depends on:
Blocks: 9443
  Show dependency tree
 
Reported: 2002-12-04 12:37 UTC by chris
Modified: 2003-04-16 12:52 UTC (History)
1 user (show)

See Also:
Issue Type: PATCH
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Patch for DATESTRING in settings.mk 643C (967 bytes, patch)
2002-12-04 12:40 UTC, chris
no flags Details | Diff
Patch for PERLINST in use_perl.mk 643C (821 bytes, patch)
2002-12-04 12:41 UTC, chris
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description chris 2002-12-04 12:37:08 UTC
When a macro is defined like this: MACRO=$(shell foo ...), the execution of the
shell command is delayed until rules evalution (I believe), and is executed once
for each rule that uses the macro.  Unfortunately I am seeing failures during
parallel builds (dmake -Pn) like this:

Project 125: scp
Subdir: /usr/local/src/ooffice/oo_cvs/scp/source/psprint

cpp.lcc -+ -P -DWEB_PRODUCT -DLINUX -DUNX -DVCL -DGCC -DC300 -DINTEL -DCVER=C300
-D_USE_NAMESPACE -DGLIBC=2 -DX86 -D_PTHREADS -D_REENTRANT -DNEW_SOLAR
-D_USE_NAMESPACE=1 -DSTLPORT_VERSION=400 -D__DMAKE -DUNIX -DCPPU_ENV=gcc3
-DSUPD=643 -DBUILD=8362 -DPRODUCT -DNDEBUG -DPRODUCT_FULL -DOPTIMIZE
-DEXCEPTIONS_OFF -DCUI -DSOLAR_JAVA -DSRX643  -DSCP_DATESTRING=02122002
-DUDK_MAJOR=3 -DUDK_MINOR=0 -DUDK_MICRO=1 -DOFFICEUPD=643 -Ugcc3 -DCOMID=gcc3
-DCOMNAME=gcc3 -D_gcc3 -DCCNUMVER=000300020001 -DLIBSTDCPP3=5.0.1
-DSHORTSTDCPP3=5 -DDLLSUFFIX=li -I. -I../../inc -I../inc -I../../unx/inc
-I../../unxlngi4.pro/inc -I.
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/external
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/unxlngi4/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/inc -I/usr/local/src/ooffice/oo_cvs/res
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solenv/inc/Xp31 -I/usr/lib/j2sdk1.3/include
-I/usr/lib/j2sdk1.3/include/linux
-I/usr/lib/j2sdk1.3/include/native_threads/include -I/usr/X11R6/include    
dir_psprint.scp > ../../unxlngi4.pro/misc/web/dir_psprint.pre
------------------------------
cpp version 17.8.1999
cpp version 17.8.1999
cpp version 17.8.1999
cpp.lcc -+ -P -DWEB_PRODUCT -DLINUX -DUNX -DVCL -DGCC -DC300 -DINTEL -DCVER=C300
-D_USE_NAMESPACE -DGLIBC=2 -DX86 -D_PTHREADS -D_REENTRANT -DNEW_SOLAR
-D_USE_NAMESPACE=1 -DSTLPORT_VERSION=400 -D__DMAKE -DUNIX -DCPPU_ENV=gcc3
-DSUPD=643 -DBUILD=8362 -DPRODUCT -DNDEBUG -DPRODUCT_FULL -DOPTIMIZE
-DEXCEPTIONS_OFF -DCUI -DSOLAR_JAVA -DSRX643  -DSCP_DATESTRING=Making script
particel: ../../unxlngi4.pro/par/osl/files_psprint.par 02122002 -DUDK_MAJOR=3
-DUDK_MINOR=0 -DUDK_MICRO=1 -DOFFICEUPD=643 -Ugcc3 -DCOMID=gcc3 -DCOMNAME=gcc3
-D_gcc3 -DCCNUMVER=000300020001 -DLIBSTDCPP3=5.0.1 -DSHORTSTDCPP3=5
-DDLLSUFFIX=li -I. -I../../inc -I../inc -I../../unx/inc -I../../unxlngi4.pro/inc
-I. -I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/external
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/unxlngi4/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/inc -I/usr/local/src/ooffice/oo_cvs/res
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solenv/inc/Xp31 -I/usr/lib/j2sdk1.3/include
-I/usr/lib/j2sdk1.3/include/linux
-I/usr/lib/j2sdk1.3/include/native_threads/include -I/usr/X11R6/include    
files_psprint.scp > ../../unxlngi4.pro/misc/web/files_psprint.pre
cpp version 17.8.1999
cpp: Can't open input file script

Note the '-DSCP_DATESTRING=Making script particel' in the middle of the command
line.  A similar corrupt commandline is generated where the $(PERLLIBDIR)
construct is used:

ccache g++-3.2 -fmessage-length=0 -c -I. -I. -I../inc -I../inc -I../unx/inc
-I../unxlngi4.pro/inc -I.
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/external
-I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/unxlngi4/inc
-I/usr/local/src/ooffice/oo_cvs/solenv/inc -I/usr/local/src/ooffice/oo_cvs/res
-I/usr/include -I/usr/local/src/ooffice/oo_cvs/solver/643/unxlngi4.pro/inc/stl
-I/usr/local/src/ooffice/oo_cvs/solenv/inc/Xp31 -I/usr/lib/j2sdk1.3/include
-I/usr/lib/j2sdk1.3/include/linux
-I/usr/lib/j2sdk1.3/include/native_threads/include -I/usr/X11R6/include     -I.
-I../res -I. -O1   -I Making:
../unxlngi4.pro/obj/gen_info.obj/usr/lib/perl/5.8.0/CORE -pipe -mcpu=pentiumpro
-fno-for-scope -fpermissive -fno-exceptions     -DLINUX -DUNX -DVCL -DGCC -DC300
-DINTEL -DCVER=C300 -D_USE_NAMESPACE -DGLIBC=2 -DX86 -D_PTHREADS -D_REENTRANT
-DNEW_SOLAR -D_USE_NAMESPACE=1 -DSTLPORT_VERSION=400 -D__DMAKE -DUNIX
-DCPPU_ENV=gcc3 -DSUPD=643 -DBUILD=8362 -DPRODUCT -DNDEBUG -DPRODUCT_FULL
-DOPTIMIZE -DEXCEPTIONS_OFF -DCUI -DSOLAR_JAVA -DSRX643  -DMULTITHREAD  -w -o
../unxlngi4.pro/obj/gen_info.o
/usr/local/src/ooffice/oo_cvs/soltools/giparser/gen_info.cxx
if ( -e ../unxlngi4.pro/obj/gen_info.o) touch ../unxlngi4.pro/obj/gen_info.obj

Since these values do not need to be evaluted more than once, a simple fix is to
use the ':=' construct instead of '='.  These also make the makefiles slightly
more efficient, since the external command only needs to be executed once.  From
the dmake manpage:

 MACRO := LINE - This form differs from the simple '=' form in that it expands
LINE prior to assigning it as the  value  of MACRO.  Future expansions of MACRO
do not recursively expand its value.

Attaching patches generated against OO643C branch.
Comment 1 chris 2002-12-04 12:40:47 UTC
Created attachment 3896 [details]
Patch for DATESTRING in settings.mk 643C
Comment 2 chris 2002-12-04 12:41:29 UTC
Created attachment 3897 [details]
Patch for PERLINST in use_perl.mk 643C
Comment 3 rt 2002-12-10 16:52:44 UTC
I'll take this one, as Ause asked me to.
Comment 4 rt 2002-12-10 17:08:40 UTC
Patch applied to OO643C branch and merged into HEAD (which at the
moment equals to SRX644). 
Chris, please verify that it's fixed and close this bug. I therefore
reassign it to you.

Rüdiger
Comment 5 rt 2002-12-10 17:24:01 UTC
Applied to OOO_STABLE_1, too.

Rüdiger
Comment 6 hjs 2003-02-10 13:33:05 UTC
isn't this one resolved now?
Comment 7 chris 2003-02-10 15:12:55 UTC
I'm not 100% sure - the patch that was applied to CVS is missing a
space, which I needed at the time I generated the patch so I need to
actually perform a build to check that this is working.

My disk is too small to build more than one tree at once, and I've
been working in STABLE_1 since mid-December.  I'm just in the process
of switching over to the new 644 branch so I will be checking this and
the rest of the parallel patches over the next few weeks again.

Chris
Comment 8 chris 2003-02-10 16:46:54 UTC
The patch for use_perl.mk did not get committed between the branches,
even though the patch to settings.mk did make it.

ause: Reassigning to you as requested.  Please commit the patch to
use_perl.mk to branch cws_srx644_ooo20030223.
Comment 9 hjs 2003-02-10 19:57:39 UTC
applied patch to "use_perl.mk" on "cws_srx644_ooo20030223".

"OOO_STABLE_1" does still contain the blank so i'll leave this issue
open, waiting for this branch to reopen.
Comment 10 hjs 2003-03-26 17:54:26 UTC
removed blank un solenv/inc/use_perl.mk on OOO_STABLE_1
Comment 11 hjs 2003-04-16 12:52:28 UTC
-