Created attachment 18512 [details]
build.pl patch

Created attachment 18513 [details]
binfilter sample patch

And, of course, I screwed up the 2nd fragment of the 1st patch, it should be:

@@ -2034,7 +2057,10 @@
     my @modules_to_build;
 #my $new_modules = '';
     foreach (@mod_array) {
-        if (/(\w+):(\S+)/o) {
+	if (/\$\((\w+)\):(\S+)/o) {
+            push(@modules_to_build, $2) if (get_env_boolean($1,1));
+	    next;
+	} elsif (/(\w+):(\S+)/o) {
             push(@modules_to_build, $2) if (defined $build_modes{$1});
             next;
         };

ie. extra brackets.

reassigned.

adding me to CC:

*** Issue 28202 has been marked as a duplicate of this issue. ***

.

It turns out I can't type:
-+ToFile( "WITH_BINFITLER",    "@WITH_BINFILTER@",   "e" );
++ToFile( "WITH_BINFILTER",    "@WITH_BINFILTER@",   "e" );

:-)

Two more patches are needed to make the binfilters really optional; otherwise 
the instsetoo_native fails. I'll attach them. 

Created attachment 18989 [details]
Disable binfilters in scp2.

Created attachment 18990 [details]
Disable creation of legacy_binfilters.rdb.

Created attachment 19186 [details]
clobber cludges in perl installer

change type to patch to get it on various radars.

Created attachment 19320 [details]
improved conditional patch

Later improved build-conditional patch much better:
  * re-factors some cut/paste coding
  * adds negation to build rules eg. !(SYSTEM_LIBXML):libxml
  * adds line-breaking to format

The line breaking would be _particularly_ helpful for release engineers merging
these files & external people patching them; this allows eg.:

--- instsetoo_native/prj/build.lst	2 Nov 2004 09:27:38 -0000	1.12
+++ instsetoo_native/prj/build.lst	15 Nov 2004 14:54:53 -0000
@@ -1,4 +1,21 @@
-oon	instsetoo_native	:	helpcontent2 UnoControls basctl configmgr cpputools
dbaccess desktop eventattacher extensions extras fileaccess forms io package
padmin readlicense_oo remotebridges scaddins scp2 setup_native SO:epm shell sc
sch chart2 sd starmath sw ucb wizards officecfg xmlhelp MathMLDTD dictionaries
bitstream_vera_fonts filter psprint_config fpicker odk embedserv pyuno crashrep
scripting embeddedobj hwpfilter unoxml slideshow binfilter msfontextract
xmlsecurity NULL
+oon	instsetoo_native  :  \	
+	helpcontent2 UnoControls basctl \
+	configmgr cpputools dbaccess \
+	desktop eventattacher extensions \
+	extras fileaccess forms io \
+	package padmin readlicense_oo \
+	remotebridges scaddins scp2 \
+	setup_native SO:epm shell sc \
+	sch chart2 sd starmath sw ucb \
+	wizards officecfg xmlhelp MathMLDTD \
+	dictionaries filter \
+	psprint_config fpicker odk embedserv \
+	pyuno crashrep scripting embeddedobj \
+	hwpfilter unoxml slideshow \
+	msfontextract xmlsecurity \
+	$(WITH_FONTS):bitstream_vera_fonts \
+	$(WITH_BINFILTER):binfilter \
+	NULL
 oon	instsetoo_native						usr1	-	all	oon_mkout NULL
 oon	instsetoo_native\packimages				nmake	-	all	oon_pack NULL
 oon	instsetoo_native\packconfig				nmake	-	all	oon_config NULL

although; this needs to replace \\\S with \\\s in the continuation code it seems.

I did an error in binfilter-solenv.diff; there has to be "ne" instead of "!=", 
of course... 

I committed this little lot, with a tweak to allow the default value for negated
parameters to cope gracefully (always eventually true) for the Sun build system
- with it's fewer environmentally adjusted things.

All in CWS buildcond01 now, the patches I applied build fine for me; but some
eg. Win32 QA appreciated, perhaps I can enlist Volker, or someone else ? ...

I can build an installset if you like. Just add me to the members of that cws
so that I don't forget it. ;)

Oh, someone already did that ;)

to mmeeks: the build list format changes cannot be accepted, because some our
inhouse tools would not understand the new format. We plan xml file format for
build.lst quite soon, so there's no use to accomodate our tools to changes in
outdated format.

I do not see the reason why the env. variables should have two or even more
values. I think that it's an overkill, it would be sufficient just to have a
varible defined or not, and write something like (in pick_for_build_type
procedure, line 2118, r1.127):
    foreach (@mod_array) {
        if (/(\w+):(\S+)/o) {
            next if (!defined $build_modes{$1});
            $_ = $2;
        };
        next if (defined $ENV{'DISABLE_BUILD_' . uc($_)});
        push(@modules_to_build, $_);
    }

No more changes in build.pl are needed, actually. Of cause, one should
accomodate configure.in for this concept.

Martin - some backup appreciated, since you read this & thought it was doable /
useful etc.

VG:
    I see the planned XML file format - but a plan doesn't help me with my
desire to shrink the 2.0 source-downloads to a manageable size for developers
quickly. Furthermore, I see no use whatsoever of any build.xlists in the tree.

> I do not see the reason why the env. variables should have two or
> even more values. I think that it's an overkill, it would be sufficient
> just to have a varible defined or not. Of cause, one should
> accomodate configure.in for this concept.

    Well - the alternative to these few lines of code here, is a huge change to
the configure system & perhaps a hundred or so makefile.mk's to reverse / alter
the polarity of the conditionals there - or - export other variables that would
stuff the environment yet fuller with cruft [ there has to be some DOS limit
we'll hit there at some stage ].

    So ... - can you enlarge as to what your in-house problems are ? - you
should be using build.pl to process the files [ surely ], and [ as you see from
the code ], I did work to ensure that the default state in your build system [
with no relevant environment variables defined ] - should be to include the
module in the build regardless of the polarity of the conditional etc. - is
there some further requirement you have there ?  

mmeeks: I do not see how build.lst format changes can help to reduce downloads.
There is no build.xlists just because od some delays with our inhouse tools.
As to environment variables, as far as can I see, there are no changes in
makefiles needed (correct me if it's not so), the only file engaged is
configure.in. I admit that there the changes can be sufficient, but they are all
homogenuos and don't require complicated logic. No additional environment
variables needed.
Moreover, I think such approach would make the build process more clear.
We have some tools, which we use to automatize our builds. Indeed, our
developers use only build tool to perform builds, in RE we have an atomatic
build process, which rely on the old build.lst format. 

mh->vg: I consider this as a valid patch for a valid concern. 

providing more managable downloads is a vaild approach to address ease of
building. This would attract new contributors.

> mmeeks: I do not see how build.lst format changes can help to reduce 
> downloads.

so - we can split the source base to remove eg. 30+Mb of (compressed) mozilla
binary that is not useful for many systems. Of course, we could do that by
leaving most of the files in the moz/ directory there - but it is more elegant /
simple just to split that top-level directory out => we need these build
changes. The binfilter likewise.

> As to environment variables, as far as can I see, there are no changes in
> makefiles needed (correct me if it's not so), the only file engaged is
> configure.in.

   Currently there is loads of code of the form:

.IF "$(SYSTEM_MOZILLA)" != "NO"
   @echo "Do nothing for mozilla"
.ENDIF

   scattered through many, many makefiles. Furthermore - since there is no
build.pl conditional support - this becomes really hard & inelegant for projects
with lots of makefiles - the patches are large & painful to manage.

   we could change configure.in to export (in addition to SYSTEM_MOZILLA) some
'DONTBUILD_MOZILLA' type environment variable that could be used to be
set/not-set but it is a hundred of lines of extra (duplicate) configure code,
many more confusing & overlapping environment variables & well - just more pain
all around.

> I admit that there the changes can be sufficient, but they are all
> homogenuos and don't require complicated logic. No additional environment
> variables needed.

     I want to use $(SYSTEM_MOZILLA) as it is, in it's current form really.

> We have some tools, which we use to automatize our builds. Indeed, our
> developers use only build tool to perform builds, in RE we have an atomatic
> build process, which rely on the old build.lst format. 

     Can you give more details of these tools - has there really been a
cut/paste job done on build.pl ? and if so - why ? surely it doesn't make much
sense to do that sort of thing.

Martin - what is your suggested action here then ? 'vq' reports a compiling 
Win32 build - vq can you confirm installing works nicely too - pwrt. binfilter.
Clearly - I'd like to get this in as a first step, I have about 6 other
conditional-* patches that depend on this ready to go into another cws.

mmeeks: Ok, i see your point. Not the best solution in my opinion, but I can
live with it. As to our internanal tools: some of them are written in c(c++) and
are quite complicated. We cannot refrain from using them. There are no
copy-paste excepts (normally), so build.lst format changes cannot be accepted.
But: we accelerate our efforts on XML format.

add me to cc:

regarding the build.lst format change:
wouldn't it be sufficient for now to use the existing variable "BUILD_TYPE", set
it according the desired "--with[out]-somthing" switches in configure and add
the prefix "BUILD_SOMETHIG:something" to the build lists definig this dependency?
this could avoid the format change.

e.g. 
"--without-system-moz" => BUILD_TYPE += MOZ
and in "xmlsecurity/prj/build.lst"
xs  xmlsecurity :   xmloff unotools offapi unoil svx MOZ:moz libxmlsec NULL

did i miss something?

> regarding the build.lst format change: wouldn't it be sufficient for now to
> use the existing variable "BUILD_TYPE", set it according the desired
> "--with[out]-somthing" switches in configure and add
> the prefix "BUILD_SOMETHIG:something" to the build lists definig this
> dependency? this could avoid the format change.

   What you suggest is almost exactly what this patch does; however, currently
BUILD_TYPE defines a single build conditional; either 'OO' or 'SO' (or something
else), and not a plural set of conditionals.

   This patch changes that; so you can do [foo]:module - extending the syntax.
Ok - so admittedly it uses $(FOO) - to make it more clear this is an environment
variable being used as distinct from some 'magic' foo - if you really hate the
$() I can trivially remove them from the patch - however the negation via '!' is
really critical to save lots of scattered makefile.mk code changes (as discussed
ad nauseum).

   I honestly can't see the:
           "it is utterly impossible for us to change our internal
            tooling, so go away"
    argument making sense since the internal tooling already has to parse /
enterpret [foo]:module; [ a new feature since 1.1.x ] and it's simply a matter
of making it default to 'always build it' if [foo] is not 'OO'; surely - this
should be trivial - and fresh in people's minds since the recent 'SO:foo' vs.
'OO:foo' change ?

> e.g. "--without-system-moz" => BUILD_TYPE += MOZ

    That looks like it would require changing Sun's internal build environment
to expand their setting of 'BUILD_TYPE' to :MOZ:FOO:BAA:BAZ _and_ - worse -
every time a new conditional was added, it would require a change to your
internal build environment. By contrast - the suggested patch does not suffer
from this limitation, by using smart defaults when environment variables are not
set.

    I _really_ would like to move on this; however vg's

            "so build.lst format changes cannot be accepted."

    Fills me with certainty that collaborating here is not going to be easy.

    Thus - I would like to propse that in addition to this patch; we create a
parallel 'build.lst2' format - we duplicate all the build.lst files, leaving
them to bit-rot until Sun sort their act out; so that we can get this committed,
shrink / split the source downloads & well - move on. Of course - this will be a
huge maintenance headache - particularly since the line-continuation feature is
_really_ useful for patching / reading / seeing what changed in long build
dependency lists - but ...

    Martin - should I create an 'every project' cws to copy build.lst ->
build.lst2s ? or is such a thing something that rel-eng could do directly on CVS
HEAD [ the cws overhead for just that would be huge ] [ of course, we could also
copy build.pl if necessary ]. I imagine there are other things that can be
improved in the build.lst2 format as well that are perhaps not possible currently.

>     Thus - I would like to propse that in addition to this patch; we create a
> parallel 'build.lst2' format - we duplicate all the build.lst files, leaving
> them to bit-rot until Sun sort their act out; ...
Not necessarily. The nasty but low impact solution that comes to mind is that
the the build.lst files for SO builds are modified with an easy
sed/perl/whatever-script that just removes the "non-SO:" before non-SO:module.

Or if you don't want it during the build, create <module>/prj/build_gen.lst
by duplicating the current build.lst files, add the FOO: whereever needed,
start your "build.lst-generate.pl" script, commit all changed build.lst /
build_gen.lst files and teach build.pl to use the build_gen.lst for OOo builds.

Not a maintainance nightmare, only a bad dream. ;) Should work as a stop-gap
solution until the xml version is ready.

To mmeeks (Sun Nov 28 20:12:36):

Just to explain our (Hamburg RE) current situation:
The solution Ause suggested would require no change at all for us. 
1) In the current implementaion  of BUILD_TYPE setting no BUILD_TYPE means
'build everything' and that's just what we do (we have to build SO as well as
OOo stuff in order to create SO and OOo install sets - the later ones needed to
provide milestone builds).
2) BUILD_TYPE is not restricted to SO and OOo. You already can use any
BUILD_TYPE - just try.
In contrast your prosed format change (splitting up the first dependency line)
would require adaptions of various tools. The magnitude of build list parsers
was the most important reason for us to change to xml based build lists. That's
why I strongly support Auses suggestion to utilize BUILD_TYPE environment variables.

Of course it would have advantages to have dependencies on multiple lines. OTOH
your proposal would address just halve of the mess, as on every other line we
still have all (inner module) dependencies in one single line. Therefore to get
this resolved I would suggest to wait for the xml based build lists.

Concerning your proposed 'patch' to hold build.lst2 files in parallel: please be
aware that this would really be a permanent maintenance nightmare. We (Hamburg)
would use old build.lst files only and not even notice if your new files get
inconsistent. Hamburg developers creating new or removing old directories would
change build.lst files first and are likely to forget your extra files (in fact
they quite often forget even build.lst files, but this gets noticed by us). For
me this does not sound like a good idea.

Ok; so BUILD_TYPE may not be restricted to SO or OO but it is restricted to
_one_ value for the entire OO.o build; nevertheless - essentially my patch (as I
explained) does just change the BUILD_TYPE.

However - I can make the patch more ugly so the syntax is:

    notSYSTEM_MOZILLA:moz
instead of:
    !$(SYSTEM_MOZILLA):moz

special case OO and SO & check for environment variables otherwise.

Of course - I can also remove the line continuation - which is an _extremely_
useful feature - since inevitably we have to patch these lists anyway ourselves.

Is that acceptable ?

Finally - I am _extremely_ scared by the potential XML format; particularly
since these project lists have to be patched by us - and I've suffered enough
pain from patching configuration .xcu/.xcs files.

I would love to see a doc. on the proposed XML syntax, pwrt. suggested
white-space & absence of any trendy / over-complex XML features such as XSLT
etc. it would be nice if the result was still comprehensible. I'm particularly
keen that the file is still reliably patchable with reasonable context, by
avoiding excessive new-lines & nesting:

     <node dep="svx"/>
     <node dep="svt"/>
+    <node dep="rtl"/>
     <node dep="sal"/>
     <node dep="baa"/>

Anyhow - I'll re-work the patch to make it use AlphaOnlyThisIsAllONE_CONDITIONAL
type variables followed by a ':'

Michael,

Sorry, but I still do not understand why you prefer your patch solution to the
existing BUILD_TYPE mechanism. BUILD_TYPE is _not_ restricted to one value for a
build.

Rüdiger

Ah; apologies - so I didn't notice that the build-type can be multiple values;

So - I guess; since I now know there are no problems with the Sun internal build
tools with adding random/crazy new BUILD_TYPE conditionals of this form - I just
need to re-write the patch / re-validate to generate all of this in configure.

I will as a convention; use the module name as the conditional where possible;
eg. MOZ:moz BINFILTER:binfilter etc.

Hope that's ok; I'll re-generate the diff as/when I get time & commit to CWS
buildcond01.

Since this was such a cock-up; I'll bin the buildcond01 cws and create a new cws
to do this work on: buildcond02 :-)

Since - I guess the buildcond02 cws looks scary - a lot of modules touched; I
attach a patch showing the complete change-set to demonstrate how extremely
trivial the majority of the changes are;

Created attachment 20323 [details]
full cws diff

fixed more elegantly in buildcond02

verified the "build.lst" changes

would you mind to change

+.IF "$(WITH_MYSPELL_DICTS)" != "YES"
+SCPDEFS+=-DWITHOUT_MYSPELL_DICTS
+.ENDIF

to 

+.IF "$(WITH_MYSPELL_DICTS)" == ""
+SCPDEFS+=-DWITHOUT_MYSPELL_DICTS
+.ENDIF

to avoid "Yes", "yes", "yeS" cinfusion?

hjs: ok - I updated the conditional, so it should work in your build system
(without that environment variable set) - sorry for that [ obviously not paying
attention somewhere ]. I used == 'NO' as is used over the place for the other
conditionals. [ I'd prefer not to get into fixing the whole yes/no/true/false
thing generically across the code just now ].

I hope that's ok. Does that mean you did the relevant QA on it ? :-)

<ause> michael_: buildcond02 is ok from my side. could you get those issues to
verified?
<michael_> ause: sure, let me do that ASAP

close

This issue is integrated into a build for OOo2.0, but the 'target milestone'
isn't set. To have a better overview about all fixed and integrated tasks in
OOo2.0, I set the field 'target milestone' to OOo2.0.