Issue 119229 - ExtensionsLst.pm does not download newer versions of extensions
Summary: ExtensionsLst.pm does not download newer versions of extensions
Status: CLOSED FIXED
Alias: None
Product: Build Tools
Classification: Code
Component: code (show other issues)
Version: current
Hardware: All All
: P3 Normal (vote)
Target Milestone: 3.4.1
Assignee: Andre
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-15 11:11 UTC by Andrea Pescetti
Modified: 2014-02-20 20:29 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---
jsc: 3.4.1_release_blocker-


Attachments
Patch to update the Spanish dictionary (476 bytes, patch)
2012-04-15 11:41 UTC, Ariel Constenla-Haile
no flags Details | Diff
Check md5sum of existing extensions (895 bytes, patch)
2012-04-15 11:43 UTC, Ariel Constenla-Haile
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description Andrea Pescetti 2012-04-15 11:11:17 UTC
The logic in ExtensionsLst.pm seems:

- if the extension has already been downloaded, use the already downloaded version
- otherwise, download it and compare the checksum with the one expected from extensions.lst

But most extensions use the same filename across releases. So the logic should be changed to:

- if the extension has already been downloaded and ( extensions.lst has no checksum for it or (extensions.lst has a checksum and it matches) ), use the already downloaded version
- otherwise [same as above]

Alternatively, checksums could be prepended to the downloaded file names; this would allow, for example, both the Italian and Spanish dictionary to be named "dict.oxt" (but this would probably break later in packaging anyway)

See http://s.apache.org/xkuN for context.
Comment 1 Andrea Pescetti 2012-04-15 11:16:59 UTC
Assigning to Ariel who has a proposed patch (but attachment was stripped from ooo-dev).
Comment 2 Ariel Constenla-Haile 2012-04-15 11:41:54 UTC
Created attachment 77443 [details]
Patch to update the Spanish dictionary
Comment 3 Ariel Constenla-Haile 2012-04-15 11:43:33 UTC
Created attachment 77444 [details]
Check md5sum of existing extensions

The current code only checks if the extension does not exist. In that case, it is pushed to the to-be-downloaded extensions.
But it can be the case that the URL/md5sum in extensions.lst has been updated, so when the extension exists (it has already been downloaded), we must get its md5sum and compare it to the one in the extensions.lst. If they do not match, the extension must be downloaded again.
Comment 4 Ariel Constenla-Haile 2012-04-15 11:48:48 UTC
My Perl skills are almost null, so assigning to André, who seems to own that code.
Comment 5 Andre 2012-04-16 05:14:46 UTC
Would it not be easier to simply delete the old version and call
bootstrap to download the new one?  After all, extensions.lst has to
be modified anyway to update to the new MD5 checksum.

This mechanism is intended to be run with every build.  Running it at
least for every update of an extension does not seem to be such a
great disadvantage.  But maybe we should put the version number into
the file name or path so that different versions can be kept on the
server and an update becomes optional instead of mandatory.
Comment 6 Andre 2012-04-16 05:17:06 UTC
Ah, just saw the discussion of the mailing list.  Please excuse my duplicate answer.
Comment 7 Andre 2012-06-15 11:54:09 UTC
Finally found the time to work on this one.
I basically applied Ariel's patch and added some minor changes.

SVN revision is 1350587.
Comment 8 SVN Robot 2012-07-16 08:07:15 UTC
"pescetti" committed SVN revision 1361930 into trunk:
#i119229# Update ES dictionary to 0.6 (first patch in issue page)Patch By: ar...
Comment 9 Andrea Pescetti 2012-07-22 21:06:23 UTC
This can be merged in AOO34 if needed/wanted; I verified it works, as discussed on ooo-dev.

$ # In a trunk checkout...
$ svn switch https://svn.apache.org/repos/asf/incubator/ooo/branches/AOO34/
[...] revision 1364388.
$ cd main/ ; autoconf
$ # Get unowinreg.dll and other technicalities, edit Ariel's build flag to keep relevant stuff only, then...
$ ./configure   --with-packager-list=/mnt/build/aoo/aoo-dev-pack.lst --with-build-version="$(date +"%Y-%m-%d %H:%M:%S (%a, %d %b %Y)") - Rev. $(echo $(svn info) | sed -e 's/^.*Last Changed Rev: //g' -e 's/ .*//g')" --enable-verbose --enable-category-b --disable-mozilla --enable-minimizer  --enable-presenter-console  --enable-wiki-publisher  --enable-bundled-dictionaries --enable-opengl  --enable-dbus --with-package-format="rpm" --with-lang="de es fr it ja pt-BR zh-CN nl"  --with-dmake-url=http://dmake.apache-extras.org.codespot.com/files/dmake-4.12.2.tar.bz2 --with-epm-url=http://ftp.easysw.com/pub/epm/3.7/epm-3.7-source.tar.gz
[...] Configure completed [...]
$ bash LinuxX86-64Env.Set.sh
$ ./bootstrap
[...] downloading 9 missing extensions [...]
[...] ** GET http://garr.dl.sourceforge.net/project/aoo-extensions/1657/1/es_ES.oxt ==> 200 OK (3s) [...]
$ # Try updating the Spanish dictionary
$ svn merge -r 1361929:1361930 https://svn.apache.org/repos/asf/incubator/ooo/trunk/main/ .
U    extensions.lst
$ svn diff
... Merged /incubator/ooo/trunk/main:r1361930
 [ language=es ]
-    5798f4a61b0f95db3d62eabcb80670eb http://sourceforge.net/projects/aoo-extensions/files/1657/1/es_ES.oxt/download "es_ES.oxt"
+    28776430bb77547fd70cf6db504bda63 http://sourceforge.net/projects/aoo-extensions/files/1657/2/es_ES.oxt/download "es_ES.oxt"
$ # Try bootstrap and it fails: bug still present in AOO34
$ ./bootstrap
all downloadable extensions present [ WRONG! ]
$ # Apply the patch to ExtensionsLst.pm
$ svn merge -r 1350586:1350587 https://svn.apache.org/repos/asf/incubator/ooo/trunk/main/ .
U    solenv/bin/modules/ExtensionsLst.pm
$ svn st
M       extensions.lst
M       solenv/bin/modules/ExtensionsLst.pm
$ # Now the build system sees the updated dictionary.
$ ./bootstrap
extension es_ES.oxt has wrong MD5 and will be updated
downloading/updating 1 extension ...
** GET http://heanet.dl.sourceforge.net/project/aoo-extensions/1657/2/es_ES.oxt ==> 200 OK (7s)
Comment 10 SVN Robot 2012-07-23 11:10:40 UTC
"jsc" committed SVN revision 1364583 into branches/AOO34:
#119229# merge fix for upgrading Spanish dictionary
Comment 11 jsc 2012-07-23 11:47:51 UTC
set release blocker flag
Comment 12 jsc 2012-07-23 11:53:12 UTC
I only included the update of the Spanish dictionary, not the patch for the ExtensionsLst.pm
Comment 13 SVN Robot 2012-07-23 12:03:23 UTC
"jsc" committed SVN revision 1364591 into branches/AOO34:
#119229# merge fix to upgrade newer extensions from trunk