| Summary: | git.eclass Invent some robust yet fast way to clone from ${DISTDIR} to ${S} | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Sergey Ilinykh <rion4ik> |
| Component: | Eclasses | Assignee: | Tomáš Chvátal (RETIRED) <scarabeus> |
| Status: | RESOLVED WONTFIX | ||
| Severity: | enhancement | CC: | dberkholz, hollow, nikoli, pesa, sputnick |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
trivial patch
fixed storing submodules sources diff git.eclass.diff git.eclass.patch |
||
|
Description
Sergey Ilinykh
2009-05-12 04:17:48 UTC
Hi, mine activity for big things expect massive breakages is much reduced until mid june. So write up patch and attach it here so we can review, i bet donnie dont have much time too :] or just let this bug wait till mid of june where i will be at full presence :] Cheers Created attachment 198718 [details, diff]
trivial patch
Added to x11 overlay for testing. Added to main tree finaly :] buggy implementation. eclass redownloads submodule sources each time on unpack, but should just update existing. please rollback this changes from main tree or do that proper way. Created attachment 210312 [details]
fixed storing submodules sources
Changed eclass - now it does not redownload all submodules every time. It works for me, but still needs more testing.
Created attachment 210313 [details, diff]
diff
Created attachment 210322 [details]
git.eclass.diff
Updated patch.
Please test and report issues.
(In reply to comment #7) > Created an attachment (id=210313) [details] > diff > I'm not sure "git submodule update" is enough to update modules. AFAIK git archive won't always work on such modules since they are detached and after next update may be broken (outdated). I don't quite understand how this works, but for me i found solution, its something like "git submodule update --merge". Ok x11 overlay got even more updates for the eclass, so anyone willing to test the feature it might be good time to report if it is not behaving well and so on :] added to main tree just to make you know 1) strange function name - git_sumbodules 2) it seems its broken again. i see submodules sources in git-src but there are just empty folders in /var/tmp/portage after unpack 3) caching fetched submodules sources never worked for me. >2) it seems its broken again. i see submodules sources in git-src but there are just empty folders in /var/tmp/portage after unpack confirm: ... A mplayer/libdvdnav/remap.h Exported revision 1188. >>> Source unpacked in /mnt/g/portage/media-video/mplayer-git-9999/work >>> Preparing source in /mnt/g/portage/media-video/mplayer-git-9999/work/mplayer-git-9999 ... * Starting bootstrap Submodule 'ffmpeg' (git://repo.or.cz/FFMpeg-mirror/mplayer-patches.git) registered for path 'ffmpeg' Submodule 'ffmpeg-mt' (git://repo.or.cz/FFMpeg-mirror/mplayer-patches.git) registered for path 'ffmpeg-mt' Submodule 'libass' (git://repo.or.cz/libass.git) registered for path 'libass' Submodule 'mplayer' (git://repo.or.cz/mplayer.git) registered for path 'mplayer' fatal: destination path 'mplayer' already exists and is not an empty directory. Traceback (most recent call last): File "./init", line 77, in <module> main() File "./init", line 59, in main git.submodule_clone('mplayer') File "/mnt/g/portage/media-video/mplayer-git-9999/work/mplayer-git-9999/script/helpers.py", line 66, in submodule_clone check_call('git clone'.split() + shallow_args + [repo_addr, name]) File "/usr/lib64/python2.6/subprocess.py", line 488, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['git', 'clone', 'git://repo.or.cz/mplayer.git\n', 'mplayer']' returned non-zero exit status 128 Already up-to-date. * Bootstrap finished ... We should not use this: # unpack to the ${S} popd &> /dev/null debug-print "git clone -l -s \"${GIT_DIR}\" \"${S}\"" git clone -l -s "${GIT_DIR}" "${S}" 'git clone -l -s' does not copy submodules, we can use 'cp -pr' or 'rsync -rlpgo', which work fine for me. I've tried it with rsync. Now qt-core fails: >>> Unpacked to /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999 /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/temp/environment: line 3772: /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999/src/plugins/codecs/kr/kr.pro: No such file or directory /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/temp/environment: line 3772: /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999/src/plugins/codecs/jp/jp.pro: No such file or directory /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/temp/environment: line 3772: /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999/src/plugins/codecs/cn/cn.pro: No such file or directory /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/temp/environment: line 3772: /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999/src/plugins/codecs/tw/tw.pro: No such file or directory >>> Source unpacked in /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work >>> Preparing source in /var/tmp/portage/x11-libs/qt-core-4.6.9999-r1/work/qt-x11-opensource-src-4.6.9999 ... Can't open perl script "bin/syncqt": No such file or directory sed: can't read configure: No such file or directory So rsync doesn't get all needed files. Note that both rsync and cp are not really acceptable, because they add several minutes to unpack time. While the approach using git clone just does a normal checkout, cp and rsync first copy the whole repository (potentially several hundred MB) and then check out the files. For each of the Qt packages, this added about 7 minutes last time I tried (with cp), which amounts to more than 90 minutes just for needlessly copying stuff around in order to emerge Qt. Not acceptable. Since this is *only* needed (it seems) for submodules, can't we make this optional and only have packages actually needing the feature use this, rather than adding a huge penalty for every package using git? I don't particularly look forward to adding several hours to my emerge @live-rebuild just because one or two packages use git submodules. Created attachment 217009 [details, diff]
git.eclass.patch
Please test this yet another attempt to make everything work.
Tested with mplayer from git, works fine for me. seems to work fine for me too =) This bug is obsoleted by git-2 eclass. Where the things are fast enough and should make everyone happy :) |
