Hi! It will be a nice thing if the webapp-config has an option to copy the files even they the source and the target are on the same filesystem. I think it isn't a big change and it can helpful some times. Thanks, Arpad Reproducible: Always Steps to Reproduce: 1. webapp-config -I -d phpmyadmin -s phpmyadmin phpmyadmin 2.11.7 Of course, you can use any other webapp enabled ebuild. Actual Results: The files in /var/www/phpmyadmin/htdocs/phpmyadmin are hard links "in to" the /usr/share/webapps. Expected Results: The files aren't hard links, they just copied.
I'm not sure about doing this. Its an old bug, are you still interested?
It was needed for our special packaging environment. I already made a workaround in our codebase to handle the hardlinks but I think this can be useful in another situation too. The decision is yours, I didn't need this fixed for my work.
Octo_Azul ~ # ls -ialh /var/www/localhost/htdocs/ total 32K 3166497 drwxr-xr-x 8 root root 4.0K Feb 18 15:19 . 2909779 drwxr-xr-x 5 root root 4.0K Feb 18 15:18 .. 3555297 drwxr-xr-x 2 root root 4.0K Feb 11 22:30 bugzilla 3549088 drwxr-xr-x 10 root root 4.0K Feb 17 21:51 drupal 3558842 drwxr-xr-x 14 root root 4.0K Feb 5 20:19 ganglia-web 3166498 drwxr-xr-x 2 root root 4.0K Jan 24 05:07 htdig 3304723 drwxr-xr-x 2 root root 4.0K Feb 18 15:03 moinmoin 4723355 drwxr-xr-x 43 root root 4.0K Feb 17 23:30 moodle Octo_Azul ~ # ls -ialh /usr/share/webapps/ total 36K 2364990 drwxr-xr-x 7 root root 4.0K Feb 18 15:03 . 1968920 drwxr-xr-x 266 root root 12K Feb 18 15:03 .. 2930387 -rw-r--r-- 1 root root 0 Feb 18 14:50 .keep_app-admin_webapp-config-0 2365340 drwxr-xr-x 3 root root 4.0K Feb 4 23:22 drupal 2365348 drwxr-xr-x 3 root root 4.0K Feb 5 20:19 ganglia-web 2361412 drwxr-xr-x 3 root root 4.0K Feb 18 15:03 moinmoin 2364860 drwxr-xr-x 3 root root 4.0K Feb 4 20:45 moodle 2755867 drwxr-xr-x 3 root root 4.0K Feb 11 21:42 phpmyadmin I don't think webapp-config makes hardlinks anymore. Seeing as the inodes for these files are not the same :)
(In reply to comment #3) > Octo_Azul ~ # ls -ialh /var/www/localhost/htdocs/ > total 32K > 3166497 drwxr-xr-x 8 root root 4.0K Feb 18 15:19 . > 2909779 drwxr-xr-x 5 root root 4.0K Feb 18 15:18 .. > 3555297 drwxr-xr-x 2 root root 4.0K Feb 11 22:30 bugzilla > 3549088 drwxr-xr-x 10 root root 4.0K Feb 17 21:51 drupal > 3558842 drwxr-xr-x 14 root root 4.0K Feb 5 20:19 ganglia-web > 3166498 drwxr-xr-x 2 root root 4.0K Jan 24 05:07 htdig > 3304723 drwxr-xr-x 2 root root 4.0K Feb 18 15:03 moinmoin > 4723355 drwxr-xr-x 43 root root 4.0K Feb 17 23:30 moodle > Octo_Azul ~ # ls -ialh /usr/share/webapps/ > total 36K > 2364990 drwxr-xr-x 7 root root 4.0K Feb 18 15:03 . > 1968920 drwxr-xr-x 266 root root 12K Feb 18 15:03 .. > 2930387 -rw-r--r-- 1 root root 0 Feb 18 14:50 > .keep_app-admin_webapp-config-0 > 2365340 drwxr-xr-x 3 root root 4.0K Feb 4 23:22 drupal > 2365348 drwxr-xr-x 3 root root 4.0K Feb 5 20:19 ganglia-web > 2361412 drwxr-xr-x 3 root root 4.0K Feb 18 15:03 moinmoin > 2364860 drwxr-xr-x 3 root root 4.0K Feb 4 20:45 moodle > 2755867 drwxr-xr-x 3 root root 4.0K Feb 11 21:42 phpmyadmin > > I don't think webapp-config makes hardlinks anymore. Seeing as the inodes > for these files are not the same :) Read the code just in case there's any left over from the hard link days. Look for any python-ism which copy directories (import shutils) versus make hard links (import os; os.link). But I think you're correct because every time I use webapp-config to install it just copies files over. Its lots of work, but I don't see an easy global way around this because, for example, each vhost instance will need its own config file in the web apps' root. I'll close this obsolete for now. Reopen if you do find anything.
> Read the code just in case there's any left over from the hard link days. > Look for any python-ism which copy directories (import shutils) versus make > hard links (import os; os.link). But I think you're correct because every > time I use webapp-config to install it just copies files over. Its lots of > work, but I don't see an easy global way around this because, for example, > each vhost instance will need its own config file in the web apps' root. > > I'll close this obsolete for now. Reopen if you do find anything. If you look at /usr/lib/python2.7/site-packages/Webapp-config/worker.py you'll see tons of commented out code for the class WebappAdd class WebappRemove but if you look at line 482 you'll see this tidbit of info: # default behaviour is to hard link (if we can), and # to copy if we cannot the latest webapp-conf (from the git head) also states this in the worker.py file: # some applications (/me points at PHP scripts) # won't run if symlinked in. # so we now support copying files in too Although, through inspection of the man webapp-config pages it doesn't really say anything about enabling a flag for copying the files when installing instead of hardlinking or softlinking. I do think that the man pages might be a little outdated though. So from what I experienced, the files are indeed copied but I don't necessarily see a flag to enable copying over the other two options.
Beyond the comment is the code there to hard-link? I would rather remove it and just do all copy. That is the safest (albeit slowest) way of doing this.
(In reply to comment #6) > Beyond the comment is the code there to hard-link? I would rather remove it > and just do all copy. That is the safest (albeit slowest) way of doing this. There is code in there to hard-link and soft-link. Do you think it'd be best to only comment out the hard-linking code? Or would you rather both be commented out and resort to directly copying as the method that webapp-config uses?
(In reply to comment #7) > (In reply to comment #6) > > Beyond the comment is the code there to hard-link? I would rather remove it > > and just do all copy. That is the safest (albeit slowest) way of doing this. > > There is code in there to hard-link and soft-link. Do you think it'd be best > to only comment out the hard-linking code? Or would you rather both be > commented out and resort to directly copying as the method that > webapp-config uses? See if you can add a flag.
Created attachment 340810 [details, diff] Adds --copy flag for webapp-config
Created attachment 340812 [details, diff] config.py: Adds copy flag for webapp-config The above file is worker.py and allows for the actual copying of the files. This allows for the flag to be used.
Created attachment 340816 [details, diff] Adds --copy flag and copying of files for webapp installation
(In reply to comment #11) > Created attachment 340816 [details, diff] [details, diff] > Adds --copy flag and copying of files for webapp installation Please update the man page and make sure this doesn't effect web-app.eclass's use of webapp-config. If it does, restrict the eclass's dependency on just versions of webapp-config which follows the current behavior.
It doesn't affect the eclass when installing webapps, I haven't tried implementing the --cp flag into the ebuild and forcing that to be used as a default, though. Without this though, the ebuild works just as it would typically work. And the man pages will be updated as soon as I figure out how to edit man pages :D Also, do you think it'd be best to implement this copying behavior into the ebuild to be used as a default? And possibly make the hardlinking installation process a flag we could use?
(In reply to comment #13) > It doesn't affect the eclass when installing webapps, I haven't tried > implementing the --cp flag into the ebuild and forcing that to be used as a > default, though. Without this though, the ebuild works just as it would > typically work. > > And the man pages will be updated as soon as I figure out how to edit man > pages :D > > Also, do you think it'd be best to implement this copying behavior into the > ebuild to be used as a default? And possibly make the hardlinking > installation process a flag we could use? hardlinking on the same filesystem saves space. i'm leaning towards keeping it the default behavior.
Created attachment 343568 [details, diff] Alters man page to make additions for --copy flag and it's uses
Created attachment 343574 [details] Alters man page to make additions for --copy flag and it's uses This is the same patch, but it was created using git format-patch, as opposed to diff -Naur. Thanks to bonsaikitten for the help.
Created attachment 343688 [details, diff] final pushed patch Take a look at this patch which combines the above and was pushed. http://git.overlays.gentoo.org/gitweb/?p=proj/webapp-config.git;a=commit;h=55d20d366085adcfd8783d56de11bb024a446bb1
this has been tested and will be out in the next release.
Created attachment 349180 [details, diff] Adds copy option in /etc/vhosts/webapp-config configuration file. This patch adds support to set vhost_link_type to "copy" as well as fixing a slight blunder on my part for the --copy flag.
(In reply to Devan Franchini from comment #19) > Created attachment 349180 [details, diff] [details, diff] > Adds copy option in /etc/vhosts/webapp-config configuration file. > > This patch adds support to set vhost_link_type to "copy" as well as fixing a > slight blunder on my part for the --copy flag. Please explain this patch. Is it a replacement of the previous one or is it in addition to?
(In reply to Anthony Basile from comment #20) > (In reply to Devan Franchini from comment #19) > > Created attachment 349180 [details, diff] [details, diff] [details, diff] > > Adds copy option in /etc/vhosts/webapp-config configuration file. > > > > This patch adds support to set vhost_link_type to "copy" as well as fixing a > > slight blunder on my part for the --copy flag. > > Please explain this patch. Is it a replacement of the previous one or is it > in addition to? It should be a replacement to the previous patch. When I created the previous patch I did not realize that config.py line 847 was very specific on how it mapped a specific value with the command line option. So when you would execute webapp-config with the --cp or --copy flag, the "g_copy" link value was not passed down throughout the rest of the program. This was then ignored in worker.py line 511 and defaulted to the hardlinking operation instead of copying the file using shutil.copy(). What I did was remove the --cp flag so that only the --copy flag existed. This made it possible for the "g_copy" value to not be ignored and then to be passed down throughout the rest of the program, which made it possible to actually copy the files instead of hardlink against them. While doing this I also noticed that I named the "g_link_type" for copying "g_clone" instead of "g_copy", so I altered that to make more sense when someone else was going through the code. I also added the ability to edit the configuration file in /etc/vhosts/webapp-config, line 154 to allow setting the vhost_link_type to "copy", this will also set the link value to "g_copy" instead of constantly using the --copy flag option. I allowed the setting by checking the configuration to see if the "vhost_link_type" is enabled, as well as the checking for the value associated with that option, on line 1001. To reproduce the behaviour of this patch do the following: For testing the --copy flag behavior: 1.) Install a webapp using the --copy flag: webapp-config -I -d <dir> <webapp name> <version> 2.) cd into the directory that holds the files for the webapp: cd /var/www/<vhost>/htdocs/<dir> 3.) Check the inodes of the files in the directory: ls -li 4.) Then check the inodes of the files in the /usr/share/webapps/<webapp name>/<version>/htdocs/: ls -li /usr/share/webapps/<webapp name>/<version>/htdocs/ Compare the two. They should not be the same. Having the same inode indicates that the two files are actually the same. For testing /etc/vhosts/webapp-config "vhost_link_type" setting: 1.) Edit you /etc/vhosts/webapp-config file on line 236, and set "vhost_link_type=copy" 2.) Repeat webapp installation process as before, but without adding the --copy flag. 3.) Check the inodes as previously described.
> 1.) Edit you /etc/vhosts/webapp-config file on line 236, and set > "vhost_link_type=copy" I meant to put line 154, my mistake.
I also need to edit the man page to reflect the removal of the --cp flag.
Created attachment 351592 [details, diff] Adds the copy option in /etc/vhosts/webapp-config as well as repairs the behavior of the --copy flag and man page alterations to relfect the removal of the --cp flag. This patch deprecates patch 349180 and 343574 by combining the man page changes and the changes to config.py and worker.py for the --copy flag.
(In reply to Devan Franchini from comment #24) > Created attachment 351592 [details, diff] [details, diff] > Adds the copy option in /etc/vhosts/webapp-config as well as repairs the > behavior of the --copy flag and man page alterations to relfect the removal > of the --cp flag. > > This patch deprecates patch 349180 and 343574 by combining the man page > changes and the changes to config.py and worker.py for the --copy flag. Devan, please make your commit messages conform to the following <one line brief summary < 72 chars> <blank line> <paragraph explaining the commit in greater detail < 80 chars wrap> Notice how your git format-patch puts everything into the subject line. That would be disasterous to commit!
Created attachment 351602 [details, diff] Adds the copy option in /etc/vhosts/webapp-config as well as repairs the behavior of the --copy flag and man page alterations to relfect the removal of the --cp flag. Cleans up git commit message for patch.
(In reply to Anthony Basile from comment #25) > (In reply to Devan Franchini from comment #24) > > Devan, please make your commit messages conform to the following > > <one line brief summary < 72 chars> > <blank line> > <paragraph explaining the commit in greater detail < 80 chars wrap> > > Notice how your git format-patch puts everything into the subject line. > That would be disasterous to commit! I will clean up the other commit messages I put in my earlier patches too :)
(In reply to Devan Franchini from comment #27) > (In reply to Anthony Basile from comment #25) > > (In reply to Devan Franchini from comment #24) > > > > Devan, please make your commit messages conform to the following > > > > <one line brief summary < 72 chars> > > <blank line> > > <paragraph explaining the commit in greater detail < 80 chars wrap> > > > > Notice how your git format-patch puts everything into the subject line. > > That would be disasterous to commit! > > I will clean up the other commit messages I put in my earlier patches too :) I pushed out webapp-config 1.51 without your latest fix and with commit http://git.overlays.gentoo.org/gitweb/?p=proj/webapp-config.git;a=commit;h=55d20d366085adcfd8783d56de11bb024a446bb1 Can we please get this issue resolved as I would like to clean it up for the next release which should go out in a month or so.
(In reply to Anthony Basile from comment #28) > (In reply to Devan Franchini from comment #27) > > (In reply to Anthony Basile from comment #25) > > > (In reply to Devan Franchini from comment #24) > > > > > > Devan, please make your commit messages conform to the following > > > > > > <one line brief summary < 72 chars> > > > <blank line> > > > <paragraph explaining the commit in greater detail < 80 chars wrap> > > > > > > Notice how your git format-patch puts everything into the subject line. > > > That would be disasterous to commit! > > > > I will clean up the other commit messages I put in my earlier patches too :) > > I pushed out webapp-config 1.51 without your latest fix and with commit > > http://git.overlays.gentoo.org/gitweb/?p=proj/webapp-config.git;a=commit; > h=55d20d366085adcfd8783d56de11bb024a446bb1 > > Can we please get this issue resolved as I would like to clean it up for the > next release which should go out in a month or so. The commit message was cleaned, the feature was tested on my system, and it should be good to go when you add it to the next release. It was good beforehand too. It could've been added to 1.51. attachment 351602 [details, diff] includes the changes to config.py, worker.py, and the man page to show the removal of the --cp flag, as well as a proper commit message.
*** Bug 196469 has been marked as a duplicate of this bug. ***
