Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 208499 - Proposed LiveUSB script
Summary: Proposed LiveUSB script
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Everything (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo Release Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-02-01 21:03 UTC by Nelson
Modified: 2013-11-20 23:54 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
liveusb script (liveusb.sh,4.28 KB, text/plain)
2008-02-01 21:03 UTC, Nelson
Details
liveUSB (liveusb.sh,4.28 KB, text/plain)
2008-02-02 00:00 UTC, Nelson
Details
diff for grub support (liveusb.sh.diff,1.75 KB, patch)
2008-02-02 02:48 UTC, Andrew Gaffney (RETIRED)
Details | Diff
new version of script (liveusb.sh,4.88 KB, text/plain)
2008-02-02 02:49 UTC, Andrew Gaffney (RETIRED)
Details
syslinux dependency removed, testing improved (liveusb.diff,2.52 KB, patch)
2008-02-02 11:34 UTC, Nelson
Details | Diff
syslinux dependency removed, added colors/presentation (liveusb.sh,5.97 KB, text/plain)
2008-02-02 16:16 UTC, Nelson
Details
Added prompt before writing to disk,minor changes (liveusb-new.sh,6.33 KB, text/plain)
2008-02-05 22:39 UTC, John Alberts
Details
added format and wipe options, added signal trap with cleanup (liveusb.sh,8.02 KB, text/plain)
2008-02-06 04:45 UTC, John Alberts
Details
added --silent option, fixed double wipe, cleaned up code (liveusb.sh,8.07 KB, text/plain)
2008-02-08 03:39 UTC, John Alberts
Details
Added --noboot, minor code optimization (liveusb.sh,7.99 KB, text/plain)
2008-02-28 23:50 UTC, Nelson
Details
forgot to wipe wipedevice lines (liveusb.sh,7.51 KB, text/plain)
2008-03-03 18:34 UTC, Nelson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nelson 2008-02-01 21:03:00 UTC
Here it is:
Comment 1 Nelson 2008-02-01 21:03:28 UTC
Created attachment 142428 [details]
liveusb script
Comment 2 John Alberts 2008-02-01 23:15:32 UTC
Great script.  Worked for me on an Intel Core2 Duo Dell Inspiron 1720 using the 700M 2007.0 amd64 live cd.
The only thing I noticed is the error message for checking the filesystem seems inappropriate.  The filesystem check is done on lines 30 & 31.
if [ "$USBFS" != "vfat" -a "$USBFS" != "msdos" ]; then
	die "USB filesystem must be vfat or ext[23]"

Seems to me that the part about 'or ext[23]' should be removed, since I believe the filesytem needs to be fat16 or fat32.  If it can be ext3, then the if statement is wrong.
Comment 3 Nelson 2008-02-02 00:00:22 UTC
Created attachment 142445 [details]
liveUSB

You're right, thanks!
Comment 4 Andrew Gaffney (RETIRED) gentoo-dev 2008-02-02 02:48:12 UTC
We're thinking about switching to grub for the x86/amd64 CDs for 2008.0. I've updated the script to support both isolinux and grub. I can't properly test this, since my only machine that can usb boot (my laptop) only does floppy emulation, which breaks it. I'll attach a diff and the new version.
Comment 5 Andrew Gaffney (RETIRED) gentoo-dev 2008-02-02 02:48:58 UTC
Created attachment 142456 [details, diff]
diff for grub support
Comment 6 Andrew Gaffney (RETIRED) gentoo-dev 2008-02-02 02:49:26 UTC
Created attachment 142457 [details]
new version of script
Comment 7 Andrew Gaffney (RETIRED) gentoo-dev 2008-02-02 04:32:43 UTC
For the grub part of the script, it really needs to be a bit more robust to allow for floppy emulation. Some machines (like my laptop) only support USB floppy emulation, so grub sees the USB stick as (fd0) instead of (hd0). This also means that it doesn't support partitions, and the fat32 filesystem has to be directly on the device instead of in a partition.
Comment 8 Nelson 2008-02-02 11:34:15 UTC
Created attachment 142469 [details, diff]
syslinux dependency removed, testing improved

Great, because syslinux apparently has a problem with large usb devices. 

I just removed the syslinux dependancy for the grub releases by using grub MBR, removed the syslinux requirement and added that either grub or syslinux must be installed (also it checks the adequate one later).

This was not tested, since I only have gentoo syslinux releases.
Comment 9 Nelson 2008-02-02 16:16:12 UTC
Created attachment 142506 [details]
syslinux dependency removed, added colors/presentation

(I prefer to post the whole file, if you also want diffs plz say)
Comment 10 John Alberts 2008-02-05 22:39:17 UTC
Created attachment 142753 [details]
Added prompt before writing to disk,minor changes

I think it's always prudent to confirm with the user before wiping a disk.  I've added a confirmation from the user before proceeding.
I also added a VERSION variable at the top of the script and used that for displaying the script version instead of embedding the version in the bottom of the script.
This is my first attempt at bash programming.  Please be gentle. :)
Comment 11 Andrew Gaffney (RETIRED) gentoo-dev 2008-02-05 22:41:54 UTC
There's one (obvious) problem...the contents of the USB stick don't get erased, but they probably should.
Comment 12 John Alberts 2008-02-06 04:45:59 UTC
Created attachment 142770 [details]
added format and wipe options, added signal trap with cleanup

I didn't even realize the script didn't format or wipe the drive before copying the files.
I've added --format and --wipe options.
--format will format the partition to fat16
--wipe will delete all files on the partition.
Also added a trap function to cleanup mount points and temp files in case a signal is caught.
Everything seems to work ok, except I'm having trouble with killing any running cp operations in the cleanup function.  Not sure the best way to do it and I'm getting sleepy.  If anyone wants to fix it, the relevant section is around line 188.
Comment 13 Nelson 2008-02-06 07:19:56 UTC
Why do we need to erase the device? If it's that necessary, I suggest asking the user if it's ok to move the existing files to a 'Docs' folder.

(In reply to comment #12)
I see several things to change in your submission but thanks! 

For example the formatting, it already detects the correct filesystem, no need to force format when it's there. Also the user should be able to begin the process without any deletion. The big_msg function is only for the title. There should be a function to ask questions instead of multiple copy/paste of code :p. No need to wipe (twice?) after formatting. You umount usbdev several times, no need ;) Don't umount cdmnt unless it's from an ISO file. 

I also would like a non interactive script, I was going to improve that and then write an interactive one to work with this one.

What is cleanup() for?


Comment 14 John Alberts 2008-02-06 14:27:09 UTC
(In reply to comment #13)
> Why do we need to erase the device? If it's that necessary, I suggest asking
> the user if it's ok to move the existing files to a 'Docs' folder.
I don't think we NEED to erase the device, but I think it's a nice option, because it allows a user to create a clean bootable usb drive without any extra files on it.  It only formats or wipes the device if the user has specified those options.

> 
> (In reply to comment #12)
> I see several things to change in your submission but thanks! 
> 
> For example the formatting, it already detects the correct filesystem, no need
> to force format when it's there.
It doesn't force a format.  It only formats the device if the user has specified --format as an option.

> Also the user should be able to begin the
> process without any deletion.
It won't delete anything if they don't specify the --format or --wipe options.

> The big_msg function is only for the title. There
> should be a function to ask questions instead of multiple copy/paste of code
> :p.
You're right.  I can make that change tonight if you want.

> No need to wipe (twice?) after formatting.
Unless I'm missing something, I don't think it wipes twice.  If both the --format and --wipe options are specified, it formats first and then sets wipe=0 so it shouldn't wipe the device after formatting.

> You umount usbdev several times,
> no need ;) Don't umount cdmnt unless it's from an ISO file. 
I believe the umounts you are referring to are all in the cleanup().  Cleanup() is assigned to the trap function, so when user aborts the process by hitting ctrl-c, cleanup() is automatically called.  The reason I did this was because I found that when I aborted the script, it left various mount points and also temporary directories.  Hence the reason for the multiple umounts, so I'm sure that no matter where the script is aborted, everything should be unmounted and all temporary files and directories have been removed.

> I also would like a non interactive script, I was going to improve that and
> then write an interactive one to work with this one.
I don't see the need to write and maintain two scripts.  Why not just add a --confirm switch to this script and we can bypass all prompts?  I agree a non-interactive option would be nice.
 
> What is cleanup() for?
See explanation of cleanup() above.
 

I apologize if the code is a little messy or if I have done something wrong in the code.  I'm pretty new at writing scripts in bash.  I just thought this script was very cool and I wanted to see if I could help improve it.  
Comment 15 Nelson 2008-02-06 15:55:19 UTC
(In reply to comment #14)

cleanup is great!

> It only formats or wipes the device if the user has specified
> those options.

Ok, it's just the initial message that's wrong :p


> > No need to wipe (twice?) after formatting.
> Unless I'm missing something, I don't think it wipes twice.  

You're missing something :p

> I don't see the need to write and maintain two scripts.  Why not just add a
> --confirm switch to this script and we can bypass all prompts?  I agree a
> non-interactive option would be nice.

Ok, can you do that? I was thinking of 'dialog' on the second script.

> 
> I apologize if the code is a little messy or if I have done something wrong in
> the code.  I'm pretty new at writing scripts in bash.  I just thought this
> script was very cool and I wanted to see if I could help improve it.  
> 

Yes and you helped. Let me know of these and the TODO's you want to do, and I'll pick up from you and correct/improve some details ;) (like you don't need a 10 lines of wipedevice() when in one well placed line you could write '[ $wipe ] && rm ${utmp}' :p cheers
Comment 16 Chris Gianelloni (RETIRED) gentoo-dev 2008-02-07 23:24:52 UTC
(In reply to comment #15)
> > I don't see the need to write and maintain two scripts.  Why not just add a
> > --confirm switch to this script and we can bypass all prompts?  I agree a
> > non-interactive option would be nice.
> 
> Ok, can you do that? I was thinking of 'dialog' on the second script.

Umm... no.

This is run on the *outside* of the CD.  This means it needs to use as few external dependencies as possible, or it won't work on nearly as many machines.  It also needs to be completely non-interactive (command line options are fine) so it can be used in scripts and also used by any potential front-ends that could call it.
Comment 17 Chris Gianelloni (RETIRED) gentoo-dev 2008-02-07 23:25:52 UTC
Oh yeah, I'd also prefer it if it didn't limit itself to only being used for USB sticks.  For example, I know of several people who like to put a copy of the minimal CD on a partition of its own, as a rescue.
Comment 18 John Alberts 2008-02-08 03:39:42 UTC
Created attachment 142954 [details]
added --silent option, fixed double wipe, cleaned up code

Ok. I figured out how I was wiping the drive twice and fixed it. :)
Added a --silent option.
Cleaned up some of my messy and lengthy code.
I'm not sure how to rewrite wipedevice() any shorter without eliminating error checks and information.
Added a TODO item about fixing the cleanup function to stop running copy operations.


(In reply to comment #17)
> Oh yeah, I'd also prefer it if it didn't limit itself to only being used for
> USB sticks.  For example, I know of several people who like to put a copy of
> the minimal CD on a partition of its own, as a rescue.

Sorry.  I really have no idea how to do this.  Also, we would have to change the name of the script to something besides liveusb.sh. ;)
Comment 19 Nelson 2008-02-28 23:50:35 UTC
Created attachment 144917 [details]
Added --noboot, minor code optimization

Ok, afaik there's nothing much to do here to correspond to that request. It still boots with slowusb, but the rest is the same I think. I just added --noboot so that it doesn't go and mess hard drives boot schemes.

John, notice how I changed the wipe function ;)
Comment 20 Nelson 2008-03-03 18:34:24 UTC
Created attachment 145227 [details]
forgot to wipe wipedevice lines
Comment 21 David Kaufman 2008-04-17 15:24:34 UTC
I used this script to create a Gentoo LiveUSB from a machine running CentOS - thanks very much, with some tweaks it worked great!

From a CentOS perspective I had to tweak the following items:
 - vold_id is not available, I just hardwired USBFS
 - mkdosfs is in /sbin
 - grub and grub-install are in /sbin, the script assumes /sbin is in PATH but it was not
 - /lib/grub is called /usr/share/grub

I just hardcoded around these, if you intend for the script to work outside Gentoo you may want to have it find the right paths for the various programs and set them to variables which are used later in the script. (Checking for programs up front would also get around the issue that the current script can run for a long time before running into, e.g., the "Grub is required" error.)

Thanks again - it's always great finding just the tool I need, already written.
Comment 22 Nelson 2008-07-19 13:17:21 UTC
(In reply to comment #21)
Thanks David, I'll have to research a bit to solve that.

Still, this is for releng, one of the purposes of this script was to be able to create a liveusb within the livecd environment. Like OpenSolaris and the upcoming Ubuntu 8.10 livecd's. For that the script is ready, can you include it in 2008.1?.

To launch it documentation should be added to the handbook as well on chapter 2 (including agaffney's comment #7). I can do it if nobody in the docs project volunteers.

Further applications like running it outside the cd and installing it in a harddrive (which must be trivial with this) can be written by people who discover this script once we get it out there.
Comment 23 Nelson 2008-08-27 10:29:19 UTC
Humm, apparently there was already a project that does all this in Linux AND Windows: http://unetbootin.sourceforge.net/

You can still include this script since it's lightweight, for the rest this looks better. I could do maintenance of both to ensure it works for every release.
Comment 24 Chí-Thanh Christopher Nguyễn gentoo-dev 2009-01-02 18:46:22 UTC
The script would become unnecessary if Gentoo could use ISOLINUX 3.72 hybrid cd-rom/hard disk mode (bug 251719)
Comment 25 pier 2009-11-13 09:40:44 UTC
Is it possible to alter the script in order to get, for example:

install-ia64-minimal-20091110.iso 

installed on a live usb stick?

Thanks
Comment 26 Fernando (likewhoa) 2009-11-13 17:30:03 UTC
(In reply to comment #24)
> The script would become unnecessary if Gentoo could use ISOLINUX 3.72 hybrid
> cd-rom/hard disk mode (bug 251719)
> 

I agree and we only need to add a few small trivial patches to catalyst script in order to take advantage of syslinux's isohybrid feature. I will be focusing on this in the coming weeks.
Comment 27 Ben Kohler gentoo-dev 2013-11-20 23:54:41 UTC
This change has been made in catalyst early this year, all current isos are dd-ready.