Bugzilla Bug 208499

Here it is:

Created an attachment (id=142428) [details]
liveusb script

Great script.  Worked for me on an Intel Core2 Duo Dell Inspiron 1720 using the
700M 2007.0 amd64 live cd.
The only thing I noticed is the error message for checking the filesystem seems
inappropriate.  The filesystem check is done on lines 30 & 31.
if [ "$USBFS" != "vfat" -a "$USBFS" != "msdos" ]; then
        die "USB filesystem must be vfat or ext[23]"

Seems to me that the part about 'or ext[23]' should be removed, since I believe
the filesytem needs to be fat16 or fat32.  If it can be ext3, then the if
statement is wrong.

Created an attachment (id=142445) [details]
liveUSB

You're right, thanks!

We're thinking about switching to grub for the x86/amd64 CDs for 2008.0. I've
updated the script to support both isolinux and grub. I can't properly test
this, since my only machine that can usb boot (my laptop) only does floppy
emulation, which breaks it. I'll attach a diff and the new version.

Created an attachment (id=142456) [details]
diff for grub support

Created an attachment (id=142457) [details]
new version of script

For the grub part of the script, it really needs to be a bit more robust to
allow for floppy emulation. Some machines (like my laptop) only support USB
floppy emulation, so grub sees the USB stick as (fd0) instead of (hd0). This
also means that it doesn't support partitions, and the fat32 filesystem has to
be directly on the device instead of in a partition.

Created an attachment (id=142469) [details]
syslinux dependency removed, testing improved

Great, because syslinux apparently has a problem with large usb devices. 

I just removed the syslinux dependancy for the grub releases by using grub MBR,
removed the syslinux requirement and added that either grub or syslinux must be
installed (also it checks the adequate one later).

This was not tested, since I only have gentoo syslinux releases.

Created an attachment (id=142506) [details]
syslinux dependency removed, added colors/presentation

(I prefer to post the whole file, if you also want diffs plz say)

Created an attachment (id=142753) [details]
Added prompt before writing to disk,minor changes

I think it's always prudent to confirm with the user before wiping a disk. 
I've added a confirmation from the user before proceeding.
I also added a VERSION variable at the top of the script and used that for
displaying the script version instead of embedding the version in the bottom of
the script.
This is my first attempt at bash programming.  Please be gentle. :)

There's one (obvious) problem...the contents of the USB stick don't get erased,
but they probably should.

Created an attachment (id=142770) [details]
added format and wipe options, added signal trap with cleanup

I didn't even realize the script didn't format or wipe the drive before copying
the files.
I've added --format and --wipe options.
--format will format the partition to fat16
--wipe will delete all files on the partition.
Also added a trap function to cleanup mount points and temp files in case a
signal is caught.
Everything seems to work ok, except I'm having trouble with killing any running
cp operations in the cleanup function.  Not sure the best way to do it and I'm
getting sleepy.  If anyone wants to fix it, the relevant section is around line
188.

Why do we need to erase the device? If it's that necessary, I suggest asking
the user if it's ok to move the existing files to a 'Docs' folder.

(In reply to comment #12)
I see several things to change in your submission but thanks! 

For example the formatting, it already detects the correct filesystem, no need
to force format when it's there. Also the user should be able to begin the
process without any deletion. The big_msg function is only for the title. There
should be a function to ask questions instead of multiple copy/paste of code
:p. No need to wipe (twice?) after formatting. You umount usbdev several times,
no need ;) Don't umount cdmnt unless it's from an ISO file. 

I also would like a non interactive script, I was going to improve that and
then write an interactive one to work with this one.

What is cleanup() for?

(In reply to comment #13)
> Why do we need to erase the device? If it's that necessary, I suggest asking
> the user if it's ok to move the existing files to a 'Docs' folder.
I don't think we NEED to erase the device, but I think it's a nice option,
because it allows a user to create a clean bootable usb drive without any extra
files on it.  It only formats or wipes the device if the user has specified
those options.

> 
> (In reply to comment #12)
> I see several things to change in your submission but thanks! 
> 
> For example the formatting, it already detects the correct filesystem, no need
> to force format when it's there.
It doesn't force a format.  It only formats the device if the user has
specified --format as an option.

> Also the user should be able to begin the
> process without any deletion.
It won't delete anything if they don't specify the --format or --wipe options.

> The big_msg function is only for the title. There
> should be a function to ask questions instead of multiple copy/paste of code
> :p.
You're right.  I can make that change tonight if you want.

> No need to wipe (twice?) after formatting.
Unless I'm missing something, I don't think it wipes twice.  If both the
--format and --wipe options are specified, it formats first and then sets
wipe=0 so it shouldn't wipe the device after formatting.

> You umount usbdev several times,
> no need ;) Don't umount cdmnt unless it's from an ISO file. 
I believe the umounts you are referring to are all in the cleanup().  Cleanup()
is assigned to the trap function, so when user aborts the process by hitting
ctrl-c, cleanup() is automatically called.  The reason I did this was because I
found that when I aborted the script, it left various mount points and also
temporary directories.  Hence the reason for the multiple umounts, so I'm sure
that no matter where the script is aborted, everything should be unmounted and
all temporary files and directories have been removed.

> I also would like a non interactive script, I was going to improve that and
> then write an interactive one to work with this one.
I don't see the need to write and maintain two scripts.  Why not just add a
--confirm switch to this script and we can bypass all prompts?  I agree a
non-interactive option would be nice.

> What is cleanup() for?
See explanation of cleanup() above.


I apologize if the code is a little messy or if I have done something wrong in
the code.  I'm pretty new at writing scripts in bash.  I just thought this
script was very cool and I wanted to see if I could help improve it.  

(In reply to comment #14)

cleanup is great!

> It only formats or wipes the device if the user has specified
> those options.

Ok, it's just the initial message that's wrong :p


> > No need to wipe (twice?) after formatting.
> Unless I'm missing something, I don't think it wipes twice.  

You're missing something :p

> I don't see the need to write and maintain two scripts.  Why not just add a
> --confirm switch to this script and we can bypass all prompts?  I agree a
> non-interactive option would be nice.

Ok, can you do that? I was thinking of 'dialog' on the second script.

> 
> I apologize if the code is a little messy or if I have done something wrong in
> the code.  I'm pretty new at writing scripts in bash.  I just thought this
> script was very cool and I wanted to see if I could help improve it.  
> 

Yes and you helped. Let me know of these and the TODO's you want to do, and
I'll pick up from you and correct/improve some details ;) (like you don't need
a 10 lines of wipedevice() when in one well placed line you could write '[
$wipe ] && rm ${utmp}' :p cheers

(In reply to comment #15)
> > I don't see the need to write and maintain two scripts.  Why not just add a
> > --confirm switch to this script and we can bypass all prompts?  I agree a
> > non-interactive option would be nice.
> 
> Ok, can you do that? I was thinking of 'dialog' on the second script.

Umm... no.

This is run on the *outside* of the CD.  This means it needs to use as few
external dependencies as possible, or it won't work on nearly as many machines.
 It also needs to be completely non-interactive (command line options are fine)
so it can be used in scripts and also used by any potential front-ends that
could call it.

Oh yeah, I'd also prefer it if it didn't limit itself to only being used for
USB sticks.  For example, I know of several people who like to put a copy of
the minimal CD on a partition of its own, as a rescue.

Created an attachment (id=142954) [details]
added --silent option, fixed double wipe, cleaned up code

Ok. I figured out how I was wiping the drive twice and fixed it. :)
Added a --silent option.
Cleaned up some of my messy and lengthy code.
I'm not sure how to rewrite wipedevice() any shorter without eliminating error
checks and information.
Added a TODO item about fixing the cleanup function to stop running copy
operations.


(In reply to comment #17)
> Oh yeah, I'd also prefer it if it didn't limit itself to only being used for
> USB sticks.  For example, I know of several people who like to put a copy of
> the minimal CD on a partition of its own, as a rescue.

Sorry.  I really have no idea how to do this.  Also, we would have to change
the name of the script to something besides liveusb.sh. ;)

Created an attachment (id=144917) [details]
Added --noboot, minor code optimization

Ok, afaik there's nothing much to do here to correspond to that request. It
still boots with slowusb, but the rest is the same I think. I just added
--noboot so that it doesn't go and mess hard drives boot schemes.

John, notice how I changed the wipe function ;)

Created an attachment (id=145227) [details]
forgot to wipe wipedevice lines

I used this script to create a Gentoo LiveUSB from a machine running CentOS -
thanks very much, with some tweaks it worked great!

From a CentOS perspective I had to tweak the following items:
 - vold_id is not available, I just hardwired USBFS
 - mkdosfs is in /sbin
 - grub and grub-install are in /sbin, the script assumes /sbin is in PATH but
it was not
 - /lib/grub is called /usr/share/grub

I just hardcoded around these, if you intend for the script to work outside
Gentoo you may want to have it find the right paths for the various programs
and set them to variables which are used later in the script. (Checking for
programs up front would also get around the issue that the current script can
run for a long time before running into, e.g., the "Grub is required" error.)

Thanks again - it's always great finding just the tool I need, already written.

(In reply to comment #21)
Thanks David, I'll have to research a bit to solve that.

Still, this is for releng, one of the purposes of this script was to be able to
create a liveusb within the livecd environment. Like OpenSolaris and the
upcoming Ubuntu 8.10 livecd's. For that the script is ready, can you include it
in 2008.1?.

To launch it documentation should be added to the handbook as well on chapter 2
(including agaffney's comment #7). I can do it if nobody in the docs project
volunteers.

Further applications like running it outside the cd and installing it in a
harddrive (which must be trivial with this) can be written by people who
discover this script once we get it out there.

Humm, apparently there was already a project that does all this in Linux AND
Windows: http://unetbootin.sourceforge.net/

You can still include this script since it's lightweight, for the rest this
looks better. I could do maintenance of both to ensure it works for every
release.

The script would become unnecessary if Gentoo could use ISOLINUX 3.72 hybrid
cd-rom/hard disk mode (bug 251719)

Is it possible to alter the script in order to get, for example:

install-ia64-minimal-20091110.iso 

installed on a live usb stick?

Thanks

(In reply to comment #24)
> The script would become unnecessary if Gentoo could use ISOLINUX 3.72 hybrid
> cd-rom/hard disk mode (bug 251719)
> 

I agree and we only need to add a few small trivial patches to catalyst script
in order to take advantage of syslinux's isohybrid feature. I will be focusing
on this in the coming weeks.