antarus did most of the work, even some files showed up in mirrors. All you have to do is figure out where it is, maybe clean the ancient output files and add a cronjob :).
Oh, and it's most likely to be on master rsync or somewhere near.
how much space will this take (since it is going to the mirrors...)? Also, we should get better approval before just adding something to the mirrors, especially with git becoming an option 'soon'.
Hmm, I'm pretty sure I answered that already. The exact amount depends on how many days we want to keep back. Around 110M for the current day snapshot + 2 * 2-3M for each day back.
In that case I don't see a reason not to keep 5 years of snapshots. With weeklys maybe.
The snapshots are direct day-to-day, so long term implies very long update.
hence the weekly comment :P
Weekly won't help you much if you're at middle of a week.
so the sync would go and fetch a couple of more incrimentals, meh
Then it's likely that those all incrementals would be larger than new snapshot :P. I've did a pretty in-depth analysis of this stuff [1,2]. [1]:http://dev.gentoo.org/~mgorny/articles/using-deltas-to-speed-up-squashfs-ebuild-repository-updates.pdf [2]:http://dev.gentoo.org/~mgorny/articles/reducing-squashfs-delta-size-through-partial-decompression.pdf
I've done a fresh manual run of the snapshots, it should be going out now, I think... I had to manually remove the old snapshots, I think the date calculations need fixing (don't look at what the old file is, do a date -> filename thing for 'yesterday' and see if that file exists, if not, do a full fresh snap). If this works out I think we will just need a cronjob for two things, cleaning up old snaps (not sure how we will do this since they kinda rely on eachother) and for doing the snap itself. I need to know how to make sure we don't eventually run out of disk space if this is to work :P
I added a cronjob manually to test things /etc/cron.d/update-masterrsync-sqfs
I don't know if you noticed but I've updated the repo with scripts, and now it does checksumming as well. You need to set up gpg to be able to sign the checksums.
I solved the signing thing, but where's the updated script?
https://bitbucket.org/mgorny/squashfs-gx86/src/master/create-snapshot.bash As for the date, I didn't make the logic rely on `date` output in case it was run before midnight UTC.
well, since I can't open issues on your repo for this, I'll submit it here :P needs * a way (can be a separate script) to keep this from growing indefinitely * I'd like about 30 days to be the limit for the diffs wants * allow me to recover if I've skipped a day (even if that means resetting to a full snapshot) I'm sure I'll think of something else.
we are creating the snapshots now and I have a todo item set to remind me in a few months to check and make sure we aren't taking up too much space. Closing :D
