Bulk download script for OS X (Panther)

[darryl]

So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link (http://www.darryl.com/getsmug.txt) that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

chmod getsmug.pl 755

Then you run it like so:

./getsmug.pl http://yourname.smugmug.com/gallery/123456

It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

Have fun!

--Darryl

#!/usr/bin/perl

$url = shift @ARGV;

if ($url eq '' || $url !~ /smugmug\.com/i ) {
die "Did you forget something like, say, the URL of a smugmug album?\n"
}

$album = $url ;
$album =~ s/^.*\/(\d+)$/$1/;

system "curl --silent --cookie 'Template=7;DefaultSize=Original' $url > /tmp/smug$album.html" ;

open (DUMP, "</tmp/smug$album.html");

mkdir $album;
chdir $album;

while (<DUMP>) {
if (/Ti.jpg/) {
s/^.+src="(.+)Ti.jpg".+$/$1O.jpg/ ;
system "curl -O $_" ;
}
}

[darryl]

BTW, JT -- I found a small error in your HTML for "All Thumbs" view. For example, in this gallery (http://gladlee.smugmug.com/gallery/132296/)

<td align="center"><center><a href="http://gladlee.smugmug.com/gallery/132296/1/4810754/Original"><img align="center" class="smborderoff" onmouseover="this.className='smborder';" onmouseout="this.className='smborderoff';" name="mainPhoto" alt="Glady, Darryl and Noah &gt; Friends photo" border="0" src="http://gladlee.smugmug.com/photos/4810754-Ti.jpg" width="66" width="100" hspace="0" vspace="0" /></a></center></td>

There's two 'width' tags within that <img> tag, and the tag is closed with a /> ? That ain't right. :-}

[{JT}]

Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

http://fog.ccsf.org/~srubin/xhtmltags.html

[darryl]

Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

http://fog.ccsf.org/~srubin/xhtmltags.html

Ah right -- I had a feeling that one was intentional. Thanks!

[Guzzler]

Ooof,

I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?

[darryl]

Ooof,

I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?

Crap -- I should have specified that it also relies on an external program, curl, which allows you to do:

curl -O http://whatever.com/foo.jpg

And it'll save foo.jpg to your current directory.

You can get it for Win32 here:

http://curl.siamu.ac.th/download.html

Within the script, you might need to specify the full path to curl (c:\program files\curl\curl.exe or whatever), as well as a different path for /tmp (c:\temp might work.)

Other than that, it's not too fancy.

Oh oh, and #!/usr/bin/perl might need to be #!c:\program files\perl\perl.exe although I believe ActivePerl is supposed to be smart enough to figure that part out, so try that last. :-}

--Darryl

[Guzzler]

Getting there, close but no cigar.

Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.

[darryl]

Getting there, close but no cigar.

Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.

Hey Guz --

Apparently Windows doesn't like single quotes, so I think you'll need to try:

system qq|curl.exe --silent --cookie "Template=7;DefaultSize=Original" $url > /tmp/smug$album.html| ;

(qq| tells Perl to treat the | character as a delimiter, instead of a double-quote, but still interpolate the $variables within the string)

--Darryl

[Guzzler]

Darryl,

We're almost home!!!

You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).

[Baldy]

Darryl,

We're almost home!!!

You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.

[Guzzler]

Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.
Aaaa, so it needs to be a wildcard then.

FYI, I'm trying to help Ricky create some animations from the Ouray adventure. I just don't feel like "right clicking, save-as" for all the pictures.

[wooac]

How do you modify the --cookie line for password protected galleries?

Thanks.

So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link (http://www.darryl.com/getsmug.txt) that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

chmod getsmug.pl 755

Then you run it like so:

./getsmug.pl http://yourname.smugmug.com/gallery/123456

It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

Have fun!

--Darryl

[wooac]

The getsmug.pl script now fails to obtain a webpage with the urls of the photos within a gallery. Has anyone updated it to obtain that information?

[darryl]

Sorry, for security reasons (http://www.dgrin.com/showthread.php?t=84543) SmugMug disabled the ability to link to a specific Style for a gallery. So I can't link to "All Thumbs" anymore (well, ok, I had some ideas (http://www.dgrin.com/showthread.php?p=843285#post843285) on how to get around this, but haven't had a chance to test them yet.)