Bulk download script for OS X (Panther)

darryldarryl Registered Users Posts: 997 Major grins
So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

chmod getsmug.pl 755

Then you run it like so:

./getsmug.pl http://yourname.smugmug.com/gallery/123456

It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

Have fun!

--Darryl
#!/usr/bin/perl

$url = shift @ARGV;

if ($url eq '' || $url !~ /smugmug\.com/i ) {
die "Did you forget something like, say, the URL of a smugmug album?\n"
}

$album = $url ;
$album =~ s/^.*\/(\d+)$/$1/;

system "curl --silent --cookie 'Template=7;DefaultSize=Original' $url > /tmp/smug$album.html" ;

open (DUMP, "</tmp/smug$album.html");

mkdir $album;
chdir $album;

while (<DUMP>) {
if (/Ti.jpg/) {
s/^.+src="(.+)Ti.jpg".+$/$1O.jpg/ ;
system "curl -O $_" ;
}
}

Comments

  • darryldarryl Registered Users Posts: 997 Major grins
    edited June 10, 2004
    BTW, JT -- I found a small error in your HTML for "All Thumbs" view. For example, in this gallery

    <td align="center"><center><a href="http://gladlee.smugmug.com/gallery/132296/1/4810754/Original"><img align="center" class="smborderoff" onmouseover="this.className='smborder';" onmouseout="this.className='smborderoff';" name="mainPhoto" alt="Glady, Darryl and Noah > Friends photo" border="0" src="http://gladlee.smugmug.com/photos/4810754-Ti.jpg&quot; width="66" width="100" hspace="0" vspace="0" /></a></center></td>

    There's two 'width' tags within that <img> tag, and the tag is closed with a /> ? That ain't right. :-}
  • {JT}{JT} Registered Users Posts: 1,016 Major grins
    edited June 14, 2004
    Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

    http://fog.ccsf.org/~srubin/xhtmltags.html
  • darryldarryl Registered Users Posts: 997 Major grins
    edited June 17, 2004
    {JT} wrote:
    Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

    http://fog.ccsf.org/~srubin/xhtmltags.html

    Ah right -- I had a feeling that one was intentional. Thanks!
  • GuzzlerGuzzler Registered Users Posts: 73 Big grins
    edited September 1, 2004
    Ooof,

    I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?
  • darryldarryl Registered Users Posts: 997 Major grins
    edited September 2, 2004
    Guzzler wrote:
    Ooof,

    I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?

    Crap -- I should have specified that it also relies on an external program, curl, which allows you to do:

    curl -O http://whatever.com/foo.jpg

    And it'll save foo.jpg to your current directory.

    You can get it for Win32 here:

    http://curl.siamu.ac.th/download.html

    Within the script, you might need to specify the full path to curl (c:\program files\curl\curl.exe or whatever), as well as a different path for /tmp (c:\temp might work.)

    Other than that, it's not too fancy.

    Oh oh, and #!/usr/bin/perl might need to be #!c:\program files\perl\perl.exe although I believe ActivePerl is supposed to be smart enough to figure that part out, so try that last. :-}

    --Darryl
  • GuzzlerGuzzler Registered Users Posts: 73 Big grins
    edited September 2, 2004
    Getting there, close but no cigar.

    Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.
  • darryldarryl Registered Users Posts: 997 Major grins
    edited September 8, 2004
    Guzzler wrote:
    Getting there, close but no cigar.

    Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.

    Hey Guz --

    Apparently Windows doesn't like single quotes, so I think you'll need to try:

    system qq|curl.exe --silent --cookie "Template=7;DefaultSize=Original" $url > /tmp/smug$album.html| ;

    (qq| tells Perl to treat the | character as a delimiter, instead of a double-quote, but still interpolate the $variables within the string)

    --Darryl
  • GuzzlerGuzzler Registered Users Posts: 73 Big grins
    edited September 8, 2004
    Darryl,

    We're almost home!!!

    You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited September 8, 2004
    Guzzler wrote:
    Darryl,

    We're almost home!!!

    You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).
    Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.
  • GuzzlerGuzzler Registered Users Posts: 73 Big grins
    edited September 8, 2004
    Baldy wrote:
    Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.
    Aaaa, so it needs to be a wildcard then.

    FYI, I'm trying to help Ricky create some animations from the Ouray adventure. I just don't feel like "right clicking, save-as" for all the pictures.
  • wooacwooac Registered Users Posts: 8 Beginner grinner
    edited December 13, 2004
    What about password protected galleries?
    How do you modify the --cookie line for password protected galleries?

    Thanks.
    darryl wrote:
    So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

    I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

    So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

    It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

    chmod getsmug.pl 755

    Then you run it like so:

    ./getsmug.pl http://yourname.smugmug.com/gallery/123456

    It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

    Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

    Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

    All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

    Have fun!

    --Darryl
  • wooacwooac Registered Users Posts: 8 Beginner grinner
    edited June 24, 2008
    Has anyone update getsmug.pl for new directory structure?
    The getsmug.pl script now fails to obtain a webpage with the urls of the photos within a gallery. Has anyone updated it to obtain that information?
  • darryldarryl Registered Users Posts: 997 Major grins
    edited July 1, 2008
    Sorry, for security reasons SmugMug disabled the ability to link to a specific Style for a gallery. So I can't link to "All Thumbs" anymore (well, ok, I had some ideas on how to get around this, but haven't had a chance to test them yet.)
Sign In or Register to comment.