Turbo-charging SmugMug

BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
edited November 13, 2008 in SmugMug Support
Hey everyone,

The most expensive project I can remember undertaking at SmugMug is our current quest for speed.

1097_3d9VN-XL-2.jpg

We haven't said much about it because it's rolling out in phases instead of a big bang. And we've been reluctant to set high expectations until we've rolled out enough pieces that you can notice.

The short story is we're joining the league of companies with fast websites like Apple who use Akamai's site acceleration. Akamai has many tricks, such as predicting which image you're likely to click on next and having it ready on a server near you before you request it.

Several improvements are live so you should be seeing faster galleries on sites that use the domain smugmug.com.

Domains that end in say, moonriverphotography.com, are not yet Akamaized.

That means the same gallery accessed via williams.smugmug.com

Should appear faster to you than the same gallery accessed via moonriverphotography.com

The further from Silicon Valley you are, the more pronounced the difference.

One of the next phases is to turbo-charge galleries accessed via your custom domains, but first we're sending an email to our customers with custom domains asking for a change in configuration. We want to allow our customers 30 days to complete it.

We didn't realize that some of our customers would see problems with their custom hostnames as we Akamaized, and to them we apologize.

It will take time to roll out all phases and see the big gains in speed we hope for, but at least now if you've been following many dgrin threads you can connect the dots. Threads about thumbnail security? Custom domain configuration changes? Akamai.

Thanks,
Baldy

Comments

  • mike.strockmike.strock Registered Users Posts: 147 Major grins
    edited August 26, 2008
    Baldy wrote:
    Hey everyone,

    The most expensive project I can remember undertaking at SmugMug is our current quest for speed.

    1097_3d9VN-XL-2.jpg

    We haven't said much about it because it's rolling out in phases instead of a big bang. And we've been reluctant to set high expectations until we've rolled out enough pieces that you can notice.

    The short story is we're joining the league of companies with fast websites like Apple who use Akamai's site acceleration. Akamai has many tricks, such as predicting which image you're likely to click on next and having it ready on a server near you before you request it.

    Several improvements are live so you should be seeing faster galleries on sites that use the domain smugmug.com.

    Domains that end in say, moonriverphotography.com, are not yet Akamized.

    That means the same gallery accessed via williams.smugmug.com

    Should appear faster to you than the same gallery accessed via moonriverphotography.com

    The further from Silicon Valley you are, the more pronounced the difference.

    One of the next phases is to turbo-charge galleries accessed via your custom domains, but first we're sending an email to our customers with custom domains asking for a change in configuration. We want to allow our customers 30 days to complete it.

    We didn't realize that some of our customers would see problems with their custom hostnames as we Akamaized, and to them we apologize.

    It will take time to roll out all phases and see the big gains in speed we hope for, but at least now if you've been following many dgrin threads you can connect the dots. Threads about thumbnail security? Custom domain configuration changes? Akamai.

    Thanks,
    Baldy
    I wasn't sure what to cut out of the quote, so I left it all in. Thanks Baldy and everyone else at SM for working on this 'secret' project. I know that it will improve things here dramatically.

    Mike.
  • bkatzbkatz Registered Users Posts: 286 Major grins
    edited August 27, 2008
    Baldy -

    Sounds great and looking forward to the speed when it catches up with custom domain names. great job. Anything which makes it faster is awesome.

    I assume that this is what held up coupons - does this mean while we wait the 30 days for custom domain names we might see movement on coupons?mwink.gif
  • NimaiNimai Registered Users Posts: 564 Major grins
    edited August 27, 2008
    I did suspect there was some CDN work going on! Thank you - good stuff!
  • SheafSheaf Registered Users, SmugMug Product Team Posts: 775 SmugMug Employee
    edited August 27, 2008
    bkatz wrote:
    Baldy -

    Sounds great and looking forward to the speed when it catches up with custom domain names. great job. Anything which makes it faster is awesome.

    I assume that this is what held up coupons - does this mean while we wait the 30 days for custom domain names we might see movement on coupons?mwink.gif

    Nope. This was mostly our operations team with Don included (he does a lot around here). The sorcerers (1, 2, 3, and another mysterious figure) are working on their own bit of, well, sorcery.
    SmugMug Product Manager
  • anderivanderiv Registered Users Posts: 80 Big grins
    edited August 27, 2008
    Awesome work, guys. CDN had been alluded to several times in recent forum posts, and I'd wondered if this was in the works.

    Thanks for continuing to raise the bar!
    Erik Anderson
    http://andersonfam.org
    http://andersonfam.smugmug.com
    D70 | SB-600 | Nifty Fifty | Tamron 17-50 f/2.8 | Nikon 70-300 f/4-5.6G
  • HaighHaigh Registered Users Posts: 64 Big grins
    edited August 27, 2008
    Great news. Speed really was an issue here in Brazil and CDN seemed just like the way to do it.

    I´m curious to know how it will perform for unfrequently visited sites. The great boost in CDNs comes for sites like apple where millions view the same pages and images. Smugmug is a bit different and the amount of content makes caching and CDNs less cost effective as they increase total bandwith usage for the network as a whole. As they try to guess what you will click on, they download stuff that won´t be viewed. With millions of viewers eventually someone will click on everything and the download won´t be wasted. With few viewers in a given location per photographer there is quite a bit of waste. Lets see.....

    I´m also curious as to what was decided on thumbanil security after all the discussion. But I know I´ll just have to sit tight and wait for the proper anouncement when the time comes.

    Contratulations on your endless efforts to improve. Makes us know our pennies are being well spent.

    clap.gifclap clap.gif
    Robert
  • onethumbonethumb Administrators Posts: 1,269 Major grins
    edited August 27, 2008
    Haigh wrote:
    Great news. Speed really was an issue here in Brazil and CDN seemed just like the way to do it.

    I´m curious to know how it will perform for unfrequently visited sites. The great boost in CDNs comes for sites like apple where millions view the same pages and images. Smugmug is a bit different and the amount of content makes caching and CDNs less cost effective as they increase total bandwith usage for the network as a whole. As they try to guess what you will click on, they download stuff that won´t be viewed. With millions of viewers eventually someone will click on everything and the download won´t be wasted. With few viewers in a given location per photographer there is quite a bit of waste. Lets see.....

    I´m also curious as to what was decided on thumbanil security after all the discussion. But I know I´ll just have to sit tight and wait for the proper anouncement when the time comes.

    Contratulations on your endless efforts to improve. Makes us know our pennies are being well spent.

    clap.gifclap clap.gif
    Robert

    We're going *way* beyond CDNs here in terms of global distribution of SmugMug's content. I'll post a in-depth analysis once I've finished tuning & tweaking our setup, but if you take a look at where your dynamic pages and stuff are coming from - they're not coming from SmugMug anymore. Your web browser is talking to a web server located somewhere very close to you for all your SmugMug traffic - not just photos and videos - if we and Akamai have done our jobs right.

    More in a blog post in the next few weeks, I hope. Stay tuned.
  • jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited August 27, 2008
    onethumb wrote:
    We're going *way* beyond CDNs here in terms of global distribution of SmugMug's content. I'll post a in-depth analysis once I've finished tuning & tweaking our setup, but if you take a look at where your dynamic pages and stuff are coming from - they're not coming from SmugMug anymore. Your web browser is talking to a web server located somewhere very close to you for all your SmugMug traffic - not just photos and videos - if we and Akamai have done our jobs right.

    More in a blog post in the next few weeks, I hope. Stay tuned.

    Sure enough. A tracert to both cdn.smugmug.com and jfriend.smugmug.com (where everything in my page is coming from) shows one hop past my ISP to akamai and no further.

    And it is now usually blazingly fast. Every once in a while, it seems to take a little while to connect to jfriend.smugmug.com, but when that doesn't happen, it's very fast. And the thumbs come popping in almost all at once compared to before where you could see them sprinkle into the page.
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited August 27, 2008
    jfriend wrote:
    And the thumbs come popping in almost all at once compared to before where you could see them sprinkle into the page.
    That's why we had that thread on thumbnail restrictions. Akamai couldn't get that kind of speed if they had to ping our Silicon Valley servers and we had to query for permissions before giving Akamai the green light. This way it's still nearly impossible to guess the URL for a thumb but Akamai can let all guns blaze.
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited August 27, 2008
    Haigh wrote:
    I´m curious to know how it will perform for unfrequently visited sites.
    Yes, solving this for a site with 350 million (mostly fat) images x a whole lot of sizes per image is a very expensive proposition... And Akamai's site accelerator is a lot spendier than a traditional CDN.

    There's also the geography question. Galleries that are frequently accessed in the U.S., will they be fast from Zambia if not many people access them there?

    But Akamai gives us lots of ways to tune, thumbnails aren't very big, and thumbs have a major influence on perception of speed. I'm guessing you'll feel like even olde dusty galleries are pretty darned fast when we get done with this.
  • CameronCameron Registered Users Posts: 745 Major grins
    edited August 28, 2008
    Great stuff - thanks for updating us on the latest developments! I'm always happy to see how SmugMug doesn't settle for being mediocre, even when it requires great effort.
    I recently read a bit about Akamai and it's cool to see this type of technology being implemented!
    thumb.gif
  • timnosenzotimnosenzo Registered Users Posts: 405 Major grins
    edited August 30, 2008
    awesome work, guys! clap.gif
  • jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited August 30, 2008
    Baldy wrote:
    That's why we had that thread on thumbnail restrictions. Akamai couldn't get that kind of speed if they had to ping our Silicon Valley servers and we had to query for permissions before giving Akamai the green light. This way it's still nearly impossible to guess the URL for a thumb but Akamai can let all guns blaze.

    What image sizes is Akamai caching? And how do you handle replace photo or edits done on the photo (which causes the image to change)?
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • SeamusSeamus Registered Users Posts: 1,573 Major grins
    edited August 30, 2008
    The galleries are very fast from Ireland. It's been very interesting watching Smugmug develop over the last 4 years.

    Here's to another 4 :seamus
  • JoeGJoeG Registered Users Posts: 81 Big grins
    edited August 30, 2008
    Holy crap... wings.gifbowdown.gif
    Joe Gearhart
    Photos | Blogs | Twitter | MySpace | Facebook
  • bhambham Registered Users Posts: 1,303 Major grins
    edited September 1, 2008
    I noticed a significant speed of the thumbs loading this evening and came to find out if it was related to the thumb security thread. The speed is really nice.

    Thanksclap.gifbowdown.gif
    "A photo is like a hamburger. You can get one from McDonalds for $1, one from Chili's for $5, or one from Ruth's Chris for $15. You usually get what you pay for, but don't expect a Ruth's Chris burger at a McDonalds price, if you want that, go cook it yourself." - me
  • bhambham Registered Users Posts: 1,303 Major grins
    edited September 1, 2008
    Seamus wrote:
    The galleries are very fast from Ireland. It's been very interesting watching Smugmug develop over the last 4 years.

    Here's to another 4 :seamus

    I think you meant 40!
    "A photo is like a hamburger. You can get one from McDonalds for $1, one from Chili's for $5, or one from Ruth's Chris for $15. You usually get what you pay for, but don't expect a Ruth's Chris burger at a McDonalds price, if you want that, go cook it yourself." - me
  • bwgbwg Registered Users, Retired Mod Posts: 2,119 SmugMug Employee
    edited September 1, 2008
    jfriend wrote:
    What image sizes is Akamai caching? And how do you handle replace photo or edits done on the photo (which causes the image to change)?
    We're still turning knobs and dials. I expect Don will detail all this in a blog post once things stabilize a bit.
    Pedal faster
  • GSPePGSPeP Registered Users Posts: 3,728 Major grins
    edited September 2, 2008
    bham wrote:
    I noticed a significant speed of the thumbs loading this evening...
    I've been looking at it for a few days now, and accessing Smugmug from several different locations (home, office, ...) The loading of the thumbs is significantly faster now. The general feeling is that everything is much faster as it used to be.

    Nice job thumb.gif
  • Jason DunnJason Dunn Registered Users Posts: 95 Big grins
    edited September 3, 2008
    clap.gif Great job! Speed, or the lack thereof to be precise, has been my #1 concern with Smugmug, so I'm *thrilled* to see you guys taking the big step up! I just took a walk through my photo site using jasondunn.smugmug.com instead of the usual photos.jasondunn.com, and the difference is simply astounding! AWESOME! I can't wait to see that kind of speed happening with my custom domain. wings.gif
    =-=-=-=-=-=-=-=-=-=-=-=
    The Photos: photos.jasondunn.com
    The Blog: www.jasondunn.com
  • Karenl39Karenl39 Registered Users Posts: 155 Major grins
    edited September 3, 2008
    Jason Dunn wrote:
    clap.gif Great job! Speed, or the lack thereof to be precise, has been my #1 concern with Smugmug, so I'm *thrilled* to see you guys taking the big step up! I just took a walk through my photo site using jasondunn.smugmug.com instead of the usual photos.jasondunn.com, and the difference is simply astounding! AWESOME! I can't wait to see that kind of speed happening with my custom domain. wings.gif

    Was I supposed to get an email about my custom domain? I don't think I did?
  • AndyAndy Registered Users Posts: 50,016 Major grins
    edited September 3, 2008
    Karenl39 wrote:
    Was I supposed to get an email about my custom domain? I don't think I did?
    It'll be going out soon thumb.gif

    http://www.dgrin.com/showthread.php?t=100417
  • Jason DunnJason Dunn Registered Users Posts: 95 Big grins
    edited September 3, 2008
    Andy wrote:
    It'll be going out soon thumb.gif

    I'm a wee bit baffled as to why it didn't go out BEFORE all these changes happened - I awoke one morning to find my photo site not working. It would have been nice to know it was going to break before hand...ya know? eek7.gif
    =-=-=-=-=-=-=-=-=-=-=-=
    The Photos: photos.jasondunn.com
    The Blog: www.jasondunn.com
  • ivarivar Registered Users Posts: 8,395 Major grins
    edited September 3, 2008
    Jason Dunn wrote:
    I'm a wee bit baffled as to why it didn't go out BEFORE all these changes happened - I awoke one morning to find my photo site not working. It would have been nice to know it was going to break before hand...ya know? eek7.gif
    Sites that were set up correctly, were not broken thumb.gif

    We allowed sites to be set up incorrectly and still function with the CNAME, but that's over now with the new system.
  • AndyAndy Registered Users Posts: 50,016 Major grins
    edited September 11, 2008
    Jason Dunn wrote:
    clap.gif Great job! Speed, or the lack thereof to be precise, has been my #1 concern with Smugmug, so I'm *thrilled* to see you guys taking the big step up! I just took a walk through my photo site using jasondunn.smugmug.com instead of the usual photos.jasondunn.com, and the difference is simply astounding! AWESOME! I can't wait to see that kind of speed happening with my custom domain. wings.gif
    Thanks, Jason!
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited November 13, 2008
    Can you tell when Akamai went live?

    416358768_exGeK-X2.png

    We're still working on getting it delivering images for custom domains. As usual, implementing this had us scrambling to implement other things as a consequence, such as investing more in data center equipment to handle the vortex.
  • DJ-S1DJ-S1 Registered Users Posts: 2,303 Major grins
    edited November 13, 2008
    Sweet! There's a strong weekly pulse there, huh. Is that Sunday that's the highest (because of lots of Sunday night uploads) or Monday (gawking at all the newly uploaded goodness)? Or another day altogether?
Sign In or Register to comment.