1/15/07 Database Reset: Http/1.1 Service Unavailable

[Zanotti]

6:45 pm EST 1/15

In the middle of a large download it quits and says:

Http/1.1 Service Unavailable

??

[Photolistic]

Http/1.1 Service Unavailable

[Seymore]

Got nothing!!! ...for about 5 mins now. Was in the middle of an upload with the DnD. Now I get the following when I try to do anything: Http/1.1 Service Unavailable.

Can someone check for a dead mouse or something blocking the wheel? :thumb

[Zanotti]

Reading some of the other threads, Yes I am in Florida but use Verizon Fios.

[ginger_55]

I can't access it at all..............from aol, didn't know if it was aol's problem. The code was the same as when you were downloading.

ginger

[Zanotti]

Same here!

[Cason]

Me too.

[Dennis]

Samehere ... can't get to anything at smugmug.com

Http/1.1 Service Unavailable

[Wireless]

We only use certified hamsters to run our site. Unfortunately 'Database Bruno' fell off his wheel and is in the process of being restored. He should be back into the swing of things shortly. Sorry for the inconvenience.

[Zanotti]

Wait, its not working from Florida and Andy's not working and IN Florida....conincidence? I think not!

[Seymore]

Houston, we have a problem!!!

[cambler]

I'm somewhat surprised that you guys push a generic "service unavailable" instead of a customized, branded "we're currently down, will be right back" page.

[ppuga]

:dunnoMe Too!

Cant Get In!:dunno

[Seymore]

LOL... Ok, thanks for the update.

We only use certified hamsters to run our site. Unfortunately 'Database Bruno' fell off his wheel and is in the process of being restored. He should be back into the swing of things shortly. Sorry for the inconvenience.

[ginger_55]

I can't access it on aol from sc.........

ginger

[devbobo]

Andy's not working

you sure about that...i think Andy might bet to differ :lol3 :lol3 :lol3

[devbobo]

I'm somewhat surprised that you guys push a generic "service unavailable" instead of a customized, branded "we're currently down, will be right back" page.

come on cambler....it's retro...so web 1.0 :thumb

[mwgrice]

We only use certified hamsters to run our site. Unfortunately 'Database Bruno' fell off his wheel and is in the process of being restored. He should be back into the swing of things shortly. Sorry for the inconvenience.

I've stopped recommending the use of hamsters, certified or otherwise, to my customers.

The reason is very simple: hamsters will occasionally escape during an upgrade or other routine maintenance. Even a certified hamster can't resist chewing through cables.

[Seymore]

I've stopped recommending the use of hamsters, certified or otherwise, to my customers.

The reason is very simple: hamsters will occasionally escape during an upgrade or other routine maintenance. Even a certified hamster can't resist chewing through cables.
R O T F L M A O P M P ! ! !

[jerryr]

The hamsters are simply saying 'feed me!'
all will be well :D

[Nikolai]

6:45 pm EST 1/15
In the middle of a large download it quits and says:
Http/1.1 Service Unavailable
??
I guess those darn hamsters decided to take the day off across the US...
BTW, did you mean "upload"? :scratch

[EIDT] it's getting better, now it's only Temporary Service Interruption

[Seymore]

[EIDT] it's getting better, now it's only Temporary Service Interruption
But, DEFINE temporary...

[ppuga]

I guess those darn hamsters decided to take the day off across the US...
BTW, did you mean "upload"? :scratch

[EIDT] it's getting better, now it's only Temporary Service Interruption
Well, hope they are negotiating with the hamsters and we get the service back shortly! :huh

[Zanotti]

I guess those darn hamsters decided to take the day off across the US...
BTW, did you mean "upload"? :scratch

[EIDT] it's getting better, now it's only Temporary Service Interruption

Upload from me, download to Smugmug........hell, I dont know up from down anymore!:rofl :rofl

[ppuga]

IT'S WORKING AGAIN! :clap


:ivar

[Barb]

Upload from me, download to Smugmug........hell, I dont know up from down anymore!:rofl :rofl

Hamster is now back on the wheel :)

[Seymore]

Hamster is now back on the wheel :)
:clap :clap :clap

:thumb

[nickphoto123]

My site visitors read this:

Temporary Service Interruption</STRONG> We're having some temporary technical difficulties. We're working on them at the moment, and expect service to return shortly. We apologize for the inconvenience. We're not happy about it, of course, but we are prepared for it and expect there to be no data loss or any long-term reduction in service. Thanks for your patience and understanding!




© 2007 SmugMug, Inc.</SPAN></STRONG>
Temporary Service Interruption</STRONG> We're having some temporary technical difficulties. We're working on them at the moment, and expect service to return shortly. We apologize for the inconvenience. We're not happy about it, of course, but we are prepared for it and expect there to be no data loss or any long-term reduction in service. Thanks for your patience and understanding!




© 2007 SmugMug, Inc.


Thank to my previous suggestion the date is now 2007.

This is still unacceptable for a Pro user to have to worry about.

My visitors know my web site name, not Smugmug. This will only confuse them.

Also there is no invitation to return after a period of time.

This lack of concern for Pro users in this situation makes me worry.

I can not accept this as a proper method of doing business.

Nicholas

[nickphoto123]

my concerns stated above are still valid and should be addressed.

Nicholas

[dogwood]

My site visitors read this:

Temporary Service Interruption</STRONG> We're having some temporary technical difficulties. We're working on them at the moment, and expect service to return shortly. We apologize for the inconvenience. We're not happy about it, of course, but we are prepared for it and expect there to be no data loss or any long-term reduction in service. Thanks for your patience and understanding!




© 2007 SmugMug, Inc.</SPAN></STRONG>
Temporary Service Interruption</STRONG> We're having some temporary technical difficulties. We're working on them at the moment, and expect service to return shortly. We apologize for the inconvenience. We're not happy about it, of course, but we are prepared for it and expect there to be no data loss or any long-term reduction in service. Thanks for your patience and understanding!




© 2007 SmugMug, Inc.


Thank to my previous suggestion the date is now 2007.

This is still unacceptable for a Pro user to have to worry about.

My visitors know my web site name, not Smugmug. This will only confuse them.

Also there is no invitation to return after a period of time.

This lack of concern for Pro users in this situation makes me worry.

I can not accept this as a proper method of doing business.

Nicholas


I agree. Should start with something like:

"The site you are trying to access is hosted by Smugmug. We're having some temporary technical difficulties..."

[Andy]

Thank to my previous suggestion the date is now 2007.
This is still unacceptable for a Pro user to have to worry about.
My visitors know my web site name, not Smugmug. This will only confuse them.
Also there is no invitation to return after a period of time.
This lack of concern for Pro users in this situation makes me worry.
I can not accept this as a proper method of doing business.
Nicholas

Hi Nick, sorry you are put out by our message. Your prior request was read, heard, and acknowledged. We try really hard to provide the best possible service. I'm sorry you had to endure this for a short time, I believe this was a really short downtime, Nick, like a few minutes or so? Anyhow, all is well again.

Thanks again for letting us know again how important this is to you.

[Andy]

my concerns stated above are still valid and should be addressed.

NicholasWe have acknowledged them, every time you've made them. (we even changed the © date after your last posting a while back :D

Thanks again, Nick. :wave

[Andy]

you sure about that...i think Andy might bet to differ :lol3 :lol3 :lol3
:thumb

[nickphoto123]

We have acknowledged them, every time you've made them. (we even changed the © date after your last posting a while back :D

Thanks again, Nick. :wave



The other issues I raised are even more important than the date issue.

Your acknowledgment does not solve the problem.

The real problem is that you have decided to cover the concerns of your larger customer base, the personal photographer ( Non-Pro) without a thought for your Pro customers.
The issues I raised as constructive suggestions were not even thought about until I raised them.

What you have to do is determine how to solve these issues to the satisfaction of your Pro and Non-Pro customers.

And so far you have not accomplished this solution.

Thank you, Nicholas

[devbobo]

The real problem is that you have decided to cover the concerns of your larger customer base, the personal photographer ( Non-Pro) without a thought for your Pro customers.
The issues I raised as constructive suggestions were not even thought about until I raised them.

:bigbs

come on Nick, you can't be serious. I seem to remember only a few months ago a whole load of stuff released for pros...backprinting, custom watermarks, etc.

Grow up...

[W.W. Webster]

my concerns stated above are still valid and should be addressed.From my experience, the frequency and typical period of SmugMug site outages seem to be well within industry norms, but I don't understand the implications from the point of view of a working pro.

Out of curiosity, based on today's event, would you be willing to share with us:

a) your average number of sales multiplied by the average net sale proceeds, calculated for the relevant day and time of day (i.e. the estimated total gross profit from sales, not the gross sales value), that you might have missed during the outage - i.e. your direct financial 'cost', assuming the sales are completely lost

b) the average number of customer visits to your site, also calculated for the relevant day and time of day, that you might have missed during the outage - i.e. the impact for your customer goodwill

c) your site URL.

Thanks

[Zanotti]

Wait, its not working from Florida and Andy's not working and IN Florida....conincidence? I think not!

It is the Andy effect. Andy, you back in NYC and now its working?

[Andy]

And so far you have not accomplished this solution.

Thank you, Nicholas
Hi Nick,
I'm sorry you are unhappy with our service. I wish I could do more for you. We have put a tremendous amoung of time, energy, resources, people, and cash into our pro business. I deal with our pros on a daily basis, these are pros who have their livelihoods depend on SmugMug for our service, our site, and our people. We take everyone's input very seriously, and we apply our development resources and energy to all facets of our business.

I hope that we can satisfy you, too. Apologies again for the few minutes downtime tonight.

[devbobo]

It is the Andy effect. Andy, you back in NYC and now its working?

No he's still in FL :D

[nickphoto123]

Hi Nick,
I'm sorry you are unhappy with our service. I wish I could do more for you. We have put a tremendous amoung of time, energy, resources, people, and cash into our pro business. I deal with our pros on a daily basis, these are pros who have their livelihoods depend on SmugMug for our service, our site, and our people. We take everyone's input very seriously, and we apply our development resources and energy to all facets of our business.

I hope that we can satisfy you, too. Apologies again for the few minutes downtime tonight.

I mentioned nothing with your downtime.

It is your posted notice when you are down that is a problem.

If you want to satisfy me then change your outage notice and don't mention the words 'Lost data' Or 'Data loss'.
Make reference to the site the visitor just typed into their browzer.
Mention a request to re-visit soon or in the near future.

Put some of your tremendous energy into the problem of this discussion,your outage notice.

Thank you, Nicholas

[Andy]

I mentioned nothing with your downtime.

It is your posted notice when you are down that is a problem.

If you want to satisfy me then change your outage notice and don't mention the words 'Lost data' Or 'Data loss'.
Make reference to the site the visitor just typed into their browzer.
Mention a request to re-visit soon or in the near future.

Put some of your tremendous energy into the problem of this discussion,your outage notice.

Thank you, NicholasHi Nick, thanks again for posting. I've made sure that our product manager has seen this. We appreciate it.

[Wireless]

Our database is cranky today.

Unfortunately, our load balancer is showing its generic message right now. We'll pop up the other shortly. Sorry for the additional outage.

[rainforest1155]

Just a couple of minutes ago it was working, now it´s down again with the message:
Http/1.1 Service Unavailable

Sebastian

[nickphoto123]

Hi Nick, thanks again for posting. I've made sure that our product manager has seen this. We appreciate it.

Here's to an ever improving Smugmug...Cheers!!!!!!!!!

Nicholas

[dogwood]

Our database is cranky today.

Unfortunately, our load balancer is showing its generic message right now. We'll pop up the other shortly. Sorry for the additional outage.
Ouch-- right in the middle of an upload. When I get cranky, sometimes it helps to eat something... wait, nevermind, we don't want the database eating anything!

Just crossing my fingers the downtime(s) today are due to implementing the latest updates soon? :dunno

Holy smokes-- you folks did listen to my suggestion for the downtime message! Wow! Thank you!

[Seymore]

Our database is cranky today.

Unfortunately, our load balancer is showing its generic message right now. We'll pop up the other shortly. Sorry for the additional outage.
:nono

[Steve Cavigliano]

Just a couple of minutes ago it was working, now it´s down again with the message:
Http/1.1 Service Unavailable

Sebastian

Yes, we are experiencing problems at this time. Our engineers are working feverishly to restore service.

Sorry :cry

Steve

[Pezpix]

Would it help if I offered a big cookie to the database gremlin? :D

http://www.dgrin.com/attachment.php?attachmentid=13710&stc=1&d=1152558860

[RichS]

I agree. Should start with something like:

"The site you are trying to access is hosted by Smugmug. We're having some temporary technical difficulties..."

So I'm reading through this thread, seeing what's going on, and I hop back over to refresh the other tab to see if it's available yet. And I get " The site you are trying to access is hosted by SmugMug."

Not bad - 2.5 hours to implement customer suggestions.

I do wish the site was back up though.

[ppuga]

So I'm reading through this thread, seeing what's going on, and I hop back over to refresh the other tab to see if it's available yet. And I get " The site you are trying to access is hosted by SmugMug."

Not bad - 2.5 hours to implement customer suggestions.

I do wish the site was back up though.

Agree, that's a fast implement of customer suggestions. But please, get us our sites back. Good luck guys! You can do it! :devbobo

[Wireless]

We're turning things back up now. Sorry for the extra wait, we were gathering some additional troubleshooting data while things were offline.

[onethumb]

We're turning things back up now. Sorry for the extra wait, we were gathering some additional troubleshooting data while things were offline.

The site is back up, but we don't know for sure if it'll stay up. Sorry. :(

We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.

We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.

Thanks for your patience.

Don

[Jeffro]

The site is back up, but we don't know for sure if it'll stay up. Sorry. :(

We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.

We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.

Thanks for your patience.

Don

Keep up the good work, but I'm putting you on the clock....my first motocross race is April 8th! I'll need to upload some shots later that day...:rofl

[photodoug]

down agin

http/1.1 service unavailable

[onethumb]

The site is back up, but we don't know for sure if it'll stay up. Sorry. :(

We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.

We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.

Thanks for your patience.

Don

We're down again, but got some useable data. Definitely looks like a hardware failure. Andrew's on his way to the datacenter now to physically take a look at what's going on.

We have a couple of avenues to take, so we'll start taking them one at a time. I wouldn't be surprised if there are some more hiccups along the way.

We'll keep you posted here.

Don

[thegrepper]

Dear Smugmug:

I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.

-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?

I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.

[Manny]

Dear Smugmug:

I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.

-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?

I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.

I think that you already answered your own questions :-)

If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.

Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.

Cheers

MG

[imageinc]

Here's hoping this works itself out as fast as possible. Just, finally, do I get the cast from my opening Big Event Shoot to finally settle down from the performing and actually *look* at the pictures...

and the damn server crashes. :-}

What the previous poster said... I'm sure you're a bit busy just now... but when you're not, those might be useful questions to address.

[onethumb]

Dear Smugmug:

I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.

-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?

I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.

Hi there,

Hopefully some of our long-term customers will weigh in here with an unbiased opinion, but the view from the inside looks something like this:

- These outages ARE NOT common. In terms of non-scheduled downtime, we aim to have less than one hour (99.99%) uptime. Last year (2006) we didn't make it because of one prolonged outage due to a distributed denial of service attack, but excluding that one, we were will within range. Previous years we made it with plenty of room.

- We don't provide an SLA. You're the first one to ask for one. I'm not opposed to implementing one, but we'd need to think about how to do it. Some estimate of the time down, multiplied by your average sales dollars per second might do the trick, but we'll have to talk about that. I'm not sure our service is really high-dollar enough to warrant an SLA, but maybe I'm wrong?

- We don't provide availability metrics, and again, you're the first to ask. We'd certainly do so if we had an SLA in place.

- We have multiple pieces of hardware standing by, but I'm not about to replace a piece of hardware without first verifying that it's the root cause. If it wasn't, I just wasted an hour or two doing nothing useful. We're still diagnosing the problem, but we're getting closer.

- At this point, I believe the only non-redundant components are core networking switches. (This is a common component to not be redundant - every major internet brand suffers from outages due to network switch failures) All of our routers, servers, and storage are redundant. Many of them are automatic fail-over, too, so you'll never notice an outage. With critical data, though, it's essential to have manual failover so we don't introduce data corruption due to split-brain or time-based latency.

When you compare our uptime to any major brand, including Google, eBay, Hotmail, or Amazon, you'll find that we're comparable. No-one meets 99.999% uptime, and most don't even come close to 99.99%.

We're doing the best we can, of course, and will continue to think about how we can improve.

Thanks for the questions, hopefully some third party validation will happen with our customers, but it's possible my world view differs from theirs. :)

Don

[thegrepper]

I think that you already answered your own questions :-)

If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.

Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.

Cheers

MG

I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.

[DavidTO]

I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.


Infrequent, and normally resolved in minutes, as far as I can see. I don't clock it, but outages are very rare and quickly recovered.

[onethumb]

I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.

Less than once a month, more than once a year. :)

Less than once a year if you count a repeated outage during a single day like this. Very rare.

Don

[devbobo]

Dear Smugmug:

I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.

-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?

I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.


thegrepper,

As a smugmug customer for around 2 years now, I can stay the availability/uptime of smugmug is second to none IMO.

While I am not a heavy uploader like some other users, I am someone who helps out with customisation and development of 3rd party apps... I can tell you that it is very rare that I couldn't access smugmug (outside of scheduled outages) and I am talking about frequent access between 7am-2am (Australian EST).

Cheers,

David

[LynD]

I have been a Smugmug user for a year and counting. I use the site on an almost daily basis. I can only remember one other instance of the site being "temporarily unavailable." The people & tech support have always been more than prompt, helpful, and otherwise amazing. So for the moment, stop griping, let them fix it, and thank them when it's all done for taking care of the matter in their usual prompt & amazing way.
I'd bring you guys coffee & chocolate if I weren't on the other side of the country.
:lust

[DJKennedy]

My site is up now - so I assume things are kosher. :clap

EDIT: and the site is FAST - faster than norm

[devbobo]

I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.

these sort of non-scheduled outages are definately not the norm and I can't remember from memory many that have been outside a planned window.

Cheers,

david

[Manny]

Hey Don,

Just want to let you know that for mission critical, there are many ways around non redundant things :-) Since I work in IT as a Director and Technical Architect for a very very large hospital (read, 24x7x365, no outages allowed when patient data is on-line :-)

--Switches. Well, yes and no. You can certainly have a hot standby system on a separate switch altogether. This can be costly but if you really have to be up, this is one way to do it.

--Database. Well, many options out there, but my favorite is the expensive solution from Golden Gate Software. They can do magical things with an SQL database... replay, caching, duplication, basic message routing etc. Look em up if you are interested in TRANSACTION level redundancy.

--Servers. Well, start out with teaming built into Windows itself. Nothing like having two NICs on the same machine connected to TWO SEPARATE switches.

--IP... BigIP from f5. Nuff said :-) They are the classic way that many people still do redundancy. We use em, as do many many other people.

--For monitoring I love InterMapper from Dartware... very cheap alternative to HP OpenView or Tivoli. I think it is MUCH better than What's Up Gold which is another favorite out there.

I may be preaching to the choir but in case you didn;t consider any of the above, I hope it helped a bit :-) You are of course welcome to tap my brain for anything IT that I can help with. For credentials I am a CCIE, MCSE, CISSP, and a few others under my belt... have designed very large networks (and implemented them as well and operated them) and recently deployed a 20 node DWDM fiber ring around Manhattan.

Good luck with your repairs :-)
Cheers


Manny

Hi there,

Hopefully some of our long-term customers will weigh in here with an unbiased opinion, but the view from the inside looks something like this:

- These outages ARE NOT common. In terms of non-scheduled downtime, we aim to have less than one hour (99.99%) uptime. Last year (2006) we didn't make it because of one prolonged outage due to a distributed denial of service attack, but excluding that one, we were will within range. Previous years we made it with plenty of room.

- We don't provide an SLA. You're the first one to ask for one. I'm not opposed to implementing one, but we'd need to think about how to do it. Some estimate of the time down, multiplied by your average sales dollars per second might do the trick, but we'll have to talk about that. I'm not sure our service is really high-dollar enough to warrant an SLA, but maybe I'm wrong?

- We don't provide availability metrics, and again, you're the first to ask. We'd certainly do so if we had an SLA in place.

- We have multiple pieces of hardware standing by, but I'm not about to replace a piece of hardware without first verifying that it's the root cause. If it wasn't, I just wasted an hour or two doing nothing useful. We're still diagnosing the problem, but we're getting closer.

- At this point, I believe the only non-redundant components are core networking switches. (This is a common component to not be redundant - every major internet brand suffers from outages due to network switch failures) All of our routers, servers, and storage are redundant. Many of them are automatic fail-over, too, so you'll never notice an outage. With critical data, though, it's essential to have manual failover so we don't introduce data corruption due to split-brain or time-based latency.

When you compare our uptime to any major brand, including Google, eBay, Hotmail, or Amazon, you'll find that we're comparable. No-one meets 99.999% uptime, and most don't even come close to 99.99%.

We're doing the best we can, of course, and will continue to think about how we can improve.

Thanks for the questions, hopefully some third party validation will happen with our customers, but it's possible my world view differs from theirs. :)

Don

[DavidTO]

I forgot to mention, how many companies would have their CEO on the forum answering any and all questions during an outage?

[thegrepper]

Less than once a month, more than once a year. :)

Less than once a year if you count a repeated outage during a single day like this. Very rare.

Don

Don,

Thanks for this and the previous detailed post. I understand the challenges and really appreciate your response given the current situation. The one thing that will absolutely keep me as a customer is your responsiveness and the SMUGMUG community.

[Cindy]

I think that you already answered your own questions :-)

If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.

Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.

Cheers

MG

Maybe it shouldn't have but just passing by this post :huh really struck my nerve as being pretty much uncalled for but than maybe your just having a really sore day... or maybe it's just me reading your post wrong. Whatever.

As a PRO hosting with a smugmug PRO account I would NOT just pop some pics up on flicker... geez. And not all PROS including myself give DVD's to clients - hello. We trust and rely on smugmug. Customers will and do understand downtime. They may not like it... hey neither do we... but stuff happens and everyone knows that.

That said... My 2 cents: Don't worry. Smugmug is rarely down but when it happens they have it fixed asap. I can assure you anytime and everytime I've ever encountered any problem with my PRO Smugmug they reply in a very PRO manner (understatement because they'll work 24/7 until it's fixed) and have me up and going in no time flat. They really do bust their buts for every customer - Pro or not. Thanks smugmug!!! I know your doing all you can to get this issue straightened out. I don't like it either but Thank God I don't have to figure it out... that's why I use smugmug for my host!!! Andy once said something along the lines of... "Your a photographer so go shoot some pics and Leave the worries to us." I encountered a problem moving 2 pics today... so I e-mailed the help desk and posted on dgrin... then went shooting. When I got home they'd figured it all out and my pics were found :bow :ivar :bow

[Manny]

Well, actually a ONCE IN A WHENEVER IT HAPPENS kind of outage. I can tell you the last one was over a year ago but then, we can have 10 in the next week... that is life :-)

If purely from an ESTABLISHED record from past experience/actual happenings, it is VERY VERY RARE.

Given that, I can say that Smugmug is not yet an Amazon nor an eBay, nor Google (wow, imagine?) so their budget and funds for THAT LEVEL of redundancy may be not here yet... (Don?) LOL.

Imagine that eBay and Google have BILLIONS of dollars in funds JUST TO KEEP THE LIGHTS ON all the time. It is their most important business and they are HUGE.

Don, can you share your traffic stats? I bet compared to Google, they pale. But, I think Smugmug will get there one day. And when they do, they will be more like Google when it comes to redundancy. Or maybe Don and crew are ready to steal all our money and bail!!!! LOL

Cheers!!

MG

P.S. I do have my own hosting site and suggest you do the same for your business. Use Smugmug as your main or not, but definitely back one up with the other.

I have my site on standby just in case. If I had customers waiting for a shoot, I would post them on my domain ... but it is rare enough I have never had to use that.

Same goes for if you use your domain as primary and Smug as backup. Your ISP can go down while Smug is up.

I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.

[Barb]

Infrequent, and normally resolved in minutes, as far as I can see. I don't clock it, but outages are very rare and quickly recovered.

:agree

[nocturne1]

In terms of non-scheduled downtime, we aim to have less than one hour (99.99%) uptime.
I think you meant "less than one hour downtime" ;)

Thanks for your efforts to get things running smoothly again!

[Manny]

Yup Cindy, you misread my post :-) I guarantee you I am having a great day. But here comes another insult (well, if you read it wrong that is) You are NO PRO if you are NOT giving DVDs to your customers when they NEED the pictures and your hosting site is down. Customer is KING... and that means handing out your right arm if you have to. Plain and simple.

Look at it this way... those Sports Illustrated PROs (and they are real PROs in my book) never even get to see their best pictures first in most cases. They are handing out their CF cards to runners and they don't even get to "chimp" their images.

Don;t worry, your images are perfectly safe ... What is wrong with giving out DVD's in an emergency? or not? :-)

Maybe you are having a bad day? If not, good, if so, my apologies for getting you hyped enough to pay attention to an idiot like me !! :-)

Cheers!
MG

Maybe it shouldn't have but just passing by this post :huh really struck my nerve as being pretty much uncalled for but than maybe your just having a really sore day... or maybe it's just me reading your post wrong. Whatever.

As a PRO hosting with a smugmug PRO account I would NOT just pop some pics up on flicker... geez. And not all PROS including myself give DVD's to clients - hello. We trust and rely on smugmug. Customers will and do understand downtime. They may not like it... hey neither do we... but stuff happens and everyone knows that.

That said... My 2 cents: Don't worry. Smugmug is rarely down but when it happens they have it fixed asap. I can assure you anytime and everytime I've ever encountered any problem with my PRO Smugmug they reply in a very PRO manner (understatement because they'll work 24/7 until it's fixed) and have me up and going in no time flat. They really do bust their buts for every customer - Pro or not. Thanks smugmug!!! I know your doing all you can to get this issue straightened out. I don't like it either but Thank God I don't have to figure it out... that's why I use smugmug for my host!!! Andy once said something along the lines of... "Your a photographer so go shoot some pics and Leave the worries to us." I encountered a problem moving 2 pics today... so I e-mailed the help desk and posted on dgrin... then went shooting. When I got home they'd figured it all out and my pics were found :bow :ivar :bow

[richW]

I forgot to mention, how many companies would have their CEO on the forum answering any and all questions during an outage? :agree Thats soooo nice.....
I’m used to dealing with 3rd party outsourced customer service representatives who don’t speak any language, don’t know anything and who don’t care.
I’ll stick With SmugMug….

[Andy]

Thanks smugmug!!!
:wave Hi Cindy, thanks for the kind words :D

[PBolchover]

Just popping in to say that I find the smugmug availability to be roughly comparable with Google Calendar...

[dogwood]

If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself.

Wow-- harsh but true. Good tip too. :D

[DavidTO]

Wow-- harsh but true. Good tip too. :D
Except that smugmug is infinitely more reliable and speedier than the mail, or even overnight delivery. By the time anyone burns a DVD, gets it in the overnight package, spends $8 or whatever it is to ship it, smugmug's been up for hours.

That and having a redundant site as backup all seem like major overkill.

[thegrepper]

MG

P.S. I do have my own hosting site and suggest you do the same for your business. Use Smugmug as your main or not, but definitely back one up with the other.

I have my site on standby just in case. If I had customers waiting for a shoot, I would post them on my domain ... but it is rare enough I have never had to use that.

Same goes for if you use your domain as primary and Smug as backup. Your ISP can go down while Smug is up.

Manny,

Again, good advice. I do have my own domain and do have a selection of photos available for viewing. Workflow does not permit maintaing two sites with all photos. The other objective of moving to SMUGMUG was to add a shopping cart for family and friends to order prints - I don't plan to implement that functionality on my site for redundancy. There is also no practical way for me to monitor SMUGMUG's service and implement my "standby" should an outage occur. I understand these things happen and hope they are infrequent. Thanks again...

[dogwood]

Except that smugmug is infinitely more reliable and speedier than the mail, or even overnight delivery. By the time anyone burns a DVD, gets it in the overnight package, spends $8 or whatever it is to ship it, smugmug's been up for hours.

That and having a redundant site as backup all seem like major overkill.
David:

The Sports Illustrated shooters mentioned in the post that inspired this don't exactly rely on the mail, but it still seems like a pretty good tip gleaned from the big time pros. :D

Sometimes a client wants the photos quickly (like getting the CF cards at an NFL game)-- or in a day or two. If they're local, a DVD is a nice backup-- even in the rare chance smugmug (or your electricity-- yeah, yeah, I can still get a broadband connection if the power is out, but some folks can't) goes out-- as long as the servers have power of course. Plus it's a nice touch for the client.

EDIT: right after I posted this-- went to my smugmug site and it's down again. If I had something due to the client in the morning-- a DVD seems like a good option. Better than staying awake to wait for the site to come back up. That's all I'm saying.

[Cindy]

You are NO PRO if you are NOT giving DVDs to your customers when they NEED the pictures and your hosting site is down.
Don;t worry, your images are perfectly safe ... What is wrong with giving out DVD's in an emergency? or not? :-)
Maybe you are having a bad day? If not, good, if so, my apologies for getting you hyped enough to pay attention to an idiot like me !! :-)

Cheers!
MG

There's not a thing wrong with giving out DVD's. I do whenever needed but I've never needed to because smugmug was down. In this case if it's only because smugmug is down... I'm quite confident that by the time I burned the DVD and got it to the customer... smugmug will have been long allready back on-line and I would have wasted time with burning and delievery and fretting all for nothing... besides it's still easier in 'most' cases for them to head to the website :D verses viewing a dvd.

flicker: Same as above. I'm sure it would likely take me longer to get the pics uploaded and inform the customer of the new site, place, etc to view them... best to just hang in there for the moment unless smugmug advises it's something extremly terrible with a potential downtime of more than 24 hours... which I don't recall ever happening.

Anyhow my point was he was posting as a new smugmug customer... worried about if this is a normal problem, how it would effect his customers... etc. I just thought you post sounded like a put down to both smugmug (their service) and to the new pro customer (implying they weren't a pro).

My day's been great but time to turn in for some zzzzzzzzz's... I'm confident the smugmug team will work this all out and grateful I don't have to loose any sleep over it :D

[Manny]

We're cool Cindy... like I said, I was basically being honest to the NEW person... and realistic. If you;re a PRO, you must have options is all I said. Heck, I back Smugmug like the best of them... and definitely take nothing away from them. Heck of a great site. My position is from a point of view of what - if. I guess you understand my point. I certainly understand yours.

Cheers
Manny

There's not a thing wrong with giving out DVD's. I do whenever needed but I've never needed to because smugmug was down. In this case if it's only because smugmug is down... I'm quite confident that by the time I burned the DVD and got it to the customer... smugmug will have been long allready back on-line and I would have wasted time with burning and delievery and fretting all for nothing... besides it's still easier in 'most' cases for them to head to the website :D verses viewing a dvd.

flicker: Same as above. I'm sure it would likely take me longer to get the pics uploaded and inform the customer of the new site, place, etc to view them... best to just hang in there for the moment unless smugmug advises it's something extremly terrible with a potential downtime of more than 24 hours... which I don't recall ever happening.

Anyhow my point was he was posting as a new smugmug customer... worried about if this is a normal problem, how it would effect his customers... etc. I just thought you post sounded like a put down to both smugmug (their service) and to the new pro customer (implying they weren't a pro).

My day's been great but time to turn in for some zzzzzzzzz's... I'm confident the smugmug team will work this all out and grateful I don't have to loose any sleep over it :D

[greenpea]

Sorry, but I had to chime in on this thread. Smugmug rarely has problems, and never any problem that has bothered me prior to today. In fact the only reason I usually know about any smugmug problems is because of all the threads that pop up on dgrin.

Then when ever there is a problem that I read on the dgrin threads, I am amused at how fired up people get for what always seems to me to be minor issue. But today through unfortunate timing on my sister-in-law's part she sent out an email to all her family and friends to announce their much-anticipated new smugmug site where everyone could go and see pictures of their new daughter.

Normally I could careless if a site where my sister-in-law's family pics are hosted goes down, but I spent a month convincing my sister-in-law and her family that smugmug is where they need to go for photo hosting, I told them how much smugmug rocks, how if you have any problem what so ever just email help@ and you would get a response almost instantly. Then she proudly announces their new site just before smugmug goes down. This was my first smugmug site sale; the first person I was able to convince to sign up with smugmug.

Yes it was just bad timing and bad luck, and I still think smugmug rocks, I will continue to support smugmug and tell everyone its the best place to host your pics. Its just that now understand a little better why people get so fired up when smugmug has problems and it effects their site.

[Matthew Saville]

Let me preface this by saying that at the end of this month I will have been with Smugmug for 3 years, most of it as a Pro account holder.

I could count the number of crashes that gave me trouble on one hand. And I spend a LOT of time on my Smugmug.

The number of crashes that gave my customers trouble, thankfully, is zero.

I know this is not the case with EVERY Smugmug user, but I just thought I'd weigh in. Smugmug is above and beyond the best value there is, none else really comes close at all. Sure you can pay a boat load to have your own private website with all sorts of complicated backups, but my Smugmug practically pays for itself every year with the referral system. I just can't beat that, so I'm not going to complain about what I've got if the grass isn't greener anywhere else!

-Matt-

[EDIT] This is always the ironic part. Each of the few times that I can remember Smugmug going down, it always ended like this- Smugmug is back up before I can even finish going on the forums to see what's happening. Back to work, everybody! Shoo!

[padge206]

my site is back up. 11:20 PM PST 01/15/07

i only noticed the outage about 20 minutes ago...
but wanted to report that my end, things are up.

even the main smugmug site is working again.
2 minutes ago i refreshed and got a "service unavailable"

:thumb

[onethumb]

Sorry, but I had to chime in on this thread. Smugmug rarely has problems, and never any problem that has bothered me prior to today. In fact the only reason I usually know about any smugmug problems is because of all the threads that pop up on dgrin.

Then when ever there is a problem that I read on the dgrin threads, I am amused at how fired up people get for what always seems to me to be minor issue. But today through unfortunate timing on my sister-in-law's part she sent out an email to all her family and friends to announce their much-anticipated new smugmug site where everyone could go and see pictures of their new daughter.

Normally I could careless if a site where my sister-in-law's family pics are hosted goes down, but I spent a month convincing my sister-in-law and her family that smugmug is where they need to go for photo hosting, I told them how much smugmug rocks, how if you have any problem what so ever just email help@ and you would get a response almost instantly. Then she proudly announces their new site just before smugmug goes down. This was my first smugmug site sale; the first person I was able to convince to sign up with smugmug.

Yes it was just bad timing and bad luck, and I still think smugmug rocks, I will continue to support smugmug and tell everyone its the best place to host your pics. Its just that now understand a little better why people get so fired up when smugmug has problems and it effects their site.

Aha! I think we know who caused our outage - it was greenpea's sister-in-law! :D

Seriously, sorry about the timing.

We're back up (again) for the time being. I'm watching it closely, we made some changes, and I'm hoping we're stable again. But if I were a betting man, I'd doubt it.

More as I get it...

Don

[Seymore]

Thanks again for the update Don... :thumb

[Greg Zillgitt]

I think that you already answered your own questions :-)

If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.

Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.

Cheers

MG

Manny you clearly don't understand the way professional event photography works. Mailing DVDs (at no additional cost) so clients can do their own printing? LMAO!! A fully functional (with fullfillment functionality I assume) alternate site while still using (and paying) smugmug? ROFLMAO!!! Yeah, I'll just create a web site with all of the free time I have left over from my 60-hour work weeks (all photographers are computer tech weenies, btw). Flickr? Please!!!! Perhaps you should constrain your condescending remarks for those subjects within your area of expertise, such as redundant network switching in very, very, very, very large hospitals.

FWIW, my paying customers get a CD with a PS-generated web gallery, so they can at least view their images off-line if smugmug is down. That doesn't give them a printing option, though (without ponying up significant $$$) and it doesn't do anything for the potentially hundreds of event guests who would view the images and hopefully buy prints. But I suppose they should they ALL get high-res DVDs.

[papajay]

...I spent a month convincing my sister-in-law and her family that smugmug is where they need to go for photo hosting... the first person I was able to convince to sign up with smugmug...

Same (almost) story here...of all days to try to convince 1)cousin, and 2) a close friend who just crossed the Pacific by sailboat (and is very dissatisfied with his yahoo picture site) to open Smugmug accounts, I had to pick a day when smugmug's service went down! Drat.

One thing that did (still does) trouble me about the "outage"...when I discovered my site was down, I immediately went to Dgrin to find out what was going on. [Was there a scheduled event taking place that I was not aware of? (I could have postponed my "promotional" efforts if I'd known)....or was it an unplanned, spontaneous outage?...to be honest, I'm still not 100% sure I know the answer, although I'm leaning toward the latter.]

It wasn't all that easy to figure it out (non-geek credentials here). Lots of the posts were tongue in cheek humor, etc., and lots were more technical than my pay grade understands. And I could not seem to find much that really appeared "official" as to what was actually happening. Andy's bio identifies him, at least, as a "Smugmug House Pro", but some others who I believe are legitimate spokespersons for smugmug have bio descriptions that are too obscure to be certain they have any real affiliation with Smugmug.

Unless one is a frequent Dgrin user, it would be difficult to know who to "listen to" about the particulars of such an event.

Bottom line, whenever there's an event like this, people just want to know what's going on, and want the information to come from an informed, "official" source. Perhaps that could be done on Dgrin with a separate "Announcement/Status" thread that only the "official" spokespersons can post to???? (just a thought), and let the rest of us rant, help, compliment, or otherwise blow off steam elsewhere (as in the current thread format).

[Manny]

Hey Greg,

CLARIFICATION TO ALL: Please stop assuming things ... I NEVER suggested that you give away the original pictures, what kind of pro would do that? It all depends on the situation... If National Geographic wanted to give me $10,000 for one of my NEFs and I gigured it was a good deal, they'd get the DVD with the NEF... it is not unlikely that someone would be in that situation. Even for a pro, you could manage to sell the rights for a price... everything has a cost to it. so that is not entirely impossible. I figured this was self explanatory... I can not believe that youwould think I meant originals? for them to print? Why would you assume that? If you and Cindy assuemd as much, that is entirely your fault :-)) I NEVER meant that... of course I meant something for them to see... a locked down, small slideshow with lockouts, watermarks whatever... I don;t care. My point was to make sure the customer sees the images. ALso, you never want to tell a customer you can;t do something such as not being able to show them their pictures because your site is down. You should have a backup is all I am saying.

Once again, why did you assume I meant originals for them to print on their own? Curious....

Cheers!
Manny

Manny you clearly don't understand the way professional event photography works. Mailing DVDs (at no additional cost) so clients can do their own printing? LMAO!! A fully functional (with fullfillment functionality I assume) alternate site while still using (and paying) smugmug? ROFLMAO!!! Yeah, I'll just create a web site with all of the free time I have left over from my 60-hour work weeks (all photographers are computer tech weenies, btw). Flickr? Please!!!! Perhaps you should constrain your condescending remarks for those subjects within your area of expertise, such as redundant network switching in very, very, very, very large hospitals.

FWIW, my paying customers get a CD with a PS-generated web gallery, so they can at least view their images off-line if smugmug is down. That doesn't give them a printing option, though (without ponying up significant $$$) and it doesn't do anything for the potentially hundreds of event guests who would view the images and hopefully buy prints. But I suppose they should they ALL get high-res DVDs.

[Manny]

Greg,

I don;t know you and you don't know me so let's start off on the right foot. When and where was i condescending to you? or anyone for that matter?

It looks like you read way too deep into my post for no reason other than to be condescending to me about whatever you are an expert at as you claim. No need for you to come telling me anything in your tone when I never directed anything at you personally. I will take the high road then and tell you this... good luck with your business and your business model. You obviously misinterpreted my posts. You didn;t have the decency to even ask me what I meant... you just went ahead and assumed whatever you wanted to state whatever you wanted.

The piece about the routing and switching and redundancy was directed at Don and no one else... why would it bother you?

Let me clarify once more. My statements were suggestions of OPTIONS in case you need to get something to a customer to see. Whatever that is. If you read that as having two completely functional and distinct sites with order fullfillment etc. that is what you understood, but it is up to you f you want to do that or not. SOme here may be happy with simply an option to say, hey, mr. customer, my site is down and I know you need to see pics. You are not in the same town so go to this link and look at them. It could be "yousendit.com" or Flickr or whatever... there is no need to interpret it as: Mr . customer, let me give you away the food from my baby's mouth" You entirely misunderstood and misinterpreted my post(s) for your benefit it seems.

Finally, my apologies for getting you upset... I never meant to harm anyone. I actually meant good by my post, not bad... I feel badly that it has somehow gone wrong with some folks...

MG :cry:dunno
Perhaps you should constrain your condescending remarks for those subjects within your area of expertise, such as redundant network switching in very, very, very, very large hospitals.

[dogwood]

I actually meant good by my post, not bad...

Manny:

I understood your post and appreciate your tips. I can't imagine handing off my CF cards or even wirelessly transmitting to a laptop while someone else handles the images-- but I know that's the way it works for many pro shooters (NFL games, NBA games, etc).

And you're right-- as a photographer, I sometimes get hired to shoot for a flat rate. The client expects to see the photos the next day and a DVD is definitely a smart idea-- especially hand delivered-- if that's what it takes. If you can't deliver your services on-time (even if your website is down), you're not likely to get more work from that client.

I have one client who needs catalog style shots of her boutique clothing for her website. We agree on a price, I shoot it, she gets all the files. I'd be an idiot to nickle and dime her by selling individual prints or files-- that's not what she needs or wants.

But obviously, this might not be a good idea for an event shooter. Though to be honest, a lot of successful wedding photographers now charge a flat rate and give all the files to their clients-- the clients then handle the printing themselves.

My point is the same as yours-- every situation is different-- but in the end, making sure the client has access to the images-- no matter what-- is really, really, really important. As we all know, servers do go down, harddrives fail, etc, etc. A backup plan is mandatory.

[irenelucier]

Posting is not my passion, but I want people who have not been here long to know just what good hands they're in. The timing of this outage was bad for me, I had just released event images to very interested parties. But my experience with SmugMug puts it into perspective.

Everything breaks. We already know this, it's why we carry our own backup equipment.

Figuring out and replacing failed network equipment without losing data is gonna take time. I've been with SmugMug long enough that when things break I am my most happy with my decision to stay here- even when I have the best reason to doubt SmugMug, they come thru.

I tumbled to this place thru google. Naturally I started at the least expensive account type and was perfectly happy with the default options as I shared photos with family.

Soon, customizing my little spot of the web appealed to me. SmugMug again was the best option I could find and the upgrade was painless.

I've encountered few problems, but whether it was my error or not, somebody at SmugMug helped like they really cared. When people started asking me to shoot for them, I upgraded to pro without bothering to look very hard at other services because I had learned that I could trust SmugMug to be there for me. My customers are thrilled with the service they get from SmugMug.

On top of the great service, SmugMug never delivers an Edsel, it's always a Cadillac. New technology can be wonderful, but it often comes with a period of instability as bugs get worked out. SmugMug works the bugs out and I hardly experience a ripple except that I keep getting new features and options. It's interesting too, to note that the feature upgrades most often delivered are those that have been requested by SmugMug users.

I could go on, ad nauseum, but I'll finish with this. Do youself a favor and find out for yourself. I predict you'll end up spending more time taking pictures than managing your SmugMug account. Isn't that the point?

[greenpea]

Aha! I think we know who caused our outage - it was greenpea's sister-in-law! :D

Seriously, sorry about the timing.

We're back up (again) for the time being. I'm watching it closely, we made some changes, and I'm hoping we're stable again. But if I were a betting man, I'd doubt it.

More as I get it...

Don

Thanks Don. I appreciate you taking the the time to respond.

I noticed that I've already received my $10 kickback from my convincing my sister-in-law to sign up, so now I don't care. :rofl

Seriously though, thanks for getting things back on line quickly!

[Andy]

an informed, "official" source. .
Papa, here are all the Smuggers, and the Dgrin names, too :) In this very thread you'll have seen official comments from Onethumb and Wireless (those closest to the issue).

Onethumb = CEO and Chief Geek
Baldy = President and Cofounder
Toni = Countess of Cash
Me = GM, House Pro
Wireless = Director of Operations
Primetime = Operations
{JT} = Web Superhero, SmugSorcerer
Bigwebguy = SmugSorcerer
Ben = Product Manager
Sheaf = Stats Geek
Markham = Director of Business Development
John Young = Captain of The Support Heroes
Steve Cavigliano, Barb Gates, Ivar, Anne, Thando, Seth, GrannyRobin = Support Heroes

[papajay]

Papa, here are all the Smuggers, and the Dgrin names, too :) In this very thread you'll have seen official comments from Onethumb and Wireless (those closest to the issue).

Onethumb = CEO and Chief Geek
Baldy = President and Cofounder
Toni = Countess of Cash
Me = GM, House Pro
Wireless = Director of Operations
Primetime = Operations
{JT} = Web Superhero, SmugSorcerer
Bigwebguy = SmugSorcerer
Ben = Product Manager
Sheaf = Stats Geek
Markham = Director of Business Development
John Young = Captain of The Support Heroes
Steve Cavigliano, Barb Gates, Ivar, Anne, Thando, Seth, GrannyRobin = Support Heroes

Thank you for the list, Andy. :thumb
I'm sure (at least hoping, anyway) that I'm not the only thick-headed numbskull that wasn't entirely certain that "Wireless - Director of Operations", for example, is in fact Smugmug's DOO (instead of DOO for some other company who just liked the title...No offense intended, Wireless.)....my point is, it didn't say "Smugmug". And didn't I see a recent post that Mike Lane is on the team? :dunno

Perhaps just a "Members of the Smugmug Team" link doubling as one of the Dgrin main pages (next to the Contact Us link, for example) would be a help to those of us who may take a little more hand-holding than others.

[Andy]

Thank you for the list, Andy. :thumb
I'm sure (at least hoping, anyway) that I'm not the only thick-headed numbskull that wasn't entirely certain that "Wireless - Director of Operations", for example, is in fact Smugmug's DOO (instead of DOO for some other company who just liked the title...No offense intended, Wireless.)....my point is, it didn't say "Smugmug". Same for Don. And didn't I see a recent post that Mike Lane is on the team? :dunno

Perhaps just a "Members of the Smugmug Team" link doubling as one of the Dgrin main pages (next to the Contact Us link, for example) would be a help to those of us who may take a little more hand-holding than others.Mike Lane is a good friend of SmugMug and Theme Creator extraordinaire, and all-round good guy and helper :D

Thanks for the suggestions.

[Andy]

Perhaps just a "Members of the Smugmug Team" link

How's this?
http://www.dgrin.com/showthread.php?p=463290#post463290

It's linked here in this sticky:
http://www.dgrin.com/showthread.php?t=31981

[papajay]

Mike Lane is a good friend of SmugMug and Theme Creator extraordinaire, and all-round good guy and helper :D

Thanks for the suggestions.

:thumb Agreed...plus a correction, Don's does indicate Smugmug. (But there are some that don't).

[Wireless]

FYI, as a byproduct of last night, we are performing some maintenance this evening:

http://www.dgrin.com/showthread.php?p=463337

[papajay]

How's this?
http://www.dgrin.com/showthread.php?p=463290#post46329

It's linked here in this sticky:
http://www.dgrin.com/showthread.php?t=31981

Andy...good info, of course...but if I read the fine print, the second was originally posted in April of 2006. I doubt if I would have found it without you leading me directly to it. It just seems to me the information should be in a prominant place, rather than tucked away. There are so many threads and forums on Dgrin, it's difficult, at times, to know where to start a search, so some of us find it easier to start a new thread than to spend ooodles of time looking for one we are not even sure exists. So the new location under the general announcements and support is a BIG step in the right direction, I think. Thanks for posting it there.

[Andy]

Andy...good info, of course...but if I read the fine print, it was originally posted in April of 2006. Papajay, I *just* posted that tonight, in response to your request. I've made it part of the general announcements sticky. What else can I do :D

[papajay]

Papajay, I *just* posted that tonight, in response to your request. I've made it part of the general announcements sticky. What else can I do :D

My mistake...the April 2006 date apparently is the date the general announcement sticky was launched, not the thread in question. :bow

[Andy]

My mistake...the April 2006 date apparently is the date the general announcement sticky was launched, not the thread in question. :bow
No worries, Papa :D Just wanted you to know that we took your suggestion seriously!

:wave

[aparmley]

I can't recall the last time, if ever seeing the site down. I've been a member for 2 years. I know I've gotten messages informing me re: adhoc maintenance and I appreciated the communication. Tonight doesn't bother me one bit - we have serious parties openly communicating about the outage, keeping us informed every step of the way. I especially liked the link to this thread via the main "Temporary Service Interruption" page. I thought that was an excellent touch.

Thanks guys. Best of luck troubleshooting. Thanks for the simple and straightforward information - good God is that refreshing.

Andy


Edit: Furthmore - I've been considering the upgrade to power-user a lot lately, I've been kinda upset I didn't go for it before the price increase, but after seeing how this outage has been handled. . . I have to support the way the entire staff has been handling this situation by upgrading my service. Thanks guys!