The other issues I raised are even more important than the date issue.
Your acknowledgment does not solve the problem.
The real problem is that you have decided to cover the concerns of your larger customer base, the personal photographer ( Non-Pro) without a thought for your Pro customers.
The issues I raised as constructive suggestions were not even thought about until I raised them.
What you have to do is determine how to solve these issues to the satisfaction of your Pro and Non-Pro customers.
And so far you have not accomplished this solution.
The real problem is that you have decided to cover the concerns of your larger customer base, the personal photographer ( Non-Pro) without a thought for your Pro customers.
The issues I raised as constructive suggestions were not even thought about until I raised them.
:bigbs
come on Nick, you can't be serious. I seem to remember only a few months ago a whole load of stuff released for pros...backprinting, custom watermarks, etc.
my concerns stated above are still valid and should be addressed.
From my experience, the frequency and typical period of SmugMug site outages seem to be well within industry norms, but I don't understand the implications from the point of view of a working pro.
Out of curiosity, based on today's event, would you be willing to share with us:
a) your average number of sales multiplied by the average net sale proceeds, calculated for the relevant day and time of day (i.e. the estimated total gross profit from sales, not the gross sales value), that you might have missed during the outage - i.e. your direct financial 'cost', assuming the sales are completely lost
b) the average number of customer visits to your site, also calculated for the relevant day and time of day, that you might have missed during the outage - i.e. the impact for your customer goodwill
Wait, its not working from Florida and Andy's not working and IN Florida....conincidence? I think not!
It is the Andy effect. Andy, you back in NYC and now its working?
It is the purpose of life that each of us strives to become actually what he is potentially. We should be obsessed with stretching towards that goal through the world we inhabit.
And so far you have not accomplished this solution.
Thank you, Nicholas
Hi Nick,
I'm sorry you are unhappy with our service. I wish I could do more for you. We have put a tremendous amoung of time, energy, resources, people, and cash into our pro business. I deal with our pros on a daily basis, these are pros who have their livelihoods depend on SmugMug for our service, our site, and our people. We take everyone's input very seriously, and we apply our development resources and energy to all facets of our business.
I hope that we can satisfy you, too. Apologies again for the few minutes downtime tonight.
Hi Nick,
I'm sorry you are unhappy with our service. I wish I could do more for you. We have put a tremendous amoung of time, energy, resources, people, and cash into our pro business. I deal with our pros on a daily basis, these are pros who have their livelihoods depend on SmugMug for our service, our site, and our people. We take everyone's input very seriously, and we apply our development resources and energy to all facets of our business.
I hope that we can satisfy you, too. Apologies again for the few minutes downtime tonight.
I mentioned nothing with your downtime.
It is your posted notice when you are down that is a problem.
If you want to satisfy me then change your outage notice and don't mention the words 'Lost data' Or 'Data loss'.
Make reference to the site the visitor just typed into their browzer.
Mention a request to re-visit soon or in the near future.
Put some of your tremendous energy into the problem of this discussion,your outage notice.
It is your posted notice when you are down that is a problem.
If you want to satisfy me then change your outage notice and don't mention the words 'Lost data' Or 'Data loss'.
Make reference to the site the visitor just typed into their browzer.
Mention a request to re-visit soon or in the near future.
Put some of your tremendous energy into the problem of this discussion,your outage notice.
Thank you, Nicholas
Hi Nick, thanks again for posting. I've made sure that our product manager has seen this. We appreciate it.
Unfortunately, our load balancer is showing its generic message right now. We'll pop up the other shortly. Sorry for the additional outage.
Ouch-- right in the middle of an upload. When I get cranky, sometimes it helps to eat something... wait, nevermind, we don't want the database eating anything!
Just crossing my fingers the downtime(s) today are due to implementing the latest updates soon?
Holy smokes-- you folks did listen to my suggestion for the downtime message! Wow! Thank you!
"The site you are trying to access is hosted by Smugmug. We're having some temporary technical difficulties..."
So I'm reading through this thread, seeing what's going on, and I hop back over to refresh the other tab to see if it's available yet. And I get " The site you are trying to access is hosted by SmugMug."
Not bad - 2.5 hours to implement customer suggestions.
So I'm reading through this thread, seeing what's going on, and I hop back over to refresh the other tab to see if it's available yet. And I get " The site you are trying to access is hosted by SmugMug."
Not bad - 2.5 hours to implement customer suggestions.
I do wish the site was back up though.
Agree, that's a fast implement of customer suggestions. But please, get us our sites back. Good luck guys! You can do it! :devbobo
We're turning things back up now. Sorry for the extra wait, we were gathering some additional troubleshooting data while things were offline.
The site is back up, but we don't know for sure if it'll stay up. Sorry.
We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.
We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.
The site is back up, but we don't know for sure if it'll stay up. Sorry.
We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.
We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.
Thanks for your patience.
Don
Keep up the good work, but I'm putting you on the clock....my first motocross race is April 8th! I'll need to upload some shots later that day...
The site is back up, but we don't know for sure if it'll stay up. Sorry.
We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.
We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.
Thanks for your patience.
Don
We're down again, but got some useable data. Definitely looks like a hardware failure. Andrew's on his way to the datacenter now to physically take a look at what's going on.
We have a couple of avenues to take, so we'll start taking them one at a time. I wouldn't be surprised if there are some more hiccups along the way.
I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.
-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?
I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.
I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.
-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?
I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.
I think that you already answered your own questions :-)
If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.
Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.
Here's hoping this works itself out as fast as possible. Just, finally, do I get the cast from my opening Big Event Shoot to finally settle down from the performing and actually *look* at the pictures...
and the damn server crashes. :-}
What the previous poster said... I'm sure you're a bit busy just now... but when you're not, those might be useful questions to address.
I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.
-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?
I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.
Hi there,
Hopefully some of our long-term customers will weigh in here with an unbiased opinion, but the view from the inside looks something like this:
- These outages ARE NOT common. In terms of non-scheduled downtime, we aim to have less than one hour (99.99%) uptime. Last year (2006) we didn't make it because of one prolonged outage due to a distributed denial of service attack, but excluding that one, we were will within range. Previous years we made it with plenty of room.
- We don't provide an SLA. You're the first one to ask for one. I'm not opposed to implementing one, but we'd need to think about how to do it. Some estimate of the time down, multiplied by your average sales dollars per second might do the trick, but we'll have to talk about that. I'm not sure our service is really high-dollar enough to warrant an SLA, but maybe I'm wrong?
- We don't provide availability metrics, and again, you're the first to ask. We'd certainly do so if we had an SLA in place.
- We have multiple pieces of hardware standing by, but I'm not about to replace a piece of hardware without first verifying that it's the root cause. If it wasn't, I just wasted an hour or two doing nothing useful. We're still diagnosing the problem, but we're getting closer.
- At this point, I believe the only non-redundant components are core networking switches. (This is a common component to not be redundant - every major internet brand suffers from outages due to network switch failures) All of our routers, servers, and storage are redundant. Many of them are automatic fail-over, too, so you'll never notice an outage. With critical data, though, it's essential to have manual failover so we don't introduce data corruption due to split-brain or time-based latency.
When you compare our uptime to any major brand, including Google, eBay, Hotmail, or Amazon, you'll find that we're comparable. No-one meets 99.999% uptime, and most don't even come close to 99.99%.
We're doing the best we can, of course, and will continue to think about how we can improve.
Thanks for the questions, hopefully some third party validation will happen with our customers, but it's possible my world view differs from theirs.
I think that you already answered your own questions :-)
If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.
Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.
Cheers
MG
I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.
I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.
Infrequent, and normally resolved in minutes, as far as I can see. I don't clock it, but outages are very rare and quickly recovered.
Comments
Thanks again, Nick.
Portfolio • Workshops • Facebook • Twitter
Portfolio • Workshops • Facebook • Twitter
The other issues I raised are even more important than the date issue.
Your acknowledgment does not solve the problem.
The real problem is that you have decided to cover the concerns of your larger customer base, the personal photographer ( Non-Pro) without a thought for your Pro customers.
The issues I raised as constructive suggestions were not even thought about until I raised them.
What you have to do is determine how to solve these issues to the satisfaction of your Pro and Non-Pro customers.
And so far you have not accomplished this solution.
Thank you, Nicholas
:bigbs
come on Nick, you can't be serious. I seem to remember only a few months ago a whole load of stuff released for pros...backprinting, custom watermarks, etc.
Grow up...
SmugMug API Developer
My Photos
Out of curiosity, based on today's event, would you be willing to share with us:
a) your average number of sales multiplied by the average net sale proceeds, calculated for the relevant day and time of day (i.e. the estimated total gross profit from sales, not the gross sales value), that you might have missed during the outage - i.e. your direct financial 'cost', assuming the sales are completely lost
b) the average number of customer visits to your site, also calculated for the relevant day and time of day, that you might have missed during the outage - i.e. the impact for your customer goodwill
c) your site URL.
Thanks
It is the Andy effect. Andy, you back in NYC and now its working?
I'm sorry you are unhappy with our service. I wish I could do more for you. We have put a tremendous amoung of time, energy, resources, people, and cash into our pro business. I deal with our pros on a daily basis, these are pros who have their livelihoods depend on SmugMug for our service, our site, and our people. We take everyone's input very seriously, and we apply our development resources and energy to all facets of our business.
I hope that we can satisfy you, too. Apologies again for the few minutes downtime tonight.
Portfolio • Workshops • Facebook • Twitter
No he's still in FL
SmugMug API Developer
My Photos
I mentioned nothing with your downtime.
It is your posted notice when you are down that is a problem.
If you want to satisfy me then change your outage notice and don't mention the words 'Lost data' Or 'Data loss'.
Make reference to the site the visitor just typed into their browzer.
Mention a request to re-visit soon or in the near future.
Put some of your tremendous energy into the problem of this discussion,your outage notice.
Thank you, Nicholas
Portfolio • Workshops • Facebook • Twitter
Unfortunately, our load balancer is showing its generic message right now. We'll pop up the other shortly. Sorry for the additional outage.
Http/1.1 Service Unavailable
Sebastian
SmugMug Support Hero
Here's to an ever improving Smugmug...Cheers!!!!!!!!!
Nicholas
Just crossing my fingers the downtime(s) today are due to implementing the latest updates soon?
Holy smokes-- you folks did listen to my suggestion for the downtime message! Wow! Thank you!
Portland, Oregon Photographer Pete Springer
website blog instagram facebook g+
Yes, we are experiencing problems at this time. Our engineers are working feverishly to restore service.
Sorry :cry
Steve
Master Of Sushi Noms
Amateur CSS Dork
So I'm reading through this thread, seeing what's going on, and I hop back over to refresh the other tab to see if it's available yet. And I get " The site you are trying to access is hosted by SmugMug."
Not bad - 2.5 hours to implement customer suggestions.
I do wish the site was back up though.
Agree, that's a fast implement of customer suggestions. But please, get us our sites back. Good luck guys! You can do it! :devbobo
www.pablopuga.com
The site is back up, but we don't know for sure if it'll stay up. Sorry.
We're getting a strange error message on one of our core database machines. It feels, to me, like a hardware problem, but we just can't be sure yet.
We do have spare hardware standing by, but switching is time-consuming, so we're doing a little experimentation first. Hopefully we can solve it without resorting to new hardware.
Thanks for your patience.
Don
Keep up the good work, but I'm putting you on the clock....my first motocross race is April 8th! I'll need to upload some shots later that day...
http/1.1 service unavailable
We're down again, but got some useable data. Definitely looks like a hardware failure. Andrew's on his way to the datacenter now to physically take a look at what's going on.
We have a couple of avenues to take, so we'll start taking them one at a time. I wouldn't be surprised if there are some more hiccups along the way.
We'll keep you posted here.
Don
Dear Smugmug:
I recently enabled and configured my pro account and have made good progress until today which has been an excercise in frustration. My customer is waiting to view their wedding photos and I'm frankly hesitant to enable the link given the instability.
-Are these outages common?
-What is the SMUGMUG's SLA?
-Do you provide availability metrics to your customers?
-If the problem is a core server, why is there no standby?
-What components are not redundant?
I understand the infrastructure challenge you face and know you will resolve this problem but I would like to better understand my exposure so I can address the concerns of my customers.
I think that you already answered your own questions :-)
If you are a Pro, you should have already mailed a DVD to your customer pronto. Nothing ever works perfectly all the time, so you must have options you can control yourself. I think your best backup is your own site and your own DVDs to send out via snail mail if needed.
Seriously, pop some small pics on to a free Flicker acct or whatever else right away and get your customer looking at the images. Worry about Smug coming back later.
Cheers
MG
http://manny.smugmug.com/
and the damn server crashes. :-}
What the previous poster said... I'm sure you're a bit busy just now... but when you're not, those might be useful questions to address.
Hi there,
Hopefully some of our long-term customers will weigh in here with an unbiased opinion, but the view from the inside looks something like this:
- These outages ARE NOT common. In terms of non-scheduled downtime, we aim to have less than one hour (99.99%) uptime. Last year (2006) we didn't make it because of one prolonged outage due to a distributed denial of service attack, but excluding that one, we were will within range. Previous years we made it with plenty of room.
- We don't provide an SLA. You're the first one to ask for one. I'm not opposed to implementing one, but we'd need to think about how to do it. Some estimate of the time down, multiplied by your average sales dollars per second might do the trick, but we'll have to talk about that. I'm not sure our service is really high-dollar enough to warrant an SLA, but maybe I'm wrong?
- We don't provide availability metrics, and again, you're the first to ask. We'd certainly do so if we had an SLA in place.
- We have multiple pieces of hardware standing by, but I'm not about to replace a piece of hardware without first verifying that it's the root cause. If it wasn't, I just wasted an hour or two doing nothing useful. We're still diagnosing the problem, but we're getting closer.
- At this point, I believe the only non-redundant components are core networking switches. (This is a common component to not be redundant - every major internet brand suffers from outages due to network switch failures) All of our routers, servers, and storage are redundant. Many of them are automatic fail-over, too, so you'll never notice an outage. With critical data, though, it's essential to have manual failover so we don't introduce data corruption due to split-brain or time-based latency.
When you compare our uptime to any major brand, including Google, eBay, Hotmail, or Amazon, you'll find that we're comparable. No-one meets 99.999% uptime, and most don't even come close to 99.99%.
We're doing the best we can, of course, and will continue to think about how we can improve.
Thanks for the questions, hopefully some third party validation will happen with our customers, but it's possible my world view differs from theirs.
Don
I can surely implement one of your suggestions but I moved to SMUGMUG to simplify workflow and improve the customer experience. My question was less about this outage and more about next time. I'd like to understand if this is once a year or once a month. Hopefully, my previous questions can be addressed once the incident is resolved.
Infrequent, and normally resolved in minutes, as far as I can see. I don't clock it, but outages are very rare and quickly recovered.
Dgrin FAQ | Me | Workshops