Camera is back and...the jury is still out.

So I got my camera back from service today and managed to take a few shots outside and then ran some tests inside.

For the inside tests, I ran the FoCal software to find the best microAF settings for all of my lenses (including the 3 loaners). Ran the AF consistency test and the camera was much improved from prior and seemed quite consistent. Note this is done in one shot mode and the camera had failed miserably here previously.

Prior to doing the inside tests, I had put the 135/2 on each camera (with both set to zero microAF adjustment) and went out shooting the kids in soft light (got home at 6pm). I jumped back and forth between cameras and, as a test, loaded them in LR with the keyword 'mine' and 'loaner' respectively. I then went to full screen and 1:1 previews and cycled through all shots by capture time. Rated 1-4 based on sharpness. All taken in AI servo mode.

Ended up taking more shots with mine than the loaner (80 vs 23). Results:
1 star (misfocus) - mine 44%; loaner 26%
2 stars (okay focus or tough to tell as eyes closed, etc) - mine 44%; loaner 35%
3 stars (great focus - spot on) - mine 11%; loaner 26%
4 stars (wow, that's sharp!) - mine 1%; loaner 13%

The n was too small to decide whether there was a real signal here.

Curiously, mine was consistently underexposing by at least 2/3 to a full stop compared to the loaner.

More testing in better light tomorrow. The saga continues...

I think the crux is going to be now/again how much validity Canon is going to give to your evidence as relevant to their response. I've always felt that the FoCal evidence is weak, given the 5D3 is not currently supported. Both sets of evidence, FoCal and your shooting examples, are vulnerable to user error/variation and subjective assessment. Versus Canon's objective and definitive testing. This is not my personal argument, but how I see the situation as an observer.

What you do have, which I think is very solid indeed, is a defacto if not explicit recognition by Canon that there were defects in critical parts of your 5D3 copy, to the extent that they have adjusted and replaced some of those parts. That you still find the camera does not perform to fit-for-purpose standard, and particularly so in comparison with your totally positive experience of the loan 5D3 copy, is something which does NOT require "proving" by their hardware testing protocols. This is a matter of your product satisfaction, which is completely valid in its own right. And put together with the history of the case which includes Canon's acceptance of a problem at their end, is now likely to outweigh any further argument Canon might consider making.

If I were you, I would ship back the defective copy to Canon without delay, with your assessment from your FoCal and real world shooting evidence, from it and the loaner, and reiterate that the defective copy is not acceptable to you as the purchaser and warranty holder for the reasons you give, and with the evidence that Canon have not been able in the case of the defective copy to find and fix the source of its performance failures in their own terms, and compared, in your experience, with the loaner. I would also make clear to Canon that the remedy process that they have put you through has now been exhausted while their obligation to satisfy your reasoned, evidenced, and legally supported expectations as purchaser and warranty holder of their product has still not been met.

The way that they respond will need care as of treading on thin ice, because it would have legal implications. I think it is likely that they will exit at this stage from contention.

Have you been able to define the proper route for escalating this? I would start along that a short time (3 days?) after you know the camera has been received back by Canon, cc to the service centre manager, if you have not received any/an acceptable response by then. You will have then opened the "battle" for them on two fronts, and put them under scrutiny of higher management.

Neil

Originally Posted by eoren1 Camera is back and...the jury is still out. So I got my camera back from service today and managed to take a few shots outside and then ran some tests inside. For the inside tests, I ran the FoCal software to find the best microAF settings for all of my lenses (including the 3 loaners). Ran the AF consistency test and the camera was much improved from prior and seemed quite consistent. Note this is done in one shot mode and the camera had failed miserably here previously. Prior to doing the inside tests, I had put the 135/2 on each camera (with both set to zero microAF adjustment) and went out shooting the kids in soft light (got home at 6pm). I jumped back and forth between cameras and, as a test, loaded them in LR with the keyword 'mine' and 'loaner' respectively. I then went to full screen and 1:1 previews and cycled through all shots by capture time. Rated 1-4 based on sharpness. All taken in AI servo mode. Ended up taking more shots with mine than the loaner (80 vs 23). Results: 1 star (misfocus) - mine 44%; loaner 26% 2 stars (okay focus or tough to tell as eyes closed, etc) - mine 44%; loaner 35% 3 stars (great focus - spot on) - mine 11%; loaner 26% 4 stars (wow, that's sharp!) - mine 1%; loaner 13% The n was too small to decide whether there was a real signal here. Curiously, mine was consistently underexposing by at least 2/3 to a full stop compared to the loaner. More testing in better light tomorrow. The saga continues...

Thanks NeiL
I agree about FoCal not being definitive. But the prior graph comparing two bodies with all else equal does support its ability to provide reliable objective measures. The 5d3 is listed as supported on their site. It is just not able to do the slick fully automated AF routine.

At this point, I love my loaner 5d3. It has proved itself more than capable over the past few weeks. I am very thankful that Canon sent it my way and really hate to part with it. Trying to blind myself to the camera and determine favorite shots further solidified that feeling.

I do want to give my camera a chance and will shoot some more tomorrow but may end up approaching Canon about swapping my camera for the loaner. If they think this camera is as good as it gets, it should be up to being a loaner for others.

On my last call, I was advised to ask for a customer relations supervisor if I needed to call again. We'll see how this evolves.

I'm really not someone who is looking for minute differences in cameras for some perverse reasons. I hope that is obvious. My camera has simply never shown me the level of satisfaction as the loaner. It would be great to have some respected reviewer take my camera and see what he would come up with...

Eyal, personally I have no doubt the original 5D3 is a dud. Re FoCal, any 3rd party app, especially one not yet fully supporting the 5D3, that could be seen to be preempting Canon's own hardware performance-diagnosis process, I think is not going to last the argument. At least not on its own. But as supporting your dissatisfaction with the original product you purchased, along with real shooting evidence, a comparison with the loaner copy, and Canon's own admission of problems with the original body, it has its place.

I very much doubt that escalating to a supervisor at the service centre is going to be seriously felt. I think it needs to be Canon USA head office. Independently of the service centre and concurrent with your dealings with it. Now.

I think the time for arguments and the normal process is over. It's now about asserting the facts of your right to satisfaction, the admission by Canon of defect and inability to correct, and the facts of your evidence, and your comparison with the loaner. Escalation on this basis will lift the situation to a different response requirement than has been made locally to date. That is where you want to be now, in my opinion.

I would avoid spinning out argument on your part as well. Clearly stating the facts as above and suggesting that the matter would end for you with exchange for the loaner, I suggest puts them in the position of having to choose to keep the argument going at the lower level, finding valid reasons for doing that, and involving themselves in potentially costlier involvement with this case - or agreeing with your position.

Neil

Originally Posted by eoren1 Thanks NeiL I agree about FoCal not being definitive. But the prior graph comparing two bodies with all else equal does support its ability to provide reliable objective measures. The 5d3 is listed as supported on their site. It is just not able to do the slick fully automated AF routine. At this point, I love my loaner 5d3. It has proved itself more than capable over the past few weeks. I am very thankful that Canon sent it my way and really hate to part with it. Trying to blind myself to the camera and determine favorite shots further solidified that feeling. I do want to give my camera a chance and will shoot some more tomorrow but may end up approaching Canon about swapping my camera for the loaner. If they think this camera is as good as it gets, it should be up to being a loaner for others. On my last call, I was advised to ask for a customer relations supervisor if I needed to call again. We'll see how this evolves. I'm really not someone who is looking for minute differences in cameras for some perverse reasons. I hope that is obvious. My camera has simply never shown me the level of satisfaction as the loaner. It would be great to have some respected reviewer take my camera and see what he would come up with...

Bit more testing (this is getting tedious) this morning with my kids. Again with the 135/2.
Misfocus: 22% with mine; 10% with loaner
Mine is consistently 1/3 to 1/2 stop underexposed
Cameras have different auto white balance biases in the same lighting conditions as well. It's subtle but definitely there

Is there really supposed to be that much variation in a top-end camera body? I would think that exposure and auto-WB readings would not vary much, if at all, between similar bodies.

Does your camera have the latest (or same) firmware as the loaner? The latest firmware does have a fix for under exposed images, but I don't think it's for mode you are using, but who knows? In any case if you are comparing the cameras they should have the same firmware.

Originally Posted by eoren1 Bit more testing (this is getting tedious) this morning with my kids. Again with the 135/2. Misfocus: 22% with mine; 10% with loaner Mine is consistently 1/3 to 1/2 stop underexposed Cameras have different auto white balance biases in the same lighting conditions as well. It's subtle but definitely there Is there really supposed to be that much variation in a top-end camera body? I would think that exposure and auto-WB readings would not vary much, if at all, between similar bodies.

Good point Dan. Just checked and my camera did come back with 1.1.3. Loaner still has 1.1.2.

Curiously, I'm finding that the new firmware underexposes more and does have a different WB. Will update the loaner in a sec.

Got off the phone with manager at Canon's NJ center. Said that, to his knowledge, they have never swapped a camera (ie loaner for customers) before. Really loving the loaner though...

Planning to shoot with mine the rest of the day and leave the loaner at home. I know how that one works. Time to see how well I can make do with 'mine'.

A couple more things you might try. Doing hand held, separate shots with what ever changes in the scene can really affect WB and AF.

Have you tried setting both cameras on a table (or two tripods if you have them) and shoot an static image like a wall, tree, etc. simultaneously.

The to see how well AF works on actions shots do the same thing but have one of you kids run through the scene and press the shutter buttons at the same time. Put a marker in the scene (maybe a chair) for the kids to run by (and maybe have the kids holding trays and wearing roller skates just a joke). Then do regular focus (not AI) and burst shots on both cameras simultaneously as the kids run through the scene. Have all focus points turned on, it theory it should focus on the nearest thing in the scene, the kids in front of the chair.

Then pick single focus point have one of your kids stand in front of the chair and pick a point that is on the head. Then do another run though.

Then compare those runs.

Use a high shutter speed, because with the cameras stationary you IS won't be a factor. In fact make sure IS is off.

That should eliminate any variability introduced by the experimenter.

Originally Posted by eoren1 Good point Dan. Just checked and my camera did come back with 1.1.3. Loaner still has 1.1.2. Curiously, I'm finding that the new firmware underexposes more and does have a different WB. Will update the loaner in a sec. Got off the phone with manager at Canon's NJ center. Said that, to his knowledge, they have never swapped a camera (ie loaner for customers) before. Really loving the loaner though... Planning to shoot with mine the rest of the day and leave the loaner at home. I know how that one works. Time to see how well I can make do with 'mine'.

One thing is for sure, I would not return the loaner until you are 100% satisfied with "your" camera. The loaner is the bargaining chip in this game. As long as you have it in your possession, you are in the driver's seat... assuming they don't have your credit card info.

Like I said before, at some point it will simply not be worth any further effort on their part, and they should just exchange the loaner for your camera at that time. Hang in there, you can outlast them.

Dan
I actually ran through extensive testing of 'one shot' focus last night using FoCal and the microAF routine. Once I got the 'sweet spot' for the lens, I ran their AF consistency module. Did one for 50 shots; others for 10. Mine was very consistent and far better than previous.
As for AI servo, I think that's the tougher test and, for good or bad, I use that mode 90% of the time. I'm going to switch from the 135/2 to the 70-200/2.8 IS II and see if the keeper rate goes up.

Jack
Have to say that I'm getting tired of doing all of this testing to 'prove' the camera is the equal of the loaner. Wish they would just do the trade and we can all be done with the saga.

I do want to tell everyone who has posted here how much I appreciate their comments and support. Really means a lot to me - quite a crew we've got here on Dgrin

Wow...
Just spent an hour on the phone with one manager then another. Somehow I ended up in Corporate Relations

Explained the situation to them in full and finished by expressing my sincere interest in keeping the loaner which has been rock-solid in exchange for the original camera. After the manager discussed this with the supervisor at NJ's service center, he called me back. In no uncertain terms, he said I would need to continue testing out the camera. Again harped on my use of 3rd party software (ignoring the images sent on CD). He insinuated that the supervisor there and the technicians know far better than I do what camera tolerances are.

I'm at my wits end. I know what is right here and I know that my camera body is incapable of achieving focus as reliably as the loaner. This is by far the most frustrating experience in customer support I have ever experienced and the most expensive item with which I have had problems to this degree.

Just really tired of this continuing saga

Small claims court.

I passed this thread on to some folks at Canon. We'll see if that helps, Eyal.

Thank you Andy and everyone else.

I have found resolution and his name is Hal! Called Beach Camera a few hours ago and they issued an RMA. Packed up the camera and lens and heading to FedEx with them (as well as the loaner 5D mk III, 70-200/2.8, 135/2 and 100/2). Ready to put this whole horrible saga behind me.

start rant - Have to say, that the most frustrating aspect in the end was the discussion I had with a manager at Canon Corporate Relations today. I admit I do not know as much about Canon gear as the techs that have been there for years but please don't insult my intelligence. I worked hard over the past 6 weeks to help pinpoint the problems with the camera and whether I used a 3rd party software (to offer whatever objective evidence I could) should not disqualify all the effort I put into this. I have been nothing but cordial in all of my verbal interactions with Canon. All I ever wanted was a single person there to sympathise with the frustrations I was expressing and help me come to a mutually agreeable resolution. - end rant

Originally Posted by eoren1 I have found resolution and his name is Hal! Called Beach Camera a few hours ago and they issued an RMA. Packed up the camera and lens and heading to FedEx with them (as well as the loaner 5D mk III, 70-200/2.8, 135/2 and 100/2). Ready to put this whole horrible saga behind me.

Really hope this works out for you. Kinda surprised you relinquished the loaner, but my fingers are crossed.

Good Luck! Be comforted that lightning never strikes the same place twice

Originally Posted by eoren1 I have found resolution and his name is Hal! Called Beach Camera

Jack
They were never going to allow me to keep the loaner. That became quite clear after talking to 'management'. Once that option was taken away, I felt really stuck. The call to Beach was like a Hail Mary pass with seconds left on the clock.

How were they going to not allow you to keep the loaner? Did they have your CC?

Well no. But they made it clear they would continue to push me to accept 'my' camera and would not authorize the loaner on exchange.

Frankly I was tired of running tests to prove to them what was clear to me - the camera's autofocus was inferior to that of the loaner. Whether that fell under 'factory tolerances', I can't answer. But for $3500 I would hope my Kepler rate would be better than a 4 year old 50d with its 9 AF points.

Well... I hope the new one works out. Let us know.