Image Exif Data Changed on Upload.....

jschoen · September 6, 2011

I am writing an application that uploads my images to SmugMug. All was going well, but I ran into 4 images (out of several hundred) that when uploaded I would get a different MD5 for it when I checked it against the original.

After some digging, I figured out the exif data on the images where different. The image that were now on SmugMug had these four additonal tags:

[Jfif] Version - 1.0
[Jfif] Resolution Units - unit
[Jfif] X Resolution - 4718664 dots
[Jfif] Y Resolution - 4718592 dots

I am sure it is by design, and not a bug, especially since it has only happened to 4 images out of several hundred. I was wondering if anyone could explain to me why this is happening it would be awesome.

Also not I manually uploaded them to a new test album I created and the same thing happened. So I am pretty sure it is not my code.

MSkaffari · September 7, 2011

jschoen wrote: »

I am writing an application that uploads my images to SmugMug. All was going well, but I ran into 4 images (out of several hundred) that when uploaded I would get a different MD5 for it when I checked it against the original.

After some digging, I figured out the exif data on the images where different. The image that were now on SmugMug had these four additonal tags:

I am sure it is by design, and not a bug, especially since it has only happened to 4 images out of several hundred. I was wondering if anyone could explain to me why this is happening it would be awesome.

Also not I manually uploaded them to a new test album I created and the same thing happened. So I am pretty sure it is not my code.

Hi

I'll gt someone to take a look at this. Could you point me to the gallery where the images are that you uploaded from your app and also for the manual upload.

Thanks

jschoen · September 7, 2011

MSkaffari wrote: »

Hi

I'll gt someone to take a look at this. Could you point me to the gallery where the images are that you uploaded from your app and also for the manual upload.

Thanks

I think I actually removed the ones I uploaded from my app, (and unfortunately can not check until I get home), but I put the 4 images I manually uploaded are in an album I cleverly called Test. The exif data for the manual upload and the upload from my app are exactly the same though.

Also worth noting (and I do not know if it is a coincidence or not) is that the 4 Jfif tags added to the 4 images all had the same values if I remember correctly.

Thanks for the help.

MSkaffari · September 7, 2011

jschoen wrote: »

I think I actually removed the ones I uploaded from my app, (and unfortunately can not check until I get home), but I put the 4 images I manually uploaded are in an album I cleverly called Test. The exif data for the manual upload and the upload from my app are exactly the same though.

Also worth noting (and I do not know if it is a coincidence or not) is that the 4 Jfif tags added to the 4 images all had the same values if I remember correctly.

Thanks for the help.

Any chance you opened and saved those 4 images in another program on your computer? You said that these images are changed, but same result through your app and manual upload. Combined with the data that these are the only files changed leads me to thinking it's somehow related to the files themselves. Either they have been changed on your machine or our system, but there is something different in these compared to the rest of the batch that causes the difference.

I can't see anything in these files that we would have added. Can you check the files with exiftool or similar to highlight the difference

jschoen · September 7, 2011

Well, I am fairly certain I have opened and saved these 4 images in another program on your computer, but it was such a long time ago, I do not know what program.

I will check them with exiftool this evening, but I am certain something happens on the upload. I can post up one before upload if that would help you. Also the change I found using a simple java app I wrote is once uploaded the images have the 4 additional [Jfifi] tags added in. I'll verify that I get the same results using exiftool.

I am sure there is a rational reason for it to happen. I am simply trying to find a way to determine when it is going to happen programmatically (not sure if that is a real word, but I'm gonna roll with it anyway) in my application. Since my app is essentially trying to keep a SmugMug account, and a local copy in sync, it uses the md5 of the images to see if they are already on SmugMug. Since I get a different md5 back, it thinks that the image has not been uploaded, and uploads it again.

jschoen · September 7, 2011

MSkaffari wrote: »

Any chance you opened and saved those 4 images in another program on your computer? You said that these images are changed, but same result through your app and manual upload. Combined with the data that these are the only files changed leads me to thinking it's somehow related to the files themselves. Either they have been changed on your machine or our system, but there is something different in these compared to the rest of the batch that causes the difference.

I can't see anything in these files that we would have added. Can you check the files with exiftool or similar to highlight the difference

MSkaffari, you wouldn't happen to know if there is a command you can use with exiftool to compare the exif data by any chance?

MSkaffari · September 8, 2011

jschoen wrote: »

MSkaffari, you wouldn't happen to know if there is a command you can use with exiftool to compare the exif data by any chance?

In case you are on a Mac or Linux you can try something like this;

diff <(exiftool photo1.jpg) <(exiftool photo2.jpg)

jschoen · September 8, 2011

Thanks for that. I should probably study up on some commandline tools, I had never seen diff before. Anyway here is the output, and apparently there is more different than I realized:

Jacob-Schoens-MacBook-Pro:~ jacobschoen$ diff <(exiftool '/Users/jacobschoen/Downloads/dsc01077-1.jpg') <(exiftool '/Users/jacobschoen/Desktop/messed up/DSC01077-1.JPG')
2,6c2,6
< File Name : dsc01077-1.jpg
< Directory : /Users/jacobschoen/Downloads
< File Size : 873 kB
< File Modification Date/Time : 2011:09:06 20:37:10-05:00
< File Permissions : rw-r--r--
---
> File Name : DSC01077-1.JPG
> Directory : /Users/jacobschoen/Desktop/messed up
> File Size : 794 kB
> File Modification Date/Time : 2011:06:15 20:54:18-05:00
> File Permissions : rwxrwxrwx
9,10d8
< JFIF Version : 1.01
< Comment : AppleMark.
63c61
< Thumbnail Offset : 2914
---
> Thumbnail Offset : 2882
64a63,69
> Comment : AppleMark.
> Image Width : 1520
> Image Height : 1625
> Encoding Process : Baseline DCT, Huffman coding
> Bits Per Sample : 8
> Color Components : 3
> Y Cb Cr Sub Sampling : YCbCr4:2:2 (2 1)
106,111d110
< Image Width : 1520
< Image Height : 1625
< Encoding Process : Baseline DCT, Huffman coding
< Bits Per Sample : 8
< Color Components : 3
< Y Cb Cr Sub Sampling : YCbCr4:2:2 (2 1)

MSkaffari · September 12, 2011

jschoen wrote: »

Thanks for that. I should probably study up on some commandline tools, I had never seen diff before. Anyway here is the output, and apparently there is more different than I realized:

For some reason, on all the files that are changed on upload have the Orientation flag defined twice in the metadata. We are changing our upload code to accommodate for this as well as some other oddities that can occur in EXIF data. I'll inform you when this has been changed on the live site.

jschoen · September 12, 2011

Thanks for the help. Sorry to make you all code around crappy input, I know it is not fun.

MSkaffari · September 13, 2011

jschoen wrote: »

Thanks for the help. Sorry to make you all code around crappy input, I know it is not fun.

Fixed and released. Let me know if there are any more issues with this one.

SamirD · September 15, 2011

I'm not sure if this affects the md5, but I do know this--if you upload an image that has an orientation bit set and SM automatically rotates the image upon upload, it will not compare correctly with the original if you download it and compare it. Even if you rotate it back, it's still different. This information might also help keep your application from re-uploading if this affects the md5.

thegoose · May 8, 2012

SamirD - This is the exact problem I just came up against. I'm writing a program to update SmugMug from files on my computer, automatically categorizing them by Year and Month. I wanted to use the MD5Sum to see if a file has been already uploaded, but SmugMug is modifying the uploaded photo's that have the Orientation bit set, thus changing the MD5Sum. :-( I guess that I'll have to use something else to determine uniqueness.

Any chance that SmugMug will stop doing this and leave the uploaded photo's untouched?

SamirD · May 9, 2012

thegoose wrote: »

SamirD - This is the exact problem I just came up against. I'm writing a program to update SmugMug from files on my computer, automatically categorizing them by Year and Month. I wanted to use the MD5Sum to see if a file has been already uploaded, but SmugMug is modifying the uploaded photo's that have the Orientation bit set, thus changing the MD5Sum. :-( I guess that I'll have to use something else to determine uniqueness.

Any chance that SmugMug will stop doing this and leave the uploaded photo's untouched?

It would be nice to have auto-rotation as a feature that could be turned off. That might be more feasible since most users probably rely on the feature working.

Image Exif Data Changed on Upload.....

Comments