Star*Explorer Thread

[technocraft]

Thanks Nik. Don't you love relying on text parsing?

[Nikolai]

Originally Posted by technocraft

Thanks Nik. Don't you love relying on text parsing?

Story of my life

[Nikolai]

Congratulations to Paul Deines, S*E Pro Customer #300!
Another nice milestone for my little app

[rainforest1155]

Originally Posted by Nikolai

Congratulations to Paul Deines, S*E Pro Customer #300!
Another nice milestone for my little app

Way to go Nik! Congrats to you and Paul!!

Cheers,
Sebastian

[Nikolai]

Originally Posted by rainforest1155

Way to go Nik! Congrats to you and Paul!!
Cheers,
Sebastian

I knew I'd hear from you on this one:-)

[rainforest1155]

Originally Posted by Nikolai

I knew I'd hear from you on this one:-)

Sure thing! Also haven't forgot the LinkedIn entry - hope I'll manage to write it before I'm hopefully leaving for Ecuador in October.

Sebastian

[Nikolai]

Many of use were caught by surprise when a couple months ago Smugmug changed the internal upload procedure.

Originally a file was brought to the Smugmug server, immediately analyzed upon delivery and then either rejected or accepted. That allowed for a file-by-file synchronous (immediate) response analysis and feedback. Both 3d party tools and SM web tools were prepared to handle this type of response.

At some point in May 2006 (I'm not sure about the exact date since I kinda missed the memo - if there was one) Smugmug changed the workflow (I think, to optimize the workload and general system responsiveness).

Now any file (valid or not) is immediately issued a valid ID upon delivery - and that's it. You can think about it as a ticket number in a triage line. This number does not guarantee your case will be accepted. It only establishes your position in the queue. Depending on the backlog, your wait time can be any from 0 sec to a few hours. In my account the longest wait was about 2.6 (!) hours, however such outrageous wait happened only once during one upload on May 29th, and only about two dozen files suffered from it.

After the wait is over, file is analyzed, sometimes rejected without a reason (again, in my case all the "erroneous" files were successfully re-uploaded later, thus making SM server a culprit, since MD5 hash should guarantee the delivery).
Then it's processed - usually almost instantly, but sometimes it may take a minute or two.

All in all, the new upload workflow essentially became asynchronous, rendering any previous file-by-file check routine useless. You upload your images and then have to wait unknown amount of time for the servers to finally get to them and issue the verdict.

The only information available to analyze is the Upload Log. It can be rather large, and looking for upload mishaps can be a very tedious task, leaving a lot of room for the operator errors. Imagine a pro with thousands and thousands of uploaded images (and many of us have way more). Say, 10,000 x 1K per image - you'd need to download 10Mb each time (!) just to see the log. Of course, the Internet cache helps and you can temporarily switch off image showing, but this is also a hassle.. Unfortunately, the log also grows with every upload

With the new version of S*E this is not gonna be an issue anymore.

You'll get two new main toolbar buttons (and two correspondent Online menu items).

One provides a direct link to your server's upload log, so, at least, you don't have to look for it the control panel.

The other - and this is the primary treat for S*E users - allows you to download the server upload log info locally (without the pictures, hence much faster:-) and then lets you view, sort, filter, export, save and, most importantly, re-upload the files that were (mistakenly) rejected by the smugmug servers.




No more manual searching for errors! Filter by Date, sort by Status or Message, or simply check Snow Errors Only - and your data will be instantly in front of your tired eyes!

Moreover, S*E even caches the last log download for you and allows you to look at it later without waisting your time to download the same info over and over again.

And since the latest versions of S*E store some upload-related info in the cache, you don't have to locate albums, retype the captions, etc. Just hit Reupload, verify that your original files are, in fact, OK, and hit green Upload button

This feature is provided for all S*E editions, from Trial to Standard to Pro.

I hope you will enjoy the new version!

As always, let me know if you have any questions or concerns.

Cheers!

[ExposeTheMoment]

Why is there an upload limit? (FILE SIZE)<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p>
<o:p></o:p>
I wanted to upload 1,010 pictures and some where shot with a 20D and some with a 5D, the 5D files were over the limit, so instead of using Star Explorer I had to use “Send to SmugMug”
<o:p></o:p>
Took me 23 hours to upload all the pictures J

[Nikolai]

Originally Posted by ExposeTheMoment

Why is there an upload limit?<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p>
<o:p></o:p>
I wanted to upload 1,010 pictures and some where shot with a 20D and some with a 5D, the 5D files were over the limit, so instead of using Star Explorer I had to use “Send to SmugMug”
<o:p></o:p>
Took me 23 hours to upload all the pictures J

Gary, this is probably some misunderstanding.
There is no, never was and never will be "an upload limit". SM servers can "act up" (in fact, they were slow this weekend), but it's not up to me...
And if you're talking about per-file limit... Well, you can control it yourself. S*E sets default to 8Mb (since it has been a common gound for all SM subscriptions), but for Pro account you can set it to 16Mb. Just check out S*E Preferences. You can set it even higher, it's just a screening filter, but AFAIK SM only allows for 16Mb...

You can remove it (by setting it to some outrageous value, like 99Mb), only to find out that your huge files were uplaoded and later rejected. Luckily, with the Upload Log Viewer you can find it out quickly...

Please let me know if you have other issues/concerns/questions.

Thank you for using Star*Explorer!

PS
I wish you knew about this setting, it would probably take you less than that :-)

[ExposeTheMoment]

Originally Posted by Nikolai

Gary, this is probably some misunderstanding.
There is no, never was and never will be "an upload limit". SM servers can "act up" (in fact, they were slow this weekend), but it's not up to me...
And if you're talking about per-file limit... Well, you can control it yourself. S*E sets default to 8Mb (since it has been a common gound for all SM subscriptions), but for Pro account you can set it to 16Mb. Just check out S*E Preferences. You can set it even higher, it's just a screening filter, but AFAIK SM only allows for 16Mb...

You can remove it (by setting it to some outrageous value, like 99Mb), only to find out that your huge files were uplaoded and later rejected. Luckily, with the Upload Log Viewer you can find it out quickly...

Please let me know if you have other issues/concerns/questions.

Thank you for using Star*Explorer!

PS
I wish you knew about this setting, it would probably take you less than that :-)

Thanks I was not aware I could change the value myself. Well when I get thru editing my other 1,000 pics for the other wedding I shot, I will be sure to let you know how long the upload took.

[Nikolai]

Originally Posted by ExposeTheMoment

Thanks I was not aware I could change the value myself. Well when I get thru editing my other 1,000 pics for the other wedding I shot, I will be sure to let you know how long the upload took.

Glad you found it!

[rainforest1155]

Originally Posted by Nikolai

[UPLOAD LOG]This feature is provided for all S*E editions, from Trial to Standard to Pro.

I hope you will enjoy the new version!

As always, let me know if you have any questions or concerns.

Cheers!

Sounds great! Thanks for the implementation! I'll check it out soon.

Cheers,
Sebastian

[Sergey Shevchenko]

Hi everybody! I'll try posting here first, although my question is also related to Smugmug, not just SE.

My problem consists of three parts:

1) When I'm uploading an image (including with Star*Explorer) with the IPTC caption already set, it gets copied to the Smugmug caption. I would expect this to work the other way, too: when I edit the Smugmug caption and then download the image (at least the original!), I get the edited caption back into the IPTC. But that's not the case, regardless of the downloading method (directly or with Star*Explorer). Is it by intention? Isn't it a design flaw then?

2) When I add files to Star*Explorer upload list, the 'Caption' field remains empty even if the IPTC caption is set. In the 'Edit caption' dialog, there's three tabs - 'Caption', 'IPTC' and 'EXIF' - the last two are filled in correctly. I now understand that Smugmug treats its own and IPTC's captions as separate entities, but since Smugmug copies IPTC when uploading, I would expect Star*Explorer to behave in the same vein, or at least provide some easy way to auto-fill 'Caption' from 'IPTC'. I haven't found such a way so far - only manually copying one tab to the other.

3) Can somebody explain to me what 'Get Images Info' feature in Star*Explorer does? I was hoping that I would be able to point the tool to my local storage, then link it to a Smugmug gallery, and then have it download the captions into my local images' IPTC or EXIF or whatever. That's was actually the main reason I bought SE, and bought the Power edition. However, I can't even figure out what kind of information this feature downloads, let alone where. Even if I pre-download the images using SE, then modify some captions online at Smugmug, then do 'Get Images Info' - nothing changes on my disk.

My workflow is usually like this: 1) Do some shooting 2) Quickly post-process if necessary 3) Upload to Smugmug for my relatives and friends to see 4) While they are looking, edit the captions online 5) Download the edited captions back to the originals on disk. So actually I would be more than happy if somebody just told me some easy automated way to perform the step 5 here.

Thanks!

[Nikolai]

Originally Posted by Sergey Shevchenko

Hi everybody! I'll try posting here first, although my question is also related to Smugmug, not just SE.

Welcome to DGrin!

Originally Posted by Sergey Shevchenko

My problem consists of three parts:
1) When I'm uploading an image (including with Star*Explorer) with the IPTC caption already set, it gets copied to the Smugmug caption. I would expect this to work the other way, too: when I edit the Smugmug caption and then download the image (at least the original!), I get the edited caption back into the IPTC. But that's not the case, regardless of the downloading method (directly or with Star*Explorer). Is it by intention? Isn't it a design flaw then?

SM always keeps your original intact. Modifying IPTC is a violation of that primary rule. I'm not even speaking of the fact that it's a highly non-trivial process (considering the fact that both SM and S*E should deal with all possible camera and scanner manufacturers, as well as all possible photo editing/managing software). So I guess the this is "an intention by design". I would not personally call it a flaw.

Originally Posted by Sergey Shevchenko

2) When I add files to Star*Explorer upload list, the 'Caption' field remains empty even if the IPTC caption is set. In the 'Edit caption' dialog, there's three tabs - 'Caption', 'IPTC' and 'EXIF' - the last two are filled in correctly. I now understand that Smugmug treats its own and IPTC's captions as separate entities, but since Smugmug copies IPTC when uploading, I would expect Star*Explorer to behave in the same vein, or at least provide some easy way to auto-fill 'Caption' from 'IPTC'. I haven't found such a way so far - only manually copying one tab to the other.

Since SM takes care of the IPTC-based info, there is no need to auto-fill anything. Just leave Captions empty, and SM will use IPTC ones. "Auto-fill" means more work for everybody: for you to deal with an extra UI element, for me to program it (as I mentioned, dealing with IPTC is not as trivial thing as it may seem), for your Internet connection to carry duplicate info both in IPTC and in Caption...

Originally Posted by Sergey Shevchenko

3) Can somebody explain to me what 'Get Images Info' feature in Star*Explorer does? I was hoping that I would be able to point the tool to my local storage, then link it to a Smugmug gallery, and then have it download the captions into my local images' IPTC or EXIF or whatever. That's was actually the main reason I bought SE, and bought the Power edition. However, I can't even figure out what kind of information this feature downloads, let alone where. Even if I pre-download the images using SE, then modify some captions online at Smugmug, then do 'Get Images Info' - nothing changes on my disk.

"Somebody: sure can:-).
"Get Images Info" delivers SM-specific info to your local MDB database, so you can use MS Access or some other ADO/ODBC tool (e.g. Crystal Reports) to run a nice sales/whatever report on your data.
The data obtained includes file name, caption, keywords, position, album, etc. Complete description can be found here.
The same info is also obtained along with actual image download, so if you do download, the info gets there along the way.

Originally Posted by Sergey Shevchenko

My workflow is usually like this: 1) Do some shooting 2) Quickly post-process if necessary 3) Upload to Smugmug for my relatives and friends to see 4) While they are looking, edit the captions online 5) Download the edited captions back to the originals on disk. So actually I would be more than happy if somebody just told me some easy automated way to perform the step 5 here.

Thanks!

Well, since all the data is now local, it is possible to enumerate the files, locate them on your HDD, find the new captions and try to put them back to IPTC. However, as I mentioned before, it not an extremely trivial process. I do not currently have a ready-to-go solution.
My only suggestion - set captions in your IPTC prior to download. Yes, you will upload a few minutes later, but at least you will have your data in sync.

HTH

Thank you for using Star*Explorer!

[Sergey Shevchenko]

Hi Nikolai,

Thanks a lot for the detailed explanation and for the great tool!

Originally Posted by Nikolai

SM always keeps your original intact. Modifying IPTC is a violation of that primary rule.

Agreed. I just didn't think of this rule, and it's a good rule indeed. But it still would be nice to have at least some button for copying the Smugmug captions back to IPTC. Perhaps it would issue ten warnings before allowing you to do that, but it's better than not having anything.

On the other hand, I realize that I'm probably asking too much from an already excellent site (I didn't choose it for nothing, right? ;-).

Originally Posted by Nikolai

I'm not even speaking of the fact that it's a highly non-trivial process (considering the fact that both SM and S*E should deal with all possible camera and scanner manufacturers, as well as all possible photo editing/managing software).

But isn't Smugmug already successfully dealing with this, since it can *read* IPTC captions from whatever you throw at it? I also thought that IPTC format was very rigid, unlike EXIF that has a free section where the camera/software maker can put their proprietary information.

Originally Posted by Nikolai

Since SM takes care of the IPTC-based info, there is no need to auto-fill anything. Just leave Captions empty, and SM will use IPTC ones. "Auto-fill" means more work for everybody: for you to deal with an extra UI element, for me to program it (as I mentioned, dealing with IPTC is not as trivial thing as it may seem), for your Internet connection to carry duplicate info both in IPTC and in Caption...

Yes, I'm more or less ok with it once I got to know exactly how it works: 'Caption' field takes priority over IPTC if explicitly entered. It's just that there's some element of surprise: you have your IPTC's in the images; upload them to Smugmug and you see them copied to 'Caption'; open them for uploading in SE and the 'Caption' field remains empty, although you'd think it is virtually mapped to Smumug's 'Caption'. Not a big deal though.

Originally Posted by Nikolai

"Get Images Info" delivers SM-specific info to your local MDB database...

Just one question: how do I get a hold of that database? SE doesn't ask for a file name and I don't see any new files in the sync tree.

Originally Posted by Nikolai

Well, since all the data is now local, it is possible to enumerate the files, locate them on your HDD, find the new captions and try to put them back to IPTC. However, as I mentioned before, it not an extremely trivial process. I do not currently have a ready-to-go solution.
My only suggestion - set captions in your IPTC prior to download. Yes, you will upload a few minutes later, but at least you will have your data in sync.

Yeah, I sort of did this manually, simply copying the captions from online to my images in ACDSee. Perhaps there's a way in ACDSee to somehow automate this using the mentioned MDB database. I'll take a look. Does anybody know a tool that can do that for sure?

Regarding pre-filling IPTC before uploading - yes, it's the best I can do as it seems. I tend to want to edit those captions online after uploading, or sometimes revise them on disk (because in my case they usually are like short stories), so they will get out-of-sync from time to time I'm sure. But that the lesser of the two evils.

Thanks again!

[Sergey Shevchenko]

Nikolai,

I have one more problem: both Smugmug.com and Star*Explorer have an inconsistent support for local alphabets, particularly cyrillic. I'll describe SE's part here, Smugmug's part will go to another thread: when obtain the list of my albums from the server, any cyrillic characters in their names get converted into something like unicode sequencies, I think. Something like "&# 1073;&# 43;&# 1085;..." without the blanks (actually, DGrin automatically converted the codes to proper cyrillic the first time I entered this message and didn't insert extra blanks!). Naturally, this coded name also appears everywhere else - in the right-click menu, on disk when doing 'Update sync tree' etc.

I have some suspicion that this problem will come down to Smugmug API eventually. If that's not so, are there any plans to fully support local alphabets in SE?

Thanks!

[Nikolai]

Originally Posted by Sergey Shevchenko

Hi Nikolai,
Thanks a lot for the detailed explanation and for the great tool!

You're more than welcome. That's what I'm here for
And if Smugmug's techsupport is the best on the planet, I sure do hope to be a close second

[quote=Sergey Shevchenko]
Agreed. I just didn't think of this rule, and it's a good rule indeed. But it still would be nice to have at least some button for copying the Smugmug captions back to IPTC. Perhaps it would issue ten warnings before allowing you to do that, but it's better than not having anything.
[quote]
I actually have this feature on my list, but it's a low priority item due a an extremely low demand (out of 300+ registered S*E users you're the second person to ask, the first one was me :-) and certain technical difficulties (such as it's not always obvious what field to use).
However, if I get some extra time on my hands I may come back to it...

Originally Posted by Sergey Shevchenko

On the other hand, I realize that I'm probably asking too much from an already excellent site (I didn't choose it for nothing, right? ;-).

It's a free market - never hurts to ask

Originally Posted by Sergey Shevchenko

But isn't Smugmug already successfully dealing with this, since it can *read* IPTC captions from whatever you throw at it? I also thought that IPTC format was very rigid, unlike EXIF that has a free section where the camera/software maker can put their proprietary information.

Yes and no. There are certain ambiguities. I'm not saying that it's a rocket science, but.. see my comment above about low priority

Originally Posted by Sergey Shevchenko

Just one question: how do I get a hold of that database? SE doesn't ask for a file name and I don't see any new files in the sync tree.

Look in C:\Documents and Settings\%your user name here%\Application Data\StarExplorer\Data\Cache. And since Application Data is a system folder, standard search usually does not go "below" it, hence you could not find it.

Originally Posted by Sergey Shevchenko

Yeah, I sort of did this manually, simply copying the captions from online to my images in ACDSee. Perhaps there's a way in ACDSee to somehow automate this using the mentioned MDB database. I'll take a look. Does anybody know a tool that can do that for sure?

I tried to work with ACDsee, especially since I was a devoted user myself, until I got enough horsepower to switch to Adobe Bridge.
Unfortunately, they change stuff from version to version, and all recent version use some proprietary database. I was trying to get ahold of their SDK several times during the last 18 months (on their website they say it's open and free for everybody), but didn't get a single reply. Probably worst customer service on the web..

Originally Posted by Sergey Shevchenko

Regarding pre-filling IPTC before uploading - yes, it's the best I can do as it seems. I tend to want to edit those captions online after uploading, or sometimes revise them on disk (because in my case they usually are like short stories), so they will get out-of-sync from time to time I'm sure. But that the lesser of the two evils.

Yes, I can understand this. Unfortunately, no easy solution at the moment.

In general, I honestly don't think it's a good idea trying to manage one's data in two (or more) completely different places and then try to sync them reciprocally.. One can get enough issues even with one-way sync, and two-way is definitely asking for troubles due to a high collision probability... Come think of it, only few most powerful database engines have replication capabilities - and that until you try to replicate cross brand (e.g. from MS SQL to Oracle and back)

Originally Posted by Sergey Shevchenko

Thanks again!

My pleasure, thank you for using Star*Explorer!

Cheers!

[Nikolai]

Originally Posted by Sergey Shevchenko

Nikolai,

I have one more problem: both Smugmug.com and Star*Explorer have an inconsistent support for local alphabets, particularly cyrillic. I'll describe SE's part here, Smugmug's part will go to another thread: when obtain the list of my albums from the server, any cyrillic characters in their names get converted into something like unicode sequencies, I think. Something like "&# 1073;&# 43;&# 1085;..." without the blanks (actually, DGrin automatically converted the codes to proper cyrillic the first time I entered this message and didn't insert extra blanks!). Naturally, this coded name also appears everywhere else - in the right-click menu, on disk when doing 'Update sync tree' etc.

I have some suspicion that this problem will come down to Smugmug API eventually. If that's not so, are there any plans to fully support local alphabets in SE?

Thanks!

Hmm, interesting.. Funny, I guess I don't have a single album in cyrillic
My bad... OK, consider it a formal request...

HTH

[Sergey Shevchenko]

Originally Posted by Nikolai

Hmm, interesting.. Funny, I guess I don't have a single album in cyrillic
My bad... OK, consider it a formal request...

HTH

Yeah, I probably wouldn't have them either (I never intentionally use cyrillic in file names), but some of my audience don't speak English, so I had to...

[Nikolai]

Originally Posted by Sergey Shevchenko

Yeah, I probably wouldn't have them either (I never intentionally use cyrillic in file names), but some of my audience don't speak English, so I had to...

I will check this thing out.
BTW, what's your default (system) locale?
Mine actually *is* 1251, so maybe that's the reason I don't see a problem (apart from that I probably never tried;-)?