APIKeys & You!

[onethumb]

So, it turns out that people are doing some crazy things with the API. Including a few attempts to basically resell sub-accounts from smugmug and things like that.

We've wanted some nice way of tracking API usage for awhile, and combined with the beginnings of some abuse issues, we thought it was time to start authorizing API usage.

For the most part, we'll probably just authorize you quickly and easily if you're doing non-commercial stuff. If you're planning on making money via your API work, though, it becomes a gray area that we'll have to take on a case-by-case basis.

Regardless, post any questions, comments, complaints, whatever here for us to discuss and we'll see how it goes.

Don

[devbobo]

Don,

I do have a concern but I will email you privately about it.

BTW, great job...the expanded API is looking great http://dgrin.com/images/smilies/thumb.gif

Cheers,

David

So, it turns out that people are doing some crazy things with the API. Including a few attempts to basically resell sub-accounts from smugmug and things like that.

We've wanted some nice way of tracking API usage for awhile, and combined with the beginnings of some abuse issues, we thought it was time to start authorizing API usage.

For the most part, we'll probably just authorize you quickly and easily if you're doing non-commercial stuff. If you're planning on making money via your API work, though, it becomes a gray area that we'll have to take on a case-by-case basis.

Regardless, post any questions, comments, complaints, whatever here for us to discuss and we'll see how it goes.

Don

[onethumb]

Should have mentioned a few more things:

- If you're building more than one app, sign up for more than one Key.

- You can see the status of your current Key applications on your control panel.

Don

[devbobo]

Should have mentioned a few more things:

- If you're building more than one app, sign up for more than one Key.

- You can see the status of your current Key applications on your control panel.

Don
Does this mean for apps like Smugmug Explorer, each end-user must get a key ??

[onethumb]

Does this mean for apps like Smugmug Explorer, each end-user must get a key ??

Nope, it means that if Nik wants to do a 2nd, different application, say "Smugmug Destroyer", he needs two keys.

Each application, service, plugin, whatever needs a key. But not each binary or anything crazy like that.

Don

[devbobo]

Nope, it means that if Nik wants to do a 2nd, different application, say "Smugmug Destroyer", he needs two keys.

Each application, service, plugin, whatever needs a key. But not each binary or anything crazy like that.

Don
ok...what's happens in the situation where you code being distributed is in script form not compiled. anyone can easily get access to an application's APIKey.

David

[onethumb]

ok...what's happens in the situation where you code being distributed is in script form not compiled. anyone can easily get access to an application's APIKey.

David

Hmm, good question. Code obfuscation tools might help, but obviously they're not impervious.

Anyone else wanna chime in here with any suggestions as to how we deal with this sort of situation?

Worst case, we shut that key down and give you a new one in case of abuse. People running your script will need to get the new copy. Temporary fix, and a pain in the butt, but I don't know what else to suggest.

APIKeys aren't designed to be Fort Knox or anything, just enough of a deterrent that we can track and squash abuse. I think it'll make the abusers, and especially profiteers, leery if their app can stop working at any time.

Don

[rutt]

Ouch! This is potentially really bad for things like smugmug.py. Here's an alternative approach. Why not register and validate smugmug users to use any API tool instead of the tools? A powerful enough tool would facilitate all kinds of abuse in the wrong hands, but might be just the nuts for the majority of users. So:

Users need to register in order to use any API tool.
An API exists to facilitate this registration.
Turn off users who abuse.


Maybe this dosen't exactly address the issue? Maybe you are looking for a bigger flyswatter?

Here is another idea. Suppose you keep a set of <user,tool> pairs. When the tool logs in it tells you it's name, which has to be registered. Do you have a way to datamine for the kind of misuse you have in mind? Anyway if you knew the name of the tool I suppose you'd have some additional info.

Can we be more concrete about exactly what you are trying to prevent? Maybe we need to take this to an email discussion for security reasons?

[flyingpylon]

I understand the need for this. I think that right now you've got a model that is similar to Amazon's. I know they require an ID (developer token) that identifies the developer of the application so that they can track usage. Or at least that's the way it was in AWS 3.0, which is now ECS or whatever they're calling it today. My point is that the developer tokens were exposed to the public in scripts, etc. and I don't recall there being a big problem with it. But depending on how Smugmug intends to use these IDs, I can see how there could be issues.

[Cameron]

Well, I think that there's NO way they could make it "bulletproof". There are always going to be those that will abuse or attempt to circumvent the system. However, if they can cut out 90% of the API usage that they don't approve of, the other 10% will be easier to sort through I imagine. I'm new to smugmug and have only brushed the surface of the API but I think it's wonderful. Keep up the great work!

[onethumb]

Well, I think that there's NO way they could make it "bulletproof". There are always going to be those that will abuse or attempt to circumvent the system. However, if they can cut out 90% of the API usage that they don't approve of, the other 10% will be easier to sort through I imagine. I'm new to smugmug and have only brushed the surface of the API but I think it's wonderful. Keep up the great work!

This is exactly the intent. :)

Thanks for the kind words.

Don

[rutt]

OK, got my key. Can you tell us what the drop dead date is for starting to use it?

One possibility for scripts: a fairly large set of valid keys. The script writer could be responsible for use a different one for each download. Then you can disable a specific one of the keys and not effect the majority of users, only those who have inheritied the script from a wrongdoer.

[Nikolai]

Nope, it means that if Nik wants to do a 2nd, different application, say "Smugmug Destroyer", he needs two keys.

Donthanks guys, I'm onto it:rofl

On a business side, having just one key is never gonna be a fool proof. Even in my own case, while it would be pretty tought to get the key from my executable (it's pretty heavily protected), it's very easy to set up a local web server, relay all the smugmug requests to it - and recieve unencrypted codes..

So unless Don is gonna use some some sort of asymmetrical encryption, this hole thing can be only used for a statistical purposes, and I would not bet my life on the results.

With public/private key, however, it becomes much more secure - while not necessarily much tougher to implement.
Yes, it does not help much script guys, since their IDs will be wide open, and any additional dll/so/exe can also be hacked.. But hey, at least a step in a right direction.

I say - let's do something and see how it goes.

[onethumb]

OK, got my key. Can you tell us what the drop dead date is for starting to use it?

For anything "new" (REST, stats calls, category calls, subcategory calls, image and album deletion), that drop-dead date is today. I'll probably even make the new method names require it later today, too.

For anything "old" (all the old methods via XML-RPC) I don't have a planned horizon. Possibly never, but certainly a long time down the road.

Don

[Nikolai]

Don, one more question:
can session IDs be shared/mixed for xmpl-prc call and REST calls?

Scenario:
I log in via REST (so my api key is confirmed)
* do smth via REST
* do smth via RPC
* logout via RPC
* login with hash via RPC

would this work? or it's unknown and I need to try..

By biggest thing here: as you know, I already have a whole bunch of code kinda field-tested to work with RPC.

I would love to add new REST-based features, and I don't mind to replace few login/logout methods, but if sessions are incompatible, this means I'd have to redo the whole app before I can use any of the new stuff...

waddayasay?

[devbobo]

With public/private key, however, it becomes much more secure - while not necessarily much tougher to implement.

Nik, but you end up in the same situation...but now you have two keys to manage and you can't keep either private.

David

[Nikolai]

Nik, but you end up in the same situation...but now you have two keys to manage and you can't keep either private.

DavidThe point of that ecryption is to avoid sending AppKey into the the open.

Here's how I thought of it:
* SM publishes public key(s), keeps private key(s)
* I'm using (a) public key to encrypt all the requests
* SM decrypts them using pertinent private key
* since my appkey is only known to SM and me (and, trust me, a hacker would have a pretty hard time to extract that appid from my binary file), they'd know that app is claiming to be me *is* me.

I'm not saying this is unbreakable - it is.
However, it would proably take a few weeks to break it. Once break is noticed, it would take SM a few minutes to change the keys, and maybe an hour for me to change the internal protection.

If someone wants to go through that hassle only to get a hold of $15..$50 worth of software intended to upload pictures to the account you'd have to pay for anyway - he can be my guest...:-)

Cheers!:1drink

[devbobo]

The point of that ecryption is to avoid sending AppKey into the the open.

Here's how I thought of it:
* SM publishes public key(s), keeps private key(s)
* I'm using (a) public key to encrypt all the requests
* SM decrypts them using pertinent private key
* since my appkey is only known to SM and me (and, trust me, a hacker would have a pretty hard time to extract that appid from my binary file), they'd know that app is claiming to be me *is* me.

I'm not saying this is unbreakable - it is.
However, it would proably take a few weeks to break it. Once break is noticed, it would take SM a few minutes to change the keys, and maybe an hour for me to change the internal protection.

If someone wants to go through that hassle only to get a hold of $15..$50 worth of software intended to upload pictures to the account you'd have to pay for anyway - he can be my guest...:-)

Cheers!:1drink Nik,

I understand encryption...i am currently doing a Masters in Information Security.

All I meant, is for the guys doing scripting...whether it be symmetric or asymmetric encryption makes no different as its difficult to obfascate the keys. In this case (IMHO), encryption is a wasted overhead.

Cheers,

David

[rutt]

So, was my idea of branding the scripts at download time with different keys a dumb idea? That would allow SM the flexibility to disable one or many of the keys and not turn off legitamate users who never shared their keys with bad abusers. It seems simple enough.

[Nikolai]

I agree.. As I mentioned before, this additional asymmetric keys approach would only work for (protected/encrypted) binaries only.

I believe the idea behind this is simply to put a gate to API usage.
However, with the scripting languages all those keys would be lying around wide open, so nothing would prevent a "bad guy" to simpley download yours or rutt's script, get the key and they use it at his discretion.

I don't know how to protect those apikeys without introducing at least some sort of local encryption.
I can come out with windows dll/activex/... (i.e. heavily secured/encypted/protected binary) which would carry such an apikey for you, but then you'd need to distibute your own license key associated with particular user account...

We're talking of pretty big can of worms here.
Yet again, only for scripted solutions I don't see any easy way, properly protected binaries do not have this issue..

[onethumb]

Don, one more question:
can session IDs be shared/mixed for xmpl-prc call and REST calls?

Scenario:
I log in via REST (so my api key is confirmed)
* do smth via REST
* do smth via RPC
* logout via RPC
* login with hash via RPC

would this work? or it's unknown and I need to try..

By biggest thing here: as you know, I already have a whole bunch of code kinda field-tested to work with RPC.

I would love to add new REST-based features, and I don't mind to replace few login/logout methods, but if sessions are incompatible, this means I'd have to redo the whole app before I can use any of the new stuff...

waddayasay?

Should work fine. Try it.

Note that there aren't any "REST-based features" ... everything REST can do, XML-RPC can do as well.

Don

[rutt]

Is the different-key-per-download idea for scripts just too stupid to deserve a comment? If so, please explain why to me. I'm not getting it.

[onethumb]

So, was my idea of branding the scripts at download time with different keys a dumb idea? That would allow SM the flexibility to disable one or many of the keys and not turn off legitamate users who never shared their keys with bad abusers. It seems simple enough.

I wouldn't say it's dumb, but it's just more work than we want to do.

The APIKeys are purely a minor deterrent, and they have other uses than preventing abuse.

If it keeps 90% of people from gaming the system, I'll consider it a runaway success.

Don

[rutt]

Please can you explain what the bad things are that people are doing or that you expect that they might do. Maybe this isn't a good thing to do in public, but I like my script and being able to script the smugmug api and I really don't want that to stop working, so I'm trying to help make keep it from doing stuff you don't like. But I don't really know what I'm trying to avoid.

Should I just not worry my little head about this? Or can/should I help?

[Nikolai]

Should work fine. Try it.
Don
I will!


Note that there aren't any "REST-based features" ... everything REST can do, XML-RPC can do as well.
DonThat's even better..:clap
It means I don't have to change my existing stuff! Shweeeet!:thumb
Cheers!:1drink

[Nikolai]

I modified SE code to get the new settings a whurl.
Most of the "old code" works - with one huge exception:
smugmug.subcatgories.getAll (aka getAllSubcategories) returns result which is stucturakky different from the previos version!!

The good news - existing version (the one that uses version 1.0 and no API Key) works.

The bad news - new one does not..:-(

Here's the log from the SE version 106 (latest deployed):


Post to http://api.smugmug.com/xmlrpc/

<?xml version="1.0"?>
<methodCall>
<methodName>getAllSubCategories</methodName>
<params>
<param><value>... session id ...</value></param>
</params>
</methodCall>
<?xml version="1.0" encoding="iso-8859-1"?>

<methodResponse>
<params>
<param>
<value>
<array>
<data>
<value>
<struct>
<member>
<name>CategoryID</name>
<value>
<int>0</int>
</value>
</member>
<member>
<name>SubCategoryID</name>
<value>
<int>53110</int>
</value>
</member>
<member>
<name>Title</name>
<value>
<string>Empty</string>
</value>
</member>
</struct>
</value>


and here's the identical log from the updated version:

Post to http://api.smugmug.com/hack/xmlrpc/

<?xml version="1.0"?>
<methodCall>
<methodName>smugmug.subcategories.getAll</methodName>
<params>
<param><value>.. another, longer session id ....</value></param>
</params>
</methodCall>

<?xml version="1.0" encoding="iso-8859-1"?>
<methodResponse>
<params>
<param>
<value>
<struct>
<member>
<name>SubCategory0</name>
<value>
<struct>
<member>
<name>CategoryID</name>
<value>
<int>0</int>
</value>
</member>
<member>
<name>SubCategoryID</name>
<value>
<int>53110</int>
</value>
</member>
<member>
<name>Title</name>
<value>
<string>Empty</string>
</value>
</member>
</struct>
</value>


As it is easy to see there are two differences
1) session IDs have different length - not that I care, but it may be a hint
2) "new code" has additional level of nesting (see Subcategory0) - of course, existing parsing code "chokes" on it..

Don, is this a new behaviour or just a bug?

Other new rpcxml methods that I tested and found working:
---------------------
login with password
login with hash
logout
get albums
create album
upload via post

[Nikolai]

I modified SE code to get the new settings a whurl.
Most of the "old code" works - with one huge exception:
smugmug.subcatgories.getAll (aka getAllSubcategories) returns result which is stucturally different from the previous version!!

setting version to 1.0 and not providing API key makes everything work as before even with new URLs and new method names

HTH

[onethumb]

I modified SE code to get the new settings a whurl.
Most of the "old code" works - with one huge exception:
smugmug.subcatgories.getAll (aka getAllSubcategories) returns result which is stucturakky different from the previos version!!


Try now.

Don

[Nikolai]

Try now.

Don
Thank you!!!:clap It works now!

[rutt]

I guess I'm not really understanding the API key issues, but I want to be as helpful as possible. Don didn't yet tell me not to worry about it so I didn't stop.

What if I stop allowing any downloads of sm_tool.py but require people to email or PM me personally to get copies and I keep a list of everyone who does so? In the case of sm_tool.py, the user commnuity is probably so small that this is easily feasible (and if not, well that's interesting information for me). I think just requiring people to contact me to get the script would discourage a huge percentage of potential misuse.

Don, do you care?

[rutt]

Don, do you care?

I take it as a "no".

[onethumb]

I guess I'm not really understanding the API key issues, but I want to be as helpful as possible. Don didn't yet tell me not to worry about it so I didn't stop.

What if I stop allowing any downloads of sm_tool.py but require people to email or PM me personally to get copies and I keep a list of everyone who does so? In the case of sm_tool.py, the user commnuity is probably so small that this is easily feasible (and if not, well that's interesting information for me). I think just requiring people to contact me to get the script would discourage a huge percentage of potential misuse.

Don, do you care?

I'm intentionally trying not to be too specific here about what I'm avoiding by using the API keys, but it's your script - you can do whatever you want with it.

I don't have a problem with it being freely passed around, in fact, that's what I'd prefer, but it's up to you.

Don

[rutt]

I'm intentionally trying not to be too specific here about what I'm avoiding by using the API keys, but it's your script - you can do whatever you want with it.

I don't have a problem with it being freely passed around, in fact, that's what I'd prefer, but it's up to you.

Don

OK, I'm going to stop thinking about this now. If you ever want me to start again, just whistle.

[rutt]

Is my API key private? Should I try to keep it a secrect? I suppose I might have been missing something, because I always assumed I'd just include it when I released a version of the python script that used an API version that required it (that hasn't happened yet.) Now I realize that the right model might be to require each user to get his/her own key and install it in the script.

Don, what do you want me to do?

[onethumb]

Is my API key private? Should I try to keep it a secrect? I suppose I might have been missing something, because I always assumed I'd just include it when I released a version of the python script that used an API version that required it (that hasn't happened yet.) Now I realize that the right model might be to require each user to get his/her own key and install it in the script.

Don, what do you want me to do?

Include it in your script.

Don

[snadboy]

I apologize if this question has been addressed previously, but I couldn't find an answer. How long does it typically take to have an API Key request approved? A couple of days? longer? And yes, I have checked the status in Control Panel...

Thanks,
snadboy

[dleewo]

The info on smugmug.com suggests that the API is really for non-commercial use, but I know there are commercial apps that currently use it.

I'm thinking of creating a cross-platform app (Windows, Mac, Linux) for uploading images to smugmug. Probably not as sophisticated as StarExplorer, but certainly more so than the Mac uploader I had tried.

The app would be primarily for use with smugmug, but again, your website states "where smugmug interaction is the primary selling point, are generally discouraged."

I would want to charge for this application. Would that be permitted? Do I need special approval?

Derek

[mpmcleod]

How do I remove a key that I don't need?

the first time I applied I saw name and blindly typed in my name instead of the app name.

Once I noticed what I did I submitted another application with the app name.

But I now have 2 keys and I only need one.

thanks,
Mike

[dac1117]

Hi Don,

I'm new to the Hacks forum and had a basic question before I apply for API authorizing usage.

Can I make SmugMug API calls from within VBScript? What would be the syntax for referencing the SmugMug application object? (I usually use the 'Set' command to reference objects (i.e. 'Set commenceDB = Application.Database').

thx -dave conrad

[iamback]

Include [the API key] in your script.

That makes (some) sense to me - if the script is an "application".

Now, what if it's just a library? I'm thinking of the following situation: for my site (travel blog), I put together a library, or a small class, in PHP to provide some functionality for embedding selected images from SmugMug. In order to run it, I'd need an API key - but the SmugMug class/library is just a small part of my travel blog application.

Now, when my class/library is reasonably complete and works well, I'd want to distribute it - even if I don't (yet) distribute my whole travel blog application - and I'd want to distribute it (of course) as open source, LPGL or similar.

The only logical thing to me seems to distribute it with an installable API key (and not with mine), and anyone using it in their own site would have to get their own API key; likewise anyone using it as part of another application (distributed, or provided as a service) would need or maybe already have(!) their own API key.

Does this make sense?

[devbobo]

That makes (some) sense to me - if the script is an "application".

Now, what if it's just a library? I'm thinking of the following situation: for my site (travel blog), I put together a library, or a small class, in PHP to provide some functionality for embedding selected images from SmugMug. In order to run it, I'd need an API key - but the SmugMug class/library is just a small part of my travel blog application.

Now, when my class/library is reasonably complete and works well, I'd want to distribute it - even if I don't (yet) distribute my whole travel blog application - and I'd want to distribute it (of course) as open source, LPGL or similar.

The only logical thing to me seems to distribute it with an installable API key (and not with mine), and anyone using it in their own site would have to get their own API key; likewise anyone using it as part of another application (distributed, or provided as a service) would need or maybe already have(!) their own API key.

Does this make sense?

yeah that makes sense, just have an api key parameter/variable that a user can assign their api key to :thumb

[rujero]

Hi Smugmug people!

I am working on an online photo magazine written by the people for the people that I have named YouTellYou (check out a prototype at http://youtellyou.com).

Users can create photo stories by selecting photos from:
- their computer
- their Smugmug account
- their flickr account
- their Facebook account
- a mixture of those
putting them in a sequence and providing captions, a title, a summary and a few other bits of information.

I am using phpSmug for the Smugmug API and I would like you guys to tell me whether I am not violating any Smugmug rule that I am not aware of.:bow

I am only using the Smugmug name to tell users that they can link to their Smugmug account and I am using a small Smugmug logo to link back to Smugmug (see this story page: http://youtellyou.com/storyRead.php?&idStory=248)

Basically what I do is I allow people to browse their photos on Smugmug and select the photos they want to place in the story. Then I store the Thumb, Medium and Large URLs and the caption of the photos in a database on YouTellYou. I also store the URL of the Smugmug page to allow readers to link back to Smugmug.

What you find at youtellyou.com is a functional prototype (nearly feature complete for the beta release). The UI is still provisional, I am working with a graphic designer on it so be forgiving for the time being. I have done everything myself and I am not really a very experienced developer.:dunno

I would very much appreciate your comments on my initiative (and your blessing, if I haven't done anything wrong).
The website is free to use. I am thinking about a possible business model for it may be involving putting stories together in a book to print but it is still very fuzzy.

BTW If Markham reads this, this is what I was trying to explain to you early this year, of course at the time it was all very much in my head and I think I did a poor job of explaining myself. I am totally committed to YouTellYou and I think it has great potential. I am very much keen on discussing any suggestion or proposal (can't stop dreaming...) you might have.

Cheers, Ruggero Domenichini (rujero)

[rujero]

YouTellYou version 1.0 is now live!

On YouTellYou you can create photo stories in minutes, linking from your Smugmug photos (or flickr or Facebook) or uploading from your computer.
Stories are published on the YouTellYou magazine and you can share yours with your friends (email, twitter, post URL) and with the world.
It's easy, it's fun, it's free! Give it a try!
Please use it. Create stories. Try to break it. Give me feedback. Ask for features.

Ruggero Domenichini (Auckland, NZ)

http://youtellyou.com (http://youtellyou.com/)