Most important: Google Adsense's crawler has been locked out by the new robots.txt

[www.paolosmeraldi.com]

It seems that with the new robots.txt file Smugmug has locked out the Google Adsense crawler.

Here is the new file: as you can see the User agent Mediapartners-Google that is, the one used by Google Adsense, has been left out.

Please allow this crawler as soon as possible as people using Adsense like me (http://photos.paolosmeraldi.com) are now publishing only public service ads (meaning we do not get any money).

Please note, this isn't the googlebot, have a look here, the two are different

https://www.google.com/adsense/support/bin/answer.py?answer=10532

Thanks



# If you're reading this, you belong at a job you love: http://www.smugmug.com/jobs/
# See http://www.smugmug.com/help/emailreal if you'd like to apply to be whitelisted for crawling this site

User-agent: baiduspider
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: Googlebot
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: msnbot
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: naverbot
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: ScoutJet
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: Slurp
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: twiceler
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/

User-agent: Yandex
Disallow: /admin/
Disallow: /cart/
Disallow: /test/
Disallow: /VIP/
Disallow: /vip/
Disallow: /search/


User-agent: *
Disallow: /

[Andy]

We'll check into this - thanks for bringing it up!

[www.paolosmeraldi.com]

Thanks Andy; it seems quite simple to me.

I have chosen Smugmug because it was possible to integrate adsense into it, and now I wish that I can continue using it as it is a very good service!

[Sheaf]

This should be fixed. Sorry!

[www.paolosmeraldi.com]

Yes, I have checked right now.

My website is starting to show ads again.

This happened right at the moment while my visits were increasing!

Well it happens, thanks.

Paolo