If you are wondering how to connect to Google APIs with the Google data Python Client Library from Google App Engine or even python in general, this might help you out with a few of the connections you will need to get it working. These connections are sparsely documented on the Google code site as of today.

I had to get one fo my Google App Engine Web-apps synced up with user G-mail contact information, which led me to the Google Data APIs. According to the docs, the way your app should retrieve Google user-data is through the URI-based APIs. If you are building a web app, you should also be using an authentication scheme which Google calls AuthSub. AuthSub is basically a shared token system where your Web App and the Google Authentication API share simple encoded tokens. These tokens exchange information between your app and Google that proves that your app has user-granted permission to access user-specific data feeds.

Once your web app has access to a client-data feed, the results will be returned as XML data structures. The XML nodes might sometimes require several iterations of API requests to fully flesh out the XML node details of say, a users G-mail contact list (contacts, details, photos, etc...).

Because this parsing, processing and authenticating can be quite a pain to write for everyone, the Google gdata API team has created a suite of client code to serve as modules/libraries in order to abstract the nitty-gritty detail work into higher-level methods.

For Googe App Engine, the API team has even created a special version of the python gdata client library that will bypass the Google App Engine socket firewalls to help your application connect to users precious data. This was exactly what I needed for my GAE web app, and I am sure you will want to use it too. Why reinvent an open-source wheel?

The python gdata client library has several classes available: from the gdata and gdata.auth modules, to the service-specific modules gdata.calendar, gdata.contacts, gdata.docs, etc... You should read the documentation on the gdata client itself, as well as the services. They are all pretty homogeneous from service to service, but some differences do exist.

With the gdata client library downloaded, installed and included my project, I set off to easily hook -up my app with the Google Contacts API.

---

Now might be the right time to explain the basics of the AuthSub authentication process. Your app will need to include an AuthSub security token for each request to a Google Gdata API service so pay attention.

If your Web app is designed to take advantage of users Google-specific data (calendars, contacts, blogger posts...), you will need to provide the hookups to the Google Data APIs in your app.

The first time a user wants to use those features in your app, they will make a request to your application for that data. If a user has no previous history with your app (or has had their previous history/token revoked), your app will not know if this user has authorized Google to let you get your hooks into their data.




Now that your app would like access to the users Google data, it must hyperlink the user to a Google-hosted access agreement page (https://www.google.com /accounts/ AuthSubRequest?....).

The python gdata client library has a method [GenerateAuthSubURL] that will create this hyperlink for you, sort of. You still are required to know and apply a scope to the request (basically the URL form of the data feed you want access to).

Once the agreement form is authorized by the user, Google redirects the user back to your web application with a temporary one-time use token appended to the URL (ie: http://www.jforsythe.com/gmailcontacts.html?token=1234).

The python gdata client library has a method [AuthSubTokenFromUrl] to easily strip the token out of the URL when the user arrives on your Web Application doorstep from Google.

Once the returning page receives this token, your Web app might want to upgrade the one-time use token to an unlimited-use token. This will let you exchange the single-use token with an unlimited-use token with the Google AuthSub api.

The python gdata client library has a method [UpgradeToSessionToken] to upgrade the one-time use token to an unlimited token in-place. All future references to the token will return the unlimited-use token.

If you want to store this unlimited-use token, it is up to your web application to create a datamodel that will hold these tokens. The gdata library will not store these session tokens for future use for you.

Once you have an upgraded unlimited-use session token, you will use this session token in each and every URL API call to Google GData APIs. The Google API endpoints will validate the session to ensure that it is still good on each request. If the session token is invalid or revoked, the remote API calls will return HTTP error codes (401, 403) instead of the typical HTTP 200 (all is fine!) response code. Additionally, as long as the user has data at the endpoint (calendars, contacts, blog posts...), it will be returned along with the successful request.

---

Onto the problem I was having with the Python gdata library...

Using the python gdata client library (which contains all sorts of helper authentication utilities and xml parsing methods), I was having no luck what-so-ever in requesting remote APIs after I had upgraded my one-time use token to a session token. I had attempted to follow all of the documentation and best-practice example Additionally, none of the examples seemed to be working from code found online. All remote requests were met with the error "401 Token Invalid - Invalid AuthSub Token."

Besides the python gdata client library, you can also request the remote Google API feeds using the GAE urlfetch class from the google.appengine.api module. This class provides a method of retrieving remote HTTP endpoints with the method urlfetch.fetch(). This method also provides the ability to apply custom HTTP headers to the HTTP process. I was able to successfully get the remote Google API data to work when I stubbed a session AuthSub token into it.

At this point I could have just migrated my code to work with the urlfetch class and abandoned the gdata library. However, I really wanted to get the gdata library working so I wouldn't have to create yet another custom parsing/processing module in python (something I will never again take for granted in the Java/asp.net work I do).

---

And finally, onto the solution...

* Update, a better solution exists, see my comments on this post.

After a few hours of re-reading all of the documentation in the python gdata client library group, installing a local HTTP sniffer and basically creating a new project to get this working outside of my project dependences, I finally discovered the dumb little problem that was preventing gdata clients from sending along the correct authentication headers.

As it turns out, after you retrieve the one-time use token from the URI and request that it be upgraded to an unlimited session token with UpgradeToSesisonToken(), the method preserves or appends the string 'AuthSub token=' to the start of the newly created Session token, creating a token that read 'AuthSub token= XXXXXXX'.

Example:

newservice = gdata.contacts.service.ContactsService()
newservice.auth_token = self.urltoken 
newservice.UpgradeToSessionToken()
print newservice.auth_token

>>> 'AuthSub token=klj23l2l2adf02_232aB'

As I was using this literal string exactly as it was generated, it turns out that the client.auth_token expects just the token 'XXXXXXX' portion of 'AuthSub token= XXXXXXX'. A quick auth_token.split('=')[1] yielded the string I needed to pass the client.auth_token property in order to get the correct unlimited-use session token into the remote HTTP API requests.

Example:

contactservice = gdata.contacts.service.ContactsService()
contactservice.auth_token = self.authsub_token.split('=')[1] 
feed = contactservice.GetContactsFeed()

---

Hoohah! As I am pretty tired now, I will track down if this is a bug in the morning. The important thing is that I can now access the data I need through the python gdata client library.


technorati: python, Google Application Engine, API, Google, gdata, library, GAE, Google App Engine
More Blog posts:


Created 24 weeks, 3 days ago