The recent decision by Twitter to turn off support for Basic Auth soon means a lot of Twitter apps are now racing to implement either full OAuth support, or the cut down xAuth designed for non-web apps. The iNewz apps fall into this last category, and an initial look at the work involved made it seem as though switching from basic auth to xAuth would be pretty straightforward. Sadly, and mostly because of poor documentation and what I consider bugs in the Twitter API implementation of OAuth, this took far longer than it should have done. Hopefully this blog post will help others looking to make this switch by providing a more complete, step-by-step description of the xAuth process. It may also help those trying to make full OAuth work, but I haven’t tried that yet.

Signing Up
First step in converting is to sign in to Twitter’s new developer site and register your app as an OAuth application. Once you have done that you need to send an email to api@twitter.com requesting permission to xAuth. Include in that email the client ID (the number at the end of the http://dev.twitter.com/apps/ URL once it’s registered as an OAuth app), and a link to your application’s website.

Until Twitter has approved your application for xAuth, you will not be able to use the xAuth protocol.

Once you register for OAuth though you will immediately have a consumer key and secret for your application. Make a note of these – you will need them for all the steps below!

xAuth Background
A useful starting point here is to get an understanding of how xAuth works. Some of this applies to OAuth as well, since after the initial steps both are pretty much the same thing.

There are two stages to xAuth/OAuth that a developer needs to think about:

1. Authorisation – the initial request to get permission to access the user’s Twitter account. Your application’s registration with Twitter will determine whether or not this is read-only or read/write access.
2. Identification – all subsequent requests using the permission must include something that proves to Twitter that you have permission.

In the basic auth model, the two were basically one step, achieved by sending the user’s username & password in each request. Now they’re two steps.

The authorisation step returns a token and a matching secret that you need to use for the second step. But unlike basic auth, you don’t pass them both back on each request, but instead use them to generate a signature. Generating that signature is something that the Twitter docs don’t help much with, so we’ll look at it in more detail later (to add to the fun, even the authorisation step requires a signature, but one that is generated slightly differently).

Authorisation
Unlike OAuth, where the authorisation process is a multi-step affair that involves sending the user to a Twitter web page, xAuth is a single request. That request is to the OAuth API endpoint that would normally be the last one in the OAuth sequence, but it includes some extra parameters.

The endpoint is https://api.twitter.com/oauth/access_token. To request your access token and secret on behalf of a user, you will need to POST a request to that URL with the following parameters:

• x_auth_username – the user’s Twitter username
• x_auth_password – the user’s Twitter password

x_auth_mode – always set to client_auth

Remember to URL encode the username and password values.

But you can’t just post those values; you also need to sign this request and add an HTTP Authorization header to your POST.

Signing The Access Token Request
First step in the signing process is to generate the base string. The base string contains all the required parameters for the request and is used as one of the inputs to the hashing function that will generate the signature.

This is one of the parts of OAuth that I feel is poorly designed, but it is what it is, so we need to jump through the necessary hoops to get the signature. The key to remember here is that Twitter is going to reconstruct this string and use that to verify your signature value; if anything is different at all, then the signatures will not match, and the request will be refused.

There are three parts to the base string, joined by ampersands (&) as follows:

method&url¶meters

For this request, method will be POST in upper case always. The URL is the URL encoded version of the access token endpoint URL:

https%3A%2F%2Fapi.twitter.com%2Foauth%2Faccess_token

The hard part is the parameter values (and this is where I think OAuth shows its first sign of poor design). The parameters must be added to the base string in strict order: alphabetic order of the parameter names for this request (requests where a parameter may be repeated must further sort those in order of the values).

The parameters you will need to get an access token, in the order they must be added to the base string, are:

• oauth_consumer_key – your application’s consumer key.
• oauth_nonce – a nonce (number used once) generated by your application (we’ll look at this more in a minute).
• oauth_signature_method – this is just set to HMAC-SHA1 (the Twitter docs say HMAC_SHA1 in some places, but that is wrong).
• oauth_timestamp – the current time in seconds since Jan 1, 1970 00:00:00 GMT. There are posts online claiming that Twitter violates the OAuth spec here by checking that this is within 5 minutes of current time. That means your app will fail on devices where the time is not set correctly, or not available.
• oauth_version – set this to ‘1.0’ (without the quotes).
• x_auth_mode – set this to ‘client_auth’ (without the quotes).
• x_auth_password – the user’s password.
• x_auth_username – the user’s Twitter username.

All the values should be URL encoded. Straightforward so far. To create a single parameters value though for substitution into our base string’s third component we need to join all that together.

Each parameter is added as key%3Dvalue (where %3D represents the URL encoding for ‘=’). So, using the oauth_signature_method as an example, for that parameter we would generate the string:

oauth_signature_method%3DHMAC-SHA1

Then we must join them all together to form a single parameter list. Do that by separating the individual parameter strings with %26 (the URL encoding for ‘&’). So the start of the list might look like this:

oauth_consumer_key%3DKEY%26oauth_nonce%3DNONCE

Where KEY and NONCE are the URL encoded values for those parameters (too long to include here and make it readable).

The Nonce
The nonce is something that your application should generate randomly ideally. The OAuth specification states:

The Consumer SHALL then generate a Nonce value that is unique for all requests with that timestamp. A nonce is a random string, uniquely generated for each request.

There is a suspicion that Twitter takes a stricter view of these values, but generating a large random number should be sufficient.

The Timestamp
The timestamp should be simple, the number of seconds since January 1, 1970 00:00:00 GMT. All the OAuth specification says about this value is:

The timestamp value MUST be a positive integer and MUST be equal or greater than the timestamp used in previous requests.

There is a belief (though I’ve not seen confirmation or denial from Twitter) that the timestamp must also be within 5 minutes of the current time. As well as being an obvious violation of the specification, it is also very poorly thought out design since it will be a problem for devices where the time is set incorrectly, or not available. Even the requirement in the OAuth specification could be problematic, and is unnecessary.

The Signature
The only algorithm Twitter currently supports is HMAC-SHA1, so this is what we need to use here. The trick (that is not easy to find in Twitter’s docs) is how you encode the ‘secret’ for this process.

Firstly though, here’s an example of the code to generate the signature (in Objective-C for the iPhone, but should be easy to port to other platforms):


NSData *dSecret = [secret dataUsingEncoding:NSUTF8StringEncoding];
NSData *dBase = [base dataUsingEncoding:NSUTF8StringEncoding];
uint8_t result[CC_SHA1_DIGEST_LENGTH];
CCHmacContext hmacCtx;
memset(&hmacCtx, 0, sizeof(hmacCtx));
CCHmacInit(&hmacCtx, kCCHmacAlgSHA1, dSecret.bytes, dSecret.length);
CCHmacUpdate(&hmacCtx, dBase.bytes, dBase.length);
CCHmacFinal(&hmacCtx, result);


The secret is normally created by joining the consumer secret for your application (from the Twitter developer site’s page for your application) and the token secret using an ampersand (&):

consumer_secret&token_secret

In this case though, since we are still requesting the token, we don’t have a token secret. So the secret in this case is the consumer secret with a trailing ampersand:

consumer_secret&

Don’t miss the trailing ampersand or it won’t work!

The Authorization Header
Once you have all this information, you can build the string that will become the ‘Authorization’ HTTP header value. That value is OAuth followed by a sequence of comma separated key value pairs, the values being URL encoded and then wrapped in double quotes:

OAuth oauth_nonce="qfQ4", oauth_signature_method="HMAC-SHA1", ...

Unlike the base string values, the order of these values doesn’t seem to matter. Also, only the oauth_ prefixed values are included here:

• oauth_nonce
• oauth_signature_method
• oauth_timestamp
• oauth_consumer_key
• oauth_signature
• oauth_version

The x_auth prefixed values are sent in the body of the POST.

The POST Response
In response to posting this information you will receive one of two things:

• An (annoyingly) URL encoded list of values, including the oauth_token and oauth_token_secret that you will need to store and use on subsequent requests to the Twitter API;
• An error message (which is unlikely to help you much, and will tell a user even less – OAuth was designed by geeks with no idea about user experience it seems);

In the case of an error, you should also get an HTTP error (400 or 401 most likely), so you can tell the difference between a successful request and an error.

You don’t need to store the Twitter username (unless you want it for some other purpose, such as displaying which account the user authorised the application to use), and must not store the Twitter password. Here’s an example response (with the token and secret masked):

oauth_token=XXXX&oauth_token_secret=YYYY&user_id=14642644&
screen_name=bluedonkey&x_auth_expires=0

That will all be on one line in the response from Twitter.

Updating User Status
Updating the user’s status (i.e. posting a tweet into the user’s timeline) is an authenticated method. Using xAuth (or OAuth, since they’re the same), we must sign the request and include the Authorization HTTP header.

The process is the same, but here are some things to be careful of:

• Once you’ve acquired the access token and corresponding secret, remember to use the combined consumer secret and token secret when generating the signature (see above);
• You will also need to add the oauth_token parameter into the list of values in the HTTP Authorization header value.
• When creating the base string, the status value (i.e. the content of the tweet) needs to be included. In the body of the POST it will be URL encoded. In the base string it needs to be double URL encoded. So, ‘Test Tweet’ becomes ‘Test%20Tweet’ for the body of the POST, and it becomes ‘Test%2520Tweet’ for the base string.

Invalid / Used Nonce Error
Another indication that Twitter’s OAuth implementation is not really ready for production use is the simple fact that almost any problem with the contents of the Authorization header results in the inaccurate ‘invalid / used nonce’ error.

In all the examples I found online, as well as my own experiences, the problem is never related to the nonce value. This error seems to highlight the least likely cause of the problem. Possible real causes of this error include:

• Timestamp values that are more than 5 minutes from the current time (check the clock on your system!);
• Using the old ‘+’ encoding for spaces instead of ‘%20’ in values included in the base string (e.g. tweet message content);
• Incorrect/invalid token value;
• Not double encoding the values of POST body parameters that are being included in the base string.

If you see this misleading error, check everything else in your base string too – it is almost certainly not a problem with your nonce as long as you have done something to make sure they’re random!