Google Services Authentication in App Engine, Part 2

The Google Developers site now has a pretty good description on how to setup OAuth 2.0. However, it still turned out to be a challenge to configure a real-life example of how it’s done, so I figured I’d document what I’ve learned.
Tutorial Scenario In the last tutorial, the project I created illustrated how to access a listing of a user’s Google Docs files. In this tutorial, I changed things up a bit, and instead use YouTube’s API to display a list of a user’s favorite videos. Accessing a user’s favorites does require authentication with OAuth, so this was a good test.

Getting Started (Eclipse project for this tutorial can be found here).

The first thing you must do is follow the steps outlined in Google’s official docs on using OAuth 2.0. Since I’m creating a web-app, you’ll want to follow the section in those docs titled ‘Web Server Applications’. In addition, the steps I talked about previously for setting up a Google App Engine are still relevant, so I’m going to jump right into the code and bypass these setup steps.

(NOTE: The Eclipse project can be found here — I again elected not to use Maven in order to keep things simple for those who don’t have it installed or are knowledgeable in Maven).

The application flow is very simple (assuming a first-time user):

1. When the user accesses the webapp (assuming you are running it locally at http://localhost:8888 using the GAE developer emulator), they must first login to Google using their gmail or Google domain account.
2. Once logged in, the user is redirected to a simple JSP page that has a link to their YouTube favorite videos.
3. When the click on the link, a servlet will initiate the OAuth process to acquire access to their YouTube account. The first part of this process is being redirected to a Google Page that prompts them whether they want to grant the application access.
4. Assuming the user responds affirmative, a list of 10 favorites will be displayed with links.
5. If they click on the link, the video will load.

Here’s the depiction of the first 3 pages flow:

And here’s the last two pages (assuming that the user clicks on a given link):

While this example is specific to YouTube, the same general principles apply for accessing any of the Google-based cloud services, such as Google+, Google Drive, Docs etc. They key enabler for creating such integrations is obviously OAuth, so let’s look at how that process works.

OAuth 2.0 Process Flow

Using OAuth can be a bit overwhelming for the new developer first learning the technology. The main premise behind it is to allow users to selectively identify which ‘private’ resources they want to make accessible to an external application, such as we are developing for this tutorial. By using OAuth, the user can avoid having to share their login credentials with a 3rd party, but instead can simply grant that 3rd party access to some of their information.

To achieve this capability, the user is navigated to the source where their private data resides, in this case, YouTube. They can then either allow or reject the access request. If they allow it, the source of the private data (YouTube) then returns a single-use authorization code to the 3rd party application. Since it’s rather tedious for the user to have to grant access every time access is desired, there is an additional call that can be played that will ‘trade-in’ their single use authorization for a longer term one. The overall flow for the web application we’re developing for this tutorial can be seen below.

OAuth Flow

The first step that takes place is to determine whether the user is already logged into Google using either their gmail or Google Domain account. While not directly tied to the OAuth process, it’s very convenient to enable users to login with their Google account as opposed to requiring them to sign up with your web site. That’s the first callout that is made to Google. Then, once logged in, the application determines whether the user has a local account setup with OAuth permissions granted. If they are logging in for the first time, they won’t. In that case, the OAuth process is initiated.

The first step of that process is to specify to the OAuth provider, in this case Google YouTube, what ‘scope’ of access is being requested. Since Google has a lot of services, they have a lot of scopes. You can determine this most easily using their OAuth 2.0 sandbox.

When you kickoff the OAuth process, you provide them the scope(s) you want access to, along with the OAuth client credentials that Google has provided you (these steps are actually rather generic to any provider that supports OAuth). For our purposes, we’re seeking access to the user’s YouTube account, so the scope provided by Google is: https://gdata.youtube.com/.

If the end-user grants access to the resource identify by the scope, Google will then post back an authorization code to the application. This is captured in a servlet. Since the returned code is only a ‘single-use’ code, it is exchanged for a longer running access token (and related refresh token). That step is represented above by the activity/box titled ‘Access & Refresh Token Requested’.

Once armed with the access token, the application can then access the users’ private data by placing an API call along with the token. If everything checks out, the API will return the results.

It’s not a terrible complicated process — it just involves a few steps. Let’s look at some of the specific implementation details, beginning with the servlet filter that determines whether the user has already logged into Google and/or granted OAuth access.

AuthorizationFilter

Let’s take a look at the first few lines of the AuthorizationFilter (to see how it’s configured as a filter, see the web.xml file).

 public void doFilter(ServletRequest req, ServletResponse res,
    FilterChain chain) throws IOException, ServletException {
 
  HttpServletRequest request = (HttpServletRequest) req;
 
  HttpServletResponse response = (HttpServletResponse) res;
 
  HttpSession session = request.getSession();
 
   if not present, add credential store to servlet context
  if (session.getServletContext().getAttribute(Constant.GOOG_CREDENTIAL_STORE) == null) {
   LOGGER.fine('Adding credential store to context ' + credentialStore);
   session.getServletContext().setAttribute(Constant.GOOG_CREDENTIAL_STORE, credentialStore);
  }
 
   if google user isn't in session, add it
  if (session.getAttribute(Constant.AUTH_USER_ID) == null) {
 
   LOGGER.fine('Add user to session');
 
   UserService userService = UserServiceFactory.getUserService();
 
   User user = userService.getCurrentUser();
 
   session.setAttribute(Constant.AUTH_USER_ID, user.getUserId());
   session.setAttribute(Constant.AUTH_USER_NICKNAME, user.getNickname());
 
    if not running on app engine prod, hard-code my email address for testing
   if (SystemProperty.environment.value() == SystemProperty.Environment.Value.Production) {
    session.setAttribute(Constant.AUTH_USER_EMAIL, user.getEmail());
   } else {
    session.setAttribute(Constant.AUTH_USER_EMAIL, 'jeffdavisco@gmail.com');
   }
 
  }

The first few lines simply cast the generic servlet request and response to their corresponding Http equivalents — this is necessary since we want access to the HTTP session. The next step is to determine whether a CredentialStore is present in the servlet context. As we’ll see, this is used to store the user’s credentials, so it’s convenient to have it readily available in subsequent servlets. The guts of the matter begin when we check to see whether the user is already present in the session using:

if (session.getAttribute(Constant.AUTH_USER_ID) == null) {

If not, we get their Google login credentials using Google’s UserService class. This is a helper class available to GAE users to fetch the user’s Google userid, email and nickname. Once we get this info from UserService, we store some of the user’s details in the session.

At this point, we haven’t done anything with OAuth, but that will change in the next series of code lines:
try {
Utils.getActiveCredential(request, credentialStore);
} catch (NoRefreshTokenException e1) {
// if this catch block is entered, we need to perform the oauth process
LOGGER.info(‘No user found – authorization URL is: ‘ +
e1.getAuthorizationUrl());
response.sendRedirect(e1.getAuthorizationUrl());
}

A helper class called Utils is used for most of the OAuth processing. In this case, we’re calling the static method getActiveCredential(). As we will see in a moment, this method will return a NoRefreshTokenException if no OAuth credentials have been previously captured for the user. As a custom exception, it will return URL value that is used for redirecting the user to Google to seek OAuth approval.

Let’s take a look at the getActiveCredential() method in more detail, as that’s where much of the OAuth handling is managed.

 public static Credential getActiveCredential(HttpServletRequest request, CredentialStore credentialStore) throws NoRefreshTokenException {
 
  String userId = (String) request.getSession().getAttribute(Constant.AUTH_USER_ID);
  Credential credential = null;
 
  try {
   if (userId != null) {
    credential = getStoredCredential(userId, credentialStore);
   }
 
   if ((credential == null || credential.getRefreshToken() == null) && request.getParameter('code') != null) {
    credential = exchangeCode(request.getParameter('code'));
 
    LOGGER.fine('Credential access token is: ' + credential.getAccessToken());
    if (credential != null) {
     if (credential.getRefreshToken() != null) {
      credentialStore.store(userId, credential);
     }
    }
   }
 
   if (credential == null || credential.getRefreshToken() == null) {
    String email = (String) request.getSession().getAttribute(Constant.AUTH_USER_EMAIL);
    String authorizationUrl = getAuthorizationUrl(email, request);
    throw new NoRefreshTokenException(authorizationUrl);
   }
 
  } catch (CodeExchangeException e) {
   e.printStackTrace();
  } 
 
  return credential;
 }

The first thing we do is fetch the Google userId from the session (they can’t get this far without it being populated). Next, we attempt to get the user’s OAuth credentials (stored in the Google class with the same name) from the CredentialStore using the Utils static method getStoredCredential(). If no credentials are found for that user, the Utils method called getAuthorizationUrl() is invoked. This method, which is shown below, is used to construct the URL that the browser is redirected to which is used to prompt the user to authorize access to their private data (the URL is served up by Google, since it will ask the user for approval).

  private static String getAuthorizationUrl(String emailAddress, HttpServletRequest request) {
  GoogleAuthorizationCodeRequestUrl urlBuilder = null;
   try {
    urlBuilder = new GoogleAuthorizationCodeRequestUrl(
     getClientCredential().getWeb().getClientId(),
     Constant.OATH_CALLBACK,
     Constant.SCOPES)
     .setAccessType('offline')
     .setApprovalPrompt('force');
   } catch (IOException e) {
     TODO Auto-generated catch block
    e.printStackTrace();
   }
   urlBuilder.set('state', request.getRequestURI());
 
  if (emailAddress != null) {
    urlBuilder.set('user_id', emailAddress);
  }
 
  return urlBuilder.build();
   }

As you can see, this method is using the class (from Google) called GoogleAuthorizationCodeRequestUrl. It constructs an HTTP call using the OAuth client credentials that is provided by Google when you sign up for using OAuth (those credentials, coincidentally, are stored in a file called client_secrets.json. Other parameters include the scope of the OAuth request and the URL that the user will be redirected back to if approval is granted by the user. That URL is the one you specified when signing up for Google’s OAuth access:

Now, if the user had already granted OAuth access, the getActiveCredential()method would instead grab the credentials from the CredentialStore.

Turning back to the URL that receives the results of the OAuth credentials, in this case, http://localhost:8888/authSub, you maybe wondering, how can Google post to that internal-only address? Well, it’s the user’s browser that is actually posting back the results, so localhost, in this case, resolves just fine. Let’s look that the servlet called OAuth2Callback that is used to process this callback (see the web.xml for how the servlet mapping for authSub is done).

 public class OAuth2Callback extends HttpServlet {
 
   private static final long serialVersionUID = 1L;
   private final static Logger LOGGER = Logger.getLogger(OAuth2Callback.class.getName());
 
   public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {
    
     StringBuffer fullUrlBuf = request.getRequestURL();
     Credential credential = null;
     
     if (request.getQueryString() != null) {
       fullUrlBuf.append('?').append(request.getQueryString());
     }
     
     LOGGER.info('requestURL is: ' + fullUrlBuf);
     
     AuthorizationCodeResponseUrl authResponse = new AuthorizationCodeResponseUrl(fullUrlBuf.toString());
     
      check for user-denied error
     if (authResponse.getError() != null) {
       LOGGER.info('User-denied access');
     } else {
       LOGGER.info('User granted oauth access');
       
       String authCode = authResponse.getCode();
       
       request.getSession().setAttribute('code', authCode);
       
       response.sendRedirect(authResponse.getState());
       
     }
   }
 }

The most important take-away from this class is the line:
AuthorizationCodeResponseUrl authResponse = new AuthorizationCodeResponseUrl(fullUrlBuf.toString()); The AuthorizationCodeResponseUrl class is provided as a convenience by Google to parse the results of the OAuth request. If the getError() method of that class isn’t null, that means that the user rejected the request. In the event that it is null, indicating the user approved the request, the method call getCode() is used to retrieve the one-time authorization code. This code value is placed into the user’s session, and when the Utils.getActiveCredential() is invoked following the redirect to the user’s target URL (via the filter), it will exchange that authorization code for a longer-term access and refresh token using the call:
credential = exchangeCode((String) request.getSession().getAttribute(‘code’));
The Utils.exchangeCode() method is shown next:

 public static Credential exchangeCode(String authorizationCode)
   throws CodeExchangeException {
    try {
  GoogleTokenResponse response = new GoogleAuthorizationCodeTokenRequest(
  new NetHttpTransport(), Constant.JSON_FACTORY, Utils
   .getClientCredential().getWeb().getClientId(), Utils
   .getClientCredential().getWeb().getClientSecret(),
   authorizationCode, Constant.OATH_CALLBACK).execute();
  return Utils.buildEmptyCredential().setFromTokenResponse(response);
     } catch (IOException e) {
  e.printStackTrace();
  throw new CodeExchangeException();
     }
 }

This method also uses a Google class called GoogleAuthorizationCodeTokenRequest that is used to call Google to exchange the one-time OAuth authorization code for the longer-duration access token.
Now that we’ve (finally) got our access token that is needed for the YouTube API, we’re ready to display to the user 10 of their video favorites.


Calling the YouTube API Services

With the access token in hand, we can now proceed to display the user their list of favorites. In order to do this, a servlet called FavoritesServlet is invoked. It will call the YouTube API, parse the resulting JSON-C format into some local Java classes via Jackson, and then send the results to the JSP page for processing. Here’s the servlet:

 public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
 
  LOGGER.fine('Running FavoritesServlet');
 
  Credential credential = Utils.getStoredCredential((String) request.getSession().getAttribute(Constant.AUTH_USER_ID), 
    (CredentialStore) request.getSession().getServletContext().getAttribute(Constant.GOOG_CREDENTIAL_STORE));
 
  VideoFeed feed = null;
 
   if the request fails, it's likely because access token is expired - we'll refresh
  try {
   LOGGER.fine('Using access token: ' + credential.getAccessToken());
   feed = YouTube.fetchFavs(credential.getAccessToken());
 
  } catch (Exception e) {
 
   LOGGER.fine('Refreshing credentials');
   credential.refreshToken();
   credential = Utils.refreshToken(request, credential);
 
   GoogleCredential googleCredential = Utils.refreshCredentials(credential);
   LOGGER.fine('Using refreshed access token: ' + credential.getAccessToken());
    retry 
   feed = YouTube.fetchFavs(credential.getAccessToken());
  }  
 
  LOGGER.fine('Video feed results are: ' + feed);
 
  request.setAttribute(Constant.VIDEO_FAVS, feed);
 
  RequestDispatcher dispatcher = getServletContext().getRequestDispatcher('htmllistVids.jsp');
 
 dispatcher.forward(request, response);  
 
 }

Since this post is mainly about the OAuth process, I won’t go into too much detail how the API call is placed, but the most important line of code is: feed = YouTube.fetchFavs(credential.getAccessToken()); Where feed is an instance of VideoFeed. As you can see, another helper class called YouTube is used for doing the heavy-lifting. Just to wrap things up, I’ll show the fetchFavs() method.

   public static VideoFeed fetchFavs(String accessToken) throws IOException, HttpResponseException {
    HttpTransport transport = new NetHttpTransport();
     final JsonFactory jsonFactory = new JacksonFactory();
     HttpRequestFactory factory = transport.createRequestFactory(new HttpRequestInitializer() {
 
       @Override
       public void initialize(HttpRequest request) {
          set the parser
         JsonCParser parser = new JsonCParser(jsonFactory);
         request.addParser(parser);
          set up the Google headers
         GoogleHeaders headers = new GoogleHeaders();
         headers.setApplicationName('YouTube Favorites1.0');
         headers.gdataVersion = '2';
         request.setHeaders(headers);
       }
     });
     
      build the YouTube URL
     YouTubeUrl url = new YouTubeUrl(Constant.GOOGLE_YOUTUBE_FEED);
     url.maxResults = 10;
     url.access_token = accessToken;
      build the HTTP GET request
     HttpRequest request = factory.buildGetRequest(url);
     
     HttpResponse response = request.execute();
      execute the request and the parse video feed
     VideoFeed feed = response.parseAs(VideoFeed.class);
     
     return feed;
   }

It uses the Google class called HttpRequestFactory to construct an outbound HTTP API call to YouTube. Since we’re using GAE, we’re limited as to which classes we can use to place such requests. Notice the line of code:
url.access_token = accessToken;
That’s where we are using the access token that was acquired through the OAuth process.

So, while it took a fair amount of code to get the OAuth stuff working correctly, once it’s in place, you are ready to rock-and-roll with calling all sorts of Google API services!

Reference: Authenticating for Google Services, Part 2 from our JCG partner Jeff Davis at the Jeff’s SOA Ruminations blog.