Generating MD5 Hashes In Java

Web applications frequently need to generate hashes for comparing data, verifying file integrity, generating keys, etc. If your application uses the Apache Commons libraries, you can use the included DigestUtils convenience class to create hashes. If you don’t use the Apache libraries, then you need an alternate way to generate MD5 hashes.

The below code snippet is a simple function that generates a MD5 hash from a given string. It’s simple to use: simply pass the string to be hashed into the function (to_be_md5), and the hash is returned. This function will throw a RuntimeException if an error is encountered (for example, if the passed-in String is null).

/**
 * Calculate a MD5 hash from the provided String. If the 
 * provided String is null, this method will throw a 
 * RuntimeException.
 * 
 * @param to_be_md5 A String to calculate a MD5 hash from.
 * @return A MD5 hash calculated from the provided String.
 * @throws RuntimeException If an error was encountered during calculating.
 */
public static String generateMD5(String to_be_md5) {
    String md5_sum = "";
    //If the provided String is null, then throw an Exception.
    if (to_be_md5 == null) {
        throw new RuntimeException("There is no string to calculate a MD5 hash from.");
    }
    try {
        MessageDigest md = MessageDigest.getInstance("MD5");
        byte[] array = md.digest(to_be_md5.getBytes("UTF-8"));
        StringBuffer collector = new StringBuffer();
        for (int i = 0; i < array.length; ++i) {
            collector.append(Integer.toHexString((array[i] & 0xFF) | 0x100).substring(1, 3));
        }
        md5_sum = collector.toString();
    }//end try
    catch (NoSuchAlgorithmException e) {
        throw new RuntimeException("Could not find a MD5 instance: " + e.getMessage());
    }
    catch (UnsupportedEncodingException e) {
        throw new RuntimeException("Could not translate UTF-8: " + e.getMessage());
    }
    return md5_sum;
}//end generateMD5

Don’t forget to import the following classes:

import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

Retrieving All Recently Created Entities

Searching the datastore using date-based entity properties can often be confusing to new App Engine developers. Here’s how to search for all recently created entities.

First, each entity has to record the time it was created. Add the following line of code when constructing your entities (entity represents the Entity object being created):

entity.setProperty("add_date", new Date());//Records the date this entity was added.

Here’s the code for the query. This code example searches for all entities created within the last 24 hours with a kind of mail. The resulting entities are placed into the variable results (which is a standard java.util.List object).

/**
 * Creates a Date object that represents a point in time 
 * 24 hours ago.
 */
Date current_date = new Date();//The current date.
long current_date_in_ms = current_date.getTime();//The current date in milliseconds.
//Take the current date in milliseconds, and subtract 24 hours worth 
//of milliseconds (1000 milliseconds per second, times 60 seconds per 
//minute, times 60 minutes per hour, times 24 hours in a day).
long one_day_ago_in_ms = current_date_in_ms - (1000 * 60 * 60 * 24);
Date one_day_ago = new Date(one_day_ago_in_ms);//Create a new Date object representing one day ago.
//Search all entities with the kind "mail". 
//We create a Filter that requires that the add_date property be greater 
//than the Date we computed above.
Query q = new Query("mail");
Query.Filter date_filter = new Query.FilterPredicate("add_date", Query.FilterOperator.GREATER_THAN_OR_EQUAL, one_day_ago);
q.setFilter(date_filter);
PreparedQuery pq = DatastoreServiceFactory.getDatastoreService().prepare(q);
List<Entity> results = pq.asList(FetchOptions.Builder.withDefaults());

 

Error Code 202

On high traffic App Engine applications, you may occasionally see a request fail with error code 202. You’ll see the following text in your logs:

A problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. (Error code 202)

The request log will look similar to the picture below:

Error code 202 is an internal App Engine failure code: an unknown error occurred within App Engine or its associated services, not from your application. This is not an error that your application can catch or recover from, nor is the application at fault for this error. The current request will fail, but the client is free to retry the request. If this request is from the task queue service, the task queue will automatically retry the request if you set the appropriate settings while creating the task.

A note about error codes: the code 202 relates to App Engine; it is NOT a HTTP status code. When a request encounters this internal error, it fails with the HTTP status code 500 (as you can see from the above picture).

HTTP Basic Access Authorization In Golang

One of the simplest and oldest methods of authorization is HTTP Basic authorization. While it isn’t as secure as the Users service of App Engine or the OAuth authorization model, it’s easy to implement and looks visually impressive depending on the user’s browser.

Here’s a picture of the HTTP Basic authorization prompt in IE10 on Windows 8:

The variable authorization contains a base64 encoded hash generated by the user’s browser, created by concatenating the username and password together with a colon: username:passwordR represents a http.Request reference, c is appengine.Context, and w is http.ResponseWriter.

//Get the authorization header.
authorization_array := r.Header["Authorization"]
if len(authorization_array) > 0 {
    authorization := strings.TrimSpace(authorization_array[0])
    c.Infof("Authorization: ", authorization)
} else {
    w.Header().Set("WWW-Authenticate", "Basic realm=\"user\"")
    http.Error(w, http.StatusText(401), 401)
}

YouTube Error Page

When designing web sites it’s always important to make every page – even error pages – user friendly. As an example, here’s a picture of YouTube’s error page. The reference to highly trained monkeys always makes me laugh.

Searching Twitter

Many web applications integrate Twitter into their pages to display realtime news and thoughts. Here’s a code snippet showing how to search Twitter using the twitter4j library.

Twitter’s search API doesn’t return all search results at once; instead it returns “pages” of results, with each page containing 100 tweets matching the search criteria. The following code snippet runs through many pages of search results using a while loop, so it may take some time to process. Run it within a backend if you’re searching for a particularly popular phrase or doing heavy processing on each tweet.

The variable twitter represents a twitter4j.Twitter object, and status represents a single tweet. You can extract tweet information from that object; for example status.getText() would return the text of the tweet. Edit the string “google app engine” to whatever text you’re searching Twitter for.

twitter4j.Query twitter_query = new twitter4j.Query("google app engine");
twitter_query.setCount(100);
//This while loop runs through each page of the returned 
//tweets. One page of results (100 tweets) is processed 
//per loop.
while (twitter_query != null) {
    //A list of the returned tweets, representing 1 page (100 tweets).
    QueryResult twitter_results = twitter.search(twitter_query);
    //Run through this page of results and access each returned tweet.
    for (Status status : twitter_results.getTweets()) {
        //Do something with the status object.
    }
    //Retrieves a Query representing the next page of results.
    twitter_query = twitter_results.nextQuery();
}//end while loop running through pages of returned results.

 

HTTP GET Using The Low Level Java App Engine API

Here’s a short code example showing how to do a HTTP GET using the low level Java API.

The variable url_string_here is the URL being retrieved as a String. It returns a byte[] array containing the content of the response. If the response code is not 200 (i.e. anything other than HTTP OK) then this code throws a RuntimeException.

URL url = new URL(url_string_here);
HTTPRequest request = new HTTPRequest(url, HTTPMethod.GET);
request.setHeader(new HTTPHeader("User-Agent", "Custom User Agent "));
//Execute request.
HTTPResponse response = URLFetchServiceFactory.getURLFetchService().fetch(request);
if (response.getResponseCode() == 200) {
    //The response was OK
    //Retrieve the content of the response.
    return response.getContent();
}//end if the response code was 200.
else {
    throw new RuntimeException("Response code was " + response.getResponseCode());
}

Capitalization In Cron Scheduling

A note on cron: schedules must be recorded in all lowercase letters. The following picture shows an application upload failure because the S in Saturday is capitalized in cron.

However the all lowercase version schedule: every saturday 9:00 works.

Posting To Twitter

Many web applications need to connect to and integrate with Twitter. In Java, the best and most widely-used library is twitter4j. The following code snippet shows how to connect to Twitter and post a tweet.

Oauth_consumer_keyoauth_consumer_secretoauth_access_token, and oauth_access_token_secret are the Twitter OAuth authentication keys. You can generate those keys by going to Twitter’s developer site and creating an application plus access tokens. The string tweet_text is the text to post to Twitter. If any problems occur while posting to Twitter, a TwitterException will be thrown.

ConfigurationBuilder config = new ConfigurationBuilder();
config.setDebugEnabled(true);
config.setOAuthConsumerKey(oauth_consumer_key);
config.setOAuthConsumerSecret(oauth_consumer_secret);
config.setOAuthAccessToken(oauth_access_token);
config.setOAuthAccessTokenSecret(oauth_access_token_secret);
Twitter twitter = (new TwitterFactory(config.build())).getInstance();
try {
    //Post to Twitter.
    twitter.updateStatus(tweet_text);
}
catch (TwitterException e) {
    //error posting to Twitter
}

Simple Logging In Java

In most applications, it’s a good idea to use a robust logging framework such as java.util.logging.* or log4j. However, when you’re writing simpler App Engine applications there is an alternative: use System.out and System.err.

App Engine catches all output written to the standard output streams System.out/System.err and prints it into the application’s logs. So logging a message is as simple as writing one line of code:

System.out.println("This is an informational logging line.");

Use the System.err stream to log an error:

System.err.println("Exception: " + e.getMessage());

If you do decide to log informational messages, you also need to set the logging level to INFO. Open up the logging.properties file in the /war/WEB-INF/ directory:

Change the .level property to INFO:

.level = INFO

Now your App Engine application will record and display all logs sent to System.out and System.err.