Posted on

CORS Preflight Request Testing In cURL

When browsers send AJAX-JSONP requests, they often send a “preflight request” before the JSONP call. This request is a HTTP OPTIONS call asking the server whether it supports the cross origin resource sharing specification (in other words, JSONP requests).

To test a server’s support for cross origin resource sharing (CORS), you can use the cURL utility to emulate a HTTP OPTIONS request. A server that supports CORS will return a number of Access-Control headers specifying the requests it supports. Here’s an example cURL command:

curl -H "Origin: http://www.example.com" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: X-Requested-With" \
  -X OPTIONS --verbose \
  http://ip.jsontest.com/

Here’s an example of a proper CORS preflight response:

Access-Control-Allow-Origin is set to a wildcard, which means that all domains are permitted to make requests to it. Access-Control-Max-Age means that the results of this preflight request can be saved for 86,400 seconds (1 day). Access-Control-Allow-Methods means that GET and POST requests are supported.

Posted on

Looking Up App Engine Issues

Found a bug within App Engine? Want to request a feature? The best way to notify Google is to file an issue within App Engine’s tracker, located at: https://code.google.com/p/googleappengine/issues/list .

You can file an issue by clicking the New Issue button in the top left corner of the page:

But before you file a new issue, make sure that someone else hasn’t already filed the same issue. You can search issues by using the search box on the upper portion of the page:

If you don’t want to search, you can browse through the issues by using the dropdown boxes in each header:

Here’s an example of what an issue looks like:

Posted on

Hanging Memcache Calls

Recently there was a discussion in the App Engine forums about memcache calls that were hanging; in one instance, a memcache async put call was taking 2 hours to complete!

This was a particularly interesting issue, and I’d like to share a number of thoughts I had while solving it:

App Engine has a number of internal rate limiting/throttling controls on services. Moving large quantities of data around can quickly cause an application to hit these limits. In fact, I suspect that this was the actual problem – the original poster’s application was storing multiple megabytes of data into memcache in multiple asynchronous calls that occurred simultaneously; this design could easily be hitting a number of different rate limits. My suggestion for solving this problem (which ultimately worked) was to add a short delay after each memcache put call and to split the data amongst an increased number of memcache put calls. The reasons for which I suggested this fix are numerous:

  1. Adding a short delay after each memcache put call buys time for App Engine’s rate limit to reset; it prevents App Engine from thinking that the application is malfunctioning or attempting to overwhelm the memcache pipeline.
  2. Delays are easy to implement – in Python it’s one call to time.sleep(number of seconds to delay)and in Java it’s a simple call to Thread.sleep(number of seconds to delay). Note that in Java, you have to catch the potential InterruptedException. The Go call is similar to Python: call time.Sleep(delay duration). In PHP a delay is even simpler than in all of the above languages: all you need to do is call sleep(delay seconds).
  3. Increasing the number of memcache put calls means that a smaller amount of data is being stored for each memcache put. This contributes to point 1: preventing the pipeline to memcache from being overwhelmed with data.
  4. The delay doesn’t need to be long: two to five seconds is more than enough. In some cases, even a one second delay is enough to work.

Fortunately, the above fix worked in this case. But if it had not, I was prepared with a number of other possible fixes. For instance, I would have suggested the use of the task queue: split the data among multiple tasks, and then have each task store their data into memcache. Since each task would constitute a separate request and may be split amongst multiple instances, there’s less of a chance for any rate limiting to kick in. If that option wasn’t palatable for any reason, then another option would be to switch to dedicated memcache; it seems to be much more forgiving in regards to usage.

If none of the above options had worked, I would have suggested dumping memcache entirely and writing to the datastore/Cloud SQL. While memcache is a terrific service, it is not reliable – persisting the data through alternative sources is a much better way to manage large quantities of information.

The short version of this post: hanging or slow memcache calls can be fixed by inserting delays after each call and decreasing the amount of data handled in each memcache call.

Posted on

Changing Front End Instance Classes

App Engine allocates a certain amount of RAM and processing power to your application. Specific sizes of memory and processing power are called instance classes. You can increase the amount of resources allocated to your app by moving to a higher instance class as shown below.

First, go to the administration console and click the Application Settings link:

Go down to the Performance section and find the dropdown box marked Frontend Instance Class.

Select the instance class you need. Note that selecting a higher instance class will cost you more money/deplete your free instance hours faster.

Save your settings by clicking the Save button.

Posted on

Error Parsing YAML File: While Scanning A Simple Key

App Engine uses the app.yaml file to route incoming requests to the appropriate handlers. It’s important to write proper YAML code in this file, otherwise your application may behave erratically or not at all.

One common problem with YAML files is failing to properly separate key:value pairs. The YAML specification requires a colon ( : ) and one space character between the key and the associated value. Here’s an example of a properly formatted YAML key:value pair:

Key: Value

Now here’s an example of a broken app.yaml file:

application: an-example-application-id
version: 1
runtime: php
api_version: 1
threadsafe:true

Notice the error? The threadsafe property has a colon, but no space separating the key ( threadsafe) and the value ( true ). Here’s a screenshot of appcfg refusing to upload this broken file:

If you receive this error, make sure that all of your YAML properties are separated by a colon and a space. One space is enough, don’t use tabs or multiple spaces.

Posted on

Bulk Adding Headers To An URL Fetch Request

A quick code example: how to easily add headers to an URL Fetch request.

First, create a java.util.Hashtable:

Hashtable<String, String> request_headers;
request_headers = new Hashtable<String, String>();

Put the headers you want into this hashtable. The keys and values of this hashtable will become the header names and values in the fetch request.

When you’re configuring the URL Fetch request, use the code below to add in all the headers:

Enumeration<String> set_header_keys = request_headers.keys();
while (set_header_keys.hasMoreElements()) {
    String key = set_header_keys.nextElement();
    String value = request_headers.get(key);
    connection.setRequestProperty(key, value);
}

The connection variable represents a java.net.HttpURLConnection object.

Posted on

Setting The Content Type In Java

In most cases servlets are used to generate and serve HTML pages. However, servlets can serve any type of data including images, plain text, PDFs, spreadsheets, Javascript code, etc. To do this, the servlet must declare the type of content being served to the web browser. This declaration is called the content type or the media type of the file.

Here’s how to set the content type of a servlet’s response. The variable resp represents a javax.servlet.http.HttpServletResponse object:

resp.setContentType(content_type);

Put the appropriate content type in the content_type variable. Some common content types are text/html(for HTML pages), text/plain (plain text), application/javascript (JS code), application/vnd.ms-excel (Excel spreadsheets), image/jpeg (JPEG images), application/pdf (PDFs); the list goes on and on. If you need to figure out the appropriate content type for your data, look it up on the Wikipedia Internet Media Types list.