Facebook Outbound Email

I’m in the middle of testing an email server on Compute Engine, and I noticed something unusual: apparently Facebook’s outbound email servers insist on using extended SMTP to send email.

With extended SMTP, an email server sends email by opening up a connection and sending the EHLO command. The proper response is either 250 (to indicate success and that extended SMTP support is available) or 550 (the responding server did not understand the command, which is another way of saying that the responding server does not support ESMTP). In case of 550 errors, the usual practice is to fall back to the original SMTP command set and to send a HELO request.

But Facebook’s outbound mail servers seem to only want to connect with ESMTP servers: FB mail servers send a quit command instead of falling back to sending a HELO command.

Another interesting oddity from watching mail logs: Google’s Gmail servers seem to be the only mail servers properly implementing the BDAT command (binary data). I never see any other mail servers attempt to use it.

Removing EXIF Data With ImageMagick

Recently I needed to find a way to mass-remove EXIF data from multiple images. EXIF – if you didn’t know – is essentially metadata included within images. This metadata can include the name of the camera, the GPS coordinates where the picture was taken, comments, etc.

The easiest way is to remove EXIF data is to download ImageMagick and use the mogrify command line tool:

mogrify -strip /example/directory/*

ImageMagick can be run within a Compute Engine VM to make it available to a web application. Here’s the command to install ImageMagick:

sudo aptitude install imagemagick

Communicating With Sockets On Compute Engine

Writing custom applications on Compute Engine requires the use of sockets to communicate with clients. Here’s a code example demonstrating how to read and write to sockets in Java.

Assume that client_socket is the socket communicating with the client:

 * Contains the socket we're communicating with.
Socket client_socket;

Create a socket by using a ServerSocket (represented by server_socket ) to listen to and accept incoming connections:

Socket client_socket = server_socket.accept();

Then extract a PrintWriter and a BufferedReader from the socket. The PrintWriter out object sends data to the client, while the BufferedReader in object lets the application read in data sent by the client:

 * Handles sending communications to the client.
PrintWriter out;
 * Handles receiving communications from the client.
BufferedReader in;
try {
    //We can send information to the client by writing to out.
    out = new PrintWriter(client_socket.getOutputStream(), true);
    //We receive information from the client by reading in.
    in = new BufferedReader(new InputStreamReader(client_socket.getInputStream()));
     * Start talking to the other server.
    //Read in data sent to us.
    String in_line = in.readLine();
    //Send back information to the client.
}//end try
catch (IOException e) {
    //A general problem was encountered while handling client communications.

A line of text sent by the client can be read in by calling in.readLine() demonstrated by the in_line string. To send a line of text to the client, call out.println(send_info) where send_info represents a string. If an error occurs during communication, an IOException will be thrown and caught by the above catchstatement.

Remember to add the following imports:

import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.io.BufferedReader;

Releasing A Static IP In Compute Engine

Compute Engine allows static IPs to be reserved, even if no VM is attached. However, IPs that are unattached to VMs or load balancers are charged at $0.01 per hour. To avoid this charge, it’s a good idea to release IPs that are no longer in use.

To release a static IP, first go to the Compute Engine section of the Google Cloud Console. Select the Networks option:

A list of the IPs allocated to the project will be displayed:

Select the IP to remove and click the link labeled Release IP address:

You’ll see a confirmation box next. Press Yes.

The console will need a few moments to release the IP:

Measuring Elapsed Time: System.nanoTime()

When I need to measure elapsed time (for example, timing how long an API call takes) I prefer to use System.nanoTime() instead of using the usual method of (new Date()).getTime().

The key idea to remember is that nanoTime() doesn’t return a timestamp – it represents a measurement of time from some arbitrary point (so for example nanoTime() could return a negative figure). This helps when writing self-documenting code: a variable storing a nanoTime() value can only be used for elapsed time comparisons, not timestamp recording.

Here’s an example of measuring elapsed time with System.nanoTime(). First, record the start time:

 * Records the start time 
 * using System.nanoTime()
Long start_time;
//Record the start time.
start_time = System.nanoTime();

Insert some time-consuming operation here, or another long-running call. Place this code where you want to end the timer:

//Calculate how long we've been running in nanoseconds.
Long diff_time = System.nanoTime() - start_time;

The variable diff_time stores the number of nanoseconds that has elapsed during the timer. Now suppose you wanted to throw an exception if the timer took more than 30 seconds (30 billion nanoseconds); here’s the example code:

//We set the maximum time in nanoseconds, multiplied by milliseconds, 
//multiplied by seconds.
Long MAXIMUM_TIME_OPEN = new Long(1000000L * 1000 * 30);
//If the runtime of this operation is longer than the time we've set, 
//throw an Exception.
if (diff_time > MAXIMUM_TIME_OPEN) {
    throw new IOException("Timeout has been exceeded.");

To keep the code easy to understand, we’re showing how the maximum time amount is computed: there are 1 million (1,000,000) nanoseconds in a millisecond, multiplied by 1 thousand milliseconds in a second (1,000), multiplied by 30 seconds.

Google Compute Engine

Google’s Compute Engine just released into General Availability, and I’ve been testing it out the last couple of days.

The one thing that blows me away is how reliably fast even the low-end instances are. I provisioned and set up a f1-micro instance – it runs great and quite consistently. That’s in sharp contrast to Amazon’s micro instance which is limited to “bursty” processing; there are spikes where processing goes quickly, then the CPU gets throttled and the instance grinds to a near-halt.

I’m considering building a mail app on top of GCE – so far, everything looks great. GCE even allows inbound SMTP connections (although unfortunately no outbound SMTP connections).