Using Google Protobuf Timestamp In PHP

This is more of a documentary post because I haven’t seen documentation on Google’s Timestamp class anywhere.

Google’s libraries – in particular, the GCP libraries for datastore/tasks/etc – use the Google/Protobuf/Timestamp class to represent time. Timestamp is a simple wrapper around the number of seconds since UNIX epoch, in the UTC timezone. For example here is how to create a Timestamp reflecting the current date and time, plus 2 minutes into the future (120 seconds):

use Google\Protobuf\Timestamp;

    $future_time_seconds = time() + 120;
    $future_timestamp = new Timestamp();
    $future_timestamp->setSeconds($future_time_seconds);
    $future_timestamp->setNanos(0);

There are equivalent classes and functions for Python/Java/Go/other languages that Google Cloud supports.

Using the Timestamp class – especially setting up future dates – is necessary for configuring my favorite Google Cloud service: Cloud Tasks. A Cloud Task can accept a future date to run at, thereby giving you a way to queue up and delay execution of an activity. For example, see the below screenshot: I’ve created 3 tasks 20 seconds ago, yet they’re set for a future execution 3 minutes 45 seconds from now:

Creating A WordPress Blog Slug Part 2 – Python

In a previous post, I posted a sample NodeJS function to assemble a WordPress blog slug. I ended up rewriting part of the larger application (and the function itself) in Python.

In the below function, source is a blog title string, and it returns a slug suitable for use in a blog URL.

def generate_slug(source):
    i = 0
    source = source.lower().strip()
    allowed = "abcdefghijklmnopqrstuvwxyz1234567890"
    slug = ""
    while i < (len(source) - 0):
        single_letter = source[i:i+1]
        if single_letter in allowed:
            slug += single_letter
        elif not slug.endswith("-"):
            #letter is not allowed
            #check that the slug doesn't already end in a dash
            slug += "-"
        i = i + 1
    return slug

Utility Functions To Save To Google Cloud Storage

Here are short utility functions used to save to Google Cloud Storage in a Java App Engine application.

These snippets require the Java Cloud Storage library ( com.google.appengine.tools.cloudstorage.GcsService ). The bucket string is the name of the bucket, no gs:// required. folder_path is the subdirectory you want to save the file under. It should end with a forward slash. For example this/is/a/subdirectory/ . If you do not want a subdirectory, pass an empty string (“”). filename is the filename of the file you want to save, while contents is the data being saved.

There are two functions here. The top is intended to quickly save simple text/html/xml/etc files, while the bottom is intended to save binary files such as images, executables, and so forth. The bottom function includes a filetype argument, which should be filled with the appropriate MIME type. For example: image/png, image/jpg, application/octet-stream.

Reminder: GCS innately views files as individual objects. When you specify a subfolder, the filename for the object itself includes the folder path. When you read the file back from Google cloud storage, you need to include the folder path: for example this/is/a/subdirectory/example.png .

	public static void saveStringToGCS(String bucket, String folder_path, String filename, String contents) throws IOException {
		GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
		GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
		GcsFileOptions file_options = GcsFileOptions.getDefaultInstance();
		storage_service.createOrReplace(to_file, file_options, ByteBuffer.wrap(contents.getBytes(StandardCharsets.UTF_8)));
	}
	
	public static void saveByteBufferToGCS(String bucket, String folder_path, String filename, ByteBuffer contents, String filetype) throws IOException {
		GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
		GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
		GcsFileOptions file_options = (new GcsFileOptions.Builder()).mimeType(filetype).build();
		storage_service.createOrReplace(to_file, file_options, contents);
	}

A HTTP GET Example Using The Apache HTTPClient Library

Here’s a quick example of using the Apache HTTPClient library to issue a simple GET request. The URL you’re connecting to is in endpoint, and the contents of the URL can be read from the BufferedReader response_reader.

		HttpGet get_request = new HttpGet(endpoint);
		get_request.addHeader(HttpHeaders.USER_AGENT, "Example Reader Service");

		HttpResponse http_response = HttpClientBuilder.create().build().execute(get_request);
		content_type_header = http_response.getFirstHeader("Content-Type").getValue();
		
		int response_code = http_response.getStatusLine().getStatusCode();

		if (response_code != 200) {
			throw new IOException("Response code is not 200 OK. Response code: " + response_code + " : " + endpoint.toString());
		}

		//Pull out text response
		InputStream response_is = http_response.getEntity().getContent();
		BufferedReader response_reader = new BufferedReader(new InputStreamReader(response_is, "UTF-8"));

Mapping Unicode Punctuation To ASCII Punctuation

The following code maps a bunch of unicode punctuation to their more normal ASCII counterparts. For example, the Unicode character LEFT DOUBLE QUOTATION MARK is mapped to the ASCII double quotation mark: “.

		Hashtable<String, String> map = new Hashtable<String, String>();
		map.put(new String(Character.toChars(0x00AB)), new String("\""));
		map.put(new String(Character.toChars(0x00AD)), "-");
		map.put(new String(Character.toChars(0x00B4)), "\'");
		map.put(new String(Character.toChars(0x00BB)), "\"");
		map.put(new String(Character.toChars(0x00F7)), "/");
		map.put(new String(Character.toChars(0x01C0)), "|");
		map.put(new String(Character.toChars(0x01C3)), "!");
		map.put(new String(Character.toChars(0x02B9)), "\'");
		map.put(new String(Character.toChars(0x02BA)), "\"");
		map.put(new String(Character.toChars(0x02BC)), "\'");
		map.put(new String(Character.toChars(0x02C4)), "^");
		map.put(new String(Character.toChars(0x02C6)), "^");
		map.put(new String(Character.toChars(0x02C8)), "\'");
		map.put(new String(Character.toChars(0x02CB)), "`");
		map.put(new String(Character.toChars(0x02CD)), "_");
		map.put(new String(Character.toChars(0x02DC)), "~");
		map.put(new String(Character.toChars(0x0300)), "`");
		map.put(new String(Character.toChars(0x0301)), "\'");
		map.put(new String(Character.toChars(0x0302)), "^");
		map.put(new String(Character.toChars(0x0303)), "~");
		map.put(new String(Character.toChars(0x030B)), "\"");
		map.put(new String(Character.toChars(0x030E)), "\"");
		map.put(new String(Character.toChars(0x0331)), "_");
		map.put(new String(Character.toChars(0x0332)), "_");
		map.put(new String(Character.toChars(0x0338)), "/");
		map.put(new String(Character.toChars(0x0589)), ":");
		map.put(new String(Character.toChars(0x05C0)), "|");
		map.put(new String(Character.toChars(0x05C3)), ":");
		map.put(new String(Character.toChars(0x066A)), "%");
		map.put(new String(Character.toChars(0x066D)), "*");
		map.put(new String(Character.toChars(0x200B)), " ");
		map.put(new String(Character.toChars(0x2010)), "-");
		map.put(new String(Character.toChars(0x2011)), "-");
		map.put(new String(Character.toChars(0x2012)), "-");
		map.put(new String(Character.toChars(0x2013)), "-");
		map.put(new String(Character.toChars(0x2014)), "-");
		map.put(new String(Character.toChars(0x2015)), "-");
		map.put(new String(Character.toChars(0x2016)), "|");
		map.put(new String(Character.toChars(0x2017)), "_");
		map.put(new String(Character.toChars(0x2018)), "\'");
		map.put(new String(Character.toChars(0x2019)), "\'");
		map.put(new String(Character.toChars(0x201A)), ",");
		map.put(new String(Character.toChars(0x201B)), "\'");
		map.put(new String(Character.toChars(0x201C)), "\"");
		map.put(new String(Character.toChars(0x201D)), "\"");
		map.put(new String(Character.toChars(0x201E)), "\"");
		map.put(new String(Character.toChars(0x201F)), "\"");
		map.put(new String(Character.toChars(0x2032)), "\'");
		map.put(new String(Character.toChars(0x2033)), "\"");
		map.put(new String(Character.toChars(0x2034)), "\'");
		map.put(new String(Character.toChars(0x2035)), "`");
		map.put(new String(Character.toChars(0x2036)), "\"");
		map.put(new String(Character.toChars(0x2037)), "\'");
		map.put(new String(Character.toChars(0x2038)), "^");
		map.put(new String(Character.toChars(0x2039)), "<");
		map.put(new String(Character.toChars(0x203A)), ">");
		map.put(new String(Character.toChars(0x203D)), "?");
		map.put(new String(Character.toChars(0x2044)), "/");
		map.put(new String(Character.toChars(0x204E)), "*");
		map.put(new String(Character.toChars(0x2052)), "%");
		map.put(new String(Character.toChars(0x2053)), "~");
		map.put(new String(Character.toChars(0x2060)), " ");
		map.put(new String(Character.toChars(0x20E5)), "\\");
		map.put(new String(Character.toChars(0x2212)), "-");
		map.put(new String(Character.toChars(0x2215)), "/");
		map.put(new String(Character.toChars(0x2216)), "\\");
		map.put(new String(Character.toChars(0x2217)), "*");
		map.put(new String(Character.toChars(0x2223)), "|");
		map.put(new String(Character.toChars(0x2236)), ":");
		map.put(new String(Character.toChars(0x223C)), "~");
		map.put(new String(Character.toChars(0x2264)), "<");
		map.put(new String(Character.toChars(0x2265)), ">");
		map.put(new String(Character.toChars(0x2266)), "<");
		map.put(new String(Character.toChars(0x2267)), ">");
		map.put(new String(Character.toChars(0x2303)), "^");
		map.put(new String(Character.toChars(0x2329)), "<");
		map.put(new String(Character.toChars(0x232A)), ">");
		map.put(new String(Character.toChars(0x266F)), "#");
		map.put(new String(Character.toChars(0x2731)), "*");
		map.put(new String(Character.toChars(0x2758)), "|");
		map.put(new String(Character.toChars(0x2762)), "!");
		map.put(new String(Character.toChars(0x27E6)), "[");
		map.put(new String(Character.toChars(0x27E8)), "<");
		map.put(new String(Character.toChars(0x27E9)), ">");
		map.put(new String(Character.toChars(0x2983)), "{");
		map.put(new String(Character.toChars(0x2984)), "}");
		map.put(new String(Character.toChars(0x3003)), "\"");
		map.put(new String(Character.toChars(0x3008)), "<");
		map.put(new String(Character.toChars(0x3009)), ">");
		map.put(new String(Character.toChars(0x301B)), "]");
		map.put(new String(Character.toChars(0x301C)), "~");
		map.put(new String(Character.toChars(0x301D)), "\"");
		map.put(new String(Character.toChars(0x301E)), "\"");
		map.put(new String(Character.toChars(0xFEFF)), " ");

Creating A WordPress Blog Slug – NodeJS

A slug in WordPress is the part of the URL that references the blog title. For example, if the URL looks like example.com/2019/this-is-my-test-post, then this-is-my-test-post would be a slug. Typically a slug is all letters and numbers, with any other character being replaced by a dash.

The below code fragment is a simple JS function to reduce a blog title down to a slug – you may want to add some additional code to guarantee a certain slug length.


var generateSlug = function generateSlug(title) {
  var allowable_characters = "abcdefghijklmnopqrstuvwxyz1234567890";
  
  //Builds our own slug
  title = title.trim().toLowerCase();
  var title_array = title.split("");
  
  var outbound_title = " ";
  for ( var i = 0; i < title_array.length; i++) {
    var title_char = title_array[i];
    if ( allowable_characters.indexOf(title_char) != -1 ) {
      outbound_title = outbound_title + title_char;
    }
    else if ( outbound_title.split("").pop() != "-" ) {
      outbound_title = outbound_title + "-";
    }
  }
  
  console.log("Generated slug: " + outbound_title);
  
  return outbound_title.trim();
  
}//end generateSlug

Querying For Previous Published WordPress Posts

The following code example pulls out the last 5 published WordPress posts by their ID, extracts the first ID (which is the last published post ID) and then extracts the post’s tags into an array.

$return_tag_array is a list of the last published post’s tags.

  //Get list of past published posts, up to 5
  $args = array(
	'numberposts' => 5,
	'offset' => 0,
	'category' => 0,
	'orderby' => 'post_date',
	'order' => 'DESC',
	'include' => '',
	'exclude' => '',
	'meta_key' => '',
	'meta_value' =>'',
	'post_type' => 'post',
	'post_status' => 'publish',
	'suppress_filters' => true
);
	
  $published_posts = wp_get_recent_posts( $args, ARRAY_A );

  //Pull off the top most post, & retrieve the ID
  $last_published_post = reset($published_posts);
  $last_published_post_id = $last_published_post["ID"];

  //TAGS	
  //Use the post ID to pull off tags
  $tags = wp_get_post_tags($last_published_post_id);
  $return_tag_array = array();	
  foreach ($tags as $tag) {
	  $tag_name = $tag->name;
	  array_push($return_tag_array, $tag_name);
  }

Adding Categories To A WordPress Post

I’ve been writing a custom WordPress plugin to accept emailed posts and insert them into a WordPress blog. One of the difficult issues I’ve encountered is that WordPress deals with tags and categories in different ways: it’s relatively easy to add and remove tags. It takes a bit of roundabout work to do the same with categories.

The below code fragment takes a string array of categories ( $to_post_categories ) and sets them onto a post (the ID of the post is in $created_post_id ).

	//Set post categories
	/**
	 * WP doesn't allow us to set categories as easily as tags. With tags we can simply declare 
	 * the names of the tags and be done with it. WP requires that we pass in the ID of the category, 
	 * and if the category doesn't exist, we need to create it. So we loop through the array of 
	 * category names, see if they exist (and if so, collect the ID) and if they don't exist, 
	 * create the category and map the ID to the post.
	 **/
	 foreach ($to_post_categories as $category_name) {
		 $category_id = get_category_by_slug($category_name);
		 if ($category_id == false) {
			 //Category doesn't exist, create it
			 $category_id = wp_create_category($category_name);
		 }
		 
		 //category_id now holds ID of right category or recently created category.
		 //Add category to post
		 wp_set_post_categories($created_post_id, array($category_id), true);
	 }//end looping through post categories sent to us

Python SQLite Table Creation Template

I often use the Python sqlite3 module: it helps save time during development as it’s a lightweight SQL engine. Even in production, some small applications can get away with running SQLite instead of a more normal SQL application.

To create a table in sqlite:

import sqlite3
def create_table():
    create_table_sql = """CREATE TABLE tweets (id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL UNIQUE,
     posted_date DATETIME, tweet_text VARCHAR(300),
     user VARCHAR(20), retweet_count int, favorite_count int,
     original_tweet_id VARCHAR(20)
     original_user VARCHAR(20));"""
    conn = sqlite3.connect("example.db")
    c = conn.cursor()
    c.execute(create_table_sql)
    conn.commit()
    conn.close()

And to execute operations against the created table, you simply need to connect to example.db and run c.execute:

# Execute into sqlite
conn = sqlite3.connect("example.db")
c = conn.cursor()