PhilipMat

See Unread Emails in iOS Mail App

I don’t aim for Inbox Zero, but I aim to read all the emails I receive, however, with default setting the iOS Mail app does not make it easy.

For a while, it seemed that the only way to find unread emails and get that red badge to disappear was to scroll through the inboxes and try to find the emails with a blue dot to their left.

Turns out there’s a better way and that indeed the iOS Mail app has a folder aka smart mailbox which tracks all unread emails across all your accounts.

To enable this mailbox:

  1. In the Mailboxes screen (the one with all the folders), tap the Edit button up in the top-right corner;
  2. The screen switches to edit mode, displaying a list of current (visible) and available mailboxes. From this list select the Unread mailbox; I also like to drag it all the way up to the top.
  3. Tap the Done button and you’re done!

Here’s a visual that shows these steps:

Steps to enable smart mailbox

The configuration does not sync across devices, so if you also have an iPad, you’ll need to repeat these steps on those other devices.

PUT and DELETE with HTML Forms in ExpressJS

In trying to send data from an HTML form to an ExpressJS backend I soon discovered two things:

  1. FORM elements only support GET and POST, not PUT, nor DELETE;
  2. ExpressJS does not have a built-in way to address this but fortunately there’s a middleware plugin, method-override, that does.

method-override’s documentation is great and very much on point, except for the FORM post override section, which I found a bit confusing on my first read (although in honesty, now it seems much clear).

As a result, and for my own clarification, I decided to put together as small example that showcases all the ways method-override can be used from within HTML pages.

General approach

Middleware in ExpressJS typically takes place early in the processing pipeline and as such method-override will attempt to identify a specific token in the received data, per configuration, and will, well, override the method aka HTTP verb used.

There are two main approaches we can use to trigger the app.put() and app.delete() route handlers from HTML code:

  1. Using AJAX
  2. Using the form’s method="POST" with a specific token.

Using AJAX

Almost all current versions of browsers support specifying an HTTP method.

// client code
var xhr = new XMLHttpRequest();
xhr.open('PUT', '/resource', true);
xhr.send();

// ---
// server code
app.put('/resource', function(req, res) {
    console.log('PUT to /resource');
});

If you only work with modern browsers, there’s nothing more required.

However, if your front-end needs to be backward compatible with older versions that don’t support HTTP methods (enterprise software developers, I feel you), then method-override can be configured to look for a token in the headers being posted and override the method being used.

// server code
var methodOverride = require('method-override');
app.use(methodOverride('X-HTTP-Method-Override'));
app.put('/resource', ...

As such, the client code needs to specify the intended method in a header:

// client code
var xhr = new XMLHttpRequest();
xhr.open('POST', '/resource', true); // method-override needs it to be POST
xhr.setRequestHeader('X-HTTP-Method-Override', 'PUT');
xhr.send();

POST-ing with specific token

If instead of AJAX, we intend to use the HTML’s FORM element to PUT or DELETE, method-override can be configured to look for a specific token either in the query string, or, with a tiny bit more code, in the data being submitted.

Specific Token in Query String

In this example we will have the form POST to /resource?_method=PUT and will configure method-override to look for _method in the query string and override the HTTP method with the indicated verb.

// server code
app.use(methodOverride('_method'));
app.put('/resource', ...

On the client side, we’ll POST to the above URL:

// client code
<form method="POST" action="/resource?_method=PUT">
...
</form>

Note: by default method-override only examines POST requests.

You can configure it to look at GET requests:

// server code
app.use(methodOverride('_method', { methods: ['POST', 'GET'] });

but this is a really bad idea for two reasons:

  1. GET-ing /resource?_method=DELETE is downright dangerous as anything unintentionally (or worse, intentionally) crawling your URLs will cause deletion of resources; this could be something as trivial as a browser pre-fetching links in order to speed up pages, or a browser extension investigating URLs for any number of reasons (phishing protection, status checking, statistics, etc).
  2. GET-ing /resource?_method=PUT with a payload makes no sense from an HTTP standard perspective. Payloads are for POST and PUT.

So just don’t.

Specific Token in Form data

The second method involved sending the token with the POST body. The most common approach is to include a hidden field.

<form method="POST" action="/log" enctype="application/x-www-form-urlencoded">
  <input type="hidden" name="_method" value="PUT">
  <button type="submit">Submit</button>
</form>

The server-side code is a little bit more involved this time as it requires another library: body-parser, which you’re likely to use anyway if you deal with form data or just HTTP body data in general.

body-parser inteprets the incoming HTTP request body and makes it available as key-value pairs in the body property of the request, req, parameter.

method-override allows one to specify a custom function to be called during the middleware execution. In this custom function we will inspect the request body, req.body, for the presence of the desired token, _method in this case, and return its value to the middleware that will in turn override the request method.

// server code
var bodyParser = require('body-parser');
app.use(bodyParser.urlencoded({ extended: false }));
app.use(methodOverride(function (req, res) {
  if (req.body && typeof req.body === 'object' && '_method' in req.body) {
    // look in urlencoded POST bodies and delete it
    var method = req.body._method;
    delete req.body._method;
    return method;
  }
}));

Composition

As a final note, it’s worth mentioning that you can have multiple method-overrides in your middleware code, thus allowing the handling of all the scenarios presented above.

Except for the GET+DELETE scenario. You should never do that.

// server code
var express = require('express');
var bodyParser = require('body-parser');
var methodOverride = require('method-override');

var app = express();

app.use(methodOverride('X-HTTP-Method-Override'));
app.use(methodOverride('_method'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(methodOverride(function (req, res) {
  if (req.body && typeof req.body === 'object' && '_method' in req.body) {
    var method = req.body._method;
    delete req.body._method;
    return method;
  }
}));

app.get('/resource', ...)
   .get('/resource/:id', ...)
   .post('/resource', ...)
   .put('/resource/:id', ...)
   .delete('/resource/:id', ...);

If you want to see all the examples above in action, clone my method-override-example repo and simply run npm start then navigate to http://localhost:3000/ to play with each of these scenarios.

django-permissivecsrf

I bet you’re here because you’ve run into the puzzling Django error

CSRF verification failed. Request aborted.

Reason given for failure:
    Referer checking failed - http://<domain>/ does not match https://<domain>/.

I’ll double down that this happens when you try to make your login form post to HTTPS, perhaps to https://<domain>/accounts/login/.

If I’m mostly right, you’ve run into the problem I’ve solved. If I’m not right and you somehow ended up on this page, I’d love to hear about it.

Short version

In the CSRF middleware, Django does an extra check when a request comes over HTTPS to ensure it comes from the same site (same origin check).

If any portion of your site benefits from HTTPS, you probably should run your entire site over HTTPS. You can use django-sslify to force your website to operate in HTTPS mode all the time.

Furthermore, if you want to disable CSRF checking for your own views, there are other methods you can use, for example the @csrf_exempt decorator.

However, if the views are not under your control and you are comfortable n trading some security for not really that much convenience, you can use django-permissivecsrf to work around this error.

Use the instructions in the README file on GitHub or on PyPI to get this project up an running. Mostly it consists of installing django-permissivecsrf and adding 'permissivecsrf.middleware.PermissiveCSRFMiddleware' to your MIDDLEWARE_CLASSES entry.

Enjoy.

Long version

The gist of why this happens is explained in point #4 of the How it works section of the Django documentation on Cross Site Request Forgery (emphasis mine):

4) In addition, for HTTPS requests, strict referer checking is done by CsrfViewMiddleware. This is necessary to address a Man-In-The-Middle attack that is possible under HTTPS when using a session independent nonce, due to the fact that HTTP ‘Set-Cookie’ headers are (unfortunately) accepted by clients that are talking to a site under HTTPS. (Referer checking is not done for HTTP requests because the presence of the Referer header is not reliable enough under HTTP.)

In other words, because the HTTPS headers are encrypted, the HTTP-Referer header is resilient against MITM attacks, so it can be safely used to check and make sure the CSRF cookie is originated by the same site that served the page and that the referring page has also been served over HTTPS, which means that page has also been protected against header injections.

The same check could be made on HTTP calls as well, but since HTTP headers are not encrypted, they could be easily faked and thus the check would be a useless placebo.

This explanation is also present, in comment form, in this f92a21daa7 commit by spookylukey aka Luke Plant, and further detailed by him in a reply to a complaint about the strictness of CSRF Referer check on the django-developers maillist.

How django-permissivecsrf works

The Django CSRF middleware performs an extra-check if the request is over HTTPS to ensure that the request came from the same site, i.e. that the referrer (HTTP-Referer header) matches the current site, and that the schema of the referrer is also HTTPS.

In other words, in ensures that the call to https://example.com/account/login came from another page of https://example.com/. As such, if you put your login form on your non-secure homepage, http://example.com/, but use a secure target for your form’s action attribute, <form action="https://example.com/account/login" method="POST">, Django’s check will fail because::

'http://example.com/' != ('https://%s/' % request.get_host())

However, Django will not perform the CSRF check at all if the request object has an attribute _dont_enforce_csrf_checks set to True. That’s what PermissiveCSRF relies on: if the request came from the same site, regardless the schema, it sets _dont_enforce_csrf_checks to True, thus telling the Django CSRF middleware to skip the CSRF check for that request.

This only happens if:

  • DEBUG == True. Your production server should always be HTTPS;
  • The HTTP-Referer header is present;
  • The request is for an HTTPS URL (i.e. request.is_secure() == True);
  • and the referrer uses HTTP.

In all other cases it defers to Django for normal processing.

Bottom line

There’s only one thing to take away from all this: in production use HTTPS (see django-sslify). Period.

Miniature Minifier Makefile

A solution that uses only make and curl:

# c/o: http://wonko.com/post/simple-makefile-to-minify-css-and-js
# Patterns matching CSS files that should be minified. Files with a -min.css
# suffix will be ignored.
CSS_FILES = $(filter-out %.min.css,$(wildcard \
	css/*.css \
	css/**/*.css \
))

# Patterns matching JS files that should be minified. Files with a -min.js
# suffix will be ignored.
JS_FILES = $(filter-out %.min.js,$(wildcard \
	js/*.js \
	js/**/*.js \
))

# Commands
CSS_MINIFIER = curl -X POST -s \
    --data-urlencode "input@CSS_TMP" \
    http://www.cssminifier.com/raw

JS_MINIFIER = curl -s -X POST \
    --data-urlencode "js_code@JS_TMP" \
    http://marijnhaverbeke.nl//uglifyjs 

CSS_MINIFIED = $(CSS_FILES:.css=.min.css)
JS_MINIFIED = $(JS_FILES:.js=.min.js)

# target: minify - Minifies CSS and JS.
minify: minify-css minify-js

# target: minify-css - Minifies CSS.
minify-css: $(CSS_FILES) $(CSS_MINIFIED)

# target: minify-js - Minifies JS.
minify-js: $(JS_FILES) $(JS_MINIFIED)

%.min.css: %.css
	@echo '  Minifying $< ==> $@'
	$(subst CSS_TMP,$(<),$(CSS_MINIFIER)) > $@
	@echo

%.min.js: %.js
	@echo '  Minifying $< ==> $@'
	$(subst JS_TMP,$(<),$(JS_MINIFIER)) > $@
	@echo

# target: clean - Removes minified CSS and JS files.
clean:
	rm -f $(CSS_MINIFIED) $(JS_MINIFIED)


# target: help - Displays help.
help:
	@egrep "^# target:" Makefile

You can also find the latest version on GitHub part of my LeaseMilesTracker project.

How Does It Work?

Surprisingly simple.

  1. We make a list of all CSS files, filtering out the *.min.css files;
  2. we ask cssminifier.com to make us a minified version;
  3. then we save the latter to a file ending in min.css.

Same with the JavaScript files, but using the service provided by UglifyJS

The obvious targets are minify-css, minify-js, and minify which is dependent on both. The less obvious targets are %.min.css and %.min.js whose purpose is to cause the re-minification of the CSS or JS files, if the normal files are newer than the minified version.

Finally, clean will remove all the minified files, that is all *.min.css and *.min.js files.

Why Would I Use This?

Ryan Grove’s “Simple makefile to minify CSS and JS”, the original inspiration for my Makefile, is an excellent solution if you’re using the YUI compressor, but YUIc requires Java (and so does Google’s closure-compiler) and I was interesting in having this not only work without Java, but more important work on a system that would require no special tools beyond what you’d find standard on most *nix distros.
After all this is 2012 and we’re supposed to be using RESTful APIs and what not.

Oh, and also Mountain Lion doesn’t come with Java installed.

Kendo Grid Select Editor

I’m going to assume you’re reading this post because you’d like to find out how to use a regular HTML SELECT as an editor in a Kendo grid, and that you already know why you want to do that - you just had a hard time figuring out how to.

If you just want to see the code and don’t care about the explanation, here is a jsFiddle to play with, also a gist for your forking pleasure.

If you wonder why you’d want to do it, read the Why would I want to do that? section that come back for the how.

I see two primary scenarios for using a SELECT aka drop-down list.

Simple Field - Extra Info

This would be the case where, for example, your model contains a field to store the user login (e.g. foo), but when displaying the drop-down to select the user you’d like to also list their name (e.g. foo - Foo Frye).

Let us assume you have the mapping of login to full name in an array of objects, and for genericality’s sake let’s assume it has a format similar to the following:

var nameList = [
  { Login: 'foo', Description: 'foo - Foo Frye' },
  ...
]

When you create your grid and specify the columns, you have the chance to pass in a function that would create a custom editor instead of the text box the grid gives you by default:

$('#grid').kendoGrid({
  ...
  dataSource: {
    data: [
      { ..., Login: 'foo', ... }
    ]
  }
  columns: [
    ...
    {
      field: 'Login',
      editor: function(container, options) {
        var s = $('<select ' +
                  ' data-bind="source: listSource.list, ' +
                    'value: ' + options.field + '" ' + 
                  ' data-text-field="Description"' + 
                  ' data-value-field="Login"' + 
                  '/>')
        options.model.listSource = kendo.observable({list: nameList})
        s.appendTo(container)
      }
    }
    ...

What’s going on here?

  1. We create a drop-down $('<select/>')
  2. Using the data-bind attribute we tell Kendo to use a specific source for the drop-down and to write the selected value to the underlying model’s field (here options.field == 'Login').
  3. We tell it to use the Description property of each element in the source for the text portion, and the Login field for the value. This would be as if we had an <option value="foo">foo - Foo Frye</option>.
  4. We create a kendo.observable containing the list of names.
  5. We append the HTML select to the container, which, if you’re curious, is the actual grid cell (TD).

There are only two tricky parts here and they both gravitate towards the use of the source directive: options.model is the object from your data source that correspond to the grid row; the source: listSource.list will look for a property called listSource on the grid row/object/model and expect it to
a) be a kendo.observable and
b) contain a sub-object named list.

Complex Field

What if your Login field was a complex object, maybe similar to one of the entries in nameList? If this was the case, how would you go about displaying it in a SELECT, but have it write back to your field the same complex object, not just a simple string.

The code looks similar, in a way even simpler:

$('#grid').kendoGrid({
  ...
  dataSource: {
    data: [
      { Complex: { Login: 'foo', Description: 'foo - Foo Frye' } },
    ]
  }
  columns: [
    ...
    {
      field: 'Complex',
      editor: function(container, options) {
        var s = $('<select ' +
                  ' data-bind="source: listSource.list, ' +
                    'value: ' + options.field + '" ' + 
                  ' data-text-field="Description"' + 
                  '/>')
        options.model.listSource = kendo.observable({list: nameList})
        s.appendTo(container)
      }
    }
    ...

The most significant difference from the simple example is the absence of the data-text-value attribute. This causes the entire underlying record to be written back into the Complex model field.

That’s it. If there was a simpler way to do this, I couldn’t find it.

Why Would I Want to Do This?

If you use Telerik’s excellent Kendo stack, you already have access to a nice drop-down editor that is part of the suite. The Kendo UI demos even show you how to use the Kendo DropDownList as a custom editor, and use it you should for it has a plethora of features.

For all its niceness, the DropDownList is built upon styled UL/LI elements, and that means it is missing a few features that a normal drop-down would have, chiefly the ability to use keyboard navigation and the two significant advantages that come with it: using keys to trigger and navigate the list (Alt-Down on Windows and Spacebar on Mac), and type-to-select when focused (for example typing Tex or TT to select Texas in a list of states).

That and the fact that it doesn’t play 100% nice with Twitter Bootstrap, in the sense that the styles don’t quite match. Oh, and on a mobile browser you don’t get the native picker.

Fortunately, the Kendo grid allows you to specify a custom editor for a column; unfortunately, the Kendo stack tends to make use of and wrap your SELECT in a kendo.observable type of object. I suspect that this is the main reason why examples of using normal HTML inputs with the Kendo grid tend to be scarce on the web.