Templating Engine for HTML (special case AMP)

I’m usually working with Angular. But recently I started an AMP Project (www.tsmean.com). For those of you who don’t know AMP, it’s short for “Accelerated Mobile Pages” where you use only the “Official AMP JavaScript Libraries” but no JavaScript of your own and only prerendered HTML. The goal of this is to make webpages blazingly fast. Since a requirement is precompiled html I can’t use Angular as a templating engine since that dynamically compiles the html. Even though it has a static option with Angular Universal (don’t get me started on this…), the Angular Universal rendered page still includes JavaScript so it’s not possible to render an AMP compatible (=JavaScript-free) page with it.

So I’m kind of back in the “dark ages” where I’m stuck with templating engines such as Mustache, Handlebars, EJS or Pug (former Jade). Why I need a templating engine should be obvious: I don’t want to copy / paste code snippets all around and duplicate them like crazy. So I need a templating engine. Just which one’s right? So I set out to choose my templating engine for html. Which has been an absolute nightmare. Let’s review our options.

Mustache

Mustache, we’ve all heard of it. Logic-less templating. So the entry point to the “mustache experience” is this one: https://mustache.github.io/

Since mustache is basically just a spec, it can be implemented in any language. That’s why you see so many languages there. I first clicked on the node.js link which leads to this godforsaken project I’ve used now for a quarter year: https://github.com/raycmorgan/Mu

The owner of the repo doesn’t seem to give two sh*ts about it. Last commit somewhen in 2016. Four unmerged pull requests https://github.com/raycmorgan/Mu/pulls. One of them is mine. No response whatsover.

Then I tried the one when clicking on “JavaScript” in mustache:  https://github.com/janl/mustache.js . Aha, that seems to be the right one with 11k github stars! Okay, I can’t concentrate because of that gif with the dude with the mustache. Is it just me, or is it really annoying to put that gif there? Maybe I’m just having a bad day. But then again, the command line tool also doesn’t seem to be ideal for me because I have to specify all used partials:

mustache -p path/to/partial1.mustache -p path/to/partial2.mustache dataView.json myTemplate.mustache

Can someone explain to me why this is necessary? The partials are already specified in the template! The first mustache compiler I mentioned didn’t require me to pass in all partials at compile time. I mean my template.mustache looks like this:

blabla
{{ > path/to/partial.html }}
blablub

Why do I have to pass this partial as a compiler option again? Especially if you have many views with many partials this becomes very tedious. And if the partial uses another partial? How am I supposed to track the usage of my partials? Unless … I don’t track it and just include all my partials everywhere. Then I’ll just have to register them ONCE in a single file, which is okay.

For those of you coming from Angular like myself, you’ll ask how can I pass data to partials?! You’d expect a syntax like

{{> mybutton title="Cancel" }}
{{> mybutton title="Save" }}

but this doesn’t exist. Instead you’ll set the variable inside the partial by the context you surround it with. An example:

{{#buttonOne}}
  {{> mybutton}}
{{/buttonOne}}
{{#buttonTwo}}
  {{> mybutton}}
{{/buttonTwo}}

and in the data.json you’d specify

{
  "buttonOne": {
    "title": "Cancel"
  },
  "buttonTwo": {
    "title": "Save"
  }
}

In the partial there could be the code

<button class="my-cool-button">{{title}}</button>

The result would print

<button class="my-cool-button">Cancel</button>
<button class="my-cool-button">Save</button>

It’s a bit of a change a first, but workable.

On a sidenote, there’s another small issue problem with the command line tool. The doc’s state:

The command line tool is basically a wrapper around Mustache.render so you get all the features.

But then their example is actually a demonstration that this doesn’t work:

var view = {
  title: "Joe",
  calc: function () {
    return 2 + 4;
  }
};

var output = Mustache.render("{{title}} spends {{calc}}", view);

where the view object is what’s supposed to go to  dataView.json (see above) which is obviously not possible since a JSON can’t hold a function value…

Handlebars

Remember how I said my project can’t have any extra JavaScript libraries? Now, the precompilation of handlebars seems to be so stupid, that it still requires a JavaScript library on runtime:

<script src="/libs/handlebars.runtime.js"></script>

Duuuudes, come on, you’re killin me…

Pug

Since the name “Jade” was copyright infringement, they changed it to pug. In case you don’t know it, it’s a html templating engine, that doesn’t look like html at all. It’s whitespace sensitive and you’ll have to learn a new language. For me, I don’t like it that they deviated so much from the standard html for a couple of reasons:

  • No copy / pasting of code snippets from nowhere (one might say that’s a actually a good thing…)
  • You’ll have to learn a “new language”
  • All team members will have to learn a new language.
  • Larger chance of getting out of fashion than when sticking with something similar to html like mustache (yet small chance, since it’s been here for a while and has too many users to be completely abandoned)
  • I don’t really find the whitespace based syntax better
  • My IDE doesn’t do it’s job as well

So all in all I have very little incentive to use pug, I just see a point of deviating that much from HTML.

EJS

EJS has horribly little documentation: http://ejs.co/

It doesn’t seem to be what I’m looking for: A command line tool to precompile some html templates. Rather you put your html as strings into into JavaScript files, which neither me nor my IDE like very much. As for storing html snippets in a separate file, then requiring them through JS, then paring to a string, and then storing that string again in a file, that’s just too complicated.

Conclusion

Mustache is the winner. Clean syntax that integrates well with html. Large community supporting it. Been here for a while. Not going anywhere anytime soon. Once you get it, you got it. However, it took me a while getting it, since the approach to handling templates and data seems weird when you come from Angular. This article on “passing variables to partials” has been a bit of an eyeopener for me, so thanks Pawel!

Blocking Browser APIs

Tip of the week:

Don’t use alert, confirm or prompt browser APIs in your production code. They are blocking and thus problematic for performant websites.

Host websites with high availability and low latency for less than 1$/month, SSL included

I’ve been deploying static websites the wrong way all of my life. My procedure used to look like this:

  • Choose a server (EC2, Digitalocean) in a region (US, Europe, …)
  • Set up some nginx / apache configuration
  • Maybe add SSL through letsencrypt

I’ve always been aware that there are several drawbacks to this:

  • The smallest EC2 / digital ocean instances cost ~5$ which is quite a lot if you have many websites.
  • To make this cheaper, you start putting everything on one server (e.g. one ec2 small instance for 20$ / month), but if this instance crashes ALL your websites are down. You’ll have a very fragile availability.
  • By selecting a server in a specific region, the other regions have high latency
  • Adding SSL is kind of a pain, even if it’s free through letsencrypt
  • It’s not scalable
  • You need to mess with complicated config files from nginx / apache and need to ssh into your instance.

I’ve just never been aware that there’s a much better and easier solution!

The solution is called … (drumroll) … : AWS CloudFront.

Or to be less product specific, the solution is called: Put a CDN in front of some highly available data store.

Here’s what that means:

Your users will access the content through servers in the region closest to them.

This eliminates all of the previously mentioned drawbacks:

  • It’s cheap: AWS charges less than 1$/month for this setup
  • It’s got high availability: S3 has 99.9999999% uptime (I’ve randomly added 9’s)
  • It’s got low latency: Content is served through the data center closest to you.
  • You get a free SSL certificate from AWS that you can easily, no config file hacking.
  • AWS takes care of scaling for you
  • It’s simple to set up & deploy!

How to set up AWS CloudFront

AWS CloudFront is pretty simple to set up, but there are a few pitfalls, so here’s a tutorial with pictures to guide you.

Step 1: Host website with AWS CloudFront

Go to https://console.aws.amazon.com/quickstart-website/home; you also have that link on your dashboard.

Next, fill out the wizard:

and hit create. Your website needs to have an index.html file for this to work.

Tada. You have a working hosted website at some crazy url.

Step 2: Setup domain

Hit “Buy domain” on the website dashboard. This is the right button even when you’ve already bought you’re website through Route 53.

If you already have a domain, you can simply select it here:

Step 3: Set up SSL

Back on the website’s main page, select “Manage settings in Amazon CloudFront”:

There, click edit:

and select Custom SSL Certificate:

If you don’t have one yet, you can request a new one. It’s easy to set up, but you’ll need to have a working email address for the domain. E.g. when you want to set up example.com, you need e.g. admin@example.com to be working. I’m not going too much into detail here, since that’s a topic on it’s own.

You can also forward HTTP to HTTPS in the CloudFront > Behaviors Tab: Edit the default switch to “Redirect HTTP to HTTPS“.

Once the certificate has been issued and connected, SSL works:

Step 4: Pretty URLs and Subdirectories

By default, the files are hosted as they are: example.com/articles/my-article.html. Now how can you get rid of the html extension? You can either just remove the html extension or have a folder example.com/articles/my-article/index.html.

Unfortunately, there’s also one setting that needs to be changed for this to work. Head over to s3 > website bucket > properties > static website hosting and copy the “Endpoint” URL:

Then in Cloudfront > Origins

select the right website and hit edit, then replace the “Origin Domain Name” with what you’ve copied from the S3 bucket (without the http://).

Congrats, you’re all set up!

Step 5: Deployment

It’s a bit annoying to always have to upload a zip if you’re more the command line type of guy. Fortunately, it’s still possible to upload through command line with AWS cli:

aws s3 cp dist s3://<bucket-name>/ --recursive

You’ll need to set up IAM for this to work. Create a new user, give him S3 access, and hit:

aws configure

once the aws command line cli is installed.

Additional Tips & Tricks

There are a few gotchas you’ll have to be aware of when setting up CloudFront.

Forwarding the Naked (Apex) Domain to the WWW domain

In order to achieve a forward from https://example.com to https://www.example.com you’ll have to use a really weird workaround: Create a new S3 bucket with the name example.com. It HAS to be example.com, you can’t give the bucket any other name. Then in properties > static website hosting redirect all requests:

Finally, in Route53 resolve example.com (or here tsmean.com) to this S3 bucket:

If the S3 bucket name IS EQUAL TO THE RECORD NAME, the S3 website bucket should show up when you select Alias Target. That’s it, that way requests to the naked (apex) domain can be forwarded to a www domain.

Cache Invalidation

If you’ve followed the “one server deployment paradigm” so far, you’re probably used to seeing your changes visible immediately after deployment. This is not the case when using AWS CloudFront. Since it’s a distributed system with servers at edge locations that cache your assets, those caches have to be invalidated first. AWShas two articles on this, first http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html and second http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ReplacingObjectsSameName.html.

They advise you against using their built-in cache invalidation mechanism since it becomes expensive above a certain number of requests. However, this free limit is 1000 requests per month as of August 2017. That’s more than enough for most people since you can declare wildcard paths and those are counted only once, even if thousands of requests are made! So you could just specify “*” as invalidation and your cache is cleared with one request:

I personally prefer this to versioned object / directory names since I find it easier.

You can also easily include this into a deploy script, that could look like this:

#!/usr/bin/env bash

# Prerequisits:
# 1) AWS CLI installation (pip install awscli --upgrade --user)
# 2) AWS CLI Login ("aws configure")
# 3) aws configure set preview.cloudfront true

CDN_DISTRIBUTION_ID="<replace with your cloudfront ID>"
S3_DIST="s3://<replace with your bucket name>/"

aws s3 cp <replace with your dist folder location> $S3_DIST --recursive
aws cloudfront create-invalidation --distribution-id $CDN_DISTRIBUTION_ID --paths "/*"
echo "Done!"

Single Page Applications

If you’re hosting a SPA (Single Page Application), you’ll run into the problem that upon reloading a non-root page (e.g. bla.com/team), you’ll get an error. To fix this, you have two options.

  1. In Cloudfront:
    – Go to Error Pages tab, click on Create Custom Error Response:
    – HTTP Error Code: 403: Forbidden (404: Not Found, in case of S3 Static Website)
    – Customize Error Response: Yes
    –  Response Page Path: /index.html
    – HTTP Response Code: 200: OK
  2. https://stackoverflow.com/a/16877231/3022127

Changing the URL (in case you need to)

If you ever want to migrate, e.g. from dev.bla.com to test.bla.com, you’ll need to:

  1. In Cloudfront, add a cname. To do so, edit the General Settings, it’s the first tab.
  2. Change in Route53
  3. Update AWS_WEBSITE_DOMAIN_NAME und S3 > website bucket > properties > tags. (for the sake of completeness)

Compressing Assets / Gzip

In order to speed up your page, it’s a good idea to compress your assets. To do so

  1. Go to CloudFront > Behaviour Tab.
  2. Select the Default Behaviour and click Edit
  3. Select “Compress Objects Automatically

Checklist

Here’s a short checklist whether you’ve completed all important steps:

  • http://www.your-domain.com is working
  • https://www.your-domain is working (SSL)
  • http://your-domain.com is working (naked)
  • https://your-domain.com is working (naked, SSL)
  • https://your-domain.com/subdirectory is working
  • Compression is enabled

Conclusion

Compared to AWS CloudFront, deploying to single instances just feels hacky and wrong now. It’s simpler, cheaper and safer with AWS CloudFront. It’s the solution I’ve always wanted, but didn’t know existed. Hope this tutorial helps someone!

Convert Latex to HTML online

Yesterday I wanted to convert a snippet of latex code to html. Unfortunately, the results when Googling for “latex to html online” are only cr*p. I knew a tool that was really great at converting latex files to html files. This tool is called Pandoc. However, it seemed like a bit of overhead to always create a file, store the latex snippet, remember what the pandoc command was to convert, enter the entire command (e.g. pandoc mysnippet.tex -o mysnippet.html), open the html file and get the snippet… That’s why I’d much prefer an online tool that does this chore for me. Since I couldn’t find any, I implemented it myself.

This is the result:

It’s open sourced, the code can be found on Github at https://github.com/bersling/latex2html.

It’s a small self-contained piece of code with one html file and one TypeScript file at its core.

But here’s the interesting part, if I hadn’t been so bad at googling, I would have found that there is the exact same functionality available at

… So there might be even a better learning for me than the additional coding experience I gained: Before you build something, Google, google, google if it already exists. For example had I tried googling “convert with pandoc online” this converter would have been the first hit. For me, it was a small price to pay, since I built the solution in little time and it was quite fun, but if you’re working on a larger project and didn’t research what’s already out there, you’re gonna have a bad time.

Tips & Tricks for Google Analytics (GA) + Google Tag Manager (GTM) + Accelerated Mobile Pages (AMP)

Setting up GTM + GA + AMP can be a “pull your hairs out” experience. So let me help you with this list of tips, tricks and pitfalls to avoid when setting up AMP with GTM and GA.

Event Logging in Google Analytics

So far the greatest pitfall has been with events. Events have 4 dimensions: category, action, label and value.

Now there is an article about events here https://support.google.com/analytics/answer/1033068?hl=en and it tells you that value needs to be an integer. But it doesn’t tell you what happens when you don’t adhere to value being an integer. What happens is that your event won’t get logged at all, even though your request to google analytics will get a 200 OK. It also doesn’t tell you, that having an undefined variable will result in the same problem. For example, if you have in GTM value = {{someValue}} , but someValue is undefined, the request being sent to GA is https://www.google-analytics.com/collect?…&ev=&… which results in the event not being logged.

Submitting information about a clicked element

Now in the regular tag manager, you get the Element Id / Click Id as built-in variable for clicked html elements. Not so for AMP. I thought you can’t transport information about a html element in AMP through GTM until I found this: https://github.com/ampproject/amphtml/blob/master/extensions/amp-analytics/analytics-vars.md

Somewhere hidden in the darkest corner of some github repository’s readme lies a very important information on how to send information about html elements: through the data-vars-* attribute. So for example you could use data-vars-event-id , which you could then use in GTM’s variables like so:

So the dash-case from the html attribute gets transformed to camel case for the variable configuration. Or as the they put it in the github doc:

The variables passed as data attributes should follow the format data-vars-*.

Example:

<span id="test1" class="box" data-vars-event-id="22"> Click here to generate an event </span>

And in the request url the token would be of the format ${eventId} (follows camelcase).

(BTW: I only stumbled on it by reading through https://github.com/ampproject/amphtml/issues/1298 . Reading through github issues is so much fun…)

Generic Click Trigger

To log every click as an event, you can use the following trigger in GTM:


Subscribe to stay posted.

AMP Project Starter

From the www.tsmean.com project, a nice little side project developed. The homepage of tsmean.com was built with AMP – Accelerated Mobile Pages. Those are pages that load freakishly fast on mobile, and hence also on the web. It’s actually really amazing for the user experience.

AMP has a quite great documentation with ampproject.org and ampbyexample.com. What was still lacking was how to build an actual website. In their example, they always only had single pages, so in order to build an actual website with multiple pages, you’ll have to do the work yourself. It’s not that it’s very complicated, after all it’s just some html and css. However, I found that I’d reuse the same basic setup on all of my amp pages and my friends started to use the same code as well. So I thought why not make an “Amp Project Starter” out of it? It gives you a working app out of the box. You can find more information about it here:

Forwarding Mail with EC2 (Ubuntu) and ElasticIP

Let’s assume you already have bought a domain – in this example we’re using tsmean.com – and you want to forward mail. You can replace all tsmean.com in this tutorial with yourdomain.com. So for example when sending an email to info@tsmean.com it should forward it to bersling@gmail.com. How can we achieve that? We can set up a mail forwarding server using an ec2 instance and postfix. To get started, create your ec2 instance, in this tutorial we’ll use Ubuntu. You can also reuse an existing instance, for example where you host some websites.

Step 1 – Open ports

You need to have port 25 open for emails to arrive at the server. Head over to the security group of the instance and open port 25:

Step 2 – DNS

We need two DNS records. One is the MX record:

Now we need to create the entry for the mailserver mail.tsmean.com. This is an A record pointing to the elastic IP:

So what this means is that:

mail sent to info@tsmean.com -> mail.tsmean.com -> elastic IP

where -> denotes “is resolved to”.

Step 3 – Install Postfix

On your EC2 instance, run

sudo apt-get install postfix

During the installation, choose “Internet Site”. For the mail choose yourdomain.com, not mail.yourdomain.com:

Step 4 – Set Postfix up

Append the following lines to /etc/postfix/main.cf:

...
virtual_alias_domains = tsmean.com
virtual_alias_maps = hash:/etc/postfix/virtual

And in the /etc/postfix/virtual (you’ll have to create it), insert:

@tsmean.com bersling@gmail.com

if you want to forward all mails or use

info@tsmean.com bersling@gmail.com

to forward a specific mail.

Step 5 – Apply the mapping

Run

postmap /etc/postfix/virtual

in the terminal.

Step 6 – Restart Postfix

sudo /etc/init.d/postfix reload

Result should look like this:

Step 7 – Test

Send an email to info@tsmean.com. Check also your Spam & Junk mail folders!

Cute Animals for tsmean!

To cutify the tsmean starter kit I have written a “cute-animals-microservice”. Here’s what it does:

 

Oooooh, that’s damn cute!

Here’s another screencap that shows more directly how the microservice is used:

So what you can see is that the service consists of two parts. The find endpoint returns a path to the animal. The animals endpoint resolves the animal’s file and serves it. And here’s what’s really interesting about it. I needed less that 50 lines of code to implement this! You can check for yourself, here’s the source: https://github.com/bersling/animals-microservice/blob/master/server.js

And of course you can create some cutsypie animals yourself if you wish, just visit

Have a great day & don’t forget to subscribe!

Material Design Notifications for Angular2+

Astonishingly, I haven’t found a good reliable library to display material-design style notifications in Angular2. So I have created this library, it can be found here:

It’s super easy to use, yet flexible. Material design is optional. The Material Design library is not required for this library to work!

You can edit styles like box-shadow, position, background or color. There are also some defaults, such as success or error.

I hope you enjoy it!

TypeScript on the Server

During the past few weeks I’ve been working on a Typescript MEAN seed-application (Mongo-Express-Angular-NodeJS). MEAN used to be popular stack even before TypeScript, but it lost some popularity when AngularJS was replaced with Angular. I gave this stack some more thought today.  Why is it that it uses a framework in the frontend (Angular), but only libraries in the backend?  I mean the frontend is stuffed with Javascript frameworks. You can choose between Angular2, React, Vue or many others. In the backend world we also have frameworks to choose from. You have Django for Python, Spring for Java or Play for Scala. But there isn’t anything anywhere near of being a framework in Typescript or for that matter, neither in Javascript. Hasn’t nodejs been somewhat of a hype now for a while? Why hasn’t anyone built a substantial framework for this?

For example the tech giants, Google and Facebook, they aren’t interested at all in typescript / javascript server side applications? They took the time to battle against each other with React and Angular2, but don’t want to claim the server side JS/TS framework throne? There’s this big discussion going on “omg node is so amazing for servers, it’s non blocking and so amazing” but no one cared enough to write a substantial framework? I know ONE framework that extends all the way front to back in JS and that is Meteor. Unfortunately it doesn’t do the best job in separating front and back and it’s a bit too much magic for me, so I’d like to see a pure server side framework written in typescript.

So if I’m asking the question “why hasn’t this happened yet”, I’m inferring that there are some benefits of using typescript in the backend. The benefits to me are first and foremost:

  • One language everywhere
  • Package management with npm
  • Vibrant open source community
  • Typescript is a good and flexible language

But of course there are also downsides of javascript on the server. What I see is:

  • Weakly typed (I know, I know, people like to argue about this. So let’s just say that’s a disadvantage for those people that prefer strongly typed languages)
  • Javascript has many “bad parts”. (Weird equality operators, global scope, etc.)
  • Breaking changes all the time in all the packages / frameworks / things

So is the answer to the question just “Why throw out the good old solid backend for a weird language, just so we have it everywhere?!”. But then again, with typescript it’s not really that weird anymore. And if the problem would be with dynamic, there wouldn’t be a python framework either. The point with the most weight here for me is that everything is constantly changing at neck-breaking speed.

There’s also the option “frameworks suck, we can just stich together libraries, that’s much more modular anyways”. Well, if that’s the case, then there’s still the question why there are frameworks for other languages but not for typescript. Apart from that, the host of libraries isn’t really baffling either.

What I see as more likely is that there just hasn’t been enough time to develop such a backend framework using TypeScript. So I forecast that someone in the not too distant future will implement something like this.

UPDATE OCTOBER 2017:

I knew about a small framwork called “Nest” when I wrote this article, but it had below 1000 Github starts and was (and still is) developed by one person, so I didn’t dare to mention it. But it seems to have hit a nerve, apparently many more people are asking the same question about a backend framework written in TypeScript. So it quickly went from under 1000 stars to 1600 as of today, rapidly gaining even more popularity. It’s still quite scary that it’s just backed by one dev, but it’s so well written and solves so many problems, that I still think it’s worth giving it a shot. So without further ado, the TypeScript framework of the future (maybe):