There is no exact match between dependencies in Maven and Gradle, however here’s a rough mapping:

compilecompile
implementationcompile
apicompile
testCompiletest
testImplementationtest
testRuntimetest
testRuntimeOnlytest
compileOnlyprovided
testCompileOnlyprovided
runtimeOnlyruntime

iCloud vs GDrive vs Dropbox vs AWS Glacier

I’ve compiled a small table for myself to check what’s best suited for me. Thought I’d share if someone tries to make the same decision.

 iCloud GDrive Dropbox AWS Glacier Possible to store in Cloud but not on local machine? Yes (append .nosync to file or folder in iCloud-WebView) Yes (pref -> google drive -> only sync these folders…) Yes Yes Share Files by Link Yes (“Add People”, works local and in webview) Yes Yes No Price for Storing 100 GB / month 3 2 10 0.5 Price for Storing 200 GB / month 3 10 10 1 Price for Storing 1 TB / month 10 10 10 5 Price for Storing 2 TB / month 10 20 20 10 Product Discontinuation Risk Low Low Medium Low Tool for managing on Mac iCloud Pref Pane, Context Menu, iCloud Drive Backup and Sync App Dropbox App Freeze for Mac ($30) or CrossFTP Configurable which folders to sync Semi: iCloudDrive synced (if selected), also an option to sync “Desktop & Documents” Yes (pref -> google drive -> only sync these folders…) No (just the dropbox folder is synced) Possible to sync files? Yes Yes Yes No Retrieval costs / 100 GB 0 0 0$10 Lock In High Medium Medium Huge (because retrieval is so expensive) Need to switch when I switch to Linux? Yes No No No Quality of the Web Interface Medium High Don’t know Low

iCloud

Most things with iCloud are fine and the functionality to backup your iPhone and sync photos when you’re a Mac user are of course cool. However, it’s not the best solution if you want to free disk space on your mac, since everything on iCloud is also synced on your Mac. You could use the ‘myfolder.nosync’ trick, but since the web-interface of iCloud isn’t that good it’s a bit problematic. For example, if you have a lot of photos, that you’d like to have in the cloud only and you have folders like 2015, 2016, 2017, 2018 on your local machine, each with sub folders of events. Then moving this to iCloud is a bit an awkward process, since you can’t just drag and drop folders to iCloud (it’ll say that’s not possible through the web interface). So you’ll have to move those directories locally to iCloud, and then add the nosync extension. If you had the photos on an external harddrive and they consume more disk space than you have on your machine, you’re a bit out of luck.

GDrive

I use Gdrive every day for my work, including Docs, Sheets, Drawings and everything. The web interface sometimes is a bit slow, but all in all solid. You can also upload directories, so it would be easy to move things from an external drive to GDrive by drag and drop.

Dropbox

I really don’t have much experience with Dropbox, so I’ll leave this part to someone else.

AWS Glacier

For the love of god, don’t use this as your personal storage solution. Downloading stuff is really slow (5h queue) and expensive, so not worth it unless you know what you’re doing!

Summary

For me as a Mac User a good setup is the following combo: get the 200 GB from iCloud to back up my Desktop, Documents, Photos and my Phone. To free disk space on my Mac, I’ll consider purchasing more space in GDrive, once I need it.

Caching with Gradle in Gitlab with AWS Autoscale

So you have a Continuous Integration Server that runs some build tasks. For example, you’re running ./gradlew buildTaskOne. Now, if you run ./gradlew buildTaskOne again you don’t want it to build again if nothing has changed, you just want it to say “up-to-date” and then move on to the next task. On your local machine this goal is achieved relatively straightforward. For Java builds this works out of the box, for other builds a small setup like the following is required:

task installNodeModules(type: Exec) {
commandLine 'npm', 'i'
inputs.files 'package.json'
outputs.dir 'node_modules'
}

The fist time this task is ran this is executed, the second time it is skipped:

That was easy! So how can you achieve the same thing when using gitlab runners in combination with AWS Autoscale? Some more steps are required to achieve this…

Set Up Distributed S3 Caches

AWS Autoscale uses distinct EC2 instances for each build stage that are dynamically spun up and torn down. This means that for sharing caches, we need a central file store. And what could be better suited for this task than S3? Also, Gitlab integrates really well with this. Simply put the following in your config.toml of your AWS Autoscale manager (bastion server):

concurrent = 10
check_interval = 0
log_level = "debug"

[[runners]]
name = "gitlab-aws-autoscaler"
token = "${RUNNER_TOKEN}" executor = "docker+machine" limit = 6 [runners.docker] image = "taskbase/gitlabrunner:2.4" privileged = true disable_cache = true volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache", "/builds:/builds"] [runners.cache] Type = "s3" ServerAddress = "s3.amazonaws.com" AccessKey = "${AWS_ACCESS_KEY_ID}"
SecretKey = "${AWS_SECRET_ACCESS_KEY}" BucketName = "your-bucket-name" BucketLocation = "eu-central-1" Shared = true The bold part is interesting for you, I’ve added the rest to give you a bit more context. The red bits need to be adjusted to your setup. You’ll need to create an S3 bucket for this to work first. Set Up Caches in .gitlab-ci.yml You’ll also need to specify what you want to cache in .gitlab-ci.yml. This is minimal setup for the node_modules example: cache: key:${CI_COMMIT_REF_SLUG}
paths:
- ./node_modules # important: don't forget to cache the output!

Those directories will get uploaded to S3. Now I’m not entirely sure about this, but I think the directories have to be within your repository. For example I tried to specify an absolute path /opt/gradle/caches but this didn’t get uploaded! In order to store gradle caches into the .gradle folder, run the following line in a before script part of gitlab ci:

before_script:
- export GRADLE_USER_HOME=pwd/.gradle # there is no $USER_HOME, and /opt/gradle/caches can't be uploaded, so local gradle home is chosen Explanation for the above setup (optional reading) On AWS Autoscale runners you don’t have a $USER_HOME variable and gradle seems to store caches by default into /opt/gradle/caches. You can verify this with the following gradle task:

task showMeCache << {
configurations.compile.each { println it }
}

But when trying to store the cache at /opt/gradle/caches nothing seems to happen, so a local gradle home is chosen. This can be achieved by setting the GRADLE_USER_HOME variable to a location within your repo. That’s what the command in the before script is good for.

It’s also important to cache the entire .gradle folder and not just ./gradle/caches/ since to check whether something is up-to-date gradle checks the file hashes in ./gradle/4.6/fileHashes, at least in gradle version 4.6. I learned this the hard way by running a git diff on the .gradle folder before and after making a change to the file declared in input.files.

Setting Up the build.gradle File

Another important concept when setting up the gradle caching mechanisms is “path sensitivity”. By default, the path sensitivity is ABSOLUTE and not RELATIVE, meaning that when you have a cache at /my/path/bla and then /your/path/bla the cache will NOT take effect. From what I’ve seen the builds on AWS Autoscale for the same branches have the same absolute path so it’s not a problem, but to be sure you could write something like the following:

@PathSensitive(PathSensitivity.RELATIVE)
@InputFiles
FileTree packageJSON() {
return fileTree('./package.json')
}
inputs.files packageJSON()
outputs.dir 'node_modules'
commandLine 'npm', 'i'
}

or

task npmInstall (type: Exec) {
inputs.files 'package.json' withPathSensitivity(PathSensitivity.RELATIVE)
outputs.dir 'node_modules'
commandLine 'npm', 'i'
}

Of course, only apply this if your input files are really path independent. But you can also leave this away and introduce it once necessary.

Checking if Everything is Working

For some reason, the cache doesn’t seem to kick in the first time it should. So when we push changes, and then on the second run it should be cached, the caches somehow don’t work yet. But on the third run, they do work.

Summary

What is minimally required to successfully employ input / output caches are the following steps:

• S3 distributed cache setup correctly
• Gradle Home within repository, tried and tested with .gradle
• Upload outputs of build task to S3 as well. The build task needs an output in order be be cachable.
• Try at least two times to check if caching works, the first time when the caches already should kick in, they don’t work for some reason…

Traefik: “Bad Gateway” for Docker Service on a Swarm Node – what to do?

I recently ran into the problem, that suddenly one of my nodes had a “nervous breakdown” and it wasn’t accessible through traefik anymore. All that was printed on the page a “Bad Gateway”. But I was sure that it was working before, I didn’t change any settings.

I tried various solutions including adding a new overlay network to traefik and the services on the problematic node. Nothing would work. So in the end I drained and restarted the node, after which the problem was gone. The commands to do so on an Ubuntu instance would be:

docker node ls # to find the name/id of the node
docker node update --availability drain worker1

# after all containers are removed
sudo reboot

Afterwards, you can simply rescale/redeploy some of your services to repopulate the instance. There currently isn’t any “rebalance” command, because the Docker team doesn’t like the idea of killing healthy containers too much.

StackOverflow Sorting is Broken

StackOverflow sorting sucks.

1. The accepted answer is always on top. This is a joke. The asker of the question usually accepts the first best thing and then heads off. Thousands of people to come will then see the first best answer on top. For example: https://stackoverflow.com/questions/16267339/s3-static-website-hosting-route-all-paths-to-index-html . How ridiculous is that?
2. Below the accepted answer, it’s sorted by upvotes. That system is also broken, because new answers will have a very hard time of getting more upvotes.

The solution? A smart sorting algorithm, like on Quora or Reddit.

So, StackOverflow, are you ever going to change this?

The Ultimate Mustache Tutorial

Learning Mustache is a lot faster if you understand properly what it’s all about and learn the underlying concepts. You’ll grasp a firmer understanding of Mustache by going through this tutorial, than by directly diving into the official docs, which often seem confusing at first.

What is Mustache?

The first thing to learn is that Mustache is NOT a templating engine. Mustache is a specification for a templating language. This specification can be found here, but don’t read that just yet. You’re not ready yet.

The idea is, that you write templates that adhere to the Mustache specification, that then can be compiled / rendered to create an output.

What’s a Templating Language and what’s it for?

Let’s say you want to send the same email to 100 people, except you want to personalize the salutation with their respective first names. You could replace the first name by some placeholder:

Dear *|FNAME|*

I hearby blablabla ...

Here I have invented (=stolen it from Mailchimp) a syntax for a placeholder: *|FNAME|*

The mustache syntax looks a bit different. This case would be covered by the following template:

Dear {{fname}}

I hearby blablabla ...

As you can see, in Mustache we have curly braces for placeholders, which look like a mustache when rotated 90 degrees clockwise:

Hence the name of the language. There’s more to the Mustache than that, but before diving any deeper into the language, it’s useful to first cover another concept.

By simply having specified a templating language and having written a template doesn’t produce 100 emails for me. Much like with a programming language, just learning the language doesn’t actually mean you know how to run a program. For example you might read in your first JavaScript tutorial that ” var x = 5 + 3 ;” is JavaScript, but you might not know exactly how you could run the code. Similarly with Mustache, you might learn about the language in a tutorial, but have no idea how to actually convert your templates to an output. Or to put it differently: How the heck can I create my 100 emails from 1 single Mustache template? You’ll need a …

Templating Engine

So once a syntax aka. templating language is defined, you need a tool that fills the placeholders in the template(s) with actual data and produces an output. This is called a templating engine:

To strech this point again, Mustache is not the templating engine. It’s the specification for how the template file must look like.

Or to come back to the example with programming languages. Mustache is the language. Not the execution environment. You can learn about the language in a book. But to actually use the language, you’ll need a computer and something that can interpret and run your code. The programming language JavaScript, for example can be ran and interpreted by Firefox, Chrome, Microsoft Edge and Safari, but they all use a different underlying engine to interpret and run it.

What’s the Templating Engine for Mustache?

Like for JavaScript there, isn’t just one thing that can compile Mustache. There’s a multitude of tools out there that can take a Mustache Template, some input data, and produce an output. The multitude of tools is actually overwhelming. There are tools that do their job well. And then there are obsolete ones that are badly maintained and don’t actually adhere to the Mustache specification. So which one should you choose from the multitude of options? The answer is: It depends on where and how you want to use Mustache. I will make some examples here, but I encourage you not to get lost in the actual links to those tools, but rather stick with the tutorial. When you leave now, you’ll be into the nitty gritty of how to use one tool without having the greater picture.

Example 1: You have a Java server and want to send out emails to users. So you use our Mustache Template from above for the emails and extend it a bit. So now you’d like to tell your Java code “hey, I’ve got a mustache template and 3 users that go by the name of [“Hans”, “Gertrude”, “Fritz”], can you send them a welcome mail”? Then your best bet would be to use the most popular Mustache Engine written in Javahttps://github.com/spullara/mustache.java . Why? Because you have the data already in your Java code. The Mustache template file you can load through regular I/O:

// DOESN'T ACTUALLY WORK THAT WAY, BUT WAS EASIER TO ILLUSTRATE
String[] users = {"Hans","Gertrude","Fritz"};
for (String user : users) {
String renderedEmail = mf.compile("emailTemplate.mustache", user);
// ...then send email to user...
}

Example 2: You have a Python server and want to send an E-Mail. Pretty much the same as above, except that you’d use the most popular Mustache Engine written for Pythonhttps://github.com/defunkt/pystache.

Example 3: You just want to print 100 personalized invitations to your birthday party, but have no idea about coding whatsoever (which is obvious, since coders don’t have 100 friends). Then unfortunately Mustache is not a good choice for you, since literally all engines for Mustache are connected to a programming language. There isn’t such a thing that takes an excel list of invitees plus a Mustache template and generates 100 pdfs out of it. At least I haven’t found something like that. Even though in principle it would work since it’s just a case of [Template + Input Data = Output] there just aren’t enough people that want to build their birthday cards that way…

Actually, this example 3 was really educational. We’ve learned that all available Mustache engines are connected to programming languages. Which ones are the available languages?

Here’s a screenshot from the official Mustache page answering that question:

Okay, that’s quite a lot of implementations. As I mentioned, some are good, some are bad. For example, the one listed for “node.js” is not maintained and buggy, while the one under “JavaScript” actually also supports node.js. So as you can see this list isn’t the be-all end-all. Maybe some of the listed links are outdated. Maybe some other ones are not included. At the same time, it’s still the best list that I’m aware of.

So now that we’ve covered the conceptual basics, we can dive deeper into the Mustache Templating language. But let’s first repeat what we’ve learned so far:

• Mustache is a templating language, not a templating engine
• There are tons of implementations for mustache engines, but be careful when choosing since not all are good.

Learning the Mustache Template Syntax

Now for the rest of the tutorial you have two choices. Either you select a templating engine from above and learn how it works. Then you can actually test the examples on your machine, which makes it a bit more tangible. But since Mustache is just a language, you might also be lying at a beach right now, slurping your drink, and reading the rest of the tutorial.

The Basics

We already had saw our very first template above, but let’s repeat it in a similar fashion because it was so beautiful:

Hello {{planet}}
{
"planet": "Earth"
}

We choose JSON to describe our data here, but this would depend on the engine you’re using.

The template plus the data would produce the output:

Hello Earth

Now depending on your tool this output might be stored in a variable of your program or written to the filesystem. For example, the JavaScript mustache engine has two modes: Either it can be used as a command line tool to produce files, or it could be used to dynamically render html files.

No spam. Just great engineering posts.

Conditionals

In Mustache, there are no if and else statements and also no loops. That’s why they call it logic-less templating. At least they’re not directly represented as keywords in the templating language. It is however possible to conditionally show or hide a block of text. And this is how:

Mustache Template:

Hello
{{#myCondition}}
la di da
{{/myCondition}}

Data:

{
"myCondition": false
}

Output:

Hello

Loops through Arrays / Lists

Loops are also not represented directly in the mustache language, but indirectly via the data and how the mustache specification specifies behaviour when the data is an array.

Data:

{
"users": ["Hans", "Fritz", "Geraldine"]
}

Mustache Template:

{{#users}}
{{.}}
{{users}}

Output:

Hans
Fritz
Geraldine

If you want all names on the same line, you’ll have to write {{#users}}{{.}} {{/users}} on one line in the template (and put a space in there for legibility).

The elements could also be objects instead of strings, in which case access is given by just writing the property names into the brackets:

Data:

{
"users": [{
"name": "Hans"
}, {
"name": "Fritz"
}, {
"name": "Geraldine"
}]
}

Mustache Template:

{{#users}}
{{name}}
{{users}}

Output:

Hans
Fritz
Geraldine

Using templates inside of templates: “Partials”

One Mustache template can load another template. The specification states that this should work with the syntax

I am a template and I'm loading
{{> template2 }}
and now I continue

This works fine as long as template2 doesn’t require template1, then you’ll get an infinite recursion which your templating engine can’t handle.

Passing Data to Partials

Passing data to partials is actually a bit tricky and one of the things that took me the longest to figure out. It works like this:

{{#buttonOne}}
{{> mybutton}}
{{/buttonOne}}
{{#buttonTwo}}
{{> mybutton}}
{{/buttonTwo}}

and in the data you’d specify

{
"buttonOne": {
"title": "Cancel"
},
"buttonTwo": {
"title": "Save"
}
}

In the partial there could be the code

<button class="my-cool-button">{{title}}</button>

The result would print

<button class="my-cool-button">Cancel</button>
<button class="my-cool-button">Save</button>


Comments begin with an exclamation point and are ignored. The following template:

<h1>Hello{{! ignore me }}.</h1>

Will render as follows:

<h1>Hello.</h1>

Summarized Learnings

Grasping the concepts at first isn’t easy because there are so many different implementations of Mustache and they’re often confused with the actual Mustache specification. Yet through a clear separation, the Mustache language can be learned independently, and then the skills can be used together with a carefully chosen engine (since not all of them are good or adhere to the official Mustache specs). Thanks for reading the tutorial, I hope you learned a lot, don’t forget to subscribe if you’re interested in Web Development and share the post with friends!

Templating Engine for HTML (special case AMP)

I’m usually working with Angular. But recently I started an AMP Project (www.tsmean.com). For those of you who don’t know AMP, it’s short for “Accelerated Mobile Pages” where you use only the “Official AMP JavaScript Libraries” but no JavaScript of your own and only prerendered HTML. The goal of this is to make webpages blazingly fast. Since a requirement is precompiled html I can’t use Angular as a templating engine since that dynamically compiles the html. Even though it has a static option with Angular Universal (don’t get me started on this…), the Angular Universal rendered page still includes JavaScript so it’s not possible to render an AMP compatible (=JavaScript-free) page with it.

So I’m kind of back in the “dark ages” where I’m stuck with templating engines such as Mustache, Handlebars, EJS or Pug (former Jade). Why I need a templating engine should be obvious: I don’t want to copy / paste code snippets all around and duplicate them like crazy. So I need a templating engine. Just which one’s right? So I set out to choose my templating engine for html. Which has been an absolute nightmare. Let’s review our options.

Mustache

Mustache, we’ve all heard of it. Logic-less templating. So the entry point to the “mustache experience” is this one: https://mustache.github.io/

Since mustache is basically just a spec, it can be implemented in any language. That’s why you see so many languages there. I first clicked on the node.js link which leads to this godforsaken project I’ve used now for a quarter year: https://github.com/raycmorgan/Mu

The owner of the repo doesn’t seem to give two sh*ts about it. Last commit somewhen in 2016. Four unmerged pull requests https://github.com/raycmorgan/Mu/pulls. One of them is mine. No response whatsover.

Then I tried the one when clicking on “JavaScript” in mustache:  https://github.com/janl/mustache.js . Aha, that seems to be the right one with 11k github stars! Okay, I can’t concentrate because of that gif with the dude with the mustache. Is it just me, or is it really annoying to put that gif there? Maybe I’m just having a bad day. But then again, the command line tool also doesn’t seem to be ideal for me because I have to specify all used partials:

mustache -p path/to/partial1.mustache -p path/to/partial2.mustache dataView.json myTemplate.mustache

Can someone explain to me why this is necessary? The partials are already specified in the template! The first mustache compiler I mentioned didn’t require me to pass in all partials at compile time. I mean my template.mustache looks like this:

blabla
{{ > path/to/partial.html }}
blablub

Why do I have to pass this partial as a compiler option again? Especially if you have many views with many partials this becomes very tedious. And if the partial uses another partial? How am I supposed to track the usage of my partials? Unless … I don’t track it and just include all my partials everywhere. Then I’ll just have to register them ONCE in a single file, which is okay.

For those of you coming from Angular like myself, you’ll ask how can I pass data to partials?! You’d expect a syntax like

{{> mybutton title="Cancel" }}
{{> mybutton title="Save" }}

but this doesn’t exist. Instead you’ll set the variable inside the partial by the context you surround it with. An example:

{{#buttonOne}}
{{> mybutton}}
{{/buttonOne}}
{{#buttonTwo}}
{{> mybutton}}
{{/buttonTwo}}

and in the data.json you’d specify

{
"buttonOne": {
"title": "Cancel"
},
"buttonTwo": {
"title": "Save"
}
}

In the partial there could be the code

<button class="my-cool-button">{{title}}</button>

The result would print

<button class="my-cool-button">Cancel</button>
<button class="my-cool-button">Save</button>

It’s a bit of a change a first, but workable.

On a sidenote, there’s another small issue problem with the command line tool. The doc’s state:

The command line tool is basically a wrapper around Mustache.render so you get all the features.

But then their example is actually a demonstration that this doesn’t work:

var view = {
title: "Joe",
calc: function () {
return 2 + 4;
}
};

var output = Mustache.render("{{title}} spends {{calc}}", view);

where the view object is what’s supposed to go to  dataView.json (see above) which is obviously not possible since a JSON can’t hold a function value…

Handlebars

Remember how I said my project can’t have any extra JavaScript libraries? Now, the precompilation of handlebars seems to be so stupid, that it still requires a JavaScript library on runtime:

<script src="/libs/handlebars.runtime.js"></script>


Duuuudes, come on, you’re killin me…

Pug

Since the name “Jade” was copyright infringement, they changed it to pug. In case you don’t know it, it’s a html templating engine, that doesn’t look like html at all. It’s whitespace sensitive and you’ll have to learn a new language. For me, I don’t like it that they deviated so much from the standard html for a couple of reasons:

• No copy / pasting of code snippets from nowhere (one might say that’s a actually a good thing…)
• You’ll have to learn a “new language”
• All team members will have to learn a new language.
• Larger chance of getting out of fashion than when sticking with something similar to html like mustache (yet small chance, since it’s been here for a while and has too many users to be completely abandoned)
• I don’t really find the whitespace based syntax better
• My IDE doesn’t do it’s job as well

So all in all I have very little incentive to use pug, I just see a point of deviating that much from HTML.

EJS

EJS has horribly little documentation: http://ejs.co/

It doesn’t seem to be what I’m looking for: A command line tool to precompile some html templates. Rather you put your html as strings into into JavaScript files, which neither me nor my IDE like very much. As for storing html snippets in a separate file, then requiring them through JS, then paring to a string, and then storing that string again in a file, that’s just too complicated.

Conclusion

Mustache is the winner. Clean syntax that integrates well with html. Large community supporting it. Been here for a while. Not going anywhere anytime soon. Once you get it, you got it. However, it took me a while getting it, since the approach to handling templates and data seems weird when you come from Angular. This article on “passing variables to partials” has been a bit of an eyeopener for me, so thanks Pawel!

What’s up with Angular Universal?

I was excited when I heard Angular 2 would support server-side rendering since SEO was a huge PITA (pain in the ass) with Angular 1. It was said that server-side rendering would be possible through Angular Universal.

Yet, this Angular Universal thing, what’s up with that? Seems to have a really small team on that and there doesn’t really seem to be any documentation. Well at least there’s a getting started repo, so I thought. However, this repo (https://github.com/robwormald/ng-universal-demo) doesn’t even have a readme…

Also, when you check out some of those files, they don’t seem very professional. For example: https://github.com/robwormald/ng-universal-demo/blob/master/src/express-engine.ts

First line of code: //hacky express wrapper thingy.

Seriously?

Then of course, there’s also some confusion as to where to go and look for resources. You’ll find quite a few.

When you just google “Angular Universal” (as of September 2017) you’ll get to https://universal.angular.io/. However, this page is a bit out of date.

Just have a look at the date:

Or the actual site:

Angular 2 is now Angular 4 and --ambient is also not a thing anymore. So the only purpose of universal.angular.io is to confuse people with outdated material and to link to github (https://github.com/angular/universal).

Github, even though being the best resource on Angular Universal, isn’t really documented either. Meh.

Well, I guess switching to React isn’t such a bad idea after all?

PS: I wrote this article in May 2017, I’ve checked again in September and now, even though the docs are still a mess, there’s a working demo here: https://github.com/angular/universal-starter/tree/master/cli

Blocking Browser APIs

Tip of the week:

Don’t use alert, confirm or prompt browser APIs in your production code. They are blocking and thus problematic for performant websites.

Host websites with high availability and low latency for less than 1$/month, SSL included I’ve been deploying static websites the wrong way all of my life. My procedure used to look like this: • Choose a server (EC2, Digitalocean) in a region (US, Europe, …) • Set up some nginx / apache configuration • Maybe add SSL through letsencrypt I’ve always been aware that there are several drawbacks to this: • The smallest EC2 / digital ocean instances cost ~5$ which is quite a lot if you have many websites.
• To make this cheaper, you start putting everything on one server (e.g. one ec2 small instance for 20$/ month), but if this instance crashes ALL your websites are down. You’ll have a very fragile availability. • By selecting a server in a specific region, the other regions have high latency • Adding SSL is kind of a pain, even if it’s free through letsencrypt • It’s not scalable • You need to mess with complicated config files from nginx / apache and need to ssh into your instance. I’ve just never been aware that there’s a much better and easier solution! The solution is called … (drumroll) … : AWS CloudFront. Or to be less product specific, the solution is called: Put a CDN in front of some highly available data store. Here’s what that means: Your users will access the content through servers in the region closest to them. This eliminates all of the previously mentioned drawbacks: • It’s cheap: AWS charges less than 1$/month for this setup
• It’s got high availability: S3 has 99.9999999% uptime (I’ve randomly added 9’s)
• It’s got low latency: Content is served through the data center closest to you.
• You get a free SSL certificate from AWS that you can easily, no config file hacking.
• AWS takes care of scaling for you
• It’s simple to set up & deploy!

How to set up AWS CloudFront

AWS CloudFront is pretty simple to set up, but there are a few pitfalls, so here’s a tutorial with pictures to guide you.

Step 1: Host website with AWS CloudFront

Next, fill out the wizard:

and hit create. Your website needs to have an index.html file for this to work.

Tada. You have a working hosted website at some crazy url.

Step 2: Setup domain

Hit “Buy domain” on the website dashboard. This is the right button even when you’ve already bought you’re website through Route 53.

If you already have a domain, you can simply select it here:

Step 3: Set up SSL

Back on the website’s main page, select “Manage settings in Amazon CloudFront”:

There, click edit:

and select Custom SSL Certificate:

If you don’t have one yet, you can request a new one. It’s easy to set up, but you’ll need to have a working email address for the domain. E.g. when you want to set up example.com, you need e.g. admin@example.com to be working. I’m not going too much into detail here, since that’s a topic on it’s own.

You can also forward HTTP to HTTPS in the CloudFront > Behaviors Tab: Edit the default switch to “Redirect HTTP to HTTPS“.

Once the certificate has been issued and connected, SSL works:

Step 4: Pretty URLs and Subdirectories

By default, the files are hosted as they are: example.com/articles/my-article.html. Now how can you get rid of the html extension? You can either just remove the html extension or have a folder example.com/articles/my-article/index.html.

Unfortunately, there’s also one setting that needs to be changed for this to work. Head over to s3 > website bucket > properties > static website hosting and copy the “Endpoint” URL:

Then in Cloudfront > Origins

select the right website and hit edit, then replace the “Origin Domain Name” with what you’ve copied from the S3 bucket (without the http://).

Congrats, you’re all set up!

Step 5: Deployment

It’s a bit annoying to always have to upload a zip if you’re more the command line type of guy. Fortunately, it’s still possible to upload through command line with AWS cli:

aws s3 cp dist s3://<bucket-name>/ --recursive

You’ll need to set up IAM for this to work. Create a new user, give him S3 access, and hit:

aws configure

once the aws command line cli is installed.

There are a few gotchas you’ll have to be aware of when setting up CloudFront.

Forwarding the Naked (Apex) Domain to the WWW domain

In order to achieve a forward from https://example.com to https://www.example.com you’ll have to use a really weird workaround: Create a new S3 bucket with the name example.com. It HAS to be example.com, you can’t give the bucket any other name. Then in properties > static website hosting redirect all requests:

Finally, in Route53 resolve example.com (or here tsmean.com) to this S3 bucket:

If the S3 bucket name IS EQUAL TO THE RECORD NAME, the S3 website bucket should show up when you select Alias Target. That’s it, that way requests to the naked (apex) domain can be forwarded to a www domain.

Cache Invalidation

If you’ve followed the “one server deployment paradigm” so far, you’re probably used to seeing your changes visible immediately after deployment. This is not the case when using AWS CloudFront. Since it’s a distributed system with servers at edge locations that cache your assets, those caches have to be invalidated first. AWShas two articles on this, first http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html and second http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ReplacingObjectsSameName.html.

They advise you against using their built-in cache invalidation mechanism since it becomes expensive above a certain number of requests. However, this free limit is 1000 requests per month as of August 2017. That’s more than enough for most people since you can declare wildcard paths and those are counted only once, even if thousands of requests are made! So you could just specify “*” as invalidation and your cache is cleared with one request:

I personally prefer this to versioned object / directory names since I find it easier.

You can also easily include this into a deploy script, that could look like this:

#!/usr/bin/env bash

# Prerequisits:
# 1) AWS CLI installation (pip install awscli --upgrade --user)
# 2) AWS CLI Login ("aws configure")
# 3) aws configure set preview.cloudfront true

aws s3 cp <replace with your dist folder location> $S3_DIST --recursive aws cloudfront create-invalidation --distribution-id$CDN_DISTRIBUTION_ID --paths "/*"
echo "Done!"



Single Page Applications

If you’re hosting a SPA (Single Page Application), you’ll run into the problem that upon reloading a non-root page (e.g. bla.com/team), you’ll get an error. To fix this, you have two options.

1. In Cloudfront:
– Go to Error Pages tab, click on Create Custom Error Response:
– HTTP Error Code: 403: Forbidden (404: Not Found, in case of S3 Static Website)
– Customize Error Response: Yes
–  Response Page Path: /index.html
– HTTP Response Code: 200: OK
2. https://stackoverflow.com/a/16877231/3022127

Changing the URL (in case you need to)

If you ever want to migrate, e.g. from dev.bla.com to test.bla.com, you’ll need to:

1. In Cloudfront, add a cname. To do so, edit the General Settings, it’s the first tab.
2. Change in Route53
3. Update AWS_WEBSITE_DOMAIN_NAME und S3 > website bucket > properties > tags. (for the sake of completeness)

Compressing Assets / Gzip

In order to speed up your page, it’s a good idea to compress your assets. To do so

1. Go to CloudFront > Behaviour Tab.
2. Select the Default Behaviour and click Edit
3. Select “Compress Objects Automatically

Checklist

Here’s a short checklist whether you’ve completed all important steps:

• http://www.your-domain.com is working
• https://www.your-domain is working (SSL)
• http://your-domain.com is working (naked)
• https://your-domain.com is working (naked, SSL)
• https://your-domain.com/subdirectory is working
• Compression is enabled

Conclusion

Compared to AWS CloudFront, deploying to single instances just feels hacky and wrong now. It’s simpler, cheaper and safer with AWS CloudFront. It’s the solution I’ve always wanted, but didn’t know existed. Hope this tutorial helps someone!