Accueil / Blog / Métier / 2017 / SEO : indexing a JavaScript application

SEO : indexing a JavaScript application

Par Simon Georges publié 29/06/2017
Contributeurs : Éric Bréhault, Régis Leroy, Alexandra Janin, Thomas Desvenain, Mathieu Dartigues
How to produce a server-side rendered version of your web application.
SEO : indexing a JavaScript application

One of the most important SEO challenge at the moment is the JavaScript applications indexing by the search engines.

Since Google announced the "Ajax-crawling scheme" indexing deprecation (= based on URL specific fragments), many experiments have been run in order to identify the actual indexing abilities of the most important search engine.

Here is for instance a very interesting JavaScript frameworks comparison:

Comparaison de l'indexation des principaux frameworks Javascript

Google claims regularly it interprets JavaScript, but there are some limitations :

  • The only indexed content is the one available in the page at load event;
  • The JavaScript must be fast enough (5 seconds seems to be the limit).

Most of the applications do not comply with those rules, so they are partially or not indexed. Some use workaround solutions like prerender.io (see below).

The best solution is to provide server-side rendering.

TL;DR: YOU MUST USE SERVER-SIDE RENDERING.

We present here an efficient solution for Angular 4 applications: Angular Universal.

How Angular Universal works (the theory)

Angular Universal used to be an Angular 2 add-on, and it is now integrated into Angular 4 core. It allows the generation on the server side of what the application should have done on the client side. Once delivered to the browser, this pre-rendered version is rehydrated with the JS app and everything that will happen next will be done client-side.

It requires to deploy NodeJS on your server (but PHP and Python implementations are planned).

Note: Angular 4 is not very different from Angular 2, it is just named that way because of semantic versionning. If you use Angular 2, you should upgrade to Angular 4.

How to setup Angular Universal (the practice)

As you could expect, there are much to explain about the practice, so we wrote a separated article about it: How to setup Angular Universal.

Does it really work?

Standard crawling

To validate Angular Universal abilities, we crawled our site with Screaming Frog SEO Spider. As we want to check how the site behaves without JavaScript, we first ran the tool with the Text Only option:

Text only rendering on Screaming Frog SEO Spider

It is roughly equivalent to a basic curl command:

curl https://your-url.com

on your site: it just donwloads the resulting HTML and does not run any JS or web rendering.

The response time is very good (HTML is delivered in less than 500ms). We first note the tool is gathering all the page links properly and is crawling the entire website. It does collect the TITLE, H1 or META Description tags and all the HTML content is compliant with what it would have been in the client-side running application. Hence, the standard page text and words ratio can be computed just like if it was a regular website.

Search engines will have no problem in indexing our site.

Sitemap.xml crawling

On a regular JS app, one of the workaround is to provide a sitemap.xml file to make sure search engines index all the pages.

It works as well with Angular Universal, but it is not necessary as all the links are discoverable in the generated pages.

JS-enabled crawling

Our second test will enable JavaScript:

Javascript crawling

As expected, the crawling is now much slower (as the tool tries to execute all the JS code). We often get some timeout and the page links are not properly crawled.

The TITLE tag seems to be processed but some parts of pages are ignored. There still would be some errors regarding our website indexation.

The drawbacks

Documentation

It is still a young technology, it is badly documented. The lack of documentation might impact the project.

Deployment

Angular Universal will require a NodeJS server on production. It can be handled of course, but that's still an extra constraint.

The user experience

You can enable Universal just for robots or for all visitors.

If you use it for all visitors, you might face annoying side effects, like some blinking if something is first run on the server, then cancelled and re-run on the client.

The angular/preboot module is designed to avoid that. With this module, once the prerender version of the page is rendered, the client-side version is launched in a hidden DOM copy. Meanwhile, all the user interactions are recorded. Once the execution is finished, the prerendered version and the client-side version are switched and the interactions are replayed.

Setting up preboot is not easy and will be the subject of a future article.

What if you don't use Angular?

The other main frameworks have equivalent solutions and when they don't, you can still use prerender.io).

ReactJS

  • The most famous server-side rendering solution for React is Next.js. It is very easy to set up but still it is recommended to set it up at beginning rather than the project is finished. Next.js proposes Link component to manage routes. It will also replace the Webpack configuration for the bundle generation.
  • With an existing app using a specific sass and webpack configuration, involving custom loaders etc., it might be easier to use Express and to adapt the configuration. In that case, you can use react-dom renderToString. And you will have to adapt react-router (if used).
In any case, you will have to cautious with Redux, and make sure it can provide data on the server side.

Vue.js

Vue.js offers several server-side rendering solutions:

  • Nuxt.js, which is the Next.js equivalent for VueJS. The nuxt/starter template allows to bootstrap a project able to generate all the pages in static version. Those files can then be served by Nginx directly.
  • Vue SSR is more complex. It is quite close to Angular Universal. You will have to adapt to the execution context to produce an accurate SSR service. Only beforeCreate et created life cycle hooks are available on the server-side.
    It provides hot-reload (so you do not have to restart your Node server everytime), which is handy.

Prerender-spa-plugin

prerender-spa-plugin is a webpack plugin. It is framework agnostic, and allows to render a collection of URLs. The set up is very quick, and can be integrated in your SPA build. The resulting static files will just have to be published by your web server.

Prerender.io

Prerender.io is THE generic solution to produce a static version of any JavaScript application.

It is a SaaS solution but the company also provides its tool as an open source project, so you can install it on your own server.

Basically, it is a multi-thread service providing PhantomJS instances in charge of rendering pages.

We recommend to run it behind a Nginx proxy (there is an "official" nginx configuration, and we have made few adaptations).

Your turn now

Did you set it up? Do you have feedback to share? Do you face issues?  if you need help, or order a SEO AUDIT!

ABONNEZ-VOUS À LA NEWSLETTER !
Voir aussi
Mise en pratique de RxJS dans Angular Mise en pratique de RxJS dans Angular 24/07/2017

Les quelques bases suffisantes pour bien utiliser RxJS dans Angular 2, 4 ou 5.

Comment mettre en place Angular Universal Comment mettre en place Angular Universal 29/06/2017

Toutes les étapes détaillées et expliquées. Les pièges à éviter.

How to setup Angular Universal How to setup Angular Universal 29/06/2017

Step by step explanation. All the pitfalls to avoid.

SEO : indexer une application Javascript SEO : indexer une application Javascript 29/06/2017

Comment utiliser le code de votre application pour un rendu server-side.

Découverte de React Native Découverte de React Native 18/04/2016

Présentation de React Native, quelles sont ses possibilités et peut-on l'utiliser en prod ?