New in Symfony 4.3: Automatic Search Engine Protection

Written by Symfony blog - - Aggregated on Wednesday March 13, 2019

Contributed by
Gary Pegeot
in #30325.

Search engines like Google, DuckDuckGo, Baidu and Yandex do a great job crawling and indexing your web sites and applications. However, this is not so great when you accidentally publish a development version of your app.

In Symfony 4.3 we improved this situation by disallowing the search engine indexing for development applications. How does it work? If the app kernel runs in debug mode (by default this happens when the Symfony environment is not prod) Symfony adds a X-Robots-Tag: noindex HTTP header to all the responses.

The X-Robots-Tag header is one of the HTTP headers that legit search engines take into account when crawling a web site. If your own app already added that header, this new feature doesn't change its value. Also, if you don't like this feature, you can disable it with this config option:

# config/packages/framework.yaml
    # ...
    disallow_search_engine_index: false
Be trained by Symfony experts - 2019-04-1 Clichy - 2019-04-4 Clichy - 2019-04-8 Clichy

« Basecamp outage: When it rains, it pours - Signal vs. Noise - Add Animations to React Apps with … »