Nginx

Nginx Module for Bot Protection

This DataDome module is to be used on Nginx.

Compatibility

This module is compatible with Nginx >= 1.5.4.

Every new release of the module is strongly tested on the the following distributions:

  • Debian 7/8/9/10/11
  • Ubuntu 14/16/18/20/22 (LTS versions)
  • Centos 6/7/8
  • SUSE 11

Installation

  1. Follow one of the options (A or B or C) below to make sure you have Nginx and DataDome libs in your system.
A: Automatic installation

Download https://package.datadome.co/autoinstall_nginx_module.sh ;
Execute it with root privileges.

curl -s https://package.datadome.co/autoinstall_nginx_module.sh > \
autoinstall_nginx_module.sh && bash autoinstall_nginx_module.sh
B: Manual Installation from repository

If you don't use 3rd party modules for Nginx, or if you want to install Nginx from scratch, you can use the Nginx DataDome repository.

We provide Nginx 1.21.3 with DataDome Module builtin as a static module.

C: Manual Installation from source files

Prerequisite
In order to build DataDome Nginx module, the following packages should be installed:

apt-get update
apt-get -y install wget gcc make libpcre3-dev libssl-dev zlib1g zlib1g-dev gnupg2 libgeoip-dev libgd-dev libxslt1-dev libxml2-dev libssl-dev
apt-get autoremove
yum update
yum install -y redhat-rpm-config curl wget gcc make pcre-devel openssl-devel libxslt-devel gd-devel perl-ExtUtils-Embed
apk --no-cache add linux-headers pcre-dev openssl-dev zlib-dev curl grep tar gcc musl-dev make
# Create a temporary directory to work in
tmp_dir=$(mktemp -d -t datadome-XXXXXXXXXX)
echo $tmp_dir

# Get the Nginx version in use
nginx_version=$(nginx -v 2>&1 | grep -oP 'nginx\/\K([0-9.]*)')
echo $nginx_version

# Download and untar the Nginx sources to compile dynamic module
curl -sLo ${tmp_dir}/nginx-${nginx_version}.tar.gz http://nginx.org/download/nginx-${nginx_version}.tar.gz
tar -C ${tmp_dir} -xzf ${tmp_dir}/nginx-${nginx_version}.tar.gz

# Download and untar DataDome module sources
curl -sLo ${tmp_dir}/datadome_nginx_module.tar.gz https://package.datadome.co/linux/DataDome-Nginx-latest.tgz
tar -C ${tmp_dir} -zxf ${tmp_dir}/datadome_nginx_module.tar.gz

# Get the compilation flags used during the compilation of nginx, and remove any --add-dynamic-module flag we find
# This is important because when compiling the modules, you have to use the same flags that have been used when compiling nginx
nginx_flags="$(nginx -V 2>&1 | grep -oP 'configure arguments: \K(.*)' | sed -e 's/--add-dynamic-module=\S*//g')"
echo $nginx_flags

# Launch the nginx configure script with same flags + the DataDome dynamic module
cd ${tmp_dir}/nginx-${nginx_version} && eval "./configure --add-dynamic-module=../ ${nginx_flags}"

# Compile the modules
make -C ${tmp_dir}/nginx-${nginx_version} -f objs/Makefile modules

# Ensure Nginx module directory is created
mkdir -p /etc/nginx/modules

# Copy the .so modules to nginx configuration
cp ${tmp_dir}/nginx-${nginx_version}/objs/ngx_http_data_dome_*.so /etc/nginx/modules/

# Then you have to add the following configuration to your nginx.conf file:
# load_module /etc/nginx/modules/ngx_http_data_dome_auth_module.so;
# load_module /etc/nginx/modules/ngx_http_data_dome_shield_module.so;
# load_module /etc/nginx/modules/ngx_http_data_dome_upstream_dynamic_servers_module.so;

# Ensure the modules are good
nginx -t

# If it's good you can go to configuration section
# Get the NGINX source
nginx -v
wget http://nginx.org/download/nginx-1.21.3.tar.gz #Replace with the correct version
tar -xzvf nginx-1.*.tar.gz
cd nginx-*

# Build DataDome Module
rm -f DataDome-Nginx-latest.tgz
wget https://package.datadome.co/linux/DataDome-Nginx-latest.tgz
tar -zxvf DataDome-Nginx-latest.tgz
./configure --add-module=/path/to/NginxDome
make
make install
  1. In nginx.conf, add the following settings:
http {
    [...]
    
      resolver 8.8.8.8;

      upstream datadome {
          dd_server api.datadome.co:443;
          keepalive 10;
      }
}
  1. In each virtual host config file, add the following code inside the serverblock:
server {
  [...]

  data_dome_auth @datadome;

  location = @datadome {
    data_dome_shield_key "KEYPROVIDEBYDATADOME";
    proxy_pass https://datadome/validate-request/;
    proxy_method POST;
    proxy_http_version 1.1;
    proxy_set_header Connection "keep-alive";
    proxy_set_header Content-Type "application/x-www-form-urlencoded";
    proxy_set_header X-DataDome-X-Set-Cookie $data_dome_header_x_set_cookie;
    proxy_set_body $data_dome_request_body;
    proxy_ignore_client_abort on;
    proxy_connect_timeout 150ms;
    proxy_read_timeout 50ms;
  }
}
  1. Modify the value of data_dome_shield_key to hold the value of your DataDome Server-side key available in your DataDome dashboard.
  2. Restart Nginx service:
service nginx start

Congrats! You can now see your traffic in your DataDome dashboard.

Configuration

By default, the configuration is located in either /usr/local/nginx/conf, /etc/nginx, or /usr/local/etc/nginx.

Refer to the next Settings section for the full list of possible configuration settings.

Settings

SettingDescriptionRequiredDefault
data_dome_shield_keyyour DataDome License keyyes
dd_serverhostname of the API Server
Available endpoints
optionalapi.datadome.co
data_dome_auth_uri_regexprocesses only matching URIs.

Note: should be added to the server block but outside the location block
optional
data_dome_auth_uri_regex_exclusionignores all matching URIs.

Note: should be added to the server block but outside the location block
optionalList of excluded static assets below
proxy_connect_timeouttimeout set for the initial opening connectionoptional150ms
proxy_read_timeouttimeout set for regular API callsoptional50ms
"\\.(avi|flv|mka|mkv|mov|mp4|mpeg|mpg|mp3|flac|ogg|ogm|opus|wav|webm|webp|bmp|gif|ico|jpeg|jpg|png|svg|svgz|swf|eot|otf|ttf|woff|woff2|css|less|js|map)$"

Exclude URIs from DataDome Bot Protection

Add this declaration to the server block but outside the location block escaping the slash/ and any other special char of the URI exclusion:

data_dome_auth_uri_regex_exclusion "\/api\/v2\/sessions|\/api\/v2\/users\/ssn$";

After that, make sure to append at the end of the regex the list of static files to be ignored, the suggestions is below:

data_dome_auth_uri_regex_exclusion "\/api\/v2\/sessions|\/api\/v2\/users\/ssn$|\\.(avi|flv|mka|mkv|mov|mp4|mpeg|mpg|mp3|flac|ogg|ogm|opus|wav|webm|webp|bmp|gif|ico|jpeg|jpg|png|svg|svgz|swf|eot|otf|ttf|woff|woff2|css|less|js|map)$;

It looks like this:

server {
  [...]

  data_dome_auth @datadome;
	data_dome_auth_uri_regex_exclusion "\/api\/v2\/sessions|\/api\/v2\/users\/ssn$|\\.(avi|flv|mka|mkv|mov|mp4|mpeg|mpg|mp3|flac|ogg|ogm|opus|wav|webm|webp|bmp|gif|ico|jpeg|jpg|png|svg|svgz|swf|eot|otf|ttf|woff|woff2|css|less|js|map)$;

  location = @datadome {
    data_dome_shield_key "KEYPROVIDEBYDATADOME";
    proxy_pass https://datadome/validate-request/;
    proxy_method POST;
    proxy_http_version 1.1;
    proxy_set_header Connection "keep-alive";
    proxy_set_header Content-Type "application/x-www-form-urlencoded";
    proxy_set_header X-DataDome-X-Set-Cookie $data_dome_header_x_set_cookie;
    proxy_set_body $data_dome_request_body;
    proxy_ignore_client_abort on;
    proxy_connect_timeout 150ms;
    proxy_read_timeout 50ms;
  }
}

Exclude a specific location DataDome from DataDome Bot Protection.

You can disable DataDome for a specified Nginx location by adding the command below:

data_dome_auth off;

You can also do this dynamically with a variable:

# disable datadome
set $is_datadome_enabled off;
# enable datadome
# set $is_datadome_enabled @datadome

data_dome_auth $is_datadome_enabled;

Activate DataDome Bot Protection on internal requests and rewrites

The module doesn't call the DataDome Bot Protection API for internal requests or rewrite(more info here).

You can switch it on by using the command bellow:

data_dome_auth_pass_internal_redirect off;

Exclude IPs from DataDome Bot Protection

You can disable DataDome Bot Protection for requests coming from IP addresses using:

http {
    ....
    geo $is_datadome_enabled {
        default        '@datadome';

        192.168.0.0/24 'off';
    }
    ....
    server {
        ....
        data_dome_auth $is_datadome_enabled;
        ....
    }
}

Logging

Enriched Headers

You can log the values of DataDome Enriched Headers inside your Nginx access logs.

http {
    ....
    log_format datadome '$request $datadome_isbot $datadome_response_time';
    access_log /var/log/nginx/datadome.log datadome;
    ....
    server {
        ....
        data_dome_auth @datadome;
        data_dome_auth_set $datadome_isbot $upstream_http_x_datadome_isbot;
        data_dome_auth_set $datadome_response_time $upstream_response_time;

        ....
    }
}

Status of the DataDome Bot Protection API's response

Just like for enriched headers, you can add the DataDome Bot Protection API Response code in your access logs. Possible values are 200, 403, or 5xx in case of timeout or connexion error.

http {
    ....
    log_format datadome '$request $datadome_status';
    access_log /var/log/nginx/datadome.log datadome;
    ....
    server {
        ....
        data_dome_auth @datadome;
        data_dome_auth_set $datadome_status $upstream_status;
        ....
    }
}

Logging level of DataDome Bot Protection Timeouts

[After version 2.45.0]

By default, some DataDome timeouts are treated as WARN to avoid verbose output in your logging solution, raising undesired alerts (punctual timeouts happen and should not be a big issue).
If you would like to have everything treated as ERRORS, you just need to add the following line in your nginx.conf file: env DATADOME_NGX_LOG_ERR=Y

...

env DATADOME_NGX_LOG_ERR=Y;

http {
  ...

FAQ

How can I change the default refresh DNS record time?

You can change the refreshing DNS time from the default 1 hour like this:

resolver 8.8.8.8;

upstream datadome {
  dd_server api.datadome.co:443 refresh_in=2h;
  keepalive 10;
}

When is the DataDome module be executed?

The DataDome module registers itself at the Nginx HTTP phase called NGX_HTTP_ACCESS_PHASE.

It takes place after URI rewrites and before content generation.

How can I expose DataDome Enriched Headers to use them inside Nginx configuration?

Our module makes sub-requests: the values of DataDome Enriched Headers only exist by default inside them.
Bot information can be forwarded to the main request for webserver logging or to be used inside the application's logic by using data_dome_auth_set.
For instance, to create an access-log file that contains the request URI, 'is it a bot', and the API server response time, you can use below code:

http {
    ....
    log_format datadome '$request $datadome_isbot $datadome_response_time';
    access_log /var/log/nginx/datadome.log datadome;
    ....
    server {
        ....
        data_dome_auth @datadome;
        data_dome_auth_set $datadome_isbot $upstream_http_x_datadome_isbot;
        data_dome_auth_set $datadome_response_time $upstream_response_time;

        ....
    }
}

How can I use our corporate proxy?

You can add use your corporate proxy with our module by changing your configuration as follow:

http {
    resolver 1.2.3.4;
    ...
    upstream datadome {
      dd_server your_proxy:3128;
      ....
    }
    ....
    server {
        ....
        data_dome_auth 'http://api.datadome.co/validate-request/';
        ....
        location = http://api.datadome.co/validate-request/ {
          internal;
          proxy_pass http://datadome;
          proxy_set_header Host "api.datadome.co";
          ....
      }
    }
}

How can I install Nginx module on AWS Linux 1 AMI?

You can either install the Nginx module as described above or use the following repository:

cd /etc/yum.repos.d/
sudo wget http://download.opensuse.org/repositories/isv:/datadome/CentOS_6/isv:datadome.repo
yum install nginx-datadome

Once the installation is done, the configuration remains the same as described above.

How can I install Nginx module on CentOS RHEL 7?

You can use the following repository:

cd /etc/yum.repos.d/
sudo wget https://download.opensuse.org/repositories/isv:/datadome/CentOS_7/isv:datadome.repo
yum install nginx-datadome

Once the installation is done, the configuration remains the same as described above.

How can I install Nginx module on Alpine Linux?

Alpine Linux is much smaller than most distribution base images (~5MB), and, because it is lightweight, requires the addition of packages in order to compile and deploy DataDome Nginx module.

Before starting the manual install the following commands should be run in order to retrieve the required packages:

# Install required packages
apk --no-cache add linux-headers pcre-dev openssl-dev zlib-dev curl grep tar gcc musl-dev make

# Add a DNS resolver
echo "nameserver 8.8.8.8" > /etc/resolv.conf