Nginx
DataDome Nginx module detects and protects against bot activity
Before the regular Nginx process, the module makes a call to the DataDome API using a keep-alive connection.
Depending on the API response, the module will either block the query or let Nginx continue the regular process.
The module has been developed to protect user experience as if any error was to occur during the process or if the timeout was reached, the module would automatically disable its blocking process and allow those requests.
Compatibility
This module is compatible with nginx >= 1.5.4.
Every new release of the module is strongly tested on the the following distributions:
- Debian 7/8/9/10/11
- Ubuntu 14/16/18/20/22 (LTS versions)
- Centos 6/7/8
- SUSE 11
Step-by-Step
There are 2 mandatory steps described below
1 - Installation : Read and choose an option that suits you the best: A, B or C
2 - Configuration : Add the DataDome configuration as described
1 . Installation
As mentioned above, you only need to pick one of the options (A or B or C).
The main goal here is to make sure you have nginx and datadome libs in your system.
A) Automatic installation
Download https://package.datadome.co/autoinstall_nginx_module.sh ;
Execute it with root privileges.
curl -s https://package.datadome.co/autoinstall_nginx_module.sh > \
autoinstall_nginx_module.sh && bash autoinstall_nginx_module.sh
B) Manual Installation from repository
If you don't use 3rd party modules for Nginx, or if you want to install Nginx from scratch, you can use the Nginx DataDome repository.
We provide Nginx 1.21.3 with DataDome Module builtin as a static module.
C) Manual Installation from source files
You can install DataDome Nginx module from source files as described below.
Prerequisite
In order to build DataDome Nginx module, the following packages should be installed:
apt-get update
apt-get -y install wget gcc make libpcre3-dev libssl-dev zlib1g zlib1g-dev gnupg2 libgeoip-dev libgd-dev libxslt1-dev libxml2-dev libssl-dev
apt-get autoremove
yum update
yum install -y redhat-rpm-config curl wget gcc make pcre-devel openssl-devel libxslt-devel gd-devel perl-ExtUtils-Embed
apk --no-cache add linux-headers pcre-dev openssl-dev zlib-dev curl grep tar gcc musl-dev make
# Create a temporary directory to work in
tmp_dir=$(mktemp -d -t datadome-XXXXXXXXXX)
echo $tmp_dir
# Get the Nginx version in use
nginx_version=$(nginx -v 2>&1 | grep -oP 'nginx\/\K([0-9.]*)')
echo $nginx_version
# Download and untar the Nginx sources to compile dynamic module
curl -sLo ${tmp_dir}/nginx-${nginx_version}.tar.gz http://nginx.org/download/nginx-${nginx_version}.tar.gz
tar -C ${tmp_dir} -xzf ${tmp_dir}/nginx-${nginx_version}.tar.gz
# Download and untar DataDome module sources
curl -sLo ${tmp_dir}/datadome_nginx_module.tar.gz https://package.datadome.co/linux/DataDome-Nginx-latest.tgz
tar -C ${tmp_dir} -zxf ${tmp_dir}/datadome_nginx_module.tar.gz
# Get the name of the DataDome module directory
datadome_dir=$(basename $(ls ${tmp_dir}/DataDome-NginxDome-* -d1))
# Get the compilation flags used during the compilation of nginx, and remove any --add-dynamic-module flag we find
# This is important because when compiling the modules, you have to use the same flags that have been used when compiling nginx
nginx_flags="$(nginx -V 2>&1 | grep -oP 'configure arguments: \K(.*)' | sed -e 's/--add-dynamic-module=\S*//g')"
echo $nginx_flags
# Launch the nginx configure script with same flags + the DataDome dynamic module
cd ${tmp_dir}/nginx-${nginx_version} && eval "./configure --add-dynamic-module=../${datadome_dir} ${nginx_flags}"
# Compile the modules
make -C ${tmp_dir}/nginx-${nginx_version} -f objs/Makefile modules
# Ensure Nginx module directory is created
mkdir -p /etc/nginx/modules
# Copy the .so modules to nginx configuration
cp ${tmp_dir}/nginx-${nginx_version}/objs/ngx_http_data_dome_*.so /etc/nginx/modules/
# Then you have to add the following configuration to your nginx.conf file:
# load_module /etc/nginx/modules/ngx_http_data_dome_auth_module.so;
# load_module /etc/nginx/modules/ngx_http_data_dome_shield_module.so;
# load_module /etc/nginx/modules/ngx_http_data_dome_upstream_dynamic_servers_module.so;
# Ensure the modules are good
nginx -t
# If it's good you can go to configuration section
# Get the NGINX source
nginx -v
wget http://nginx.org/download/nginx-1.21.3.tar.gz #Replace with the correct version
tar -xzvf nginx-1.*.tar.gz
cd nginx-*
# Build DataDome Module
rm -f DataDome-Nginx-latest.tgz
wget https://package.datadome.co/linux/DataDome-Nginx-latest.tgz
tar -zxvf DataDome-Nginx-latest.tgz
./configure --add-module=/path/to/NginxDome
make
make install
2. Configuration
After going through the installation process (you already have nginx and datadome libs in your system) it is now time to configure nginx.
In nginx.conf, add the following settings.
Make sure to select the best API Server endpoint
http {
[...]
resolver 8.8.8.8;
upstream datadome {
dd_server api.datadome.co:443;
keepalive 10;
}
}
In each virtual host config file, set the Key provided by DataDome in data_dome_shield_key
:
server {
[...]
data_dome_auth @datadome;
location = @datadome {
data_dome_shield_key "KEYPROVIDEBYDATADOME";
proxy_pass https://datadome/validate-request/;
proxy_method POST;
proxy_http_version 1.1;
proxy_set_header Connection "keep-alive";
proxy_set_header Content-Type "application/x-www-form-urlencoded";
proxy_set_header X-DataDome-X-Set-Cookie $data_dome_header_x_set_cookie;
proxy_set_body $data_dome_request_body;
proxy_ignore_client_abort on;
proxy_connect_timeout 150ms;
proxy_read_timeout 50ms;
}
}
[After version 2.45.0]
By default, some datadome timeouts are treated as WARN to avoid verbose output in your logging solution, raising undesired alerts (punctual timeouts happen and should not be a big issue).
If you would like to have everything treated as ERRORS, you just need to add the following line in your nginx.conf file: env DATADOME_NGX_LOG_ERR=Y
...
env DATADOME_NGX_LOG_ERR=Y;
http {
...
Settings
Setting | Description | Required | Default |
---|---|---|---|
data_dome_shield_key | your DataDome License key | yes | |
dd_server | hostname of the API Server Available endpoints | optional | api.datadome.co |
data_dome_auth_uri_regex | processes only matching URIs. Note: should be added to the server block but outside the location block | optional | |
data_dome_auth_uri_regex_exclusion | ignores all matching URIs. Note: should be added to the server block but outside the location block | optional | exclude static assets |
proxy_connect_timeout | timeout set for the initial opening connection | optional | 150ms |
proxy_read_timeout | timeout set for regular API calls | optional | 50ms |
FAQ
Can I disable DataDome on a specific location?
You can disable DataDome for specified location by add command below
data_dome_auth off;
By default, the module is in auto mode.
Can I disable DataDome dynamically with a variable (Lua for instance)?
You can set a variable to disable dynamically the DataDome module.
# disable datadome
set $is_datadome_enabled off;
# enable datadome
# set $is_datadome_enabled @datadome
data_dome_auth $is_datadome_enabled;
Can I activate DataDome for internal call?
The module doesn't call the API server for internal requests. You can switch it on by command bellow:
data_dome_auth_pass_internal_redirect off;
Can I change refresh DNS record time?
If you use dd_server
, you can change refresh time from default 1 hours by command below:
resolver 8.8.8.8;
upstream datadome {
dd_server api.datadome.co:443 refresh_in=2h;
keepalive 10;
}
Can I disable or enable the module on an IP range?
The easy way is doing this by ngx_http_geo_module.
You can find an example below that disables DataDome module for request coming from IP addresses 192.168.0.0/24
http {
....
geo $is_datadome_enabled {
default '@datadome';
192.168.0.0/24 'off';
}
....
server {
....
data_dome_auth $is_datadome_enabled;
....
}
}
When will the DataDome module be executed?
The DataDome module registers itself at the Nginx HTTP phase called NGX_HTTP_ACCESS_PHASE
. It takes place after URI rewrites and before content generation.
How can I expose DataDome enriched headers to use them inside Nginx configuration?
Our module makes sub-requests: the values of enriched headers only exist by default inside them.
Bot information can be forwarded to the main request for webserver logging or to be used inside the application's logic by using data_dome_auth_set
.
For instance, to create an access-log file that contains the request URI, 'is it a bot', and the API server response time, you can use below code:
http {
....
log_format datadome '$request $datadome_isbot $datadome_response_time';
access_log /var/log/nginx/datadome.log datadome;
....
server {
....
data_dome_auth @datadome;
data_dome_auth_set $datadome_isbot $upstream_http_x_datadome_isbot;
data_dome_auth_set $datadome_response_time $upstream_response_time;
....
}
}
How can I add DataDome response status inside my nginx logs ?
Just like for enriched headers, you can add the DataDome API Response code in logs. Possible values are 200, 403, or 5xx in case of timeout or connexion error
http {
....
log_format datadome '$request $datadome_status';
access_log /var/log/nginx/datadome.log datadome;
....
server {
....
data_dome_auth @datadome;
data_dome_auth_set $datadome_status $upstream_status;
....
}
}
How can I use our corporate proxy?
You can add use your corporate proxy with our module by changing your configuration as follow:
http {
resolver 1.2.3.4;
...
upstream datadome {
dd_server your_proxy:3128;
....
}
....
server {
....
data_dome_auth 'http://api.datadome.co/validate-request/';
....
location = http://api.datadome.co/validate-request/ {
internal;
proxy_pass http://datadome;
proxy_set_header Host "api.datadome.co";
....
}
}
}
How can I install Nginx module on AWS Linux 1 AMI?
You can either install the Nginx module as described above or use the following repository:
cd /etc/yum.repos.d/
sudo wget http://download.opensuse.org/repositories/isv:/datadome/CentOS_6/isv:datadome.repo
yum install nginx-datadome
Once the installation is done, the configuration remains the same as described above.
How can I install Nginx module on CentOS RHEL 7?
You can use the following repository:
cd /etc/yum.repos.d/
sudo wget https://download.opensuse.org/repositories/isv:/datadome/CentOS_7/isv:datadome.repo
yum install nginx-datadome
Once the installation is done, the configuration remains the same as described above.
How can I install Nginx module on Alpine Linux?
Alpine Linux is much smaller than most distribution base images (~5MB), and, because it is lightweight, requires the addition of packages in order to compile and deploy DataDome Nginx module.
Before starting the manual install the following commands should be run in order to retrieve the required packages:
# Install required packages
apk --no-cache add linux-headers pcre-dev openssl-dev zlib-dev curl grep tar gcc musl-dev make
# Add a DNS resolver
echo "nameserver 8.8.8.8" > /etc/resolv.conf
You can now resume with the Manual Install from Source .
Updated 5 days ago