Reverse proxy

About a reverse proxy

A reverse proxy sits between your site and the internet. With the robots.nxt reverse proxy, all of your traffic is routed through robots.nxt, where we can detect bots and impose restrictions and rules on the bots.

To set up the reverse proxy, you need to update your DNS settings to point your domain to robots.nxt, and set up a private subdomain for your site. This will prevent anyone from being able to access your site directly, which enables robots.nxt to control bot traffic.

When you create a site, the setup flow will provide you with the DNS settings you need to update, as well as the private subdomain for your site. This includes a CNAME record for your site, an A record for your IP address, and a subdomain name to receive your traffic from the reverse proxy.

Example setup

In this example, we will be setting up a reverse proxy for the domain example.com.

1. Create account at paynpoint.net
2. Create an organization
3. Create a site example.com

4. Configure your site to accept traffic on a private subdomain, in this example we we will use proxybackend.example.com
5. Update your Proxy URL on the settings to point to your websites private subdomain. proxybackend.example.com

6. Configure your DNS records based on the instructions found in your proxy settings page. Settings

Once your DNS settings have been updated, your site will be accessible through the reverse proxy. You can now use the robots.nxt dashboard to view your site's traffic.