Who's really in your nginx logs?
Paste access log lines to see humans, search engines, AI crawlers, scanners and fake bots. Everything runs in your browser. Your logs are never uploaded.
Your log lines
Works with nginx's default combined format, with or without $host in front. Reads up to 200,000 lines, including .gz files. To grab recent lines on your server:
sudo tail -n 20000 /var/log/nginx/access.logHow the first line was read
If these fields look wrong, your log format differs from combined.
Where your requests come from
| Class | Requests | Share | Bandwidth | Share |
|---|
Fake bots
Requests that claim to be a known crawler but come from an IP outside that operator's published ranges.
Crawlers by name
Every declared crawler, what it is for, and whether its IPs checked out.
Scanners
IPs probing paths like /.env, /.git/ or /wp-login.php, or using known scanning tools.
Per site
Bot share for each site. Needs $host at the start of each log line.
This was one server, one time.
Trafficsort does this continuously for all your sites. It verifies every bot, shows what each one costs you in bandwidth, blocks what you choose with safe nginx presets, and emails you when a new crawler or a fake-bot surge shows up. One dashboard covers up to 20 sites.
Join the waitlistHow the checker decides
- Known crawlers are matched by user agent, then their IP is checked against the operator's published IP list. In the IP list: verified. Outside it: fake.
- Unverified means the operator publishes no IP list this page can use (for example Bytespider), or the lists did not load. A browser cannot do reverse DNS, so Applebot, YandexBot and similar stay unverified here.
- Scanners are flagged by probe paths and scanner tool names. On a real WordPress site, humans also visit
/wp-login.php, so check the IPs before blocking. - Browser means the request claims to be a normal browser. That includes real people, AI browsers and scrapers that fake a browser user agent. Logs alone cannot fully separate them.
- Bandwidth is the response body size nginx logged (
$body_bytes_sent).