WhoKnew Shield™ | Troubleshooting: Scraper Trap Not Blocking

Skip to main content
Table of Contents
Print

You’ve enabled Scraper Trap™ (Pro) but bots don’t seem to be getting blocked, the Blocked IPs list stays empty, or you’re not sure if the honeypot is working. This checklist covers the main reasons blocking might not appear to work and what to verify so you can fix it without opening a ticket.

Checklist Overview

Blocking only happens when (1) your Pro license is active, (2) Scraper Trap is enabled and settings are saved, (3) a bot or visitor actually hits the honeypot URL (from your page or from robots.txt). Use the steps below in order.

SymptomWhat to check
No Scraper Trap tab or it’s lockedScraper Trap is Pro-only and only runs when your license is active. Go to WhoKnew Shield → License, activate your key, and ensure the domain matches. See Troubleshooting: Pro Features Missing.
Blocked IPs list is always emptyConfirm Enable Scraper Trap is on and you’ve clicked Save Scraper Trap Settings. Blocking only starts after a bot or visitor triggers the honeypot link. If no bot has hit the trap yet, the list will be empty.
I tested with my browser and wasn’t blockedLogged-in administrators and whitelisted IPs are never blocked. Honeypots are also not shown to them. To test the block experience, use the Test Honeypot Block Page link in a browser where you’re not logged in (e.g. private window), and temporarily remove your IP from the whitelist—or use a different IP (e.g. mobile data). See Getting Started with Scraper Trap.
Blocks expire too quicklyBlock Duration may be set to 10 Minutes (Test Mode). For real protection, use 24 Hours (Recommended) or longer. Test Mode is for initial testing only.
I want more bots to hit the trap / protect SEOAdd the honeypot URL to robots.txt via Robots.txt Integration. The Disallow line tells Google and other legitimate crawlers not to request the trap URL—so they won’t be blocked (important for search engine optimization). Bad bots that ignore robots.txt often target disallowed URLs and will still hit the trap. Use Auto-Fix Robots.txt (Add Entry Now) or add the line manually. See robots.txt and Scraper Trap (Pro).

Step 1: Pro License Active

Scraper Trap (honeypot output, block check, and Blocked IPs list) only runs when WhoKnew Shield™ Pro is licensed for your site. If the license expired, wasn’t activated, or the domain doesn’t match, the Scraper Trap tab may be locked or show the upsell, and no honeypots will be added to your pages—so nothing can trigger a block. Go to WhoKnew Shield → License, enter your license key, and activate. After activation, open the Scraper Trap™ tab and confirm you see the full settings (Enable Scraper Trap, Block Duration, IP Whitelist, etc.). If the tab stays locked, see Troubleshooting: Pro Features Missing.

Step 2: Enable Scraper Trap and Save

In WhoKnew Shield → Scraper Trap™, turn on Enable Scraper Trap. Set Block Duration (e.g. 24 Hours (Recommended); use 10 Minutes (Test Mode) only for quick testing, then switch back). Click Save Scraper Trap Settings at the bottom. Until you save, honeypots are not added to the front end and no IP will be recorded when something hits the trap URL. The Blocked IPs list only fills when a visitor or bot actually requests the honeypot link—so if traffic is low or no bot has found the trap yet, the list can stay empty even when everything is configured correctly.

Step 3: Whitelist and Testing

Your IP is whitelisted — Whitelisted IPs are never blocked and do not receive the honeypot in the page output. So when you browse your site while logged in (or from a whitelisted IP), you will never see yourself in the Blocked IPs list. That’s by design. To verify that blocking works, use the Test Honeypot Block Page URL shown in the Scraper Trap tab: open it in a private or incognito window (or a browser/device where you’re not logged in), and temporarily remove your IP from the IP Whitelist first—then you’ll see the block page and your IP will appear in Blocked IPs. Re-add your IP to the whitelist afterward so you can access the admin again. See Getting Started with Scraper Trap and Viewing Your Site as a Visitor / Confirming Protection.

Step 4: Block Duration

If you set Block Duration to 10 Minutes (Test Mode), blocked IPs are automatically unblocked after 10 minutes. For ongoing protection, choose 24 Hours (Recommended), 1 Week, 1 Month, or Permanent. The Blocked IPs list still shows temporary blocks until they expire; expired blocks are cleaned up automatically and no longer count as “active.” So “not blocking” is not the same as “blocks expired”—check that duration is not Test Mode if you want blocks to last.

The honeypot link is included in your page output (footer, header, or body start, depending on Output Location in Honeypot Stealth Settings). Bots that crawl your HTML can find and trigger it without robots.txt. Adding the honeypot URL to robots.txt with a Disallow line does two things: legitimate crawlers (e.g. Google, Bing) that respect robots.txt will avoid the trap, so they won’t be blocked—important for search engine optimization. Bad bots that ignore robots.txt often seek out disallowed URLs and will hit the trap and get blocked. In the Scraper Trap tab, open Robots.txt Integration. If you have a physical robots.txt file, the plugin will show Manual Setup Required and a line to add; click Auto-Fix Robots.txt (Add Entry Now) to insert it, or add it manually. After changing robots.txt, clear any CDN or server cache for robots.txt and do a hard refresh when viewing it. See robots.txt and Scraper Trap (Pro).

Recommendation: Add the robots.txt entry so Google and other search-engine crawlers don’t follow the honeypot link. Legitimate crawlers obey Disallow and will skip the trap, which keeps your site from accidentally blocking them—that’s important for search engine optimization. Bad bots that ignore robots.txt will still find and trigger the trap. So robots.txt isn’t optional if you care about SEO; set it up in Robots.txt Integration and use Auto-Fix or add the line manually.

Tip: Blocking only happens when something requests the honeypot URL. If your site has little bot traffic, the Blocked IPs list may stay empty for a while. Enable Email Alerts in Scraper Trap so you’re notified when a scraper is caught; that confirms the trap is working.

Need Help?

Getting Started with Scraper Trap — enable safely, whitelist, test. robots.txt and Scraper Trap (Pro) — Auto-Fix, Verify, View Full. Customizing the Scraper Trap Block Message (Pro) — block page text and Test Honeypot Block Page. Viewing Your Site as a Visitor / Confirming Protection — testing without locking yourself out. Troubleshooting: Pro Features Missing — license tab, activation, domain.

Share