Cloudflare Worker Setup for AgentFi Integration
AgentFi provides a one-click deployment for Cloudflare Workers to optimize websites for AI search engines. Users connect their accounts via a Cloudflare API Token to automatically proxy requests for AI-specific files like llms.txt and robots.txt. The Worker tracks AI bot activity from GPTBot, ClaudeBot, and PerplexityBot while performing health checks every 10 minutes. This integration requires an active Cloudflare domain and an AgentFi account with a previously crawled site.
What are the prerequisites for AgentFi Worker setup?
To use the automatic setup, your environment must meet specific Cloudflare and AgentFi configuration requirements.
* Cloudflare account with an active domain (zone)
* Domain must use Cloudflare DNS (orange-clouded)
* Free or paid Cloudflare Workers plan
* AgentFi account with one crawled site
How do I create a Cloudflare API Token for AgentFi?
The deployment process requires a specific API token that grants AgentFi permission to manage Workers and routes on your behalf.
1. Access the Cloudflare API Tokens page.
2. Click Create Token and select the Edit Cloudflare Workers template.
3. Select the specific account containing your domain under Account Resources.
4. Select "All zones" or a specific domain under Zone Resources.
5. Review permissions and click Create Token.
6. Copy and securely store the token immediately.
How is the AgentFi Worker deployed?
Deployment is managed through the AgentFi dashboard and involves connecting the API token and selecting a target zone.
Connecting to the Dashboard
* Navigate to Dashboard → AIO → [Your Site] → Integration.
* Select the Automatic (API Token) tab.
* Paste the Cloudflare API Token and click Connect.
Executing Deployment
* Select the zone matching your domain from the dropdown.
* Click Deploy Worker to start the automated script upload.
* AgentFi creates route bindings to handle domain requests.
* The system runs a verification check to confirm status.
What does the AgentFi Worker do?
The Worker acts as a transparent proxy that intercepts requests for AI-optimized discovery files while passing all other traffic to the origin server.
| Path | Description |
|---|---|
| /llms.txt | AI-optimized discovery file |
| /llms-full.txt | Full context discovery file |
| /sitemap.xml | Sitemap with AI-optimized pages |
| /robots.txt | Dynamic robots.txt with agent directives |
| /.well-known/ucp | Unified Commerce Protocol profile |
| /agentfi/pages/* | AI-optimized page versions |
| /agentfi/health | Integration health check endpoint |
The Worker also tracks visits from the following AI bots for dashboard analytics:
* GPTBot
* ClaudeBot
* PerplexityBot
Is the AgentFi Cloudflare integration secure?
AgentFi uses minimal permission sets and secure storage protocols to maintain the security of your Cloudflare account.
* API tokens are stored securely by AgentFi.
* Tokens only grant Worker script and route write access.
* Users can revoke tokens via Cloudflare at any time.
* Revocation prevents AgentFi from updating or redeploying the Worker.
* Workers can be removed via the AgentFi Integration page.
How does AgentFi monitor integration health?
AgentFi maintains a continuous monitoring schedule to ensure the proxy remains active and correctly configured.
* Health checks run automatically every 10 minutes.
* Dashboard displays warnings if the Worker is unreachable.
* Integrations pause automatically if the route is changed.
* Users can click Redeploy Worker to fix connectivity issues.
Source: https://agentfi.tech/docs/cloudflare-worker-setup