Cloudflare Worker Setup for AgentFi
AgentFi enables automated AI search engine optimization through a one-click Cloudflare Worker deployment using a secure API Token. The Worker serves as a transparent proxy, delivering AI-optimized discovery files such as /llms.txt and /sitemap.xml directly from your domain. Once deployed, the system performs automated health checks every 10 minutes to ensure continuous integration. This setup requires no manual server configuration and supports free Cloudflare Workers plans.
What are the prerequisites for deployment?
To use the automatic setup, you must have an active AgentFi account with a crawled site and a managed Cloudflare domain.
* A Cloudflare account with an active, orange-clouded domain (zone).
* The domain must utilize Cloudflare DNS services.
* A Cloudflare Workers plan (the free tier is sufficient).
* An AgentFi account with at least one site added and crawled.
For users who prefer not to share API credentials, AgentFi offers a Manual Worker Setup option.
How do I set up the Cloudflare Worker automatically?
Automatic deployment is completed through the AgentFi dashboard by connecting a Cloudflare API Token with specific "Edit Cloudflare Workers" permissions.
1. Create a Cloudflare API Token
Users must generate a token via the Cloudflare API Tokens page using the following steps:
* Select the "Edit Cloudflare Workers" template.
* Assign the specific Account and Zone resources for your domain.
* Verify permissions include Workers script and route write access.
* Copy and secure the token immediately upon creation.
2. Connect the AgentFi Dashboard
As of 2024, the integration is finalized within the AgentFi interface:
* Navigate to Dashboard → AIO → [Your Site] → Integration.
* Select the "Automatic (API Token)" tab.
* Submit the token to validate account connectivity.
3. Deploy and Verify
AgentFi automates the final technical configuration:
* Select the matching Cloudflare zone from the dropdown menu.
* Click "Deploy Worker" to upload the script and bind routes.
* The system runs an immediate verification to confirm the endpoint is live.
How does the AgentFi Worker function?
The AgentFi Worker operates as a transparent proxy built on Cloudflare Workers to intercept specific AI-related requests while passing all other traffic to your origin.
| Path | Description | Specification |
|---|---|---|
/llms.txt | AI-optimized discovery file | llmstxt.org |
/llms-full.txt | Full context discovery file | llmstxt.org |
/sitemap.xml | Optimized sitemap for AI crawlers | Sitemaps Protocol |
/robots.txt | Dynamic directives for AI agents | Robots Exclusion Protocol |
/.well-known/ucp | Unified Commerce Protocol profile | UCP Spec |
/agentfi/pages/* | AI-optimized page versions | Internal |
/agentfi/health | Integration health check endpoint | Internal |
The Worker also logs visits from major AI agents, including GPTBot, ClaudeBot, and PerplexityBot, to provide specialized analytics within your dashboard.
Is the AgentFi Worker integration secure?
Integration security is maintained by adhering to the principle of least privilege and utilizing standard Cloudflare security protocols.
* Tokens only grant minimum permissions required for script and route management.
* API tokens are stored securely and used exclusively for Worker updates.
* Users retain full control and can revoke tokens at any time.
* Disconnecting the service automatically removes the Worker from the Cloudflare account.
How is integration health monitored?
AgentFi maintains integration reliability through a continuous, automated monitoring schedule.
* The system performs health checks every 10 minutes.
* Automated alerts trigger if the Worker becomes unreachable or routes change.
* Integrations are paused automatically if critical errors are detected.
* The "Redeploy Worker" feature allows for instant resolution of configuration drift.