Scrape Any Website
Your employees can read the full content of any public webpage. Just give them a URL and they'll extract clean, readable text.
TL;DR
Every employee can read the full content of any public webpage. Paste a URL and they extract clean, readable text with no ads or clutter. No setup required.
How It Works
The employee fetches the page, strips boilerplate (headers, footers, ads, navigation), and returns clean text you can work with.
What It Can Do
| Capability | Example |
|---|---|
| Read articles | "Read this blog post and summarize the key points" |
| Extract data | "Pull the pricing from this page" |
| Research competitors | "Read their features page and compare to ours" |
| Process documentation | "Read this API doc and write me a quick-start guide" |
| Digest long content | "Read this 5,000-word report and give me the top 3 takeaways" |
How to Set It Up
Nothing to do. Web Scraper is enabled for every employee at hire.
You can ask the employee to manage their own tools, or do it manually:
- Select the employee
- Click the Tools tab
- Find "Web Scraper" in the Actions section
| Action | What it does | How |
|---|---|---|
| Enable / Disable | Controls whether the employee can use this tool | Toggle the switch |
| Tool Rules | Custom instructions that guide how the employee uses this specific tool, e.g. "always extract tables as markdown" or "skip navigation content" | Expand the tool, then write your rules in the text field |
| Delete | Permanently removes the tool from the employee | Click the delete button |
Tips & Tricks
- Give the direct URL. Not a search results page or redirect link
- Ask for specific extraction. "Read this page and pull out only the pricing table" works better than "read this"
- Combine with search. "Search for X, then read the top 3 results" lets the employee use both tools together
- Say when a site is a JavaScript-heavy app. If you know the page is a single-page app or dashboard, mention that and the employee will render it with a headless browser right away instead of trying the fast path first
Behind the Scene
| How it works | A staged extraction pipeline. Each stage only runs if the one before it comes up empty or thin |
| Stage | Method | Handles |
|---|---|---|
| 1. Fast text extraction | Lightweight fetch, no browser | Most pages: articles, docs, blogs, news |
| 2. Browser User-Agent retry | Same fetch, browser-like headers | Sites that block bots but don't require JavaScript |
| 3. Headless browser rendering | A real headless browser we run ourselves | JavaScript-heavy single-page apps and dashboards. No Desktop companion app needed |
| 4. Desktop companion browser | Your own browser, via the Desktop companion app | The rare page that defeats stages 1-3, only if you have the companion app connected |
Most pages resolve on stage 1 in under a second. Stage 3 handles the majority of JavaScript-heavy sites automatically, no extra setup required.
Web Scraper vs Web Search vs Browser Controller
| Web Scraper | Web Search | Browser Controller | |
|---|---|---|---|
| Purpose | Read full content from a specific URL | Find information via search engine | Control a real browser. Navigate, click, fill forms |
| Input | A URL you already have | A question or topic | Instructions like "go to this site and..." |
| Output | Full page text, clean, no ads or nav | Titles, snippets, and source links | Screenshots, extracted data, completed actions |
| Best for | "Read this article for me" | "What's happening with X?" | "Log into this site and download the report" |
| When to use | You know the exact page | You don't know where to look | You need to interact with a page |
| Requires | Nothing, built in | Nothing, built in | Desktop companion app |
The employee often combines these automatically, searching first to find URLs, then reading the best results in full.
What It Costs
| Cost | Free. Reading a page costs no credits |
| Exception | If every other stage fails and the employee falls back to your Desktop companion browser, that fallback is billed as Browser Controller usage, not as Web Scraper |
| Rate limits | None from our side, but the target website may block rapid consecutive requests |
| Truncation | Extremely long pages are truncated with a notice rather than cut silently. In practice, almost no real page hits the limit |
Is It Safe
- Public pages only. The scraper can only access publicly available content. No login-protected or paywalled content is accessible
- Results in your chat. Scraped content is summarized in the employee's response, which is saved in your conversation history like any other message
Good to Know
- Clean text only. The scraper strips all HTML, scripts, styles, and navigation. You get readable text, not raw markup
- No login-protected content. The scraper can only access publicly available pages. For authenticated content, use Browser Controller with the Desktop companion app
- JavaScript-heavy sites work without the Desktop app. Single-page apps and dashboards are rendered by a headless browser we run ourselves. The Desktop companion app is only needed for the rare page that defeats every other stage, or for content behind a login
Frequently Asked Questions
Q: Can the employee read PDFs from a URL? A: The scraper is optimized for HTML web pages. For PDFs, upload the file directly to the employee's chat. They can read uploaded documents natively.
Q: What happens with pages behind a login? A: The scraper can only access public pages. For authenticated content, use Browser Controller. It uses your actual browser session, so any site you're logged into is accessible.
Q: Does the employee cache scraped pages? A: No. Every scrape fetches the live page, so you always get current content.
Q: Can I scrape multiple pages at once? A: Yes. Give the employee a list of URLs and they'll read each one. They may do this automatically when combining Web Search and Web Scraper.
Q: Why is the content truncated? A: Only extremely long pages get truncated, and it happens with a clear notice rather than silently. If you need the full page, ask the employee to focus on a specific section or split the request.