MCP Webscan Server

MCP Webscan Server is an MCP server that provides web content scanning, crawling, and analysis capabilities for Model Context Protocol clients. It connects directly to live web pages via HTTP requests, allowing AI assistants to process unstructured web data into structured Markdown. Designed for researchers, developers, and autonomous agents, the server enables automated exploration of external web domains directly from LLM interfaces. Users can convert full web pages or targeted CSS elements into Markdown, extract internal and external hyperlinks with associated anchor text, and conduct recursive crawls up to specified link depths. Furthermore, MCP Webscan Server identifies broken links across target sites, matches URL paths against regular expressions, and crawls domains to generate standard XML sitemaps. By utilizing standard input/output transport, it executes locally without requiring third-party cloud scraping services, giving users direct visibility into site architectures and page content within desktop LLM environments.

Category: Browser & Web Automation

Tags: data-extraction, html extraction, web crawling, web scraping

Visit MCP Webscan Server

How to install and configure MCP Webscan Server

1. Install the server automatically using Smithery by running: bash npx -y @smithery/cli install mcp-server-webscan --client claude 2. Alternatively, clone the repository and build manually: bash git clone https://github.com/bsmi021/mcp-server-webscan cd mcp-server-webscan npm install npm run build 3. Add the server to your Claude Desktop configuration file: json { "mcpServers": { "webscan": { "command": "node", "args": ["path/to/mcp-server-webscan/build/index.js"], "env": { "NODE_ENV": "development" } } } } 4. Restart Claude Desktop to use the webscan tools.

What you can do with MCP Webscan Server

• Convert targeted web pages into clean Markdown using specific CSS selectors for direct LLM synthesis. - Crawl full websites recursively up to specified depth limits to map internal page structures. - Extract all hyperlinks and anchor text from web pages to compile external source lists. - Scan domains to verify link integrity and automatically flag broken or unreachable URLs. - Search website link structures for specific patterns using JavaScript-compatible regular expressions. - Crawl a root domain to automatically generate and format XML sitemaps for site management.

Key facts

• https://github.com/bsmi021/mcp-server-webscan
• Browser & Web Automation, Web Search & Research
• data-extraction, html extraction, web crawling, web scraping

Part of MCP Servers

How do I install MCP Webscan Server?

You can install it automatically for Claude Desktop using Smithery by running npx -y @smithery/cli install mcp-server-webscan --client claude. Alternatively, clone the repository locally, install dependencies with npm install, run npm run build, and reference the build/index.js file in your MCP client configuration.

What tools are included in MCP Webscan Server?

The server includes six distinct tools: fetch-page to convert pages to Markdown, extract-links to collect page links and text, crawl-site to recursively crawl domains, check-links to test for broken URLs, find-patterns to filter links by regex, and generate-site-map to output XML sitemaps.

Which MCP clients work with MCP Webscan Server?

The server uses standard input and output (stdio) transport, making it compatible with Claude Desktop and any client or agent framework that supports the Model Context Protocol stdio specification.

What are the system requirements for running MCP Webscan Server?

The server requires Node.js version 18 or higher and npm installed on your system to run the build commands and execute the Node runtime environment.

Is MCP Webscan Server open source?

Yes, MCP Webscan Server is open source and licensed under the MIT License according to the project documentation.