WebScraping.AI
Web scraping API to extract data from any website with one call. We handle proxies, browsers, CAPTCHAs and parsing. Get HTML, text, or AI-extracted JSON.
Authentication
| Method | Kind | Status | Details |
|---|---|---|---|
| API Key | api_key | available | — |
Call a tool
import { createClient } from "@open-connector/sdk";const oc = createClient({ baseUrl: "https://api.openconnector.dev", apiKey: process.env.OPEN_CONNECTOR_API_KEY!,});const result = await oc.executeTool({ slug: "WEBSCRAPING_AI_ACCOUNT", connectedAccountId: "conn_...", arguments: { /* match this tool's input schema */ },});import Composio from "@composio/client";const composio = new Composio({ baseURL: "https://api.openconnector.dev/composio", apiKey: process.env.OPEN_CONNECTOR_API_KEY!,});const result = await composio.tools.execute("WEBSCRAPING_AI_ACCOUNT", { connected_account_id: "conn_...", arguments: { /* match this tool's input schema */ },});oc tools execute WEBSCRAPING_AI_ACCOUNT --data '{ }'Tool catalog
Available tools
7 callable operations
Information about your account calls quotaWEBSCRAPING_AI_ACCOUNTReturns information about your account, including the remaining API credits quota, the next billing cycle start time, and the remaining concurrent requests. The response is in JSON format.Connection
Returns information about your account, including the remaining API credits quota, the next billing cycle start time, and the remaining concurrent requests. The response is in JSON format.
Authentication
Connected account requiredTags
Extract structured data fields from a web pageWEBSCRAPING_AI_GET_FIELDSReturns structured data fields extracted from the webpage using an LLM model. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.Connection
Returns structured data fields extracted from the webpage using an LLM model. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.
Authentication
Connected account requiredTags
Page HTML by URLWEBSCRAPING_AI_GET_HTMLReturns the full HTML content of a webpage specified by the URL. The response is in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.Connection
Returns the full HTML content of a webpage specified by the URL. The response is in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.
Authentication
Connected account requiredTags
Get an answer to a question about a given web pageWEBSCRAPING_AI_GET_QUESTIONReturns the answer in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing, then the answer is extracted using an LLM model.Connection
Returns the answer in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing, then the answer is extracted using an LLM model.
Authentication
Connected account requiredTags
HTML of a selected page area by URL and CSS selectorWEBSCRAPING_AI_GET_SELECTEDReturns HTML of a selected page area by URL and CSS selector. Useful if you don't want to do the HTML parsing on your side.Connection
Returns HTML of a selected page area by URL and CSS selector. Useful if you don't want to do the HTML parsing on your side.
Authentication
Connected account requiredTags
HTML of multiple page areas by URL and CSS selectorsWEBSCRAPING_AI_GET_SELECTED_MULTIPLEReturns HTML of multiple page areas by URL and CSS selectors. Useful if you don't want to do the HTML parsing on your side.Connection
Returns HTML of multiple page areas by URL and CSS selectors. Useful if you don't want to do the HTML parsing on your side.
Authentication
Connected account requiredTags
Page text by URL (Markdown)WEBSCRAPING_AI_GET_TEXTConverts a webpage to clean Markdown ("URL to Markdown") - boilerplate is stripped and the document structure (headings, lists, tables, links) is preserved. Can be used to feed data to LLM models and RAG pipelines. text_format=plain (default) returns the raw Markdown; "json" and "xml" wrap the same Markdown content with the page title and description. Proxies and Chromium JavaScript rendering are used for page retrieval and processing. Returns JSON on error.Connection
Converts a webpage to clean Markdown ("URL to Markdown") - boilerplate is stripped and the document structure (headings, lists, tables, links) is preserved. Can be used to feed data to LLM models and RAG pipelines. text_format=plain (default) returns the raw Markdown; "json" and "xml" wrap the same Markdown content with the page title and description. Proxies and Chromium JavaScript rendering are used for page retrieval and processing. Returns JSON on error.
Authentication
Connected account requiredTags