WebScraping.AI
Web scraping API to extract data from any website with one call. We handle proxies, browsers, CAPTCHAs and parsing. Get HTML, text, or AI-extracted JSON.
认证方式
| 方式 | 底层类型 | 状态 | 说明 |
|---|---|---|---|
| API Key | api_key | available | — |
调用示例
import { createClient } from "@open-connector/sdk";const oc = createClient({ baseUrl: "https://api.openconnector.dev", apiKey: process.env.OPEN_CONNECTOR_API_KEY!,});const result = await oc.executeTool({ slug: "WEBSCRAPING_AI_ACCOUNT", connectedAccountId: "conn_...", arguments: { /* match this tool's input schema */ },});import Composio from "@composio/client";const composio = new Composio({ baseURL: "https://api.openconnector.dev/composio", apiKey: process.env.OPEN_CONNECTOR_API_KEY!,});const result = await composio.tools.execute("WEBSCRAPING_AI_ACCOUNT", { connected_account_id: "conn_...", arguments: { /* match this tool's input schema */ },});oc tools execute WEBSCRAPING_AI_ACCOUNT --data '{ }'Tool 目录
可用 Tools
7 个可调用操作
Information about your account calls quotaWEBSCRAPING_AI_ACCOUNTReturns information about your account, including the remaining API credits quota, the next billing cycle start time, and the remaining concurrent requests. The response is in JSON format.需要连接
Returns information about your account, including the remaining API credits quota, the next billing cycle start time, and the remaining concurrent requests. The response is in JSON format.
认证
需要 Connected AccountTags
Extract structured data fields from a web pageWEBSCRAPING_AI_GET_FIELDSReturns structured data fields extracted from the webpage using an LLM model. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.需要连接
Returns structured data fields extracted from the webpage using an LLM model. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.
认证
需要 Connected AccountTags
Page HTML by URLWEBSCRAPING_AI_GET_HTMLReturns the full HTML content of a webpage specified by the URL. The response is in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.需要连接
Returns the full HTML content of a webpage specified by the URL. The response is in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.
认证
需要 Connected AccountTags
Get an answer to a question about a given web pageWEBSCRAPING_AI_GET_QUESTIONReturns the answer in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing, then the answer is extracted using an LLM model.需要连接
Returns the answer in plain text. Proxies and Chromium JavaScript rendering are used for page retrieval and processing, then the answer is extracted using an LLM model.
认证
需要 Connected AccountTags
HTML of a selected page area by URL and CSS selectorWEBSCRAPING_AI_GET_SELECTEDReturns HTML of a selected page area by URL and CSS selector. Useful if you don't want to do the HTML parsing on your side.需要连接
Returns HTML of a selected page area by URL and CSS selector. Useful if you don't want to do the HTML parsing on your side.
认证
需要 Connected AccountTags
HTML of multiple page areas by URL and CSS selectorsWEBSCRAPING_AI_GET_SELECTED_MULTIPLEReturns HTML of multiple page areas by URL and CSS selectors. Useful if you don't want to do the HTML parsing on your side.需要连接
Returns HTML of multiple page areas by URL and CSS selectors. Useful if you don't want to do the HTML parsing on your side.
认证
需要 Connected AccountTags
Page text by URL (Markdown)WEBSCRAPING_AI_GET_TEXTConverts a webpage to clean Markdown ("URL to Markdown") - boilerplate is stripped and the document structure (headings, lists, tables, links) is preserved. Can be used to feed data to LLM models and RAG pipelines. text_format=plain (default) returns the raw Markdown; "json" and "xml" wrap the same Markdown content with the page title and description. Proxies and Chromium JavaScript rendering are used for page retrieval and processing. Returns JSON on error.需要连接
Converts a webpage to clean Markdown ("URL to Markdown") - boilerplate is stripped and the document structure (headings, lists, tables, links) is preserved. Can be used to feed data to LLM models and RAG pipelines. text_format=plain (default) returns the raw Markdown; "json" and "xml" wrap the same Markdown content with the page title and description. Proxies and Chromium JavaScript rendering are used for page retrieval and processing. Returns JSON on error.
认证
需要 Connected AccountTags