Web Scraper connector.

Scrape website pages, discover script-generated catalog routes, filter fallback duplicates, and format clean JSON records for DynamoDB or AI context.

Download connector See all connectors

Connector download

Download Web Scraper

Versioned ZIP for your CXFabric tenant.

Register the ZIP in your tenant, or add a new version if the connector is already installed.
Create or select a credential if the action requires one.
In a Draft flow, add an available action, choose its version, connect it, and map inputs.
Test the step and flow; review output and failures before activation.

See readme.md in the ZIP for settings and limitations.

Version history

v0.2.0.2Latest · 2026-10-25 · Implemented source
Security update: public HTTPS/DNS checks, robots rules, disabled redirects, bounded requests/results/time, linear URL matching and safer diagnostics.
Full release notes View download
v0.2.0.1Earlier · 2026-10-25 · Implemented source
First public tenant-based release: compatible named inputs, explicit complete-fetch failures, updated dependencies and documented crawl limits.
Full release notes View download

Included in this version

Implemented in v0.2.0.2. Use a separate flow trigger; no webhook trigger is included.

Scrape https://ianswer.io/ and return a JSON object under ianswer_site.

Scrape a static website and return AI-ready JSON context records.

Vendor API

Implementation considerations

Supported scope: Fetch approved static HTML websites, follow links and optionally discover literal script routes into structured content records.
Public HTTPS hostnames only, socket DNS checks reject private addresses, redirects/proxies are disabled. robots.txt is checked before HTML and scripts; disallowed resources and positive crawl-delay fail closed.
Up to25 page visits,64 requests including robots/scripts and200 discovered URLs within an8-second maximum action budget. Input, response and result size limits apply. UTF-8 uncompressed responses only.
Only Accept-Language custom headers are supported. URL patterns use linear-time RE2 matching without lookaround/backreferences. Diagnostics omit URL query values and raw transport errors.
Fetched and curated records carry provenance. The legacy iAnswer FAQ remains curated compatibility content. Local extraction/security tests passed; this new version has not been imported or executed in a tenant.

Support

Tell us what you need to connect or change.