Apify Ultimate Scraper
Universal AI-powered web scraper for any platform.
- Rating
- 4.7 (463 reviews)
- Downloads
- 6,119 downloads
- Version
- 1.0.0
Overview
Universal AI-powered web scraper for any platform.
✨Key Features
Understand User Goal and Select Actor
Fetch Actor Schema
Ask User Preferences
Run the Script
Summarize Results and Offer Follow-ups
Complete Documentation
View Source →
Universal Web Scraper
AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.
Prerequisites
APIFY_TOKENconfigured in OpenClaw settings- Node.js 20.6+
mcpcCLI (auto-installed via skill metadata)
Input Sanitization Rules
Before substituting any value into a bash command:
- ACTOR_ID: Must be either a technical name (
owner/actor-name— alphanumeric, hyphens, dots, one slash) or a raw ID (exactly 17 alphanumeric characters, e.g.,oeiQgfg5fsmIJB7Cn). Reject values containing shell metacharacters (`; | & $( ) { } < > ! \n `). - SEARCH_KEYWORDS: Plain text words only. Reject shell metacharacters.
- JSON_INPUT: Must be valid JSON. Must not contain single quotes (use escaped double quotes). Validate structure before use.
- Output filenames: Must match YYYY-MM-DD_descriptive-name.{csv,json}
. No path separators (/,..), no spaces, no metacharacters.
Workflow
Copy this checklist and track progress:
Task Progress:
- [ ] Step 1: Understand user goal and select Actor
- [ ] Step 2: Fetch Actor schema via mcpc
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the scraper script
- [ ] Step 5: Summarize results and offer follow-ups
Step 1: Understand User Goal and Select Actor
First, understand what the user wants to achieve. Then select the best Actor from the options below.
#### Instagram Actors (12)
| Actor ID | Best For |
|---|---|
| apify/instagram-profile-scraper | Profile data, follower counts, bio info |
| apify/instagram-post-scraper | Individual post details, engagement metrics |
| apify/instagram-comment-scraper | Comment extraction, sentiment analysis |
| apify/instagram-hashtag-scraper | Hashtag content, trending topics |
| apify/instagram-hashtag-stats | Hashtag performance metrics |
| apify/instagram-reel-scraper | Reels content and metrics |
| apify/instagram-search-scraper | Search users, places, hashtags |
| apify/instagram-tagged-scraper | Posts tagged with specific accounts |
| apify/instagram-followers-count-scraper | Follower count tracking |
| apify/instagram-scraper | Comprehensive Instagram data |
| apify/instagram-api-scraper | API-based Instagram access |
| apify/export-instagram-comments-posts | Bulk comment/post export |
| Actor ID | Best For |
|---|---|
| apify/facebook-pages-scraper | Page data, metrics, contact info |
| apify/facebook-page-contact-information | Emails, phones, addresses from pages |
| apify/facebook-posts-scraper | Post content and engagement |
| apify/facebook-comments-scraper | Comment extraction |
| apify/facebook-likes-scraper | Reaction analysis |
| apify/facebook-reviews-scraper | Page reviews |
| apify/facebook-groups-scraper | Group content and members |
| apify/facebook-events-scraper | Event data |
| apify/facebook-ads-scraper | Ad creative and targeting |
| apify/facebook-search-scraper | Search results |
| apify/facebook-reels-scraper | Reels content |
| apify/facebook-photos-scraper | Photo extraction |
| apify/facebook-marketplace-scraper | Marketplace listings |
| apify/facebook-followers-following-scraper | Follower/following lists |
| Actor ID | Best For |
|---|---|
| clockworks/tiktok-scraper | Comprehensive TikTok data |
| clockworks/free-tiktok-scraper | Free TikTok extraction |
| clockworks/tiktok-profile-scraper | Profile data |
| clockworks/tiktok-video-scraper | Video details and metrics |
| clockworks/tiktok-comments-scraper | Comment extraction |
| clockworks/tiktok-followers-scraper | Follower lists |
| clockworks/tiktok-user-search-scraper | Find users by keywords |
| clockworks/tiktok-hashtag-scraper | Hashtag content |
| clockworks/tiktok-sound-scraper | Trending sounds |
| clockworks/tiktok-ads-scraper | Ad content |
| clockworks/tiktok-discover-scraper | Discover page content |
| clockworks/tiktok-explore-scraper | Explore content |
| clockworks/tiktok-trends-scraper | Trending content |
| clockworks/tiktok-live-scraper | Live stream data |
| Actor ID | Best For |
|---|---|
| streamers/youtube-scraper | Video data and metrics |
| streamers/youtube-channel-scraper | Channel information |
| streamers/youtube-comments-scraper | Comment extraction |
| streamers/youtube-shorts-scraper | Shorts content |
| streamers/youtube-video-scraper-by-hashtag | Videos by hashtag |
| Actor ID | Best For |
|---|---|
| compass/crawler-google-places | Business listings, ratings, contact info |
| compass/google-maps-extractor | Detailed business data |
| compass/Google-Maps-Reviews-Scraper | Review extraction |
| poidata/google-maps-email-extractor | Email discovery from listings |
| Actor ID | Best For |
|---|---|
| apify/google-search-scraper | Google search results |
| apify/google-trends-scraper | Google Trends data |
| voyager/booking-scraper | Booking.com hotel data |
| voyager/booking-reviews-scraper | Booking.com reviews |
| maxcopell/tripadvisor-reviews | TripAdvisor reviews |
| vdrmota/contact-info-scraper | Contact enrichment from URLs |
#### Actor Selection by Use Case
| Use Case | Primary Actors |
|---|---|
| Lead Generation | compass/crawler-google-places, poidata/google-maps-email-extractor, vdrmota/contact-info-scraper |
| Influencer Discovery | apify/instagram-profile-scraper, clockworks/tiktok-profile-scraper, streamers/youtube-channel-scraper |
| Brand Monitoring | apify/instagram-tagged-scraper, apify/instagram-hashtag-scraper, compass/Google-Maps-Reviews-Scraper |
| Competitor Analysis | apify/facebook-pages-scraper, apify/facebook-ads-scraper, apify/instagram-profile-scraper |
| Content Analytics | apify/instagram-post-scraper, clockworks/tiktok-scraper, streamers/youtube-scraper |
| Trend Research | apify/google-trends-scraper, clockworks/tiktok-trends-scraper, apify/instagram-hashtag-stats |
| Review Analysis | compass/Google-Maps-Reviews-Scraper, voyager/booking-reviews-scraper, maxcopell/tripadvisor-reviews |
| Audience Analysis | apify/instagram-followers-count-scraper, clockworks/tiktok-followers-scraper, apify/facebook-followers-following-scraper |
#### Multi-Actor Workflows
For complex tasks, chain multiple Actors:
| Workflow | Step 1 | Step 2 |
|---|---|---|
| Lead enrichment | compass/crawler-google-places → | vdrmota/contact-info-scraper |
| Influencer vetting | apify/instagram-profile-scraper → | apify/instagram-comment-scraper |
| Competitor deep-dive | apify/facebook-pages-scraper → | apify/facebook-posts-scraper |
| Local business analysis | compass/crawler-google-places → | compass/Google-Maps-Reviews-Scraper |
If none of the Actors above match the user's request, search the Apify Store directly:
mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="SEARCH_KEYWORDS" limit:=10 offset:=0 category:="" | jq -r '.content[0].text'
Replace SEARCH_KEYWORDS with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").
Step 2: Fetch Actor Schema
Fetch the Actor's input schema and details dynamically using mcpc:
mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"
Replace ACTOR_ID with the selected Actor (e.g., compass/crawler-google-places).
This returns:
- Actor description and README
- Required and optional input parameters
- Output fields (if available)
Step 3: Ask User Preferences
Before running, ask:
- Output format:
- Quick answer - Display top few results in chat (no file saved)
- CSV - Full export with all fields
- JSON - Full export in JSON format
- Number of results: Based on character of use case
Step 4: Run the Script
Quick answer (display in chat, no file):
node {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT'
CSV:
node {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT' \
--output 'YYYY-MM-DD_OUTPUT_FILE.csv' \
--format csv
JSON:
node {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT' \
--output 'YYYY-MM-DD_OUTPUT_FILE.json' \
--format json
Step 5: Summarize Results and Offer Follow-ups
After completion, report:
- Number of results found
- File location and name
- Key fields available
- Suggested follow-up workflows based on results:
| If User Got | Suggest Next |
|---|---|
| Business listings | Enrich with vdrmota/contact-info-scraper or get reviews |
| Influencer profiles | Analyze engagement with comment scrapers |
| Competitor pages | Deep-dive with post/ad scrapers |
| Trend data | Validate with platform-specific hashtag scrapers |
Security & Data Privacy
This skill instructs the agent to select an Apify Actor, fetch its schema (via mcpc), and run scrapers. The included script communicates only with api.apify.com and writes outputs to files under the current working directory; it does not access unrelated system files or other environment variables.
Apify Actors only scrape publicly available data and do not collect private or personally identifiable information beyond what is openly accessible on the target platforms. For additional security assurance, you can check an Actor's permission level by querying https://api.apify.com/v2/acts/:actorId — an Actor with LIMITED_PERMISSIONS operates in a restricted sandbox, while FULL_PERMISSIONS indicates broader system access. For full details, see Apify's General Terms and Conditions.
Error Handling
APIFY_TOKEN not found - Ask user to configure APIFY_TOKEN in OpenClaw settings
mcpc not found - Run npm install -g @apify/mcpc
Actor not found - Check Actor ID spelling
Run FAILED - Ask user to check Apify console link in error output
Timeout - Reduce input size or increase --timeout`
Installation
openclaw install apify-ultimate-scraper
💻Code Examples
- [ ] Step 5: Summarize results and offer follow-ups
### Step 1: Understand User Goal and Select Actor
First, understand what the user wants to achieve. Then select the best Actor from the options below.
#### Instagram Actors (12)
| Actor ID | Best For |
|----------|----------|
| `apify/instagram-profile-scraper` | Profile data, follower counts, bio info |
| `apify/instagram-post-scraper` | Individual post details, engagement metrics |
| `apify/instagram-comment-scraper` | Comment extraction, sentiment analysis |
| `apify/instagram-hashtag-scraper` | Hashtag content, trending topics |
| `apify/instagram-hashtag-stats` | Hashtag performance metrics |
| `apify/instagram-reel-scraper` | Reels content and metrics |
| `apify/instagram-search-scraper` | Search users, places, hashtags |
| `apify/instagram-tagged-scraper` | Posts tagged with specific accounts |
| `apify/instagram-followers-count-scraper` | Follower count tracking |
| `apify/instagram-scraper` | Comprehensive Instagram data |
| `apify/instagram-api-scraper` | API-based Instagram access |
| `apify/export-instagram-comments-posts` | Bulk comment/post export |
#### Facebook Actors (14)
| Actor ID | Best For |
|----------|----------|
| `apify/facebook-pages-scraper` | Page data, metrics, contact info |
| `apify/facebook-page-contact-information` | Emails, phones, addresses from pages |
| `apify/facebook-posts-scraper` | Post content and engagement |
| `apify/facebook-comments-scraper` | Comment extraction |
| `apify/facebook-likes-scraper` | Reaction analysis |
| `apify/facebook-reviews-scraper` | Page reviews |
| `apify/facebook-groups-scraper` | Group content and members |
| `apify/facebook-events-scraper` | Event data |
| `apify/facebook-ads-scraper` | Ad creative and targeting |
| `apify/facebook-search-scraper` | Search results |
| `apify/facebook-reels-scraper` | Reels content |
| `apify/facebook-photos-scraper` | Photo extraction |
| `apify/facebook-marketplace-scraper` | Marketplace listings |
| `apify/facebook-followers-following-scraper` | Follower/following lists |
#### TikTok Actors (14)
| Actor ID | Best For |
|----------|----------|
| `clockworks/tiktok-scraper` | Comprehensive TikTok data |
| `clockworks/free-tiktok-scraper` | Free TikTok extraction |
| `clockworks/tiktok-profile-scraper` | Profile data |
| `clockworks/tiktok-video-scraper` | Video details and metrics |
| `clockworks/tiktok-comments-scraper` | Comment extraction |
| `clockworks/tiktok-followers-scraper` | Follower lists |
| `clockworks/tiktok-user-search-scraper` | Find users by keywords |
| `clockworks/tiktok-hashtag-scraper` | Hashtag content |
| `clockworks/tiktok-sound-scraper` | Trending sounds |
| `clockworks/tiktok-ads-scraper` | Ad content |
| `clockworks/tiktok-discover-scraper` | Discover page content |
| `clockworks/tiktok-explore-scraper` | Explore content |
| `clockworks/tiktok-trends-scraper` | Trending content |
| `clockworks/tiktok-live-scraper` | Live stream data |
#### YouTube Actors (5)
| Actor ID | Best For |
|----------|----------|
| `streamers/youtube-scraper` | Video data and metrics |
| `streamers/youtube-channel-scraper` | Channel information |
| `streamers/youtube-comments-scraper` | Comment extraction |
| `streamers/youtube-shorts-scraper` | Shorts content |
| `streamers/youtube-video-scraper-by-hashtag` | Videos by hashtag |
#### Google Maps Actors (4)
| Actor ID | Best For |
|----------|----------|
| `compass/crawler-google-places` | Business listings, ratings, contact info |
| `compass/google-maps-extractor` | Detailed business data |
| `compass/Google-Maps-Reviews-Scraper` | Review extraction |
| `poidata/google-maps-email-extractor` | Email discovery from listings |
#### Other Actors (6)
| Actor ID | Best For |
|----------|----------|
| `apify/google-search-scraper` | Google search results |
| `apify/google-trends-scraper` | Google Trends data |
| `voyager/booking-scraper` | Booking.com hotel data |
| `voyager/booking-reviews-scraper` | Booking.com reviews |
| `maxcopell/tripadvisor-reviews` | TripAdvisor reviews |
| `vdrmota/contact-info-scraper` | Contact enrichment from URLs |
---
#### Actor Selection by Use Case
| Use Case | Primary Actors |
|----------|---------------|
| **Lead Generation** | `compass/crawler-google-places`, `poidata/google-maps-email-extractor`, `vdrmota/contact-info-scraper` |
| **Influencer Discovery** | `apify/instagram-profile-scraper`, `clockworks/tiktok-profile-scraper`, `streamers/youtube-channel-scraper` |
| **Brand Monitoring** | `apify/instagram-tagged-scraper`, `apify/instagram-hashtag-scraper`, `compass/Google-Maps-Reviews-Scraper` |
| **Competitor Analysis** | `apify/facebook-pages-scraper`, `apify/facebook-ads-scraper`, `apify/instagram-profile-scraper` |
| **Content Analytics** | `apify/instagram-post-scraper`, `clockworks/tiktok-scraper`, `streamers/youtube-scraper` |
| **Trend Research** | `apify/google-trends-scraper`, `clockworks/tiktok-trends-scraper`, `apify/instagram-hashtag-stats` |
| **Review Analysis** | `compass/Google-Maps-Reviews-Scraper`, `voyager/booking-reviews-scraper`, `maxcopell/tripadvisor-reviews` |
| **Audience Analysis** | `apify/instagram-followers-count-scraper`, `clockworks/tiktok-followers-scraper`, `apify/facebook-followers-following-scraper` |
---
#### Multi-Actor Workflows
For complex tasks, chain multiple Actors:
| Workflow | Step 1 | Step 2 |
|----------|--------|--------|
| **Lead enrichment** | `compass/crawler-google-places` → | `vdrmota/contact-info-scraper` |
| **Influencer vetting** | `apify/instagram-profile-scraper` → | `apify/instagram-comment-scraper` |
| **Competitor deep-dive** | `apify/facebook-pages-scraper` → | `apify/facebook-posts-scraper` |
| **Local business analysis** | `compass/crawler-google-places` → | `compass/Google-Maps-Reviews-Scraper` |
#### Can't Find a Suitable Actor?
If none of the Actors above match the user's request, search the Apify Store directly:mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="SEARCH_KEYWORDS" limit:=10 offset:=0 category:="" | jq -r '.content[0].text'
Replace `SEARCH_KEYWORDS` with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").
### Step 2: Fetch Actor Schema
Fetch the Actor's input schema and details dynamically using mcpc:mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"
Replace `ACTOR_ID` with the selected Actor (e.g., `compass/crawler-google-places`).
This returns:
- Actor description and README
- Required and optional input parameters
- Output fields (if available)
### Step 3: Ask User Preferences
Before running, ask:
1. **Output format**:
- **Quick answer** - Display top few results in chat (no file saved)
- **CSV** - Full export with all fields
- **JSON** - Full export in JSON format
2. **Number of results**: Based on character of use case
### Step 4: Run the Script
**Quick answer (display in chat, no file):**Task Progress:
- [ ] Step 1: Understand user goal and select Actor
- [ ] Step 2: Fetch Actor schema via mcpc
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the scraper script
- [ ] Step 5: Summarize results and offer follow-upsnode {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT'node {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT' \
--output 'YYYY-MM-DD_OUTPUT_FILE.csv' \
--format csvnode {baseDir}/reference/scripts/run_actor.js \
--actor 'ACTOR_ID' \
--input 'JSON_INPUT' \
--output 'YYYY-MM-DD_OUTPUT_FILE.json' \
--format jsonTags
Quick Info
Ready to Install?
Get started with this skill in seconds
Related Skills
4claw
4claw — a moderated imageboard for AI agents.
Aap Passport
Agent Attestation Protocol - The Reverse Turing Test.
Acestep Lyrics Transcription
Transcribe audio to timestamped lyrics using OpenAI Whisper or ElevenLabs Scribe API.
Adaptive Suite
A continuously adaptive skill suite that empowers Clawdbot.