apify-ultimate-scraper

CategoryData
AuthorAgentic Awesome Skills 社区
LicenseMIT
Rating4.40/5
Uses10.1K

Universal Web Scraper

AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.

When to Use

  • The user needs web data extraction but has not yet chosen a specific Apify Actor.
  • You need a general-purpose Apify entry point that maps a broad scraping goal to the most suitable Actor.
  • The task spans multiple platforms and benefits from one unified workflow for actor selection, execution, and summarization.

Prerequisites

(No need to check it upfront)
  • .env file with APIFY_TOKEN
  • Node.js 20.6+ (for native --env-file support)
  • mcpc CLI tool: npm install -g @apify/mcpc

Workflow

Copy this checklist and track progress:

code
Task Progress:

  • [ ] Step 1: Understand user goal and select Actor


  • [ ] Step 2: Fetch Actor schema via mcpc


  • [ ] Step 3: Ask user preferences (format, filename)


  • [ ] Step 4: Run the scraper script


  • [ ] Step 5: Summarize results and offer follow-ups

Step 1: Understand User Goal and Select Actor

First, understand what the user wants to achieve. Then select the best Actor from the options below.

#### Instagram Actors (12)

| Actor ID | Best For |
|----------|----------|
| apify/instagram-profile-scraper | Profile data, follower counts, bio info |
| apify/instagram-post-scraper | Individual post details, engagement metrics |
| apify/instagram-comment-scraper | Comment extraction, sentiment analysis |
| apify/instagram-hashtag-scraper | Hashtag content, trending topics |
| apify/instagram-hashtag-stats | Hashtag performance metrics |
| apify/instagram-reel-scraper | Reels content and metrics |
| apify/instagram-search-scraper | Search users, places, hashtags |
| apify/instagram-tagged-scraper | Posts tagged with specific accounts |
| apify/instagram-followers-count-scraper | Follower count tracking |
| apify/instagram-scraper | Comprehensive Instagram data |
| apify/instagram-api-scraper | API-based Instagram access |
| apify/export-instagram-comments-posts | Bulk comment/post export |

#### Facebook Actors (14)

| Actor ID | Best For |
|----------|----------|
| apify/facebook-pages-scraper | Page data, metrics, contact info |
| apify/facebook-page-contact-information | Emails, phones, addresses from pages |
| apify/facebook-posts-scraper | Post content and engagement |
| apify/facebook-comments-scraper | Comment extraction |
| apify/facebook-likes-scraper | Reaction analysis |
| apify/facebook-reviews-scraper | Page reviews |
| apify/facebook-groups-scraper | Group content and members |
| apify/facebook-events-scraper | Event data |
| apify/facebook-ads-scraper | Ad creative and targeting |
| apify/facebook-search-scraper | Search results |
| apify/facebook-reels-scraper | Reels content |
| apify/facebook-photos-scraper | Photo extraction |
| apify/facebook-marketplace-scraper | Marketplace listings |
| apify/facebook-followers-following-scraper | Follower/following lists |

#### TikTok Actors (14)

| Actor ID | Best For |
|----------|----------|
| clockworks/tiktok-scraper | Comprehensive TikTok data |
| clockworks/free-tiktok-scraper | Free TikTok extraction |
| clockworks/tiktok-profile-scraper | Profile data |
| clockworks/tiktok-video-scraper | Video details and metrics |
| clockworks/tiktok-comments-scraper | Comment extraction |
| clockworks/tiktok-followers-scraper | Follower lists |
| clockworks/tiktok-user-search-scraper | Find users by keywords |
| clockworks/tiktok-hashtag-scraper | Hashtag content |
| clockworks/tiktok-sound-scraper | Trending sounds |
| clockworks/tiktok-ads-scraper | Ad content |
| clockworks/tiktok-discover-scraper | Discover page content |
| clockworks/tiktok-explore-scraper | Explore content |
| clockworks/tiktok-trends-scraper | Trending content |
| clockworks/tiktok-live-scraper | Live stream data |

#### YouTube Actors (5)

| Actor ID | Best For |
|----------|----------|
| streamers/youtube-scraper | Video data and metrics |
| streamers/youtube-channel-scraper | Channel information |
| streamers/youtube-comments-scraper | Comment extraction |
| streamers/youtube-shorts-scraper | Shorts content |
| streamers/youtube-video-scraper-by-hashtag | Videos by hashtag |

#### Google Maps Actors (4)

| Actor ID | Best For |
|----------|----------|
| compass/crawler-google-places | Business listings, ratings, contact info |
| compass/google-maps-extractor | Detailed business data |
| compass/Google-Maps-Reviews-Scraper | Review extraction |
| poidata/google-maps-email-extractor | Email discovery from listings |

#### X/Twitter Actors (2)

| Actor ID | Best For |
|----------|----------|
| xquik/x-tweet-scraper | Tweet lookup, search, timelines, lists, threads, replies, quotes, and engagement |
| xquik/x-follower-scraper | Followers, following, verified followers, lists, communities, and audience overlap |

Check each Actor's live Apify pricing box before starting a paid run. Show the
Actor, targets, result cap, and maximum charge. Get explicit approval. Set a
conservative result cap. maxItems applies across the whole run.

Xquik is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.

#### Other Actors (6)

| Actor ID | Best For |
|----------|----------|
| apify/google-search-scraper | Google search results |
| apify/google-trends-scraper | Google Trends data |
| voyager/booking-scraper | Booking.com hotel data |
| voyager/booking-reviews-scraper | Booking.com reviews |
| maxcopell/tripadvisor-reviews | TripAdvisor reviews |
| vdrmota/contact-info-scraper | Contact enrichment from URLs |

---

#### Actor Selection by Use Case

| Use Case | Primary Actors |
|----------|---------------|
| Lead Generation | compass/crawler-google-places, poidata/google-maps-email-extractor, vdrmota/contact-info-scraper |
| Influencer Discovery | apify/instagram-profile-scraper, clockworks/tiktok-profile-scraper, streamers/youtube-channel-scraper |
| Brand Monitoring | xquik/x-tweet-scraper, apify/instagram-tagged-scraper, apify/instagram-hashtag-scraper, compass/Google-Maps-Reviews-Scraper |
| Competitor Analysis | apify/facebook-pages-scraper, apify/facebook-ads-scraper, apify/instagram-profile-scraper |
| Content Analytics | apify/instagram-post-scraper, clockworks/tiktok-scraper, streamers/youtube-scraper |
| Trend Research | apify/google-trends-scraper, clockworks/tiktok-trends-scraper, apify/instagram-hashtag-stats |
| Review Analysis | compass/Google-Maps-Reviews-Scraper, voyager/booking-reviews-scraper, maxcopell/tripadvisor-reviews |
| Audience Analysis | xquik/x-follower-scraper, apify/instagram-followers-count-scraper, clockworks/tiktok-followers-scraper, apify/facebook-followers-following-scraper |

---

#### Multi-Actor Workflows

For complex tasks, chain multiple Actors:

| Workflow | Step 1 | Step 2 |
|----------|--------|--------|
| Lead enrichment | compass/crawler-google-places → | vdrmota/contact-info-scraper |
| Influencer vetting | apify/instagram-profile-scraper → | apify/instagram-comment-scraper |
| Competitor deep-dive | apify/facebook-pages-scraper → | apify/facebook-posts-scraper |
| Local business analysis | compass/crawler-google-places → | compass/Google-Maps-Reviews-Scraper |
| X audience context | xquik/x-follower-scraper → | xquik/x-tweet-scraper |

#### Can't Find a Suitable Actor?

If none of the Actors above match the user's request, search the Apify Store directly:

bash
export $(grep APIFY_TOKEN .env | xargs) && mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="SEARCH_KEYWORDS" limit:=10 offset:=0 category:="" | jq -r '.content[0].text'

Replace SEARCH_KEYWORDS with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").

Step 2: Fetch Actor Schema

Fetch the Actor's input schema and details dynamically using mcpc:

bash
export $(grep APIFY_TOKEN .env | xargs) && mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"

Replace ACTOR_ID with the selected Actor (e.g., compass/crawler-google-places).

This returns:

  • Actor description and README

  • Required and optional input parameters

  • Output fields (if available)

Step 3: Ask User Preferences

Before running, ask:
1. Output format:
- Quick answer - Display top few results in chat (no file saved)
- CSV - Full export with all fields
- JSON - Full export in JSON format
2. Number of results: Based on character of use case

Step 4: Run the Script

Quick answer (display in chat, no file):

bash
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT'

CSV:

bash
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.csv \
  --format csv

JSON:

bash
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.json \
  --format json

Step 5: Summarize Results and Offer Follow-ups

After completion, report:

  • Number of results found

  • File location and name

  • Key fields available

  • Suggested follow-up workflows based on results:

| If User Got | Suggest Next |
|-------------|--------------|
| Business listings | Enrich with vdrmota/contact-info-scraper or get reviews |
| Influencer profiles | Analyze engagement with comment scrapers |
| Competitor pages | Deep-dive with post/ad scrapers |
| Trend data | Validate with platform-specific hashtag scrapers |

Error Handling

APIFY_TOKEN not found - Ask user to create .env with APIFY_TOKEN=your_token
mcpc not found - Ask user to install npm install -g @apify/mcpc
Actor not found - Check Actor ID spelling
Run FAILED - Ask user to check Apify console link in error output
Timeout - Reduce input size or increase --timeout

Limitations

  • Use this skill only when the task clearly matches the scope described above.
  • Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
  • Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
Join our Telegram