Playwright Stealth: Scraping Modern React/Vue Apps Without Getting Blocked

I spent two weeks trying to scrape a client's React dashboard. Requests returned empty divs. Selenium got blocked immediately. Regular Playwright lasted about 20 requests before hitting a 403.

Eventually figured out the problem: modern SPAs (React, Vue, Angular) render everything client-side, and they've gotten really good at detecting automation. The content you want appears seconds after page load, and if you're using WebDriver, they know.

Why SPAs break traditional scrapers

The HTML source never contains the product data. It arrives via XHR calls after page load, then gets inserted into the DOM. You need a browser that executes JavaScript.

Installing the tools

pip install playwright playwright-stealth
playwright install chromium

playwright-stealth is a Python port of the puppeteer-extra stealth plugin. It's maintained separately from Playwright itself.

Getting stealth to work

Basic setup to avoid detection:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()

    # This patches the browser
    stealth_sync(page)

    page.goto('https://facebook.com')
    page.wait_for_selector('.data-loaded')

    content = page.inner_text('.data-loaded')
    print(content)

    browser.close()

Always test with headless=False first. You can see what's happening and verify stealth is working.

How stealth patches the browser

The plugin modifies several things bot detection scripts look for:

Use the code recorder

This is the killer feature. No more inspecting elements or guessing CSS selectors:

# Start recording
playwright codegen https://facebook.com

What happens:

1. Browser opens with the site loaded
2. You click around, fill forms, scroll
3. Code generates in the sidebar in real-time
4. Copy and paste when done

Generated output:

from playwright.sync_api import Page, expect

def run(page: Page) -> None:
    page.goto("https://facebook.com/")
    page.get_by_role("button", name="Load More").click()
    page.wait_for_selector(".product-card")

    products = page.locator(".product-card").all()
    for product in products:
        print(product.inner_text())

Handling async content

React/Vue apps load data asynchronously. Multiple ways to wait:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    stealth_sync(page)

    page.goto('https://facebook.com')

    # Option 1: Wait for element to appear
    page.wait_for_selector('.loaded-content')

    # Option 2: Wait for no network requests
    page.wait_for_load_state('networkidle')

    # Option 3: Wait for specific text
    page.wait_for_selector('text=Data loaded')

    # Option 4: Wait for API response
    with page.expect_response('/api/data') as response_info:
        page.click('button:has-text("Refresh")')
    response = response_info.value

    # Now scrape
    items = page.locator('.item').all()
    print(f"Found {len(items)} items")

I usually use networkidle + selector wait. Belt and suspenders approach.

Selecting elements in React apps

Playwright has special React support. These selectors are more stable than CSS:

# By role (accessible name)
submit = page.get_by_role('button', name='Submit')

# By test ID (if developers added them)
element = page.get_by_test_id('submit-button')

# By text content
title = page.get_by_text('Welcome back')

# Combine filters
sale_items = page.get_by_role('listitem').filter(
    has_text='sale'
)

# React DevTools locator
from playwright.sync_api import Page
page.get_by_test_id('user-profile')

Dealing with infinite scroll

Most modern shops use this. Here's a pattern that works:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync
import time

def scrape_infinite_scroll(url):
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=False)
        page = browser.new_page()
        stealth_sync(page)

        page.goto(url)

        items = set()

        while True:
            # Wait for items to load
            page.wait_for_selector('.item')
            current = page.locator('.item').all_text_contents()
            new_items = set(current) - items

            if not new_items:
                break  # Reached the end

            items.update(new_items)
            print(f"Collected {len(items)} items...")

            # Scroll down to trigger more
            page.evaluate('window.scrollTo(0, document.body.scrollHeight)')
            time.sleep(2)

        browser.close()
        return list(items)

Intercept API responses

Sometimes it's easier to grab the JSON directly:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    stealth_sync(page)

    api_responses = []

    def handle_response(response):
        if '/api/products' in response.url:
            api_responses.append(response.json())

    page.on('response', handle_response)

    page.goto('https://facebook.com')

    # Trigger the API call
    page.click('button:has-text("Load")')
    page.wait_for_load_state('networkidle')

    # Process the JSON data directly
    for product in api_responses[0]['results']:
        print(f"{product['name']}: ${product['price']}")

This skips all the HTML parsing. The data arrives already structured.

Errors I encountered

# Wait for element to be attached AND visible
page.wait_for_selector('.item', state='attached', state='visible')

# Or wait for it to NOT be loading
page.wait_for_selector('.content:not(.loading)')

# Don't use headless mode
browser = p.chromium.launch(headless=False)

# Set a realistic viewport
page.set_viewport_size({'width': 1920, 'height': 1080})

# Add user agent
page = browser.new_context(
    user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64)'
).new_page()

# Wait for spinner to disappear first
page.wait_for_selector('.spinner', state='hidden', timeout=15000)

# Then wait for actual content
page.wait_for_selector('.content', state='visible')

# Replace with stable selectors
page.click('button:has-text("Submit")')
page.get_by_role('button', name='Submit').click()

Extra stealth measures

For sites with aggressive detection:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync

with sync_playwright() as p:
    browser = p.chromium.launch(
        headless=False,
        args=[
            '--disable-blink-features=AutomationControlled',
            '--disable-infobars',
        ]
    )

    context = browser.new_context(
        viewport={'width': 1920, 'height': 1080},
        user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64)',
        locale='en-US',
        timezone_id='America/New_York',
    )

    page = context.new_page()
    stealth_sync(page)

    # Scroll gradually like a human would
    page.goto('https://facebook.com')
    for i in range(5):
        page.evaluate(f'window.scrollTo(0, {i * 300})')
        page.wait_for_timeout(800)

Which approach to use

Not every site needs stealth:

• requests + BeautifulSoup - Static HTML, server-side rendering
• Regular Playwright - Simple JS sites without detection
• Playwright + stealth - Bot detection, login-protected content
• Official API - When available and rate limits work for you