Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ Commands with JSON output support:
- `captcha-solver` - Template demonstrating Kernel's auto-CAPTCHA solver
- `stagehand` - Template with Stagehand SDK (TypeScript only)
- `browser-use` - Template with Browser Use SDK (Python only)
- `lead-scraper` - Google Maps lead scraper using Browser Use (Python only)
- `anthropic-computer-use` - Anthropic Computer Use prompt loop
- `openai-computer-use` - OpenAI Computer Use Agent sample
- `gemini-computer-use` - Implements a Gemini computer use agent (TypeScript only)
Expand Down Expand Up @@ -449,6 +450,9 @@ kernel create --name my-cu-app --language py --template anthropic-computer-use

# Create a Claude Agent SDK app (TypeScript or Python)
kernel create --name my-claude-agent --language ts --template claude-agent-sdk

# Create a Google Maps Lead Scraper (Python)
kernel create --name my-lead-scraper --language python --template lead-scraper
```

### Deploy with environment variables
Expand Down
2 changes: 2 additions & 0 deletions pkg/templates/python/lead-scraper/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Copy this file to .env and fill in your API key
OPENAI_API_KEY=your_openai_api_key_here
113 changes: 113 additions & 0 deletions pkg/templates/python/lead-scraper/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Kernel Lead Scraper Template - Google Maps

A ready-to-use lead scraper that extracts local business data from Google Maps using [browser-use](https://github.com/browser-use/browser-use) and the Kernel platform.

## What It Does

This template creates an AI-powered web scraper that:
1. Navigates to Google Maps
2. Searches for businesses by type and location
3. Scrolls through results to load more listings
4. Extracts structured lead data (name, phone, address, website, rating, reviews)
5. Returns clean JSON ready for your CRM or outreach tools

## Quick Start

### 1. Install Dependencies

```bash
uv sync
```

### 2. Set Up Environment

```bash
cp .env.example .env
# Edit .env and add your OpenAI API key
```

### 3. Deploy to Kernel

```bash
kernel deploy main.py -e OPENAI_API_KEY=$OPENAI_API_KEY
```

### 4. Run the Scraper

```bash
kernel run lead-scraper scrape-leads \
--data '{"query": "restaurants", "location": "Austin, TX", "max_results": 10}'
```

## API Reference

### Action: `scrape-leads`

**Input Parameters:**

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `query` | string | ✅ | - | Business type to search (e.g., "plumbers", "gyms") |
| `location` | string | ✅ | - | Geographic location (e.g., "Miami, FL") |
| `max_results` | integer | ❌ | 20 | Maximum leads to scrape (1-50) |

**Example Output:**

```json
{
"leads": [
{
"name": "Joe's Pizza",
"phone": "(512) 555-0123",
"address": "123 Main St, Austin, TX 78701",
"website": "https://joespizza.com",
"rating": 4.5,
"review_count": 234,
"category": "Pizza restaurant"
}
],
"total_found": 1,
"query": "pizza restaurants",
"location": "Austin, TX"
}
```

## Use Cases

- **Sales Teams**: Build targeted prospect lists for cold outreach
- **Marketing Agencies**: Find local businesses needing marketing services
- **Service Providers**: Identify potential B2B clients in your area
- **Market Research**: Analyze competitor density and ratings by location

## Customization

### Modify the Search Prompt

Edit the `SCRAPER_PROMPT` in `main.py` to customize what data the AI extracts:

```python
SCRAPER_PROMPT = """
Navigate to Google Maps and search for {query} in {location}.
# Add your custom extraction instructions here
"""
```

### Add New Fields

1. Update `BusinessLead` model in `models.py`
2. Modify the prompt to extract the new fields
3. Redeploy with `kernel deploy main.py`

## Troubleshooting

| Issue | Solution |
|-------|----------|
| No results found | Try a broader search query or different location |
| Timeout errors | Reduce `max_results` or check your network |
| Rate limiting | Add delays between requests in production |

## Resources

- [Kernel Documentation](https://www.kernel.sh/docs)
- [Browser Use Docs](https://docs.browser-use.com)
- [Pydantic Models](https://docs.pydantic.dev)
79 changes: 79 additions & 0 deletions pkg/templates/python/lead-scraper/_gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg

# Virtual Environment
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
.project
.pydevproject
.settings/

# Testing
.coverage
htmlcov/
.pytest_cache/
.tox/
.nox/
coverage.xml
*.cover
.hypothesis/

# Logs
*.log
logs/

# OS
.DS_Store
Thumbs.db

# Browser Use specific
.playwright-screenshots/
.playwright-videos/
.playwright-report/
test-results/
blob-report/
playwright/.cache/
playwright/.local-browsers/

# Lead Scraper specific
leads_output/
*.csv
*.json

# Misc
.cache/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.temp/
.tmp/
Loading