URL Import

Overview

URL Import lets you create products by pasting a product page URL. Alana uses Bright Data’s scraping infrastructure to extract structured data from the page, map it to the Alana schema, and run it through the Bronze → Silver pipeline automatically.

How it works

You submit a product page URL
Bright Data fetches the page (handling JavaScript rendering, CAPTCHAs, and geo-restrictions)
The scraper extracts: title, description, images, price, brand, specifications
Extracted data is mapped to the Alana product schema (Bronze)
Silver normalizes the result automatically
The product appears in your catalog

Scraping methods

Method	Description	Best for
`web_scraper`	Full-page JavaScript rendering, structured data extraction	Product pages with dynamic content
`web_unlocker`	Bypasses anti-bot protections	Retailers with aggressive bot detection
`crawl`	Follows links to extract multiple products from a category page	Category or collection pages

Import a single URL

Via UI

Open your catalog
Click Add Products → Import from URL
Paste the product page URL
Select the scraping method (default: web_scraper)
Click Import
A job is created — the product appears in the catalog within 30–90 seconds

Via API

curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/products/running-shoes-pro",
    "catalogId": "CATALOG_ID",
    "method": "web_scraper"
  }'

const response = await fetch(
  `https://app.alana.shopping/api/workspace/${workspaceId}/url-import`,
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      url: 'https://www.example.com/products/running-shoes-pro',
      catalogId,
      method: 'web_scraper',
    }),
  }
);
const { jobId, status } = await response.json();
console.log(`Import job ${jobId} started with status: ${status}`);

import requests

response = requests.post(
    f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import",
    headers={"Authorization": f"Bearer {api_key}"},
    json={
        "url": "https://www.example.com/products/running-shoes-pro",
        "catalogId": catalog_id,
        "method": "web_scraper",
    }
)
job = response.json()
print(f"Import job {job['jobId']} started")

Response

{
  "jobId": "job_9x8k2m",
  "status": "processing",
  "url": "https://www.example.com/products/running-shoes-pro",
  "estimatedSeconds": 45
}

Import multiple URLs (bulk)

curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/bulk" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://www.example.com/products/item-1",
      "https://www.example.com/products/item-2",
      "https://www.example.com/products/item-3"
    ],
    "catalogId": "CATALOG_ID",
    "method": "web_scraper"
  }'

const response = await fetch(
  `https://app.alana.shopping/api/workspace/${workspaceId}/url-import/bulk`,
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      urls: [
        'https://www.example.com/products/item-1',
        'https://www.example.com/products/item-2',
        'https://www.example.com/products/item-3',
      ],
      catalogId,
      method: 'web_scraper',
    }),
  }
);
const { batchJobId, urlCount } = await response.json();

import requests

response = requests.post(
    f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import/bulk",
    headers={"Authorization": f"Bearer {api_key}"},
    json={
        "urls": [
            "https://www.example.com/products/item-1",
            "https://www.example.com/products/item-2",
        ],
        "catalogId": catalog_id,
        "method": "web_scraper",
    }
)
batch = response.json()
print(f"Batch job {batch['batchJobId']} for {batch['urlCount']} URLs")

Crawl a category page

Use the crawl method to import all products from a category or collection page:

curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/category/running-shoes",
    "catalogId": "CATALOG_ID",
    "method": "crawl",
    "crawlOptions": {
      "maxProducts": 100,
      "followPagination": true
    }
  }'

const response = await fetch(
  `https://app.alana.shopping/api/workspace/${workspaceId}/url-import`,
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      url: 'https://www.example.com/category/running-shoes',
      catalogId,
      method: 'crawl',
      crawlOptions: { maxProducts: 100, followPagination: true },
    }),
  }
);

import requests

response = requests.post(
    f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import",
    headers={"Authorization": f"Bearer {api_key}"},
    json={
        "url": "https://www.example.com/category/running-shoes",
        "catalogId": catalog_id,
        "method": "crawl",
        "crawlOptions": {"maxProducts": 100, "followPagination": True},
    }
)

Check job status

curl "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/JOB_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"

const response = await fetch(
  `https://app.alana.shopping/api/workspace/${workspaceId}/url-import/${jobId}`,
  { headers: { 'Authorization': `Bearer ${apiKey}` } }
);
const { status, productId, error } = await response.json();

import requests

response = requests.get(
    f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import/{job_id}",
    headers={"Authorization": f"Bearer {api_key}"}
)
job = response.json()
print(f"Status: {job['status']}, Product ID: {job.get('productId')}")

Job status values

Status	Description
`processing`	Bright Data is fetching and parsing the page
`success`	Product created in catalog
`partial`	Product created with some fields missing
`failed`	Page could not be scraped (see `error` field)

Rate limits and cost

Metric	Limit
Single URL imports	100/hour per workspace
Bulk imports	500 URLs/request
Crawl max products	500/crawl
Concurrent jobs	10 per workspace

Cost per import is deducted from your Bright Data credit balance. Costs vary by method:

Method	Approximate cost
`web_scraper`	0.001 credits/page
`web_unlocker`	0.005 credits/page
`crawl`	0.001 credits/product found

View your Bright Data credit usage in Settings → Integrations → Bright Data.

Best practices

Test with a single URL before bulk import

Always test one URL first to confirm the scraper correctly extracts the fields you need. Different retailers have different page structures.

Use web_unlocker for major retailers

Sites like Amazon, Walmart, and major fashion retailers have bot detection. Use web_unlocker to avoid failed imports.

Use crawl for category-level imports

When you want all products in a category, crawl is more efficient than pasting each product URL individually.

Check partial results

A partial status means the product was created but some fields couldn’t be extracted. Review these products in Canvas and fill missing fields manually.

Getting Started

Core Concepts

Guides

Overview

How it works

Scraping methods

Import a single URL

Via UI

Via API

Response

Import multiple URLs (bulk)

Crawl a category page

Check job status

Job status values

Rate limits and cost

Best practices

​Overview

​How it works

​Scraping methods

​Import a single URL

​Via UI

​Via API

​Response

​Import multiple URLs (bulk)

​Crawl a category page

​Check job status

​Job status values

​Rate limits and cost

​Best practices

Overview

How it works

Scraping methods

Import a single URL

Via UI

Via API

Response

Import multiple URLs (bulk)

Crawl a category page

Check job status

Job status values

Rate limits and cost

Best practices