Skip to main content

​
Overview

URL Import lets you create products by pasting a product page URL. Alana uses Bright Data’s scraping infrastructure to extract structured data from the page, map it to the Alana schema, and run it through the Bronze → Silver pipeline automatically.

​
How it works

  1. You submit a product page URL
  2. Bright Data fetches the page (handling JavaScript rendering, CAPTCHAs, and geo-restrictions)
  3. The scraper extracts: title, description, images, price, brand, specifications
  4. Extracted data is mapped to the Alana product schema (Bronze)
  5. Silver normalizes the result automatically
  6. The product appears in your catalog

​
Scraping methods

MethodDescriptionBest for
web_scraperFull-page JavaScript rendering, structured data extractionProduct pages with dynamic content
web_unlockerBypasses anti-bot protectionsRetailers with aggressive bot detection
crawlFollows links to extract multiple products from a category pageCategory or collection pages

​
Import a single URL

​
Via UI

  1. Open your catalog
  2. Click Add Products → Import from URL
  3. Paste the product page URL
  4. Select the scraping method (default: web_scraper)
  5. Click Import
  6. A job is created — the product appears in the catalog within 30–90 seconds

​
Via API

curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/products/running-shoes-pro",
    "catalogId": "CATALOG_ID",
    "method": "web_scraper"
  }'

​
Response

{
  "jobId": "job_9x8k2m",
  "status": "processing",
  "url": "https://www.example.com/products/running-shoes-pro",
  "estimatedSeconds": 45
}

​
Import multiple URLs (bulk)

curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/bulk" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://www.example.com/products/item-1",
      "https://www.example.com/products/item-2",
      "https://www.example.com/products/item-3"
    ],
    "catalogId": "CATALOG_ID",
    "method": "web_scraper"
  }'

​
Crawl a category page

Use the crawl method to import all products from a category or collection page: 
curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/category/running-shoes",
    "catalogId": "CATALOG_ID",
    "method": "crawl",
    "crawlOptions": {
      "maxProducts": 100,
      "followPagination": true
    }
  }'

​
Check job status

curl "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/JOB_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"

​
Job status values

StatusDescription
processingBright Data is fetching and parsing the page
successProduct created in catalog
partialProduct created with some fields missing
failedPage could not be scraped (see error field)

​
Rate limits and cost

MetricLimit
Single URL imports100/hour per workspace
Bulk imports500 URLs/request
Crawl max products500/crawl
Concurrent jobs10 per workspace
Cost per import is deducted from your Bright Data credit balance. Costs vary by method:
MethodApproximate cost
web_scraper0.001 credits/page
web_unlocker0.005 credits/page
crawl0.001 credits/product found
View your Bright Data credit usage in Settings → Integrations → Bright Data.

​
Best practices

Always test one URL first to confirm the scraper correctly extracts the fields you need. Different retailers have different page structures.
Sites like Amazon, Walmart, and major fashion retailers have bot detection. Use web_unlocker to avoid failed imports.
When you want all products in a category, crawl is more efficient than pasting each product URL individually.
A partial status means the product was created but some fields couldn’t be extracted. Review these products in Canvas and fill missing fields manually.
Last modified on March 18, 2026