> ## Documentation Index
> Fetch the complete documentation index at: https://docs.alana.shopping/llms.txt
> Use this file to discover all available pages before exploring further.

# URL Import

> Import products directly from web pages using Bright Data's web scraping infrastructure — paste a URL and the pipeline handles the rest.

## Overview

**URL Import** lets you create products by pasting a product page URL. Alana uses Bright Data's scraping infrastructure to extract structured data from the page, map it to the Alana schema, and run it through the Bronze → Silver pipeline automatically.

***

## How it works

```mermaid theme={null}
graph LR
    A["Paste URL"] --> B["Bright Data\nScrapes Page"]
    B --> C["Bronze\n(raw HTML → schema)"]
    C --> D["Silver\n(normalize fields)"]
    D --> E["Gold\n(optional score)"]
    E --> F["Product appears\nin catalog"]
```

1. You submit a product page URL
2. Bright Data fetches the page (handling JavaScript rendering, CAPTCHAs, and geo-restrictions)
3. The scraper extracts: title, description, images, price, brand, specifications
4. Extracted data is mapped to the Alana product schema (Bronze)
5. Silver normalizes the result automatically
6. The product appears in your catalog

***

## Scraping methods

| Method         | Description                                                     | Best for                                |
| -------------- | --------------------------------------------------------------- | --------------------------------------- |
| `web_scraper`  | Full-page JavaScript rendering, structured data extraction      | Product pages with dynamic content      |
| `web_unlocker` | Bypasses anti-bot protections                                   | Retailers with aggressive bot detection |
| `crawl`        | Follows links to extract multiple products from a category page | Category or collection pages            |

***

## Import a single URL

### Via UI

1. Open your catalog
2. Click **Add Products** → **Import from URL**
3. Paste the product page URL
4. Select the scraping method (default: `web_scraper`)
5. Click **Import**
6. A job is created — the product appears in the catalog within 30–90 seconds

### Via API

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://www.example.com/products/running-shoes-pro",
      "catalogId": "CATALOG_ID",
      "method": "web_scraper"
    }'
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch(
    `https://app.alana.shopping/api/workspace/${workspaceId}/url-import`,
    {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        url: 'https://www.example.com/products/running-shoes-pro',
        catalogId,
        method: 'web_scraper',
      }),
    }
  );
  const { jobId, status } = await response.json();
  console.log(`Import job ${jobId} started with status: ${status}`);
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import",
      headers={"Authorization": f"Bearer {api_key}"},
      json={
          "url": "https://www.example.com/products/running-shoes-pro",
          "catalogId": catalog_id,
          "method": "web_scraper",
      }
  )
  job = response.json()
  print(f"Import job {job['jobId']} started")
  ```
</CodeGroup>

### Response

```json theme={null}
{
  "jobId": "job_9x8k2m",
  "status": "processing",
  "url": "https://www.example.com/products/running-shoes-pro",
  "estimatedSeconds": 45
}
```

***

## Import multiple URLs (bulk)

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/bulk" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "urls": [
        "https://www.example.com/products/item-1",
        "https://www.example.com/products/item-2",
        "https://www.example.com/products/item-3"
      ],
      "catalogId": "CATALOG_ID",
      "method": "web_scraper"
    }'
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch(
    `https://app.alana.shopping/api/workspace/${workspaceId}/url-import/bulk`,
    {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        urls: [
          'https://www.example.com/products/item-1',
          'https://www.example.com/products/item-2',
          'https://www.example.com/products/item-3',
        ],
        catalogId,
        method: 'web_scraper',
      }),
    }
  );
  const { batchJobId, urlCount } = await response.json();
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import/bulk",
      headers={"Authorization": f"Bearer {api_key}"},
      json={
          "urls": [
              "https://www.example.com/products/item-1",
              "https://www.example.com/products/item-2",
          ],
          "catalogId": catalog_id,
          "method": "web_scraper",
      }
  )
  batch = response.json()
  print(f"Batch job {batch['batchJobId']} for {batch['urlCount']} URLs")
  ```
</CodeGroup>

***

## Crawl a category page

Use the `crawl` method to import all products from a category or collection page:

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://www.example.com/category/running-shoes",
      "catalogId": "CATALOG_ID",
      "method": "crawl",
      "crawlOptions": {
        "maxProducts": 100,
        "followPagination": true
      }
    }'
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch(
    `https://app.alana.shopping/api/workspace/${workspaceId}/url-import`,
    {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        url: 'https://www.example.com/category/running-shoes',
        catalogId,
        method: 'crawl',
        crawlOptions: { maxProducts: 100, followPagination: true },
      }),
    }
  );
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import",
      headers={"Authorization": f"Bearer {api_key}"},
      json={
          "url": "https://www.example.com/category/running-shoes",
          "catalogId": catalog_id,
          "method": "crawl",
          "crawlOptions": {"maxProducts": 100, "followPagination": True},
      }
  )
  ```
</CodeGroup>

***

## Check job status

<CodeGroup>
  ```bash curl theme={null}
  curl "https://app.alana.shopping/api/workspace/WORKSPACE_ID/url-import/JOB_ID" \
    -H "Authorization: Bearer YOUR_API_KEY"
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch(
    `https://app.alana.shopping/api/workspace/${workspaceId}/url-import/${jobId}`,
    { headers: { 'Authorization': `Bearer ${apiKey}` } }
  );
  const { status, productId, error } = await response.json();
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      f"https://app.alana.shopping/api/workspace/{workspace_id}/url-import/{job_id}",
      headers={"Authorization": f"Bearer {api_key}"}
  )
  job = response.json()
  print(f"Status: {job['status']}, Product ID: {job.get('productId')}")
  ```
</CodeGroup>

### Job status values

| Status       | Description                                   |
| ------------ | --------------------------------------------- |
| `processing` | Bright Data is fetching and parsing the page  |
| `success`    | Product created in catalog                    |
| `partial`    | Product created with some fields missing      |
| `failed`     | Page could not be scraped (see `error` field) |

***

## Rate limits and cost

| Metric             | Limit                  |
| ------------------ | ---------------------- |
| Single URL imports | 100/hour per workspace |
| Bulk imports       | 500 URLs/request       |
| Crawl max products | 500/crawl              |
| Concurrent jobs    | 10 per workspace       |

Cost per import is deducted from your Bright Data credit balance. Costs vary by method:

| Method         | Approximate cost            |
| -------------- | --------------------------- |
| `web_scraper`  | 0.001 credits/page          |
| `web_unlocker` | 0.005 credits/page          |
| `crawl`        | 0.001 credits/product found |

View your Bright Data credit usage in **Settings** → **Integrations** → **Bright Data**.

***

## Best practices

<AccordionGroup>
  <Accordion title="Test with a single URL before bulk import">
    Always test one URL first to confirm the scraper correctly extracts the fields you need. Different retailers have different page structures.
  </Accordion>

  <Accordion title="Use web_unlocker for major retailers">
    Sites like Amazon, Walmart, and major fashion retailers have bot detection. Use `web_unlocker` to avoid failed imports.
  </Accordion>

  <Accordion title="Use crawl for category-level imports">
    When you want all products in a category, `crawl` is more efficient than pasting each product URL individually.
  </Accordion>

  <Accordion title="Check partial results">
    A `partial` status means the product was created but some fields couldn't be extracted. Review these products in Canvas and fill missing fields manually.
  </Accordion>
</AccordionGroup>