Downloader

API Documentation

Website Downloader API v2.6.0

Base URL: https://downloader.dynomapper.com/wp-json/wd/v1

📢 New in v2.6.0: Downloads now default to HTML-only mode (no images, CSS, or JavaScript). This makes downloads 10-100x faster. To include assets, set advanced_options.assets.mode = "full_site".

Authentication

All API requests require an API key (Agency plan). Include it in the header:

Authorization: Bearer YOUR_API_KEY

or

X-API-Key: YOUR_API_KEY

Quick Start

1. Create a Download Job

curl -X POST "https://downloader.dynomapper.com/wp-json/wd/v1/downloads" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

2. Poll for Completion

curl "https://downloader.dynomapper.com/wp-json/wd/v1/downloads/123" \
  -H "Authorization: Bearer YOUR_API_KEY"

3. Download the Archive

When status is "ready", use the download_url to fetch the ZIP file.

Endpoints

POST /downloads

Create a new download job.

Request Body

FieldTypeRequiredDefaultDescription
urlstringRequired—Website URL to download
depthstringNosinglesingle, shallow, full, custom
storagestringNolocallocal (Downloader, 7-day retention), google_drive, dropbox, onedrive
advanced_optionsobjectNo—Advanced crawler settings
client_job_idstringNo—New Idempotency key (max 64 chars)
validate_onlybooleanNofalseNew Validate without creating job
callback_urlstringNo—New Webhook URL (HTTPS)

GET /downloads/{id}

Get job status and details.

Response

{
  "success": true,
  "job": {
    "id": 123,
    "url": "https://example.com",
    "status": "ready",
    "depth": "shallow",
    "storage": "local",
    "download_url": "https://.../example.com_20241208.zip",
    "error": null,
    "total_pages": 42,
    "total_assets": 0,
    "total_bytes": 1250000,
    "created_at": "2024-12-08T20:00:00",
    "started_at": "2024-12-08T20:00:05",
    "completed_at": "2024-12-08T20:01:30"
  }
}

Status Values

StatusDescription
queuedJob is waiting to be processed
runningJob is currently downloading
readyJob completed, download_url available
failedJob failed, error contains reason

GET /downloads

List jobs with optional filtering.

Query Parameters

ParameterTypeDefaultDescription
statusstring—Filter by status
limitinteger50Max results (1-100)
offsetinteger0Pagination offset

GET /usage

Get rate limit and quota usage.

GET /meta New

Get API capabilities and limits. No authentication required.

{
  "version": "2.6.0",
  "max_depth": 20,
  "supports": {
    "advanced_options": true,
    "idempotency": true,
    "validation_mode": true,
    "webhooks": true
  },
  "storage_destinations": ["local", "google_drive", "dropbox", "onedrive"],
  "status_values": ["queued", "running", "ready", "failed"]
}

Download Modes

Default: HTML Only
Downloads are now HTML-only by default, which is 10-100x faster and creates much smaller files. Most use cases only need the HTML content.
ModeDescriptionUse Case
html_onlyDefault HTML files onlyContent extraction, text analysis, SEO audits
full_siteHTML + images, CSS, JS, fontsOffline viewing, full archive, design review
customSpecify file types to include/excludeSelective downloads

HTML Only (Default)

curl -X POST ".../downloads" \
  -H "Authorization: Bearer KEY" \
  -d '{"url": "https://example.com"}'

Full Site with Assets

curl -X POST ".../downloads" \
  -H "Authorization: Bearer KEY" \
  -d '{
    "url": "https://example.com",
    "advanced_options": {
      "assets": {"mode": "full_site"}
    }
  }'

Custom File Types

curl -X POST ".../downloads" \
  -H "Authorization: Bearer KEY" \
  -d '{
    "url": "https://example.com",
    "advanced_options": {
      "assets": {"mode": "custom"},
      "file_types": {
        "accept": ["html", "css"],
        "reject": ["jpg", "png", "gif"]
      }
    }
  }'

Advanced Options

{
  "advanced_options": {
    "depth": {
      "mode": "single|shallow|full|custom",
      "custom_level": 3,
      "span_hosts": false
    },
    "assets": {
      "mode": "html_only|full_site|custom"
    },
    "file_types": {
      "accept": ["html", "css"],
      "reject": ["jpg", "png"]
    },
    "speed": {
      "wait_seconds": 0.5,
      "random_wait": true,
      "limit_rate_kb": 500,
      "respect_robots": true
    },
    "user_agent": {
      "mode": "default|browser_like|custom",
      "custom_string": "MyBot/1.0"
    },
    "url_filters": {
      "include_paths": ["/blog"],
      "exclude_paths": ["/wp-admin"]
    },
    "auth": {
      "enabled": true,
      "type": "basic",
      "username": "user",
      "password": "pass"
    }
  }
}

AI-Agent Features

Idempotency (client_job_id)

Prevent duplicate jobs when retrying requests:

{"url": "https://example.com", "client_job_id": "unique-abc123"}

If a job with this ID exists, it returns the existing job with "idempotent_match": true.

Validation Mode (validate_only)

Test a request without creating a job:

{"url": "https://example.com", "validate_only": true}

Returns validated, errors, warnings, and can_create_job.

Webhooks (callback_url)

Receive a POST notification when the job completes:

{"url": "https://example.com", "callback_url": "https://your-server.com/hook"}

Error Handling

{
  "success": false,
  "error": {
    "code": "rate_limit_exceeded",
    "message": "You have exceeded your daily limit.",
    "data": {
      "status": 429,
      "retry_hint": "wait_60_seconds"
    }
  }
}

Retry Hints

HintAction
wait_60_secondsWait and retry
upgrade_planUser needs to upgrade
connect_storageConnect storage in dashboard
invalid_request_no_retryFix request, don’t retry
contact_supportContact support

Rate Limits

LimitValue
Requests per second1
Requests per minute3
Requests per hour20
Requests per day50
Concurrent jobs3
Monthly downloadsPlan-dependent

Rate limit headers are included in responses: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset

Scroll to Top