Managing Jobs

This guide shows you how to monitor and manage asynchronous jobs in your Marqo ecommerce search index. Learn about job statuses, how to retrieve job information, and best practices for tracking document operations.

Prerequisites

• A Marqo Cloud account
• Your Marqo API key
• An existing ecommerce index

Understanding Asynchronous Jobs

When you perform document operations (adding, updating, or deleting products), these operations are asynchronous. This means:

• Immediate Response: Your request is accepted immediately and you receive a job ID
• Background Processing: The actual operation happens in the background
• Status Tracking: You can monitor progress using the job ID
• Error Handling: Success or failure details are available once processing completes

Job Operations

Get All Jobs

Retrieve all jobs for your index:

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs \
-H "Authorization: Bearer {api_key}"

Get Specific Job

Retrieve details for a specific job using its job ID:

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs/{job_id} \
-H "Authorization: Bearer {api_key}"

Filter Jobs by Status

Get Active Jobs

Monitor currently running jobs:

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=active \
-H "Authorization: Bearer {api_key}"

Get Failed Jobs

Check for jobs that encountered errors:

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=failed \
-H "Authorization: Bearer {api_key}"

Note: Replace {index_name} with your actual index name, {api_key} with your API key, and {job_id} with the specific job ID you want to check.

Job Statuses

Jobs can have the following statuses:

Status	Description
COMPLETED	Job finished processing (check failedItems to see if there were errors)
RUNNING	Job is currently being processed
QUEUED	Job is waiting to be processed
FAILED	Job encountered a system error and could not complete

Response Fields

Field	Type	Description
jobId	String	Unique identifier for the job
jobType	String	Type of operation (typically BULK for document operations)
jobStatus	String	Current status of the job
totalItems	Integer	Total number of documents in the job
processedItems	Integer	Number of documents successfully processed
failedItems	Integer	Number of documents that failed processing
errorMessage	String/null	General error message if the job failed
failedItemsDetails	Object	Detailed error information for failed documents
errorSummary	Object	Summary of error types and counts
progressPercentage	Float	Completion percentage (0.0 to 100.0)
estimatedTimeMinutes	Integer	Estimated time for job completion
createdAt	String	ISO timestamp when job was created
startedAt	String	ISO timestamp when job started processing
completedAt	String	ISO timestamp when job completed

Response Examples

All Jobs Response

{
  "jobs": [
    {
      "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521",
      "jobType": "BULK",
      "jobStatus": "COMPLETED",
      "totalItems": 3,
      "processedItems": 3,
      "failedItems": 0,
      "errorMessage": null,
      "failedItemsDetails": {},
      "errorSummary": {},
      "createdAt": "2025-08-12T14:50:22.272035+00:00",
      "startedAt": "2025-08-12T14:50:24.936806+00:00",
      "completedAt": "2025-08-12T14:50:35.980820+00:00",
      "progressPercentage": 100.0,
      "estimatedTimeMinutes": 1
    },
    {
      "jobId": "fe5fbc8d-59ec-49bf-8521-8e970bb60e12",
      "jobType": "BULK",
      "jobStatus": "COMPLETED",
      "totalItems": 3,
      "processedItems": 3,
      "failedItems": 0,
      "errorMessage": null,
      "failedItemsDetails": {},
      "errorSummary": {},
      "createdAt": "2025-08-12T14:42:15.191116+00:00",
      "startedAt": "2025-08-12T14:42:17.714932+00:00",
      "completedAt": "2025-08-12T14:42:19.370160+00:00",
      "progressPercentage": 100.0,
      "estimatedTimeMinutes": 1
    }
  ],
  "total": null,
  "hasActiveJob": false
}

Specific Job Response

{
  "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521",
  "jobType": "BULK",
  "jobStatus": "COMPLETED",
  "totalItems": 3,
  "processedItems": 3,
  "failedItems": 0,
  "errorMessage": null,
  "failedItemsDetails": {},
  "errorSummary": {},
  "createdAt": "2025-08-12T14:50:22.272035+00:00",
  "startedAt": "2025-08-12T14:50:24.936806+00:00",
  "completedAt": "2025-08-12T14:50:35.980820+00:00",
  "progressPercentage": 100.0,
  "estimatedTimeMinutes": 1
}

Workflow Example

Here's a typical workflow for adding products and monitoring the job:

1. Add Products (Returns Job ID)

curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/documents \
    -H "Authorization: Bearer {api_key}" \
    -H "Content-Type: application/json" \
    -d '{"documents": [
{
    "_id": "light-blue-shirt",
    "parentProductId": "parent-shirt", 
    "productTitle": "Shirt",
    "variantTitle": "Shirt - Light Blue",
    "variantImageUrl": "https://cdn.shopify.com/s/files/1/0705/8276/3687/files/light-blue-tshirt.jpg?v=1754569578",
    "price": 100,
    "color": "Light Blue",
    "productCollections": ["shirts"]
}
]}'

Response:

{
  "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521",
  "jobStatus": "QUEUED",
  "message": "Documents queued for processing"
}

2. Check Job Status

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs/4eb23b8c-8d50-4a39-9bb1-490d8e6df521 \
-H "Authorization: Bearer {api_key}"

3. Monitor Until Complete

Keep checking the job status until it shows COMPLETED. Check the failedItems count to see if any documents failed processing.

Best Practices

Job Monitoring

• Check Status Regularly: Monitor job progress, especially for large batches
• Handle Failures: Check failed jobs for error details and retry if needed
• Batch Appropriately: Keep document batches to 25-75 documents for optimal performance

Error Handling

• Parse Error Messages: Failed jobs contain detailed error information
• Retry Failed Operations: Fix issues and resubmit failed documents
• Validate Before Submitting: Ensure documents meet all requirements