Guides Handling large catalogs

Handling large catalogs

Indexing, memory, and performance notes for catalogs past 5k SKUs.

Updated

searchabee is built for catalogs that outgrow Shopify’s native page-scoped filters. This guide covers what to expect at scale.

Index size

Each shop gets a dedicated Meilisearch index. Index size roughly scales linearly with:

  • Number of products + variants
  • Number of indexed fields (more fields = larger index)
  • Filter source count (each adds a faceted attribute)

A typical 50k-SKU apparel catalog indexes to ~500MB. A 200k-SKU catalog indexes to ~2GB. Meilisearch handles both well.

Sync performance

Initial backfill processes Shopify’s GraphQL catalog API. Expect:

  • 10k SKUs → ~2 minutes
  • 50k SKUs → ~8 minutes
  • 200k SKUs → ~30 minutes

After backfill, incremental sync is webhook-driven and completes in seconds.

Query latency

Meilisearch typical latency at catalog sizes we target:

  • 10k SKUs: sub-5ms
  • 100k SKUs: sub-15ms
  • 500k SKUs: sub-30ms

These are index-side times. Add ~30–80ms of network round trip from the shopper’s browser to our API.

Best practices

  • Limit indexed fields. Each added field (beyond the defaults) adds latency. Only index what you actually filter/sort/search on.
  • Use metafield filters sparingly. Metafields are the heaviest source type because they require denormalizing. Prefer product fields and options where possible.
  • Avoid deeply nested filter trees. Past 15 filters per tree, shoppers rarely scroll that far. Consolidate.
  • Rebuild rarely. Our sync is incremental. Triggering a full rebuild is a last resort, not a regular operation.

Monitoring

searchabee console → Health shows index size, last sync time, and recent query latency percentiles. If latency p95 starts creeping up, reach out — often it’s addressable by dropping an unused filter source.