APIbatch time series forecasting api

Forecasting 500 SKUs: batching, chunking and the numbers behind it

Pierre N. · Founder, VecTime Cloud28 August 2026 · 6 min read

A catalogue of 500 SKUs is not 500 forecasting problems. It is one request per chunk of series, where the chunk size is a number your plan tells you and the model does the whole chunk in a single pass. On the Pro plan that is four requests. This post shows the one measurement we have of what batching saves, the loop that chunks a catalogue to the plan ceiling, and the two things that go wrong when people write that loop themselves.

The short versionOne series took 1.52 s of inference. Four series in one request took 2.08 s — four times the work for 1.36× the time, or 520 ms per series. Read your ceiling from GET /v1/models, chunk to it, and one request is one metered call regardless of how many series it carries.

Why one request beats five hundred

Two reasons, and they are independent. The first is cost: a request is the unit of metering, so 500 SKUs in four requests spend four calls of quota, not five hundred. The second is time: a foundation model forecasts a batch of series in a single forward pass, so the fixed cost of the pass — loading the context, running the layers — is paid once. Per series, that cost shrinks as the batch grows, right up to the point where the batch no longer fits the hardware.Neither reason depends on the series being related. The model does not learn across the batch; each series is forecast on its own history. Batching is a transport optimisation, not a modelling one, which is also why you can mix unrelated series freely.

The measurement

One data point, honestly labelled. The same 96-point weekly series was sent once, then four copies of it were sent in a single request, on a free-tier key against the CPU deployment the marketing assets are rendered from. inference_ms is the model time the API reports in every response; the wall-clock latency was about 50 ms more in both cases.

Request	inference_ms	Per series	Metered calls
1 series × 96 points, horizon 12	1525	1525 ms	1
4 series × 96 points, horizon 12	2081	520 ms	1

What this does not show: how the curve bends at 32, 128 or 512 series, or what a GPU deployment does to the absolute numbers. Four is the free tier's ceiling and the free tier is what we measure with, so that the measurement is one you can repeat. If you run it at a larger batch, the per-series figure should keep falling until the batch saturates the device — but that is the expectation, not a measurement, and this table does not claim it.

Chunking to your plan ceiling

The ceiling differs by plan, and the right way to learn it is to ask the API rather than copy it from the pricing page into a constant that goes stale on the day you upgrade.

Plan	Series / request	Requests / minute	500 SKUs
Free	4	10	125 requests
Starter	32	60	16 requests
Pro	128	300	4 requests
Enterprise	512+	2000	1 request

python
import os, requests
API = "https://api.vectime.cloud"
H = {"Authorization": f"Bearer {os.environ['VECTIME_API_KEY']}"}
# Unmetered: does not count against the monthly quota.
model = requests.get(f"{API}/v1/models", headers=H, timeout=30).json()["models"][0]
MAX_SERIES  = model["max_series_per_request"]   # 4 / 32 / 128 / 512
MAX_CONTEXT = model["max_context"]              # 1024 / 2048 / 4096
MAX_HORIZON = model["max_horizon"]

python
def chunks(items, n):
    for i in range(0, len(items), n):
        yield items[i:i + n]
skus = list(catalogue.items())          # [(sku_id, [oldest, ..., newest]), ...]
results = {}
for batch in chunks(skus, MAX_SERIES):
    payload = {
        # Each series is trimmed to the plan's context window; a longer one is
        # rejected with context_limit_exceeded rather than silently truncated.
        "series": [hist[-MAX_CONTEXT:] for _, hist in batch],
        "horizon": 12,
        "quantiles": True,
    }
    r = requests.post(f"{API}/v1/forecast", headers=H, json=payload, timeout=120)
    r.raise_for_status()
    for (sku, _), fc in zip(batch, r.json()["forecasts"]):
        results[sku] = fc          # fc["index"] matches the position in "series"

Two details in that loop are load-bearing. The results come back in the order the series went in, with an index on each, so the zip is safe — but keep the index check if you ever filter series out before sending. And the history is trimmed to the context window on the client, because the server rejects a series that is too long with context_limit_exceeded instead of quietly dropping the oldest points. That is deliberate: silent truncation would make a forecast depend on a limit you never saw.

Ragged histories

A real catalogue has products with four years of history next to products launched last quarter. They can share a request. Each series is validated on its own length — at least two points, at most the context window — so there is no padding to do and no need to sort the catalogue by history length first.

python
# Series in one request may have different lengths. Each is validated on its
# own: at least 2 points, at most the plan's context window.
payload["series"] = [
    catalogue["A-100"][-1024:],   # a mature product, 4 years of weekly data
    catalogue["N-901"],           # a launch: 9 points, that is all there is
]

What you should expect from the nine-point series is a flat median and a wide band. That is the model declining to extrapolate from almost nothing, and it is the correct answer; we have a note on what to do instead when the history is that short.

Throughput and the rate limit

Batching also changes the arithmetic on the per-minute limit. The limit counts requests, not series, so at 128 series per request the Pro plan's 300 requests a minute is 38,400 series a minute before the bucket empties — far more than the model will return in that time. In practice the model, not the limiter, is the bottleneck for a batched client, and you will hit 503 model_unavailable on a cold start long before you hit a 429. Both are retryable; the rate-limit post has the loop.Run chunks sequentially unless you have measured that concurrency helps. On a CPU deployment two requests in flight share the same cores, and the second one mostly waits.

When not to batch

You need one answer now. A forecast behind an interactive request should go alone. Waiting to fill a batch adds latency to the one series someone is looking at.The series have different horizons. The horizon is per request. Two series that need 6 and 52 steps go in two requests, or in one at 52 with the extra steps discarded — which spends forecast points you did not need.One bad series should not fail the rest. A request is validated as a whole: a single series with a NaN in it rejects all 128 with invalid_request. Clean the batch first, or accept that the retry after a validation error re-sends everything.

Run it on your own series.

If you want to try this without hosting a model, VecTime Cloud's free tier is 300 forecasts a month and needs no card.

Start free Read the API docs

Pierre N.Founder, VecTime CloudBuilds VecTime Cloud — the Go API and its metering, the Python inference service, and the benchmarks published here, including the runs that go against us.