Benchmarkseasonal naive forecast baseline

Beat seasonal-naive first. On a third of monthly series, we didn't.

Pierre N. · Founder, VecTime Cloud2 September 2026 · 7 min read

Seasonal-naive means: predict that the next cycle looks exactly like the last one. It fits nothing, costs nothing and is the baseline every published M4 result is scaled against. It is also, on the 300 series in our benchmark, the method that beat a 200-million-parameter foundation model on 20% of hourly series, 32% of weekly series and 34% of monthly series. If you are choosing a forecasting method, this is the comparison to run before any other, and this post is the per-series evidence for why.

The short versionAverages flatter the model. Per series, TimesFM zero-shot cleared the in-sample naive bar (MASE below 1) on 78, 43 and 64 of 100 series by frequency, against 38, 23 and 53 for seasonal-naive itself. The median series is a clear win; the upper quartile on weekly and monthly data is a loss. Run the baseline on your own series and ship the model only where it wins.

What seasonal-naive is

Take the last full season of length m — 24 for hourly data with a daily cycle, 12 for monthly data with a yearly one — and repeat it for as many steps as you need. For weekly and daily data M4 sets m to 1, which turns the method into “next week equals this week”. That sounds like a straw man until you try to beat it on a series with a strong cycle and weak trend, which describes most operational data.It is the right baseline for two reasons. It encodes the one thing almost every business series has — periodicity — and nothing else, so beating it proves the model has learned something beyond the calendar. And it is deterministic, so a comparison against it is reproducible by anyone with the series and ten lines of NumPy.

Why MASE is the right ruler

MASE divides a method's mean absolute error by the mean absolute error of seasonal-naive on the training part of the same series. A value of 1 means the method did as well out of sample as naive did in sample; below 1 is better, above 1 is worse. It is scale-free, so an error on a series measured in thousands and one measured in tenths sit on the same axis, and it is defined against the baseline this article is about, so the threshold is meaningful rather than arbitrary.One subtlety that catches people: seasonal-naive's own out-of-sample MASE is not 1. It is scaled by its in-sample error, and the future is usually harder than the past, so it lands above 1 on most series — 62 of 100 hourly series, for instance. That is why the table below reports both methods against the same bar rather than treating naive as the bar.

Per series, not on average

The summary table in the benchmark article says TimesFM won every frequency on mean MASE. True, and not very informative: a mean over 100 series can be carried by a dozen large wins. These are the same 300 series, counted.

Frequency	TimesFM MASE < 1	Naive MASE < 1	TimesFM at under half naive's error	Neither under 1
Hourly	78 / 100	38 / 100	45 / 100	16 / 100
Weekly	43 / 100	23 / 100	24 / 100	54 / 100
Monthly	64 / 100	53 / 100	13 / 100	32 / 100

Bars span the middle half of series; the thick mark is the median. Anything left of the dashed line is a series where TimesFM beat seasonal-naive; the right end of the Weekly and Monthly bars crosses it.

Hourly is the clean case: the model halves the naive error on 45 series and loses on 20. On weekly and monthly data the middle of the distribution is a modest win — a median ratio of 0.86 on both — and the upper quartile is a loss. Read that as: on a typical monthly series the model shaves 14% off the naive error, and on one series in three it adds to it.

Where the model loses

The last column of the table is the one to sit with. On 54 of 100 weekly series and 32 of 100 monthly series, neither method got under 1 — the future was simply harder than the past for both, and the question of which lost by less is secondary to the fact that both lost. Those are series with level shifts, ends of trends, or a last season that was not representative, and no method that only sees the history can do much with them.When TimesFM does lose to naive, it loses narrowly: the median losing margin is 1.46× on hourly, 1.21× on weekly and 1.14× on monthly. The wins are larger than the losses, which is how the means come out in the model's favour while a third of series do not. The worst single series for TimesFM scored a MASE of 12.1 on weekly data; naive's worst on the same frequency was 14.2. Neither is a number you would want in production, and both are in the per-series CSV.

Run the baseline in ten lines

There is no library to install. The baseline and the metric together are shorter than most import blocks.

python
import numpy as np
def seasonal_naive(y: np.ndarray, horizon: int, m: int) -> np.ndarray:
    """Repeat the last seasonal cycle of length m. m=1 is 'tomorrow equals today'."""
    last_cycle = y[-m:]
    reps = int(np.ceil(horizon / m))
    return np.tile(last_cycle, reps)[:horizon]
def mase(y_train: np.ndarray, y_true: np.ndarray, y_hat: np.ndarray, m: int) -> float:
    """M4's definition: error scaled by the in-sample seasonal-naive error."""
    scale = np.mean(np.abs(y_train[m:] - y_train[:-m]))
    return np.mean(np.abs(y_true - y_hat)) / scale

Then hold out the last horizon of every series, forecast it with the baseline and with whatever you are evaluating, and compare the two columns series by series. Not the mean of the columns — the columns.

python
# Hold out the last H points of every series you care about, then:
for name, y in catalogue.items():
    train, test = y[:-H], y[-H:]
    base = seasonal_naive(train, H, m)
    model = forecast(train, H)["forecasts"][0]["point"]   # or Prophet, or anything
    print(name, mase(train, test, base, m), mase(train, test, model, m))
# Ship the model only on the series where its column is lower. Keep the
# baseline on the rest — it is free, and on those series it is also better.

The published benchmark script does this for M4 with one command(opens in new tab); the per-series output it wrote is at m4-300-per-series.csv(opens in new tab), and every count in this article is a groupby over it.

When naive is the right answer

The series is strongly periodic and flat. Hourly traffic to a stable service, weekly sales of a mature product: last cycle is a very good guess, and a model that mostly reproduces it is paying for nothing. Keep naive, and spend the forecasting budget on the series where it fails.You need to explain the number. “Same as last week” is an explanation everyone in the room accepts. A foundation model's output is not, and on a series where the two are within a few percent, the explainable one wins the meeting.You have not measured yet. The most common failure is not choosing the wrong method; it is choosing any method without the baseline column next to it. Until that column exists, naive is the honest default — it is what you would have to beat to justify anything else.

Run it on your own series.

If you want to try this without hosting a model, VecTime Cloud's free tier is 300 forecasts a month and needs no card.

Start free Read the API docs

Pierre N.Founder, VecTime CloudBuilds VecTime Cloud — the Go API and its metering, the Python inference service, and the benchmarks published here, including the runs that go against us.