Google Releases Time-Series Foundation Model
Google Research has introduced TimesFM, a decoder-only foundation model designed for time-series forecasting. The model was trained on a diverse dataset of 100 billion real-world and synthetic time points to enable zero-shot forecasting for tasks like sales, traffic, and financial predictions. Google has open-sourced the model on GitHub to facilitate rapid prototyping and research.
- The model's architecture is a 200-million-parameter, decoder-only transformer, similar to those used in LLMs like GPT. It processes time-series data by grouping contiguous time-points into "patches," a technique adapted from Vision Transformers (ViTs) to efficiently capture temporal patterns. - Its training corpus of 100 billion data points includes a mix of real-world and synthetic data. Key real-world sources were Google Trends data (0.5 billion points) and Wikipedia Pageviews (300 billion points), supplemented by public forecasting datasets like M4, Electricity, and Traffic. - In zero-shot scenarios, TimesFM's performance is competitive with state-of-the-art supervised models that were trained specifically on the target datasets. On benchmarks like the Monash Archive and the Informer benchmark, it consistently ranks as a top-3 model without any dataset-specific fine-tuning. - The model is designed for univariate time-series forecasting, predicting a single variable using only its past observations; it does not natively incorporate external factors or covariates like price changes or holidays. - TimesFM 1.0 can process a context length of up to 512 time points and generates forecasts in output patches (e.g., 128 time points at once) rather than generating autoregressively one point at a time, which helps reduce error propagation. - The model is part of a growing landscape of time-series foundation models, competing with others like Amazon's Chronos and Nixtla's TimeGPT. Benchmarks indicate TimesFM has leading accuracy, while TimeGPT-1 was noted for its combination of accuracy and inference speed. - For enterprise applications, TimesFM is integrated into Google Cloud's BigQuery ML, allowing users to run forecasts on millions of time series with a single SQL function (`AI.FORECAST`) without managing model infrastructure. - The research, authored by a team at Google including Abhimanyu Das and Rajat Sen, was accepted and presented at the International Conference on Machine Learning (ICML) in 2024.
