Federated Foundation Models on Heterogeneous Time Series (FFTS)

AAAI 2025 |

Official implementation for the AAAI'25 paper "Federated Foundation Models on Heterogeneous Time Series"

---

📌 Table of Contents

• 🚀 Overview
• ✨ Key Features
• 🎯 Method
• 🛠️ Installation
• ⚡ Quick Start
• 📁 Repository Structure
• 🎓 Citation
• 🤝 Acknowledgments
• 📜 License

---

🚀 Overview

Training general-purpose time series foundation models across diverse domains is challenging due to severe statistical heterogeneity. FFTS tackles this with a federated learning formulation where each dataset owner is a client with its own local model.

🎯 Core Innovation

Challenge	Solution
⚠️ Statistical heterogeneity across datasets	✅ Client-specific local models
⚠️ Domain-specific patterns loss	✅ Shared knowledge alignment
⚠️ Limited generalization	✅ Dual-side regularization
⚠️ Single-task limitations	✅ Unified adaptation architecture

🔄 How It Works

┌─────────────────────────────────────────────────────────────┐
│                     Central Server                          │
│  ┌─────────────────────────────────────────────────────┐   │
│  │  Global Model Aggregation + Knowledge Alignment     │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────┬───────────────────────────────────┘
                          │
          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
    ┌──────────┐    ┌──────────┐    ┌──────────┐
    │ Client 1 │    │ Client 2 │    │ Client N │
    │  Domain  │    │  Domain  │    │  Domain  │
    └──────────┘    └──────────┘    └──────────┘

The resulting foundation model generalizes well across forecasting, imputation, and anomaly detection tasks.

---

✨ Key Features

🌟 Highlights

graph TD
    A[FFTS Framework] --> B[Federated Foundation Model]
    A --> C[Client-Specific Local Models]
    A --> D[Unified Adaptation Architecture]
    A --> E[Learnable Time-Scale Weights]

    B --> B1[Heterogeneous Dataset Support]
    C --> C1[Domain-Specific Pattern Preservation]
    D --> D1[Multi-Task Adaptation]
    E --> E1[Temporal Pattern Learning]

    style A fill:#6e42f5,color:#fff
    style B fill:#e7f2ff
    style C fill:#e7f2ff
    style D fill:#e7f2ff
    style E fill:#e7f2ff

Loading

📋 Feature Details

Feature	Description	Benefit
🔗 Federated Learning	Each dataset owner operates as an independent client	Privacy-preserving collaboration
🧠 Client-Specific Models	Local models preserve dataset-specific characteristics	Better domain adaptation
🤝 Knowledge Alignment	Client and server regularization align shared knowledge	Effective cross-domain learning
🎯 Unified Adaptation	Single architecture for multiple downstream tasks	Efficient fine-tuning
⏰ Learnable Time-Scale Weights	ATM module with adaptive temporal weights	Enhanced pattern recognition

---

🎯 Method

Architecture Overview

Pretraining Datasets

Training Pipeline

🔧 Detailed Training Process

graph LR
    A[Data Collection] --> B[Local Preprocessing]
    B --> C[Client Model Training]
    C --> D[Local Regularization]
    D --> E[Model Upload]
    E --> F[Server Aggregation]
    F --> G[Global Regularization]
    G --> H[Model Distribution]
    H --> C

    style A fill:#e1f5ff
    style C fill:#e1f5ff
    style F fill:#fff4e1
    style H fill:#e1ffe1

Loading

Key Components:

1. Client-Side Training

   • Local model optimization on private data
   • Client-specific pattern preservation
   • Local regularization for knowledge alignment

2. Server-Side Aggregation

   • Secure model averaging
   • Global knowledge alignment
   • Federated model distribution

3. Regularization Mechanism

   • Dual-side alignment (client + server)
   • Balances shared vs. domain-specific knowledge
   • Ensures generalization across domains

---

🛠️ Installation

Prerequisites

• Python >= 3.8
• PyTorch >= 2.0
• CUDA (for GPU acceleration, recommended)

Setup

# Clone the repository
git clone https://github.com/shengchaochen82/FFTS.git
cd FFTS

# Create virtual environment
python -m venv ffts_env
source ffts_env/bin/activate  # On Windows: ffts_env\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Dependencies

📦 Requirements List

torch>=2.0.0
numpy>=1.21.0
pandas>=1.3.0
scikit-learn>=1.0.0
matplotlib>=3.4.0
tensorboard>=2.8.0

---

⚡ Quick Start

1️⃣ Prepare Data

# Download datasets from Monash Time Series Repo
# Visit: https://forecastingdata.org/

# Follow preprocessing steps in the notebook
jupyter notebook preprocessing.ipynb

💡 Tip: The preprocessing.ipynb notebook provides unified preprocessing for all supported datasets.

2️⃣ Run Training

# Basic pretraining command
python main.py \
  --task pretrain \
  --task_note demo_run \
  --is_training 1 \
  --algorithm FFTS \
  --dataset weather \
  --global_rounds 10 \
  --local_epochs 5

3️⃣ Configuration Options

⚙️ Advanced Configuration

Argument	Description	Default
--task	Task type (pretrain, forecasting, imputation, anomaly_detection)	pretrain
--dataset	Dataset name (weather, traffic, electricity, etc.)	weather
--algorithm	Federated algorithm (FFTS, FedAvg)	FFTS
--global_rounds	Number of federated learning rounds	10
--local_epochs	Local training epochs per client	5
--batch_size	Training batch size	32
--learning_rate	Learning rate	0.001
--num_clients	Number of federated clients	10

Example: Forecasting Task

python main.py \
  --task long_term_forecast \
  --task_note weather_forecast \
  --is_training 1 \
  --algorithm FFTS \
  --dataset weather \
  --pred_len 96 \
  --seq_len 96

Example: Imputation Task

python main.py \
  --task imputation \
  --task_note weather_impute \
  --is_training 1 \
  --algorithm FFTS \
  --dataset weather \
  --mask_rate 0.2

Example: Anomaly Detection

python main.py \
  --task anomaly_detection \
  --task_note traffic_anomaly \
  --is_training 1 \
  --algorithm FFTS \
  --dataset traffic

---

📁 Repository Structure

FFTS/
├── 📂 data_provider/           # Dataset loading and preprocessing
│   ├── data_base.py           # Base dataset class
│   ├── monash_data.py         # Monash dataset loader
│   └── pre_loader.py          # Data preprocessing utilities
│
├── 📂 flcore/                  # Federated learning core components
│   ├── servers/               # Server implementations
│   │   ├── serveravg.py       # FedAvg server
│   │   └── serverffts.py      # FFTS server
│   ├── clients/               # Client implementations
│   │   ├── clientbase.py      # Base client class
│   │   └── clientavg.py       # FedAvg client
│   └── layers/                # Neural network layers
│       ├── Transformer_EncDec.py
│       ├── SelfAttention_Family.py
│       └── Embed.py
│
├── 📂 models/                  # Model definitions
│   └── ffts_model.py          # FFTS model architecture
│
├── 📂 utils/                   # Utilities and helpers
│   ├── metrics.py             # Evaluation metrics
│   ├── tools.py               # Helper functions
│   ├── losses.py              # Loss functions
│   └── timefeatures.py        # Time feature extraction
│
├── 📂 assest/                  # Assets (images, diagrams)
│   ├── difference.png         # Architecture diagram
│   ├── adaption.png           # Adaptation architecture
│   └── pretrain_data.png      # Dataset visualization
│
├── preprocessing.ipynb         # Unified preprocessing notebook
├── main.py                     # Training entry point
├── requirements.txt            # Python dependencies
├── LICENSE                     # MIT License
└── README.md                   # This file

---

📜 Citation

If you find this work useful, please cite our paper:

@inproceedings{chen2025federated,
  title={Federated foundation models on heterogeneous time series},
  author={Chen, Shengchao and Long, Guodong and Jiang, Jing and Zhang, Chengqi},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={39},
  number={15},
  pages={15839--15847},
  year={2025},
  organization={AAAI Press}
}

📊 Altmetrics

---

🤝 Acknowledgments

Note

Development Status: We are continuously improving the codebase. Some interfaces may change as we enhance the framework.

This work was inspired and supported by:

• 🙏 Time-Series-Library - Excellent time series modeling framework
• 🙏 PFLlib - Personalized federated learning library
• 🙏 Monash Time Series Repository - Comprehensive time series datasets

---

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

📬 Contact

• Shengchao Chen - Bio
• For questions, please open an issue

---

🗺️ Roadmap

🚀 Development Timeline

timeline
    title FFTS Development Roadmap
    section 2024
        Dec 2024 : Paper accepted at AAAI 2025
        Dec 2024 : Preprint posted on arXiv
    section 2025
        Jan 2025 : Pretraining datasets available
        Jan 2025 : Preprocessing tutorials released
        Aug 2025 : Codebase restructured
        Aug 2025 : Learnable time-scale weights added
    section Future
        Q1 2026 : Extended experiments
        Q2 2026 : Additional dataset support
        Q3 2026 : Documentation enhancement

Loading

Completed ✅

• Release core codebase
• Release detailed training tutorials
• Pretraining data download and tutorials
• Release AAAI 2025 paper
• Implement federated learning framework
• Add ATM module with learnable weights

---

🔗 Related Projects

• Time-Series-Library - A unified library for time series analysis
• PFLlib - Personalized federated learning library
• FederatedScope - A comprehensive federated learning platform

---

⬆ Back to Top

Made with ❤️ by Shengchao Chen