Apache Wayang

Learn Apache Wayang from its creators. Tutorials, architecture guides, and benchmarks for cross-platform data processing.

Distributed Processing for LLM Training: Unify Your Data Lakes

Large language models require diverse training data scattered across your organization. Distributed data processing with Apache Wayang lets you unify data lakes for LLM fine-tuning without moving sensitive data—enabling compliant AI training across healthcare, finance, and government sectors.

Author:

Alexander Alten

Cross-Platform Analytics Benchmark: 10x Performance Gains

We benchmarked Scalytics Connect against standalone Spark, Flink, and Java across TPC-H and ML workloads. Results: cross-platform optimization delivered 7-10x faster execution on complex joins and 40% lower infrastructure costs through intelligent workload distribution.

Author:

Dr. Zoi Kaoudi

AI Query Optimizer: ML-Powered Data Processing Speed

Traditional query optimizers fail with cross-platform workloads. Our learning-based query optimizer uses machine learning to automatically select optimal execution plans across Spark, Flink, and Postgres—reducing query times by up to 10x on complex analytical workloads.

Author:

Dr. Zoi Kaoudi

Apache Wayang: Complete Guide by Its Creators

Apache Wayang enables unified cross-platform data processing across Spark, Flink, and Java in a single pipeline. Created by Scalytics founders, now an Apache TLP. Learn how to reduce processing costs by 40% while eliminating vendor lock-in with this comprehensive implementation guide.

Author:

Alexander Alten

Scalytics Architecture Deep Dive: Wayang & Federated AI

A technical deep dive into the Scalytics architecture: how Apache Wayang powers our federated data layer, the ML-based cost optimizer, and our approach to privacy-preserving AI. Essential reading for data architects evaluating cross-platform processing solutions.

Author:

Dr. Jorge Quiané