Scalytics Connect Release Notes

New Features & Performance Enhancements Updates

Scalytics Connect v. 1.02 / September 2023

Scalytics Core

Apache Wayang is at the core of our products. Apache Wayang (incubating) is the only open-source, cross-platform data processing engine. Application developers specify applications using Apache Wayang’s API. 

Unique features:
  • AI-based optimizer automatically picks an optimal configuration of class data processing frameworks, such as Java Streams or Apache Spark, to run applications on.
  • Scalytics Core performs the program execution. It abstracts the different platform specific APIs and coordinates inter-platform communication.
  • Applications can run on multiple data processing platforms, without changing the native code of the underlying platforms.

Data sources:
  • PostgresSQL
  • Columnar Data Files (e.g., CSV, Iceberg, Parquet, ORC)
  • SQlite (e.g. Mobiles, Embedded)
  • Local file systems
  • Distributed file systems (e.g., HDFS, S3)

Data Processing Platforms:
  • Java 8 Streams
  • Apache Spark / DataBricks
  • Postgres
  • SQLite
  • Apache Flink / Confluent, Decodable

Programming APIs
  • Java
  • Scala
  • Basic SQL

Scalytics Studio

Scalytics Studio is a cloud-native, low code add-on to Scalytics Core for designing AI enhanced data management and ETL pipelines. 

Scalytics Studio delivers a graphical user interface that allows you to connect to and locally query different data sources or join data across multiple data sources in a very intuitive way. It also supports complex data transformations that can be processed on platforms of your choice.

Supported Data Sources:
  • PostgresSQL
  • Files on local or distributed filesystems (e.g., HDFS)

Supported Platforms
  • Java 8 Streams
  • Apache Spark / DataBricks

Supported Data Transformations
  • Map
  • Filter 
  • Reduce
  • GroupBy
  • Join
  • Union
  • Cross

Compliance

Scalytics tests and verifies the platform security, privacy, and compliance to help you meet regulatory and policy objectives, including unique compliance needs of highly-regulated industries.
GDPR logoHIPAA logo
Don't just dream of better data pipelines, make it happen.