Skip to main content
Version: latest

Beta 2

New Features​

  • Support for customizing JSON serialization and deserialization

Improvements​

  • CLI: Operators logs works for both run and deployment mode including cloud.
  • CLI: Hub package list now shows public or private packages.
  • Hub: Private package works with hub.
  • Deployment: Separate run vs deployment mode.
  • Deployment: Dataflows can be stopped and started in the worker.
  • Deployment: dataflow command uses fully qualified package name to avoid conflicts.
  • Deployment: show dataflow now display the status and last updated time.
  • Deployment: Dataflow can be stopped and restarted.
  • Deployment: Import errors are properly reported.
  • Worker: Worker listing with -all shows the version.
  • Config: Producer config now supports compression, isolation, and timeout.
  • Cloud: Cloud supports larger worker storage.
  • Package: Generated codes are consolidated and hidden from developers.
  • Operator: Operator error messages are sent to log.

Bug Fixes​

  • CLI show list of created topics

Breaking Changes​

V4 of dataflow is no longer supported. Please upgrade to V5.

Package migration​

Since beta2, the packages functions are consolidated in a single rust crate on generation. This allows us to share types and logic between different package functions.

In order to migrate your existing packages, you need to generate your package (sdf generate) using the sdf-beta2 version and then copy the code in the new location of the generated code.

In other words, for each function, the logic in rust/<function-name>/src/lib.rs must be copied on rust/<package-name>/src/<function_name>.rs. But please, make sure that all the attributes in #[sdf] attribute macro matches with the attributes that we get on code generation.