You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today, the C* source connectors only allows 1:1 between tables and sinks. In order to increate the utilization of the underling resources associated with a single source instance (e.g. Memory footprint a single sink is ~500MB, which does not scale well if the user has 10s or 100s of tables), the proposal is to enable users to configure multiple tables in their source config.
Alternatively, we can keep the config as close as possible to today's single table configs by replacing data.topic with destination-topic-name
aymkhalil
changed the title
[Source][Utilization] Enable processing multiple C* in a single sink instance
[Source][Utilization] Enable processing multiple C* in a single source instance
Mar 15, 2023
Today, the C* source connectors only allows 1:1 between tables and sinks. In order to increate the utilization of the underling resources associated with a single source instance (e.g. Memory footprint a single sink is ~500MB, which does not scale well if the user has 10s or 100s of tables), the proposal is to enable users to configure multiple tables in their source config.
Proposed source config:
The text was updated successfully, but these errors were encountered: