You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we rebalance shards any time we notice kinesis shard counts change. We have noticed that at high shard counts (100+), kinesis' scaling can take a very long time - up to a couple of hours - before the shard count is stable again.
This will cause kinsumer to constantly stop-start itself, which can be an issue for apps using it.
One solution to work around this is to check the stream status at the same time that we check the shard count, and only rebalance once it's done scaling.
The text was updated successfully, but these errors were encountered:
Currently we rebalance shards any time we notice kinesis shard counts change. We have noticed that at high shard counts (100+), kinesis' scaling can take a very long time - up to a couple of hours - before the shard count is stable again.
This will cause kinsumer to constantly stop-start itself, which can be an issue for apps using it.
One solution to work around this is to check the stream status at the same time that we check the shard count, and only rebalance once it's done scaling.
The text was updated successfully, but these errors were encountered: