- Upgrade from Python 3.8 to Python 3.10
- add 2.x api backport support
- some bug fix
Enhancements in the JVM Part:
- Core Component Reconstruction: The
cluster-manager
andnode-manager
components have been entirely rebuilt using Java, ensuring uniformity and enhanced performance. - Transport Component Modification: The
rollsite
transport component has been removed and replaced with the more efficientosx
component. - Improved Process Management: Advanced logic has been implemented to manage processes more effectively, significantly reducing the risk of process leakage.
- Enhanced Data Storage Logic: Data storage mechanisms have been refined for better performance and reliability.
- Concurrency Control Improvements: We've upgraded the logic for concurrency control in the original components, leading to performance boosts.
- Visualization Component: A new visualization component has been added for convenient monitoring of computational information.
- Refined Logging: The logging system has been enhanced for more precise outputs, aiding in rapid anomaly detection.
Upgrades in the Python Part:
- Reconstruction of
roll_pair
andegg_pair
: These components now support serialization and partition methods controlled by the caller. Serialization safety is uniformly managed by the caller. - Automated Cleanup of Intermediate Tables: The issue of automatic cleaning for intermediate tables between federation and computing has been resolved, eliminating the need for extra operations by the caller.
- Unified Configuration Control: A flexible configuration system is introduced, supporting direct pass-through, configuration files, and environment variables to cater to diverse requirements.
- Client-Side PyPI Installation: Eggroll 3.0 supports easy installation via PyPI for clients.
- Optimized Log Configuration: Callers can now customize log formats according to their needs.
- Code Structure Refinement: The codebase has been streamlined for clarity, removing a substantial amount of redundant code.