x deeplearning

We have hosted the application x deeplearning in order to run this application in our online workstations with Wine or directly.

Quick description about x deeplearning:

X-DeepLearning (XDL for short) is a complete set of deep optimization solutions for high-dimensional sparse data scenarios (such as advertising/recommendation/search, etc.). XDL version 1.2 has been released recently. Performance optimization for large batch/low concurrency scenarios, 50-100% performance improvement in such scenarios. Storage and communication optimization, parameters are automatically allocated globally without manual intervention, and requests are merged to completely eliminate computing/storage/communication hotspots of ps. Complete streaming training features including feature admission, feature elimination, model incremental export, feature counting statistics, etc. Background: XDL1.0 focuses on throughput optimization and adopts the one request per thread processing model, which can significantly improve the limit throughput under ultra-high concurrency.

Features:

• Performance optimization of large batch/single-sample massive feature scenarios
• Parameter assignment and communication optimization
• Unified storage and average distribution of parameters: ensure the average distribution of computing, communication, and memory on all servers
• Automatically analyze and merge undependent communication nodes in the calculation graph to reduce the number of communications and improve communication efficiency
• Simplify user usage costs, no longer need to provide the estimated value of the embedding parameter size, and no longer need to do regular rebalance
• In scenarios with massive features, performance and scalability are further improved

Categories: