GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Resource orchestration optimization of kubernetes cluster in large scale Patrickxie ( 谢谆志） Background Cloud has been the general trend. How to manage so many clusters ,resources and businesses How to ensure load balancing of cluster nodes 1 2 Improper resource requests 3 Multi-tenant resource preemption How to expand horizontally more quickly and flexibly 4 Region1 How do you manage K8S scheduling is based on the resource request of Pod. However, in many cases, some nodes have low resource requests but high load, while some nodes have high resource requests but low load. Dynamic-Scheduler

C++17: • std::pmr::memory_resource • std::pmr::polymorphic_allocator • std::pmr::vector • std::pmr::monotonic_buffer_resource • … std::pmr 16 |// gpu unified_memory_resource mem; std::pmr::vector std::pmr::vector y(N, &mem); // … Memory Allocation 17 |struct unified_memory_resource : std::pmr::memory_resource { void* do_allocate(std::size_t, std::size_t); void do_deallocate( do_is_equal( const std::pmr::memory_resource& other) const noexcept; }; A Unified Memory Resource 18 |struct unified_memory_resource : std::pmr::memory_resource { void* do_allocate(std::size_t

under synchronization/Hazptr.h Is Asynchronous Reclamation Always Enough? delete ptr shutdown_resource_X() Hazard Pointer Synchronous Reclamation Beyond Concurrency TS2 – Maged Michael Example: Removed reclamation ~Foo() { use_resource_X (); } Foo* ptr = new Foo; Remove ptr Is Asynchronous Reclamation Always Enough? No Asynchronous reclamation: delete ptr Using unavailable resource X Hazard Pointer Synchronous Michael Same example but using hazard pointer deferred reclamation ptr-retire() shutdown_resource_X() ~Foo() { use_resource_X (); } Foo* ptr = new Foo; Remove ptr Some use cases need synchronous reclamation

alias. P ∗ Q Separation Logic (Iris) P Q v x y P −∗ Q Magic wand Consuming resource “P”, we can reach resource “Q”. Q P what we acually own Logics (x ↦ v) ∗ (x ↦ w) ⟹ False (x ↦ v) ∧ (x

concept: mutex, latches, barriers, …  Two types:  counting semaphore: models a non-negative resource count  binary semaphore: only has 1 slot, i.e. only has two states: free slot or no free slot

P2300: STD::EXECUTION Proposes: • A set of concepts that represent: • A handle to a compute resource (aka, scheduler) • A unit of lazy async work (aka, sender) • A completion handler (aka, receiver)

，就没必要去搞封装，只会让程序员 变得痛苦，同时还有一定性能损失：特别 是如果 getter/setter 函数分离了声明和定 义，实现在另一个文件时！ C++ 思想： RAII （ Resource Acquisition Is Initialization ） 资源获取视为初始化，反之，资源释放视为销毁 C++ 除了用于初始化的构造函数（ constructor ） 还包括了用于销毁的解构函数（