GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

assumptions are being made? /* Opens the device with name ‘dev’. On failure to open, returns SIZE_MAX. */ size_t open_device(char const* dev); size_t open_device(std::wstring_view const dev); The types same as “width”.What isn’t being declared? struct data_t { int a; int b; }; /* Get data from device ‘dev’. */ data_t get_data_from(size_t dev); What defines how dev is passed or data_t is returned data_t { int a; int b; }; /* Get data from device ‘dev’. */ data_t get_data_from(size_t dev); class dev_t { public: /* Get data from this device. */ virtual data_t get_data_from() = 0; }; Assuming

to be? Swift/Obj-C Java .NET We ❤ Web. But… We need: • Better offline support • Access to all device APIs • Home screen availability • Push notifications • App monetization • App store discovery "Promise" 100% Web 100% Nativ e Hybri d Reach Code/Skill Reuse Richness Premium experience Device APIs Best of both? Compromises. Mobile Web - PWA Hybrid Native “1st Gen” X-Plat Native Binary Reuse existing skills/teams • Reuse existing libraries • Native UI (no WebView!) • Full access to device APIs • Immediate access to new OS features ? Fast to market ? Best experience Intro to NativeScript

Python/Matlab/R Java/C/C++/Restful IoT Application Visualization Tool Grafana Web Based Management Tool 21,400+ 4,600+ 18,500+ 社区版开源 2019.07.12 集群版开源 2020.08.03 云原生版开源 2022.08.13 注 : GitHub 为截止 {"names_from": "field1", "values_from": "field2"}} {"pivot_longer": {"select": ["device1", "device2", "device3", "device4"], "names_to": "device", "values_to": "observation" }} {"parse": {"field2": { "regex": "(?P\d+)\s+(

Single code base for both platforms Extended gesture and action support Image comparison Device management Reporting nativescript-dev-appium Installation npm install nativescript-dev-appium --save-dev Platform name Platform version Device name Application path https://appium.io/slate/en/master/?javascript#appium-server-capabilities Demo Ready To Execute Note: Device name App path $ npm run e2e

correctness and performance • Compiled to native code offering performance similar to C • Memory management without garbage collection • Designed for systems programming Rust is a multi-paradigm programming vmlinux and bzImage images as well as some other helpers for kernel commandline. • vm-virtio: virtio device trait and implementation for virtio primitives such as virtqueues and descriptor chain. crates linux-loader kvm-ioctls vm-virtio vsock net VFIO APIC vhost Cloud Hypervisor KVM File System Device Driver Host Linux Kernel vCPU block Memory Guest VM containerd + Kata-runtime create start VM

• Introduction • Memory • Host vs Device Functions • Return on Investment • Concluding remarks Outline 2 |• I work the RiskLab team at CSIRO on applied mathematics for Financial Risk. • The aim of key is that the system automatically migrates data allocated in Unified Memory between host and device...” -- https://developer.nvidia.com/blog/unified-memory-in-cuda-6/ Unified Memory System Memory Unified Memory between host and device….” -- https://developer.nvidia.com/blog/unified-memory-in-cuda-6/ Unified Memory System Memory GPU Memory 24 |struct device_memory_resource : std::pmr::memory_resource

的一大好处， CUDA 和 C++ 的关 系就像 C++ 和 C 的关系一样，大部分都兼容 ，因此能很方便地重用 C++ 现有的任何代码库 ，引用 C++ 头文件等。 • host 代码和 device 代码写在同一个文件内，这 是 OpenCL 做不到的。 编写一段在 GPU 上运行的代码 • 定义函数 kernel ，前面加上 __global__ 修 饰符，即可让他在 GPU 上执行。 用，可以有参数，不可以有返回值。 • 而 __device__ 则用于定义设备函数，他在 GPU 上执行，但是从 GPU 上调用的，而 且不需要三重尖括号，和普通函数用起来一 样，可以有参数，有返回值。 • 即： host 可以调用 global ； global 可以调 用 device ； device 可以调用 device 。 声明为内联函数 • 注意， inline 在现代 上的主机函数 • __device__ 将函数定义在 GPU 上，而 __host__ 则相反，将函数定义在 CPU 上。 定义在 CPU 上的主机函数 • CUDA 完全兼容 C++ ，因此任何函数如 果没有指明修饰符，则默认就是 __host__ ，即 CPU 上的函数。 同时定义在 CPU 和 GPU 上 • 通过 __host__ __device__ 这样的双重修 饰符，可以把函数同时定义在

applications is a breeze. Easy The needs of a server application differ from that of an embedded device. Although Tokio comes with defaults that work well out of the box, it also provides the knobs

in Software Engineering Mike Ballou“There is no single development, in either technology or management technique, which by itself promises even one order-of- magnitude improvement within a decade abstract away its complexity often abstract away its essence • Not only technical problems but management problems as well comes from the complexity • It creates the tremendous learning and understanding I can help them I need…” • to Containerize the software, but to run that I need… • a Container Management Platform, but to network it I need… • a Service Mesh, but to secure it I need… • an automated Certificate