比較基本的模組： Hadoop HDFS (Storage) Hadoop MapReduce (Computing Engine + Resource Management + Job Scheduling / Monitoring + ...) 比較明顯的限制： 每個 Cluster 大概就是 4,000 - 4,500 個 Node JobTracker 是架構瓶頸，Concurrent Hadoop HDFS (Storage) Hadoop MapReduce (Computing Engine) Hadoop YARN (Resource Management + Job Scheduling / Monitoring) 比較沒人知道的事： Hadoop 2.x 也默默地做了四五年了 ... 雖然是大修，但是 Backward Compatibility 做的很棒 Yahoo

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1120 49 Tasks 1126 49.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1132 49.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are sampling tasks that are running—a wall-time task profiler sam- ples tasks independently of their scheduling state. For example, tasks that are sleeping on a synchroniza- tion primitive at the time the profiler

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1119 49 Tasks 1125 49.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1131 49.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are sampling tasks that are running—a wall-time task profiler sam- ples tasks independently of their scheduling state. For example, tasks that are sleeping on a synchroniza- tion primitive at the time the profiler

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1119 49 Tasks 1125 49.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1131 49.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are sampling tasks that are running—a wall-time task profiler sam- ples tasks independently of their scheduling state. For example, tasks that are sleeping on a synchroniza- tion primitive at the time the profiler

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1111 48 Tasks 1117 48.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1123 48.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are sampling tasks that are running—a wall-time task profiler sam- ples tasks independently of their scheduling state. For example, tasks that are sleeping on a synchroniza- tion primitive at the time the profiler

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1079 48 Tasks 1085 48.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1090 48.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are key, or lack thereof, is restored afterwards. Useful for emulating dynamic scoping. source 48.1 Scheduling Base.yield – Function. yield()CHAPTER 48. TASKS 1091 Switch to the scheduler to allow another

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1080 48 Tasks 1085 48.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1090 48.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are key, or lack thereof, is restored afterwards. Useful for emulating dynamic scoping. source 48.1 Scheduling Base.yield – Function. yield()CHAPTER 48. TASKS 1091 Switch to the scheduler to allow another

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1080 48 Tasks 1085 48.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1090 48.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are key, or lack thereof, is restored afterwards. Useful for emulating dynamic scoping. source 48.1 Scheduling Base.yield – Function. yield()CHAPTER 48. TASKS 1091 Switch to the scheduler to allow another

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1123 49 Tasks 1128 49.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1134 49.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are sampling tasks that are running—a wall-time task profiler sam- ples tasks independently of their scheduling state. For example, tasks that are sleeping on a synchroniza- tion primitive at the time the profiler

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 935 47 Tasks 939 47.1 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944 47.2 Synchronization occur at intervals (by default, 1 ms on Unix systems and 10 ms on Windows, although the actual scheduling is subject to operating system load). Moreover, as discussed further below, because samples are is restored afterwards. Useful for emulating dynamic scoping. sourceCHAPTER 47. TASKS 944 47.1 Scheduling Base.yield – Function. yield() Switch to the scheduler to allow another scheduled task to run