scope  Bottom-up approach  #3 Benchmarks  Evaluate on community/own benchmarks  Examples: TPC-C, TPC-H, TPC-DS, JOB, MLPerf  #4 End-to-end Applications  Evaluate in larger scope of real datasets Benchmarks can define a field and foster innovation  #1 Data Management  Query processing: 007, TPC-C, TPC-E, TPC-H, TPC-DS (w/ audit)  Join ordering: JOB  #2 “Big Data”  MR/Spark: BigBench, HiBench