Homepage#
Benchmarking, measuring, and optimizing are fundamental activities in human being's lives. Starting as nine BPOE Workshop series and SDBA Workshop in conjunction with ASPLOS, VLDB, and ICS, this symposium (Bench 18) is organized by International Open Benchmarking Council (BenchCouncil) and dedicated to benchmarking, measuring, and optimizing complex systems, including (but not limited to) Big Data, AI, chainblock, datacenter, cloud and warehouse-scale computing, high performance computing, Mobile Robotics, edge and fog computing,Big Scientific data, IoT, and the other miscellaneous things, i.e., education systems, financial systems, and power systems. In addition, we encourage publishing benchmark specifications and the summaries of the open-source benchmarking projects.
Program#
Preliminary Bench'18 program is relased.
Slides:
Benchmarking Opportunities and Challenges: present and future of BenchCouncil (Prof. Jianfeng Zhan)
Big Data Benchmarking: Applications and Systems (Prof. Geoffrey Fox)
DataMotif: A Benchmark Proposal for Big Data and AI (Dr. Wanling Gao)
A Benchmark proposal for Datacenter Computing (Dr. Chen Zheng)
TS-benchmark: a benchmark proposal for time series databases (Prof. Yueguo Chen)
Benchmarking SpMV on Many-Core Architecture (Dr. Zhen Jia)
DCMIX: Generating Mixed Workloads for the Cloud Data Center (Prof. Yi Liang)
PeakBench: A Benchmark for Scalable Transaction Processing Systems (Prof. Weining Qian)
IBDB: A Benchmark Suite for Industrial Big Data System (Yijian Cheng)
Power Characterization of Memory Intensive Applications: Analysis and Implications
Scalability Evaluation of Big Data Processing Services in Clouds
Important Dates#
Call for Benchmarking Proposals#
Benchmarking proposals are solicited for BenchCouncil open meeting, which will take place in Seattle, WA, on Dec 10-13. Benchmark proposals are solicited on any topic that is relevant to Bench 2018 audience, including (but not limited to):
- Artificial Intelligence
- NewSQL and distributed database systems
- Blockchain
- Datacenter, cloud and warehouse-scale computing
- High performance computing
- Mobile Robotics
- Egde and fog computing
- Big Scientific Data
- Internet of Things (IoT)
- Education, financial and power system
Once a proposal is accepted, at least one of the authors should present their proposal. Meanwhile, a breakout session will be organized to discuss the benchmarking targets, major challenges, and plans. Finally, for each invited talk, a breakout report will be presented by the same speaker.
After the conference, the authors should form a position paper that will be published in the proceeding of Bench 2018. Finally, for each benchmarking plan, a working group will be established, and each published position paper will act as a starting point for a formal benchmark specification, that will be released by BenchCouncil in future.
Proposals should provide the following information:
- Title.
- Authors and contact info.
- 1 paragraph abstract suitable for benchmark publicity.
- biography per author suitable for publicity.
Proposals should be submitted in PDF format via e-mail to Dr. Chen Zheng (zhengchen@ict.ac.cn) and Dr. Zhen Jia (zhenj@cs.princeton.edu) with the subject "BenchCouncil Proposal". Submissions will be acknowledged via e-mail.
Important Dates#
Submission deadline: Nov 10, 2018 Notification: Nov 13, 2018
Call for Papers#
Topics of Interest#
We solicit papers in all areas related to benchmarking, measuring, and optimizing systems. Short papers are encouraged, and the authors can extend their short papers to full versions after the conference. Topics of interest include (but are not limited to):
- Big data domains
- Artificial intelligence
- NewSQL and distributed database systems
- Blockchain
- Datacenter, cloud and warehouse-scale computing
- High performance computing
- Mobile Robotics
- Edge and fog computing
- Big scientific data
- Internet of Things (IoT)
- Education, financial and power system
Submission Site#
Bench 2018 Submission Site: https://wi-lab.com/cyberchair/2018/bigdata18/scripts/submit.php?subarea=SP04&undisplay_detail=1&wh=/cyberchair/2018/bigdata18/scripts/ws_submit.php
Papers must be submitted in PDF. Short papers (no more than 4 pages in standard two-column IEEE conference format or 6 pages in LNCS format, not including references) are encouraged, and the authors can extend their short paper to 12 pages in LNCS format, not including references, in the final publication. If a full paper is submitted, the page limit is 8 pages in standard two-column IEEE conference format or 12 pages in LNCS format, not including references. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, revised papers will be published by Springer LNCS (www.springer.com/lncs, indexed by EI). Please note that LNCS format is the final one for publishing.
LNCS Copyright Form for authors:
Formatting Instructions
IEEE conference format:
8.5" x 11" (DOC, PDF)
LaTex Formatting Macros
LNCS format: LaTex Template
Please make sure your submission satisfies ALL of the following requirements:
- Paper must be submitted in printable PDF format.
- Please number the pages of your submission.
- The submission must be formatted for black-and-white printers. Please make sure your figures are readable when printed in black-and-white.
- The submission must describe unpublished work that is not currently under review of any other conference or journal venues.
Call for Tutorial#
Tutorial proposals are solicited for Bench 2018, which will take place in Seattle, WA. Tutorials will be held on Dec 10-12. Proposals for both half- and full-day tutorials are solicited on any topic that is relevant to the Bench 2018 audience. In our experiences, tutorials seeking to achieve either of the following goals have been particularly successful:
- Describe an important piece of research infrastructure.
- Educate the community on an emerging topic.
Submission Procedures
Proposals should provide the following information:
- Title.
- Presenter(s) and contact information.
- Proposed duration (full day, half day).
- 1-2 paragraph abstract suitable for tutorial publicity.
- 1 paragraph biography per presenter suitable for tutorial publicity.
- 1-3 page description (for evaluation). This should include:
- Tutorial scope and objectives,
- Topics to be covered,
- Target audience,
- If the tutorial has been held previously, the location (i.e., conference), date, and number of attendees.
Proposals should be submitted in PDF format via e-mail to Lei Wang (wangle_2011@ict.ac.cn ) and Chen Zheng (zhengchen@ict.ac.cn) with the subject "Bench’18 Tutorial Proposal". Submissions will be acknowledged via e-mail.
Important Dates
Submission deadline: Oct 6, 2018 Notification: October 24th, 2018
Organization#
Program Chair#
- Prof. Jianfeng Zhan, ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
Program Committee#
- Lizy K. John, The University of Texas at Austin USA
- Vijay Janapa Reddi, University of Texas at Austin USA and Google
- Wenguang Chen, Tsinghua University China
- Woongki Baek, UNIST Korea
- Yunji Chen, ICT, CAS China
- Piotr Luszczek, University of Tennessee USA
- Rui Hou, Institute of Information Engineering, CAS China
- Yueguo Chen, Renmin University of China
- H. Peter Hofstee, IBM USA
- Cheqing Jin, East China Normal University China
- Trevor E. Carlson, National University of Singapore (NUS) Singapore Junzhao Du, Xidian University China
- Hyogi Sim, Oak Ridge National Laboratory USA
- Weining Qian, East China Normal University China
- Zheng Cao, Alibaba China
- Matthias Nicola, IBM Germany
- Zhihui Du, Tsinghua University China
- Chunjie Luo, ICT, CAS
- Bin Ren, College of William and Mary USA
- Hua Chen, ICT, CAS China
- Jiaquan Gao, Nanjing Normal University China
- Jiannan Ouyang, Facebook USA
- Xueming Si, Fudan University China
- Yanjun Wu, Institute of Software Chinese Academy of Sciences China
- Lucas Mello Schnorr, Federal University of Rio Grande do Sul (UFRGS) Brazil
- Yunquan Zhang, ICT, CAS China
- Shengzhong Feng, Shenzhen Institutes of Advanced Technology, CAS China
- Zhen Jia, Princeton University USA
- Zhibin Yu, Shenzhen Institutes of Advanced Technology, CAS China
- Dapeng Wang, Institute of Software Chinese Academy of Sciences China
- Bo Wu, Colorado School of Mines USA
- Shaoliang Peng, National University of Defense Technology China
- Yong Qi, Xi'an jiaotong University China
- Gwangsun Kim, Arm Inc., Austin, Texas USA
- Rui Ren, ICT, CAS China
- Zujie Ren, Hangzhou Dianzi University China
- Xiaoyi Lu, The Ohio State University USA
- Lei Wang, ICT, CAS China
- Jianhui Li, Computer Network Information Center, CAS China
- Jianwu Wang, University of Maryland, Baltimore County USA
- Kenli Li, Hunan University China
- Suzanne Rivoire, Sonoma State University USA Rui Han, ICT, CAS China
- Jungang Xu, Chinese academy of sciences China
- Xu Liu, College of William and Mary USA
- Tong Wu, National Institute of Metrology, China
- Li ZHA, University of the Chinese Academy of Sciences
- Lei Liu, ICT, CAS China
- Chen Zheng, ICT, CAS China
Tutorial Co-Chairs#
- Dr. Lei Wang ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
- Dr. Chen Zheng ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
Publication Chair#
- Prof. Xiaohui Hu Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
Web Chairs#
- Mr. Fei Tang University of Chinese Academy of Sciences
Publicity Chairs#
- Miss. Wanling Gao, University of Chinese Academy of Sciences
- Miss. Rui Ren, ICT, Chinese Academy of Sciences
- Dr. Zhen Jia, Princeton University
Submission Co-Chairs#
- Miss. Rui Wang Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
- Miss. Ruiying Li Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
Sponsorship Chair#
- Dr. Biwei Xie ICT, Chinese Academy of Sciences
Sponsors#
Best Paper Awards#
Previous Events#
-
J. Zhan, R. Han, C. Weng. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 8807, 2014.
-
J. Zhan, R. Han, R. V. Zicari. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 9495, 2016.
Location#
Seattle, WA





