2018 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing
Bench 2018

2018 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing

2018-11-10 - 2018-11-13

Seattle, WA, USA

Description

No event description available yet.

Homepage#

Description

Benchmarking, measuring, and optimizing are fundamental activities in human being's lives. Starting as nine BPOE Workshop series and SDBA Workshop in conjunction with ASPLOS, VLDB, and ICS, this symposium (Bench 18) is organized by International Open Benchmarking Council (BenchCouncil) and dedicated to benchmarking, measuring, and optimizing complex systems, including (but not limited to) Big Data, AI, chainblock, datacenter, cloud and warehouse-scale computing, high performance computing, Mobile Robotics, edge and fog computing,Big Scientific data, IoT, and the other miscellaneous things, i.e., education systems, financial systems, and power systems. In addition, we encourage publishing benchmark specifications and the summaries of the open-source benchmarking projects.

Program#

Program

Preliminary Bench'18 program is relased.

PDF Version

Slides:

Benchmarking Opportunities and Challenges: present and future of BenchCouncil (Prof. Jianfeng Zhan)

Big Data Benchmarking: Applications and Systems (Prof. Geoffrey Fox)

DataMotif: A Benchmark Proposal for Big Data and AI (Dr. Wanling Gao)

Characterizing and Benchmarking Deep Learning Systems on Modern Data Center Architectures (Prof. Xiaoyi Lu)

A Benchmark proposal for Datacenter Computing (Dr. Chen Zheng)

MLPerf: The Vision Behind an ML Benchmark Suite for Measuring the Performance of ML Software Frameworks, ML Hardware Accelerators, and ML Cloud and Edge Platforms (Prof. Vijay Janapa Reddi)

TS-benchmark: a benchmark proposal for time series databases (Prof. Yueguo Chen)

Benchmarking SpMV on Many-Core Architecture (Dr. Zhen Jia)

DCMIX: Generating Mixed Workloads for the Cloud Data Center (Prof. Yi Liang)

PeakBench: A Benchmark for Scalable Transaction Processing Systems (Prof. Weining Qian)

EC-Bench: Benchmarking Onload and Offload Erasure Coders on Modern Hardware Architectures (Haiyang Shi)

IBDB: A Benchmark Suite for Industrial Big Data System (Yijian Cheng)

Power Characterization of Memory Intensive Applications: Analysis and Implications

Scalability Evaluation of Big Data Processing Services in Clouds

Testing Raft-replicated Database Systems

Important Dates#

TimeEvent
Oct 10, 2018Electronic submission of full papers
Nov 9, 2018Notification of paper acceptance
Dec 10-12, 2018Conference
Feb 9, 2019Camera-ready of extended papers

Call for Benchmarking Proposals#

Benchmarking proposals are solicited for BenchCouncil open meeting, which will take place in Seattle, WA, on Dec 10-13. Benchmark proposals are solicited on any topic that is relevant to Bench 2018 audience, including (but not limited to):

  • Artificial Intelligence
  • NewSQL and distributed database systems
  • Blockchain
  • Datacenter, cloud and warehouse-scale computing
  • High performance computing
  • Mobile Robotics
  • Egde and fog computing
  • Big Scientific Data
  • Internet of Things (IoT)
  • Education, financial and power system

Once a proposal is accepted, at least one of the authors should present their proposal. Meanwhile, a breakout session will be organized to discuss the benchmarking targets, major challenges, and plans. Finally, for each invited talk, a breakout report will be presented by the same speaker.

After the conference, the authors should form a position paper that will be published in the proceeding of Bench 2018. Finally, for each benchmarking plan, a working group will be established, and each published position paper will act as a starting point for a formal benchmark specification, that will be released by BenchCouncil in future.

Proposals should provide the following information:

  • Title.
  • Authors and contact info.
  • 1 paragraph abstract suitable for benchmark publicity.
  • biography per author suitable for publicity.

Proposals should be submitted in PDF format via e-mail to Dr. Chen Zheng (zhengchen@ict.ac.cn) and Dr. Zhen Jia (zhenj@cs.princeton.edu) with the subject "BenchCouncil Proposal". Submissions will be acknowledged via e-mail.

Important Dates#

Submission deadline: Nov 10, 2018 Notification: Nov 13, 2018

Call for Papers#

Topics of Interest#

We solicit papers in all areas related to benchmarking, measuring, and optimizing systems. Short papers are encouraged, and the authors can extend their short papers to full versions after the conference. Topics of interest include (but are not limited to):

  • Big data domains
  • Artificial intelligence
  • NewSQL and distributed database systems
  • Blockchain
  • Datacenter, cloud and warehouse-scale computing
  • High performance computing
  • Mobile Robotics
  • Edge and fog computing
  • Big scientific data
  • Internet of Things (IoT)
  • Education, financial and power system

Submission Site#

Bench 2018 Submission Site: https://wi-lab.com/cyberchair/2018/bigdata18/scripts/submit.php?subarea=SP04&undisplay_detail=1&wh=/cyberchair/2018/bigdata18/scripts/ws_submit.php

Papers must be submitted in PDF. Short papers (no more than 4 pages in standard two-column IEEE conference format or 6 pages in LNCS format, not including references) are encouraged, and the authors can extend their short paper to 12 pages in LNCS format, not including references, in the final publication. If a full paper is submitted, the page limit is 8 pages in standard two-column IEEE conference format or 12 pages in LNCS format, not including references. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, revised papers will be published by Springer LNCS (www.springer.com/lncs, indexed by EI). Please note that LNCS format is the final one for publishing.

Gallery image 1

LNCS Copyright Form for authors:

Consent to Publish

Formatting Instructions

IEEE conference format:

8.5" x 11" (DOC, PDF)

PDF

LaTex Formatting Macros

LNCS format: LaTex Template

Please make sure your submission satisfies ALL of the following requirements:

  • Paper must be submitted in printable PDF format.
  • Please number the pages of your submission.
  • The submission must be formatted for black-and-white printers. Please make sure your figures are readable when printed in black-and-white.
  • The submission must describe unpublished work that is not currently under review of any other conference or journal venues.

Call for Tutorial#

Tutorial proposals are solicited for Bench 2018, which will take place in Seattle, WA. Tutorials will be held on Dec 10-12. Proposals for both half- and full-day tutorials are solicited on any topic that is relevant to the Bench 2018 audience. In our experiences, tutorials seeking to achieve either of the following goals have been particularly successful:

  • Describe an important piece of research infrastructure.
  • Educate the community on an emerging topic.

Submission Procedures

Proposals should provide the following information:

  • Title.
  • Presenter(s) and contact information.
  • Proposed duration (full day, half day).
  • 1-2 paragraph abstract suitable for tutorial publicity.
  • 1 paragraph biography per presenter suitable for tutorial publicity.
  • 1-3 page description (for evaluation). This should include:
  1. Tutorial scope and objectives,
  2. Topics to be covered,
  3. Target audience,
  4. If the tutorial has been held previously, the location (i.e., conference), date, and number of attendees.

Proposals should be submitted in PDF format via e-mail to Lei Wang (wangle_2011@ict.ac.cn ) and Chen Zheng (zhengchen@ict.ac.cn) with the subject "Bench’18 Tutorial Proposal". Submissions will be acknowledged via e-mail.

Important Dates

Submission deadline: Oct 6, 2018 Notification: October 24th, 2018

Organization#

Program Chair#

  • Prof. Jianfeng Zhan, ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences

Program Committee#

  • Lizy K. John, The University of Texas at Austin USA
  • Vijay Janapa Reddi, University of Texas at Austin USA and Google
  • Wenguang Chen, Tsinghua University China
  • Woongki Baek, UNIST Korea
  • Yunji Chen, ICT, CAS China
  • Piotr Luszczek, University of Tennessee USA
  • Rui Hou, Institute of Information Engineering, CAS China
  • Yueguo Chen, Renmin University of China
  • H. Peter Hofstee, IBM USA
  • Cheqing Jin, East China Normal University China
  • Trevor E. Carlson, National University of Singapore (NUS) Singapore Junzhao Du, Xidian University China
  • Hyogi Sim, Oak Ridge National Laboratory USA
  • Weining Qian, East China Normal University China
  • Zheng Cao, Alibaba China
  • Matthias Nicola, IBM Germany
  • Zhihui Du, Tsinghua University China
  • Chunjie Luo, ICT, CAS
  • Bin Ren, College of William and Mary USA
  • Hua Chen, ICT, CAS China
  • Jiaquan Gao, Nanjing Normal University China
  • Jiannan Ouyang, Facebook USA
  • Xueming Si, Fudan University China
  • Yanjun Wu, Institute of Software Chinese Academy of Sciences China
  • Lucas Mello Schnorr, Federal University of Rio Grande do Sul (UFRGS) Brazil
  • Yunquan Zhang, ICT, CAS China
  • Shengzhong Feng, Shenzhen Institutes of Advanced Technology, CAS China
  • Zhen Jia, Princeton University USA
  • Zhibin Yu, Shenzhen Institutes of Advanced Technology, CAS China
  • Dapeng Wang, Institute of Software Chinese Academy of Sciences China
  • Bo Wu, Colorado School of Mines USA
  • Shaoliang Peng, National University of Defense Technology China
  • Yong Qi, Xi'an jiaotong University China
  • Gwangsun Kim, Arm Inc., Austin, Texas USA
  • Rui Ren, ICT, CAS China
  • Zujie Ren, Hangzhou Dianzi University China
  • Xiaoyi Lu, The Ohio State University USA
  • Lei Wang, ICT, CAS China
  • Jianhui Li, Computer Network Information Center, CAS China
  • Jianwu Wang, University of Maryland, Baltimore County USA
  • Kenli Li, Hunan University China
  • Suzanne Rivoire, Sonoma State University USA Rui Han, ICT, CAS China
  • Jungang Xu, Chinese academy of sciences China
  • Xu Liu, College of William and Mary USA
  • Tong Wu, National Institute of Metrology, China
  • Li ZHA, University of the Chinese Academy of Sciences
  • Lei Liu, ICT, CAS China
  • Chen Zheng, ICT, CAS China

Tutorial Co-Chairs#

  • Dr. Lei Wang ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
  • Dr. Chen Zheng ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences

Publication Chair#

  • Prof. Xiaohui Hu Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences

Web Chairs#

  • Mr. Fei Tang University of Chinese Academy of Sciences

Publicity Chairs#

  • Miss. Wanling Gao, University of Chinese Academy of Sciences
  • Miss. Rui Ren, ICT, Chinese Academy of Sciences
  • Dr. Zhen Jia, Princeton University

Submission Co-Chairs#

  • Miss. Rui Wang Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
  • Miss. Ruiying Li Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences

Sponsorship Chair#

  • Dr. Biwei Xie ICT, Chinese Academy of Sciences

Sponsors#

Gallery image 1
Gallery image 2

Best Paper Awards#

Gallery image 1

Previous Events#

WorkshopDatesLocation
BPOE-1October 7, 2013IEEE Big Data Conference, San Jose, CA
BPOE-2October 31, 2013CCF HPC China, Guilin, China
BPOE-3December 5,2013CCF Big Data Technology Conference 2013, BeiJing, China
BPOE-4March 1, 2014ASPLOS 2014, Salt Lake City, Utah, USA
BPOE-5September 5, 2014VLDB 2014, Hangzhou, Zhejiang Province, China
BPOE-6Spetember 4th, 2015VLDB 2015, Hilton, Waikoloa hotel·Kohala Coast, Hawai’i
BPOE-7April 3, 2016ASPLOS 2016, Atlanta, Georgia, USA
BPOE-8April 9, 2017ASPLOS 2017, Xi’an, China
BPOE-9March 25, 2018ASPLOS 2018, Williamsburg, VA, USA
SDBA-1June 12, 2018ICS 2018, Beijing, China

  1. J. Zhan, R. Han, C. Weng. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 8807, 2014.

  2. J. Zhan, R. Han, R. V. Zicari. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 9495, 2016.

Location#

Seattle, WA

Maintenance

Created: 2026-07-15Updated: 2026-07-15Details Presentation Updated: 2026-07-15Details Presentation Created: 2026-07-15
Previous EventBench 2019 | 2019 BenchCouncil International Symposium on Benchmarking, Measuring and OptimizingNext EventBPOE-9: The Ninth Workshop On Big Data BenchmarksPerformance, Optimization and Emerging Hardware