Friday, June 9, 2017

All about Linkbench

Linkbench is a benchmark for social graph transaction processing. It lead to a paper, a post, benchmark client in Java and many blog posts from me. As a bonus, I also got to know the person who did the work for it - thanks Tim. The benchmark client has moved to my Github account because the upstream project has been archived (no comment). The repo is not active but someone recently added support for Postgres, someone else added support for multiple schemas and I fixed it to work with MySQL 8. Percona forked it to add support for MongoDB but that really needs proper support for multi-document transactions in MongoDB.

I run Linkbench via helper scripts. These include files to create tables for MyRocks and InnoDB. There are two phases Linkbench: load and transaction. My helper scripts run the load phase and then transactions. From past experience results become stable after a few hours of transactions so I run it for 24 hours in 1-hour loops and compute metrics per hour to understand whether performance and efficiency change over time. My helper script all.sh uses load.sh and run.sh to do that while also computing hardware efficiency metrics. A typical command line for a small database is below. The last argument is the number of clients to concurrent load link and count tables while the load of the node table is always done by 1 thread So when the last argument is 1 there will be 2 concurrent clients for most of the load (1 for link/count and 1 for node). Note that the load of the link and count tables can finish before the load of the node table.
bash all.sh rx ~/bin/mysql /data/m/data 2000001 sdb 1 3600 mysql lb.sql.inno 24 127.0.0.1 1
Load phase

The load.sh script creates a result summary file with the name l.r.$something and an example is here. I extract the most interesting numbers from that file into a one-line summary that I archive. An example summary line is listed below. The legend for the results is:
  • ips - average insert rate for the link table. This is less than the real insert rate during the test because it doesn't count inserts to the count and node tables. Don't forget this when looking at the results below that are divided by ips.
  • r/i, rkb/i, wkb/i - iostat r/s, iostat rKB/s and iostat wKB/s divided by ips.
  • Mcpu/i - vmstat us + sy columns divided by ips and then multiplied by a constant. Note that this value can only be compared between servers using the same CPU.
  • size - database size in GB at the end of the load.
  • rss - mysqld RSS size in GB at the end of the load.
  • wMB/s - average of iostat wMB/s during the load. I really need to update this script to compute r/s and rMB/s.
  • cpu - average of vmstat us + sy columns during the test
  • engine - notes on the storage engine and configuration

ips    r/i    rkb/i  wkb/i  Mcpu/i  size   rss    wMB/s  cpu    engine
29810  0      0      1.33   1311    2.8g   0.17   39.6   39.1   MyRocks

Transaction phase

The run.sh script runs transaction phases in a loop for a fixed amount of time per loop and creates results summary files with the name r.r.$something.L$n.P$x where $n is the loop number. When it runs with 16 clients for 24 1-hour loops then there will be files like r.r.$something.L1.P16, r.r.$something.L2.P16, etc. An example is here.

I extract interesting results from that file to create a one line summary as shown below. The legend for the results is:
  • tps - average transaction rate.
  • r/t, rkb/t, wkb/t - iostat r/s, iostat rKB/s and iotat wKB/s divided by tps.
  • Mcpu/t - vmstat us and sy columns divided by tps and then multipled by a constant. Note that this value can only be compared between servers using the same CPU.
  • size - database size in GB at test end.
  • rss - mysqld RSS size in GB at test end.
  • un, gn, ul, gl - 99th percentile response time in milliseconds for the most frequent transactions: un is UPDATE_NODE, gn is GET_NODE, ul is UPDATE_LINK, gl is GET_LINKS_LIST. The types of transactions are explained in the paper and implemented for MySQL in LinkStoreMysql.java.
  • r/s, rMB/s, wMB/s - average for iostat r/s, rMB/s and wMB/s during the test.
  • cpu - average of vmstat us + sy columns during the test
  • engine - notes on the storage engine and configuration

tps   r/t   rkb/t wkb/t Mcpu/t  size  rss   un    gn   ul   gl   r/s   rMB/s wMB/s cpu   engine
3868  0      0    1.41  8154    3.0g  4.04  0.2   0.1  0.6  0.5  10.4  0     5.4   31.5  MyRocks

No comments:

Post a Comment