Replaced dd w/ fio; Added fio/iperf3 binaries; Added 32-bit and IPv6 support

This commit is contained in:
Mason Rowe 2020-02-04 21:43:28 -05:00
parent a6ca7214aa
commit 2c74b1a9b6
12 changed files with 337 additions and 144 deletions

View File

@ -10,104 +10,113 @@ This script isn't an attempt to be a golden standard. It's just yet another benc
`curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash`
This script has been tested on CentOS 7, CentOS 8, Debian 9, Debian 10, Fedora 30, Ubuntu 16.04, and Ubuntu 18.04. It is designed to not require any external dependencies to be installed nor elevated privileges.
This script has been tested on CentOS 7, CentOS 8, Debian 9, Debian 10, Fedora 30, Ubuntu 16.04, and Ubuntu 18.04. It is designed to not require any external dependencies to be installed nor elevated privileges to run.
*IPv6-Only Machines*: The above command will not work on IPv6-only machines. [See below](#ipv6-only-machines)
### Skipping Tests
By default, the script runs all three tests described in the next section below. In the event that you wish to skip one or more of the tests, use the commands below:
```
curl https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh -o yabs.sh; chmod +x yabs.sh
./yabs.sh -{dig}
curl https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash -s -- -{fig}
```
* `-d` this option disables the dd (disk performance) test
* `-f`/`-d` this option disables the fio (disk performance) test
* `-i` this option disables the iperf (network performance) test
* `-g` this option disables the Geekbench (system performance) test
Options can be grouped together to skip multiple tests, i.e. `./yabs -dg` to skip the disk and system performance tests (effectively only testing network performance).
Options can be grouped together to skip multiple tests, i.e. `-fg` to skip the disk and system performance tests (effectively only testing network performance).
## Tests Conducted
* **dd** - the dd utility is utilized to test sequential write and read disk performance.
* **fio** - the most comprehensive I/O testing software available, fio grants the ability to evaluate disk performance in a variety of methods with a variety of options. Four random read and write fio disk tests are conducted as part of this script with 4k, 64k, 512k, and 1m block sizes. The tests are designed to evaluate disk throughput in near-real world (using random) scenarios with a 50/50 split (50% reads and 50% writes per test).
* **iperf3** - the industry standard for testing download and upload speeds to various locations. This script utilizes iperf3 with 8 parallel threads and tests both download and upload speeds. If an iperf server is busy after 10 tries, the speed test for that location/direction is skipped.
* **Geekbench 4** - Geekbench is a benchmarking program that measures system performance, which is widely used in the tech community. The web URL is displayed to be able to see complete test and individual benchmark results and allow comparison to other geekbench'd systems. The claim URL to add the Geekbench 4 result to your Geekbench profile is written to a file in the directory that this script is executed from.
### Note on Disk Performance Test
This script uses dd's sequential throughput test in order to test both write and read speeds to the disk. It is well known that sequential disk speeds are not necessarily indicative of actual, real-world performance. A superior method to testing disk performance using real-world scenarios is [fio (Flexible I/O Tester)](https://github.com/axboe/fio). Fio was not utilized within this script because the program needs to be compiled and installed on the user's system to run correctly, thus clashes with YABS' tenet to not require any dependencies (installed or compiled) or admin rights to run the script. The sequential dd tests are merely provided as a convienence for the end user.
### Security Notice
This script relies on two external binaries in order to complete the network and system performance tests. For the network test, an iperf3 binary and shared library are downloaded from the official source at iperf.fr. For the system test, a Geekbench 4 tarball is downloaded, extracted, and the resulting binary is run. The security risks of running these binaries are minimal, however, use this script at your own risk as you would with any script publicly available on the net.
This script relies on external binaries in order to complete the performance tests. The network (iperf3) and disk (fio) tests use binaries that are compiled by myself utilizing a [Holy Build Box](https://github.com/phusion/holy-build-box) compiliation environment to ensure binary portability. The reasons for doing this include ensuring standardized (parsable) output, allowing support of both 32-bit and 64-bit architectures, bypassing the need for prerequisites to be compiled and/or installed, among other reasons. For the system test, a Geekbench 4 tarball is downloaded, extracted, and the resulting binary is run. Use this script at your own risk as you would with any script publicly available on the net. Additional information regarding the binaries, including compilation notes and steps, can be found in the bin directory's [README page](bin/README.md).
## Example Output
```
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-01-08 #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Wed 08 Jan 2020 07:33:21 PM UTC
Tue Feb 4 19:04:24 UTC 2020
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
CPU cores : 8 @ 4098.759 MHz
CPU cores : 8 @ 800.098 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 31Gi
RAM : 31G
Swap : 0B
Disk : 221G
dd Sequential Disk Speed Tests:
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
| Test 1 | Test 2 | Test 3 | Avg
| | | |
Write | 291 MB/s | 286 MB/s | 281 MB/s | 286.00 MB/s
Read | 179 MB/s | 188 MB/s | 179 MB/s | 182.00 MB/s
Block Size | 4kb (IOPS) | 64kb (IOPS)
------ | --- ---- | ---- ----
Read | 69.37 MB/s (17.3k) | 106.51 MB/s (1.6k)
Write | 69.57 MB/s (17.3k) | 107.07 MB/s (1.6k)
Total | 138.94 MB/s (34.7k) | 213.59 MB/s (3.3k)
| |
Block Size | 512kb (IOPS) | 1mb (IOPS)
------ | ----- ---- | --- ----
Read | 133.52 MB/s (260) | 141.90 MB/s (138)
Write | 140.61 MB/s (274) | 151.35 MB/s (147)
Total | 274.13 MB/s (534) | 293.26 MB/s (285)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 2.93 Gbits/sec | 7.80 Gbits/sec
Online.net | Paris, FR (10G) | 7.79 Gbits/sec | 5.20 Gbits/sec
Severius | The Netherlands (10G) | 8.98 Gbits/sec | 2.53 Gbits/sec
Worldstream | The Netherlands (10G) | 8.65 Gbits/sec | 8.57 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 7.80 Gbits/sec | 9.03 Gbits/sec
Biznet | Bogor, Indonesia (1G) | 752 Mbits/sec | busy
Hostkey | Moscow, RU (1G) | 905 Mbits/sec | 449 Mbits/sec
Vultr | Piscataway, NJ, US (1G) | 448 Mbits/sec | 51.6 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 1.74 Gbits/sec | 1.61 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 1.61 Gbits/sec | 106 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 28.2 Mbits/sec | 476 Mbits/sec
Bouygues Telecom | Paris, FR (10G) | 1.41 Gbits/sec | 1.13 Gbits/sec
Online.net | Paris, FR (10G) | 1.44 Gbits/sec | 1.29 Gbits/sec
Worldstream | The Netherlands (10G) | 1.18 Gbits/sec | 1.22 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 805 Mbits/sec | 1.13 Gbits/sec
Biznet | Bogor, Indonesia (1G) | 768 Mbits/sec | 38.5 Mbits/sec
Hostkey | Moscow, RU (1G) | 503 Mbits/sec | 686 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 2.74 Gbits/sec | 2.67 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 3.36 Gbits/sec | 963 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 6.34 Gbits/sec | 3.76 Gbits/sec
iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 7.78 Gbits/sec | 6.00 Gbits/sec
Online.net | Paris, FR (10G) | 2.86 Gbits/sec | 5.74 Gbits/sec
Severius | The Netherlands (10G) | 6.96 Gbits/sec | 2.38 Gbits/sec
Worldstream | The Netherlands (10G) | 7.29 Gbits/sec | 6.02 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 4.64 Gbits/sec | 8.93 Gbits/sec
Vultr | Piscataway, NJ, US (1G) | 97.5 Mbit/sec | 37.3 Mbits/sec
Bouygues Telecom | Paris, FR (10G) | 1.44 Gbits/sec | 1.25 Gbits/sec
Online.net | Paris, FR (10G) | 1.36 Gbits/sec | 972 Mbits/sec
Worldstream | The Netherlands (10G) | 1.19 Gbits/sec | 1.20 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 826 Mbits/sec | 1.14 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | busy | busy
Hurricane Electric | Fremont, CA, US (10G) | 348 Mbits/sec | 505 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 6.36 Gbits/sec | 2.95 Gbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 5714
Multi Core | 19758
Full Test | https://browser.geekbench.com/v4/cpu/15115430
Single Core | 5587
Multi Core | 19093
Full Test | https://browser.geekbench.com/v4/cpu/15200550
```
## IPv6 Only Machines
GitHub's CDN does not resolve via IPv6. You will need to run the following command to download and run the script.
`curl -s -k -g --header 'Host: raw.githubusercontent.com' https://[2a04:4e42::133]/masonr/yet-another-bench-script/master/yabs.sh | bash`
(2a04:4e42::133 is fastly.net's [GitHub's CDN Provider] IPv6 address)
## Acknoledgements
This script was inspired by several great benchmarking scripts out there, including, but not limited to, [bench.sh](https://bench.sh/), [nench.sh](https://github.com/n-st/nench), [ServerBench](https://github.com/K4Y5/ServerBench), among others. Members of both the [HostBalls](https://hostballs.com) and [LowEndTalk](https://www.lowendtalk.com) hosting-related communities play a pivotal role in testing, evaluating, and shaping this script as it matures.

View File

@ -1,42 +1,29 @@
## YABS Static Binaries
## YABS Pre-Compiled Binaries
This directory contains all of the static binaries required to run the benchmarking tests. Naturally, there is a security risk to your machine and its contents by running this script since, after all, this is just a script on the internet. You'll simply have to have confidence that I don't have malicious intent and am semi-competent at writing a bash script. The script is made public so you can look at the code yourself. The static binaries were compiled on local Debian 7 VMs. The compiled binary version numbers and compilations steps are noted below. Please open an issue if the compiled version is out of date and lacking any security-related and/or performance updates.
This directory contains all of the binaries required to run the benchmarking tests. Naturally, there is a security risk to your machine and its contents by running this script since, after all, this is just a script on the internet. You'll simply have to have confidence that I don't have malicious intent and am semi-competent at writing a bash script. The script is made public so you can look at the code yourself. The static binaries were compiled using a [Holy Build Box](https://github.com/phusion/holy-build-box) compilation environment in order to ensure the most portability. The compiled binary version numbers and compilations steps are noted below. Please open an issue if the compiled version is out of date and lacking any security-related and/or performance updates.
### Static Binaries
### Binaries
* **fio_x64.static** - v3.17-66-gb7ed (compiled 13 Jan 2020) - 64-bit version
* **fio_x86.static** - v3.17-67-g7eff0 (compiled 13 Jan 2020) - 32-bit version
* **iperf_x64.static** - v3.7+ (compiled 13 Jan 2020) - 64-bit version
* **iperf_x86.static** - v3.7+ (compiled 13 Jan 2020) - 32-bit version
* **fio_x64** - v3.17 (compiled 28 Jan 2020) - 64-bit version
* **fio_x86** - v3.17 (compiled 28 Jan 2020) - 32-bit version
* **iperf_x64** - v3.7 (compiled 28 Jan 2020) - 64-bit version
* **iperf_x86** - v3.7 (compiled 28 Jan 2020) - 32-bit version
### Compile Notes
**Pre-reqs**:
* Docker - https://www.docker.com/
```sh
apt install build-essential git libc6-dev libaio-dev zlib1g-dev libssl-dev
**Compiling 64-bit binaries**:
```
docker run -t -i --rm -v `pwd`:/io phusion/holy-build-box-64:latest bash /io/compile.sh
```
_(or equivalents in other package repos)_
**Compiling 32-bit binaries**:
**fio**:
```sh
git clone https://github.com/axboe/fio
cd fio
./configure --build-static
make
```
docker run -t -i --rm -v `pwd`:/io phusion/holy-build-box-32:latest linux32 bash /io/compile.sh
```
fio static binary will be in current dir
**iperf3**:
```sh
git clone https://github.com/esnet/iperf
cd iperf
./configure "LDFLAGS=--static" --disable-shared --diable-profiling
make
```
iperf3 static binary will be in the src dir
64-bit and 32-bit binaries will be placed in the current directory.

56
bin/compile.sh Executable file
View File

@ -0,0 +1,56 @@
#!/bin/bash
set -e
# Activate Holy Build Box lib compiliation environment
source /hbb/activate
set -x
yum install -y xz
# determine arch
ARCH=$(uname -m)
if [[ $ARCH = *x86_64* ]]; then
# container is 64-bit
ARCH="x64"
elif [[ $ARCH = *i?86* ]]; then
# container is 32-bit
ARCH="x86"
fi
# download, compile, and install libaio as static library
cd ~
curl -L http://ftp.de.debian.org/debian/pool/main/liba/libaio/libaio_0.3.112.orig.tar.xz -o "libaio.tar.xz"
tar xf libaio.tar.xz
cd libaio-*/src
ENABLE_SHARED=0 make prefix=/hbb_exe install
# Activate Holy Build Box exe compilation environment
source /hbb_exe/activate
# download and compile fio
cd ~
curl -L https://github.com/axboe/fio/archive/fio-3.17.tar.gz -o "fio.tar.gz"
tar xf fio.tar.gz
cd fio-fio*
./configure
make
# verify no external shared library links
libcheck fio
# copy fio binary to mounted dir
cp fio /io/fio_$ARCH
# download and compile iperf
cd ~
curl -L https://github.com/esnet/iperf/archive/3.7.tar.gz -o "iperf.tar.gz"
tar xf iperf.tar.gz
cd iperf*
./configure --disable-shared --disable-profiling
make
# verify no external shared library links
libcheck src/iperf3
# copy iperf binary to mounted dir
cp src/iperf3 /io/iperf3_$ARCH

BIN
bin/fio_x64 Executable file

Binary file not shown.

Binary file not shown.

BIN
bin/fio_x86 Executable file

Binary file not shown.

Binary file not shown.

BIN
bin/iperf3_x64 Executable file

Binary file not shown.

Binary file not shown.

BIN
bin/iperf3_x86 Executable file

Binary file not shown.

Binary file not shown.

289
yabs.sh
View File

@ -1,21 +1,21 @@
#!/bin/bash
# Yet Another Bench Script by Mason Rowe
# Initial Oct 2019; Updated Jan 2020
# Initial Oct 2019; Updated Feb 2020
#
# Disclaimer: This project is a work in progress. Any errors or suggestions should be
# relayed to me via the GitHub project page linked below.
#
# Purpose: The purpose of this script is to quickly gauge the performance of a Linux-
# based server by benchmarking network performance via iperf3, CPU and
# overall system performance via Geekbench 4, and simple sequential disk
# performance via dd. The script is designed to not require any dependencies
# overall system performance via Geekbench 4, and sequential + random disk
# performance via fio. The script is designed to not require any dependencies
# - either compiled or installed - nor admin privileges to run.
#
echo -e '# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #'
echo -e '# Yet-Another-Bench-Script #'
echo -e '# v2020-01-08 #'
echo -e '# v2020-02-04 #'
echo -e '# https://github.com/masonr/yet-another-bench-script #'
echo -e '# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #'
@ -25,6 +25,20 @@ date
# override locale to eliminate parsing errors (i.e. using commas a delimiters rather than periods)
export LC_ALL=C
# determine architecture of host
ARCH=$(uname -m)
if [[ $ARCH = *x86_64* ]]; then
# host is running a 64-bit kernel
ARCH="x64"
elif [[ $ARCH = *i?86* ]]; then
# host is running a 32-bit kernel
ARCH="x86"
else
# host is running a non-supported kernel
echo -e "Architecture not supported by YABS."
exit 1
fi
# gather basic system information (inc. CPU, AES-NI/virt status, RAM + swap + disk size)
echo -e
echo -e "Basic System Information:"
@ -62,89 +76,204 @@ rm $DATE.test
mkdir -p $YABS_PATH
# flags to skip certain performance tests
SKIP_DISK=""
SKIP_FIO=""
SKIP_IPERF=""
SKIP_GEEKBENCH=""
# get any arguments that were passed to the script and set the associated skip flags (if applicable)
while getopts 'dig' flag; do
while getopts 'fdig' flag; do
case "${flag}" in
d) SKIP_DISK="True" ;;
f) SKIP_FIO="True" ;;
d) SKIP_FIO="True" ;;
i) SKIP_IPERF="True" ;;
g) SKIP_GEEKBENCH="True" ;;
*) exit 1 ;;
esac
done
# test if the host has IPv4/IPv6 connectivity
IPV4_CHECK=$(curl -s -4 -m 4 icanhazip.com 2> /dev/null)
IPV6_CHECK=$(curl -s -6 -m 4 icanhazip.com 2> /dev/null)
# format_speed
# Purpose: This method is a convienence function to format the output of the fio disk tests which
# always returns a result in KB/s. If result is >= 1 GB/s, use GB/s. If result is < 1 GB/s
# and >= 1 MB/s, then use MB/s. Otherwise, use KB/s.
# Parameters:
# 1. RAW - the raw disk speed result (in KB/s)
# Returns:
# Formatted disk speed in GB/s, MB/s, or KB/s
function format_speed {
RAW=$1 # disk speed in KB/s
RESULT=$RAW
local DENOM=1
local UNIT="KB/s"
# ensure raw value is not null, if it is, return blank
if [ -z "$RAW" ]; then
echo ""
return 0
fi
# check if disk speed >= 1 GB/s
if [ "$RAW" -ge 1000000 ]; then
DENOM=1000000
UNIT="GB/s"
# check if disk speed < 1 GB/s && >= 1 MB/s
elif [ "$RAW" -ge 1000 ]; then
DENOM=1000
UNIT="MB/s"
fi
# divide the raw result to get the corresponding formatted result (based on determined unit)
RESULT=$(awk -v a="$RESULT" -v b="$DENOM" 'BEGIN { print a / b }')
# shorten the formatted result to two decimal places (i.e. x.xx)
RESULT=$(echo $RESULT | awk -F. '{ printf "%0.2f",$1"."substr($2,1,2) }')
# concat formatted result value with units and return result
RESULT="$RESULT $UNIT"
echo $RESULT
}
# format_iops
# Purpose: This method is a convienence function to format the output of the raw IOPS result
# Parameters:
# 1. RAW - the raw IOPS result
# Returns:
# Formatted IOPS (i.e. 8, 123, 1.7k, 275.9k, etc.)
function format_iops {
RAW=$1 # iops
RESULT=$RAW
# ensure raw value is not null, if it is, return blank
if [ -z "$RAW" ]; then
echo ""
return 0
fi
# check if IOPS speed > 1k
if [ "$RAW" -ge 1000 ]; then
# divide the raw result by 1k
RESULT=$(awk -v a="$RESULT" 'BEGIN { print a / 1000 }')
# shorten the formatted result to one decimal place (i.e. x.x)
RESULT=$(echo $RESULT | awk -F. '{ printf "%0.1f",$1"."substr($2,1,1) }')
RESULT="$RESULT"k
fi
echo $RESULT
}
# disk_test
# Purpose: This method is designed to test the disk performance of the host using the partition that the
# script is being run from using dd sequential speed tests and averaging the results.
# script is being run from using fio sequential and random read/write speed tests.
# Parameters:
# - (none)
function disk_test {
I=0
DISK_WRITE_TEST_RES=()
DISK_READ_TEST_RES=()
DISK_WRITE_TEST_AVG=0
DISK_READ_TEST_AVG=0
# run a quick test to generate the fio test file to be used by the actual tests
echo -en "Generating fio test file..."
$DISK_PATH/fio --name=setup --ioengine=libaio --rw=read --bs=4k --iodepth=64 --numjobs=2 --size=2G --runtime=1 --gtod_reduce=1 --filename=$DISK_PATH/test.fio --direct=1 --minimal &> /dev/null
echo -en "\r\033[0K"
# run the disk speed tests (write and read) thrice over
while [ $I -lt 3 ]
do
# write test using dd, "direct" flag is used to test direct I/O for data being stored to disk
DISK_WRITE_TEST=$(dd if=/dev/zero of=$DISK_PATH/$DATE.test bs=64k count=16k oflag=direct |& grep copied | awk '{ print $(NF-1) " " $(NF)}')
VAL=$(echo $DISK_WRITE_TEST | cut -d " " -f 1)
[[ "$DISK_WRITE_TEST" == *"GB"* ]] && VAL=$(awk -v a="$VAL" 'BEGIN { print a * 1000 }')
DISK_WRITE_TEST_RES+=( "$DISK_WRITE_TEST" )
DISK_WRITE_TEST_AVG=$(awk -v a="$DISK_WRITE_TEST_AVG" -v b="$VAL" 'BEGIN { print a + b }')
# read test using dd using the 1G file written during the write test
DISK_READ_TEST=$(dd if=$DISK_PATH/$DATE.test of=/dev/null bs=8k |& grep copied | awk '{ print $(NF-1) " " $(NF)}')
VAL=$(echo $DISK_READ_TEST | cut -d " " -f 1)
[[ "$DISK_READ_TEST" == *"GB"* ]] && VAL=$(awk -v a="$VAL" 'BEGIN { print a * 1000 }')
DISK_READ_TEST_RES+=( "$DISK_READ_TEST" )
DISK_READ_TEST_AVG=$(awk -v a="$DISK_READ_TEST_AVG" -v b="$VAL" 'BEGIN { print a + b }')
# run rand read/write mixed 4kb fio test
echo -en "Running fio random mixed read + write disk test with 4kb blocks..."
DISK_RW4_TEST=$(timeout 35 $DISK_PATH/fio --name=rand_rw_4k --ioengine=libaio --rw=randrw --rwmixread=50 --bs=4k --iodepth=64 --numjobs=2 --size=2G --runtime=30 --gtod_reduce=1 --direct=1 --filename=$DISK_PATH/test.fio --group_reporting --minimal 2> /dev/null | grep rand_rw_4k)
DISK_RW4_IOPS_R=$(echo $DISK_RW4_TEST | awk -F';' '{print $8}')
DISK_RW4_IOPS_W=$(echo $DISK_RW4_TEST | awk -F';' '{print $49}')
DISK_RW4_IOPS=$(format_iops $(awk -v a="$DISK_RW4_IOPS_R" -v b="$DISK_RW4_IOPS_W" 'BEGIN { print a + b }'))
DISK_RW4_IOPS_R=$(format_iops $DISK_RW4_IOPS_R)
DISK_RW4_IOPS_W=$(format_iops $DISK_RW4_IOPS_W)
DISK_RW4_TEST_R=$(echo $DISK_RW4_TEST | awk -F';' '{print $7}')
DISK_RW4_TEST_W=$(echo $DISK_RW4_TEST | awk -F';' '{print $48}')
DISK_RW4_TEST=$(format_speed $(awk -v a="$DISK_RW4_TEST_R" -v b="$DISK_RW4_TEST_W" 'BEGIN { print a + b }'))
DISK_RW4_TEST_R=$(format_speed $DISK_RW4_TEST_R)
DISK_RW4_TEST_W=$(format_speed $DISK_RW4_TEST_W)
echo -en "\r\033[0K"
I=$(( $I + 1 ))
done
# calculate the write and read speed averages using the results from the three runs
DISK_WRITE_TEST_AVG=$(awk -v a="$DISK_WRITE_TEST_AVG" 'BEGIN { print a / 3 }')
DISK_READ_TEST_AVG=$(awk -v a="$DISK_READ_TEST_AVG" 'BEGIN { print a / 3 }')
# run rand read/write mixed 64kb fio test
echo -en "Running fio random mixed read + write disk test with 64kb blocks..."
DISK_RW64_TEST=$(timeout 35 $DISK_PATH/fio --name=rand_rw_64k --ioengine=libaio --rw=randrw --rwmixread=50 --bs=64k --iodepth=64 --numjobs=2 --size=2G --runtime=30 --gtod_reduce=1 --direct=1 --filename=$DISK_PATH/test.fio --group_reporting --minimal 2> /dev/null | grep rand_rw_64k)
DISK_RW64_IOPS_R=$(echo $DISK_RW64_TEST | awk -F';' '{print $8}')
DISK_RW64_IOPS_W=$(echo $DISK_RW64_TEST | awk -F';' '{print $49}')
DISK_RW64_IOPS=$(format_iops $(awk -v a="$DISK_RW64_IOPS_R" -v b="$DISK_RW64_IOPS_W" 'BEGIN { print a + b }'))
DISK_RW64_IOPS_R=$(format_iops $DISK_RW64_IOPS_R)
DISK_RW64_IOPS_W=$(format_iops $DISK_RW64_IOPS_W)
DISK_RW64_TEST_R=$(echo $DISK_RW64_TEST | awk -F';' '{print $7}')
DISK_RW64_TEST_W=$(echo $DISK_RW64_TEST | awk -F';' '{print $48}')
DISK_RW64_TEST=$(format_speed $(awk -v a="$DISK_RW64_TEST_R" -v b="$DISK_RW64_TEST_W" 'BEGIN { print a + b }'))
DISK_RW64_TEST_R=$(format_speed $DISK_RW64_TEST_R)
DISK_RW64_TEST_W=$(format_speed $DISK_RW64_TEST_W)
echo -en "\r\033[0K"
# run rand read/write mixed 512kb fio test
echo -en "Running fio random mixed read + write disk test with 512kb blocks..."
DISK_RW512_TEST=$(timeout 35 $DISK_PATH/fio --name=rand_rw_512k --ioengine=libaio --rw=randrw --rwmixread=50 --bs=512k --iodepth=64 --numjobs=2 --size=2G --runtime=30 --gtod_reduce=1 --direct=1 --filename=$DISK_PATH/test.fio --group_reporting --minimal 2> /dev/null | grep rand_rw_512k)
DISK_RW512_IOPS_R=$(echo $DISK_RW512_TEST | awk -F';' '{print $8}')
DISK_RW512_IOPS_W=$(echo $DISK_RW512_TEST | awk -F';' '{print $49}')
DISK_RW512_IOPS=$(format_iops $(awk -v a="$DISK_RW512_IOPS_R" -v b="$DISK_RW512_IOPS_W" 'BEGIN { print a + b }'))
DISK_RW512_IOPS_R=$(format_iops $DISK_RW512_IOPS_R)
DISK_RW512_IOPS_W=$(format_iops $DISK_RW512_IOPS_W)
DISK_RW512_TEST_R=$(echo $DISK_RW512_TEST | awk -F';' '{print $7}')
DISK_RW512_TEST_W=$(echo $DISK_RW512_TEST | awk -F';' '{print $48}')
DISK_RW512_TEST=$(format_speed $(awk -v a="$DISK_RW512_TEST_R" -v b="$DISK_RW512_TEST_W" 'BEGIN { print a + b }'))
DISK_RW512_TEST_R=$(format_speed $DISK_RW512_TEST_R)
DISK_RW512_TEST_W=$(format_speed $DISK_RW512_TEST_W)
echo -en "\r\033[0K"
# run rand read/write mixed 1mb fio test
echo -en "Running fio random mixed read + write disk test with 1mb blocks..."
DISK_RW1M_TEST=$(timeout 35 $DISK_PATH/fio --name=rand_rw_1m --ioengine=libaio --rw=randrw --rwmixread=50 --bs=1m --iodepth=64 --numjobs=2 --size=2G --runtime=30 --gtod_reduce=1 --direct=1 --filename=$DISK_PATH/test.fio --group_reporting --minimal 2> /dev/null | grep rand_rw_1m)
DISK_RW1M_IOPS_R=$(echo $DISK_RW1M_TEST | awk -F';' '{print $8}')
DISK_RW1M_IOPS_W=$(echo $DISK_RW1M_TEST | awk -F';' '{print $49}')
DISK_RW1M_IOPS=$(format_iops $(awk -v a="$DISK_RW1M_IOPS_R" -v b="$DISK_RW1M_IOPS_W" 'BEGIN { print a + b }'))
DISK_RW1M_IOPS_R=$(format_iops $DISK_RW1M_IOPS_R)
DISK_RW1M_IOPS_W=$(format_iops $DISK_RW1M_IOPS_W)
DISK_RW1M_TEST_R=$(echo $DISK_RW1M_TEST | awk -F';' '{print $7}')
DISK_RW1M_TEST_W=$(echo $DISK_RW1M_TEST | awk -F';' '{print $48}')
DISK_RW1M_TEST=$(format_speed $(awk -v a="$DISK_RW1M_TEST_R" -v b="$DISK_RW1M_TEST_W" 'BEGIN { print a + b }'))
DISK_RW1M_TEST_R=$(format_speed $DISK_RW1M_TEST_R)
DISK_RW1M_TEST_W=$(format_speed $DISK_RW1M_TEST_W)
echo -en "\r\033[0K"
}
# if the skip disk flag was set, skip the disk performance test, otherwise test disk performance
if [ -z "$SKIP_DISK" ]; then
echo -en "\nPerforming disk performance test. This may take a couple minutes to complete..."
if [ -z "$SKIP_FIO" ]; then
echo -en "\nPreparing system for disk tests..."
# create temp directory to store disk write/read test files
DISK_PATH=$YABS_PATH/disk
mkdir -p $DISK_PATH
# download fio binary
if [ ! -z "$IPV4_CHECK" ]; then # if IPv4 is enabled
curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/bin/fio_$ARCH -o $DISK_PATH/fio
else # no IPv4, use IPv6 - below is necessary since raw.githubusercontent.com has no AAAA record
curl -s -k -g --header 'Host: raw.githubusercontent.com' https://[2a04:4e42::133]/masonr/yet-another-bench-script/master/bin/fio_$ARCH -o $DISK_PATH/fio
fi
chmod +x $DISK_PATH/fio
echo -en "\r\033[0K"
# execute disk performance test
disk_test
# format the speed averages by converting to GB/s if > 1000 MB/s
if [ $(echo $DISK_WRITE_TEST_AVG | cut -d "." -f 1) -ge 1000 ]; then
DISK_WRITE_TEST_AVG=$(awk -v a="$DISK_WRITE_TEST_AVG" 'BEGIN { print a / 1000 }')
DISK_WRITE_TEST_UNIT="GB/s"
else
DISK_WRITE_TEST_UNIT="MB/s"
if [ -z "$DISK_RW4_TEST" ]; then # fio was killed or returned an error
echo -e "fio disk speed tests failed. Run manually to determine cause."
else # fio tests completed sucessfully, print results
# print disk speed test results
echo -e "fio Disk Speed Tests (Mixed R/W 50/50):"
echo -e "---------------------------------"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Block Size" "4kb" "(IOPS)" "64kb" "(IOPS)"
printf "%-10s | %-11s %8s | %-11s %8s\n" " ------" "---" "---- " "----" "---- "
printf "%-10s | %-11s %8s | %-11s %8s\n" "Read" "$DISK_RW4_TEST_R" "($DISK_RW4_IOPS_R)" "$DISK_RW64_TEST_R" "($DISK_RW64_IOPS_R)"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Write" "$DISK_RW4_TEST_W" "($DISK_RW4_IOPS_W)" "$DISK_RW64_TEST_W" "($DISK_RW64_IOPS_W)"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Total" "$DISK_RW4_TEST" "($DISK_RW4_IOPS)" "$DISK_RW64_TEST" "($DISK_RW64_IOPS)"
printf "%-10s | %-20s | %-20s\n"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Block Size" "512kb" "(IOPS)" "1mb" "(IOPS)"
printf "%-10s | %-11s %8s | %-11s %8s\n" " ------" "-----" "---- " "---" "---- "
printf "%-10s | %-11s %8s | %-11s %8s\n" "Read" "$DISK_RW512_TEST_R" "($DISK_RW512_IOPS_R)" "$DISK_RW1M_TEST_R" "($DISK_RW1M_IOPS_R)"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Write" "$DISK_RW512_TEST_W" "($DISK_RW512_IOPS_W)" "$DISK_RW1M_TEST_W" "($DISK_RW1M_IOPS_W)"
printf "%-10s | %-11s %8s | %-11s %8s\n" "Total" "$DISK_RW512_TEST" "($DISK_RW512_IOPS)" "$DISK_RW1M_TEST" "($DISK_RW1M_IOPS)"
fi
if [ $(echo $DISK_READ_TEST_AVG | cut -d "." -f 1) -ge 1000 ]; then
DISK_READ_TEST_AVG=$(awk -v a="$DISK_READ_TEST_AVG" 'BEGIN { print a / 1000 }')
DISK_READ_TEST_UNIT="GB/s"
else
DISK_READ_TEST_UNIT="MB/s"
fi
# print dd sequential disk speed test results
echo -en "\r\033[0K"
echo -e "dd Sequential Disk Speed Tests:"
echo -e "---------------------------------"
printf "%-6s | %-6s %-4s | %-6s %-4s | %-6s %-4s | %-6s %-4s\n" "" "Test 1" "" "Test 2" "" "Test 3" "" "Avg" ""
printf "%-6s | %-6s %-4s | %-6s %-4s | %-6s %-4s | %-6s %-4s\n"
printf "%-6s | %-11s | %-11s | %-11s | %-6.2f %-4s\n" "Write" "${DISK_WRITE_TEST_RES[0]}" "${DISK_WRITE_TEST_RES[1]}" "${DISK_WRITE_TEST_RES[2]}" "${DISK_WRITE_TEST_AVG}" "${DISK_WRITE_TEST_UNIT}"
printf "%-6s | %-11s | %-11s | %-11s | %-6.2f %-4s\n" "Read" "${DISK_READ_TEST_RES[0]}" "${DISK_READ_TEST_RES[1]}" "${DISK_READ_TEST_RES[2]}" "${DISK_READ_TEST_AVG}" "${DISK_READ_TEST_UNIT}"
fi
# iperf_test
@ -173,7 +302,7 @@ function iperf_test {
# run the iperf test sending data from the host to the iperf server; includes
# a timeout of 15s in case the iperf server is not responding; uses 8 parallel
# threads for the network test
IPERF_RUN_SEND="$(LD_LIBRARY_PATH=$IPERF_PATH timeout 15 $IPERF_PATH/iperf3 $FLAGS -c $URL -p $PORT -P 8)"
IPERF_RUN_SEND="$(timeout 15 $IPERF_PATH/iperf3 $FLAGS -c $URL -p $PORT -P 8 2> /dev/null)"
# check if iperf exited cleanly and did not return an error
if [[ "$IPERF_RUN_SEND" == *"receiver"* && "$IPERF_RUN_SEND" != *"error"* ]]; then
# test did not result in an error, parse speed result
@ -201,7 +330,7 @@ function iperf_test {
# run the iperf test recieving data from the iperf server to the host; includes
# a timeout of 15s in case the iperf server is not responding; uses 8 parallel
# threads for the network test
IPERF_RUN_RECV="$(LD_LIBRARY_PATH=$IPERF_PATH timeout 15 $IPERF_PATH/iperf3 $FLAGS -c $URL -p $PORT -P 8 -R)"
IPERF_RUN_RECV="$(timeout 15 $IPERF_PATH/iperf3 $FLAGS -c $URL -p $PORT -P 8 -R 2> /dev/null)"
# check if iperf exited cleanly and did not return an error
if [[ "$IPERF_RUN_RECV" == *"receiver"* && "$IPERF_RUN_RECV" != *"error"* ]]; then
# test did not result in an error, parse speed result
@ -248,8 +377,8 @@ function launch_iperf {
IPERF_RECVRESULT_VAL=$(echo $IPERF_RECVRESULT | awk '{ print $6 }')
IPERF_RECVRESULT_UNIT=$(echo $IPERF_RECVRESULT | awk '{ print $7 }')
# if the results are blank, then the server is "busy" and being overutilized
[[ -z $IPERF_SENDRESULT_VAL || "$IPERF_SENDRESULT_VAL" == *"0.00"* ]] && IPERF_SENDRESULT_VAL="busy"
[[ -z $IPERF_RECVRESULT_VAL || "$IPERF_RECVRESULT_VAL" == *"0.00"* ]] && IPERF_RECVRESULT_VAL="busy"
[[ -z $IPERF_SENDRESULT_VAL || "$IPERF_SENDRESULT_VAL" == *"0.00"* ]] && IPERF_SENDRESULT_VAL="busy" && IPERF_SENDRESULT_UNIT=""
[[ -z $IPERF_RECVRESULT_VAL || "$IPERF_RECVRESULT_VAL" == *"0.00"* ]] && IPERF_RECVRESULT_VAL="busy" && IPERF_RECVRESULT_UNIT=""
# print the speed results for the iperf location currently being evaluated
printf "%-25s | %-25s | %-15s | %-15s\n" "${IPERF_LOCS[i*5+2]}" "${IPERF_LOCS[i*5+3]}" "$IPERF_SENDRESULT_VAL $IPERF_SENDRESULT_UNIT" "$IPERF_RECVRESULT_VAL $IPERF_RECVRESULT_UNIT"
fi
@ -258,18 +387,20 @@ function launch_iperf {
# if the skip iperf flag was set, skip the network performance test, otherwise test network performance
if [ -z "$SKIP_IPERF" ]; then
# create a temp directory to house the required iperf binary and library
IPERF_PATH=$YABS_PATH/iperf
mkdir -p $IPERF_PATH
# download the publicly available iperf files from project's official source (iperf.fr)
curl -s -o $IPERF_PATH/libiperf.so.0 https://iperf.fr/download/ubuntu/libiperf.so.0_3.1.3 > /dev/null
curl -s -o $IPERF_PATH/iperf3 https://iperf.fr/download/ubuntu/iperf3_3.1.3 > /dev/null
# download iperf3 binary
if [ ! -z "$IPV4_CHECK" ]; then # if IPv4 is enabled
curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/bin/iperf3_$ARCH -o $IPERF_PATH/iperf3
else # no IPv4, use IPv6 - below is necessary since raw.githubusercontent.com has no AAAA record
curl -s -k -g --header 'Host: raw.githubusercontent.com' https://[2a04:4e42::133]/masonr/yet-another-bench-script/master/bin/iperf3_$ARCH -o $IPERF_PATH/iperf3
fi
chmod +x $IPERF_PATH/iperf3
# test if the host has IPv4/IPv6 connectivity
IPV4_CHECK=$(curl -s -4 -m 4 icanhazip.com 2> /dev/null)
IPV6_CHECK=$(curl -s -6 -m 4 icanhazip.com 2> /dev/null)
# array containing all currently available iperf3 public servers to use for the network test
# format: "1" "2" "3" "4" "5" \
# 1. domain name of the iperf server
@ -281,12 +412,10 @@ if [ -z "$SKIP_IPERF" ]; then
"bouygues.iperf.fr" "5200-5209" "Bouygues Telecom" "Paris, FR (10G)" "IPv4|IPv6" \
"ping.online.net" "5200-5209" "Online.net" "Paris, FR (10G)" "IPv4" \
"ping6.online.net" "5200-5209" "Online.net" "Paris, FR (10G)" "IPv6" \
"speedtest.serverius.net" "5002-5002" "Severius" "The Netherlands (10G)" "IPv4|IPv6" \
"iperf.worldstream.nl" "5201-5201" "Worldstream" "The Netherlands (10G)" "IPv4|IPv6" \
"speedtest.wtnet.de" "5200-5209" "wilhelm.tel" "Hamburg, DE (10G)" "IPv4|IPv6" \
"iperf.biznetnetworks.com" "5201-5203" "Biznet" "Bogor, Indonesia (1G)" "IPv4" \
"speedtest.hostkey.ru" "5200-5203" "Hostkey" "Moscow, RU (1G)" "IPv4" \
"mirror.square-r00t.net" "5201-5201" "Vultr" "Piscataway, NJ, US (1G)" "IPv4|IPv6" \
"iperf3.velocityonline.net" "5201-5210" "Velocity Online" "Tallahassee, FL, US (10G)" "IPv4" \
"iperf.airstreamcomm.net" "5201-5205" "Airstream Communications" "Eau Claire, WI, US (10G)" "IPv4|IPv6" \
"iperf.he.net" "5201-5201" "Hurricane Electric" "Fremont, CA, US (10G)" "IPv4|IPv6" \
@ -309,15 +438,27 @@ if [ -z "$SKIP_GEEKBENCH" ]; then
# create a temp directory to house all geekbench files
GEEKBENCH_PATH=$YABS_PATH/geekbench
mkdir -p $GEEKBENCH_PATH
# download the latest Geekbench 4 tarball and extract to geekbench temp directory
curl -s http://cdn.geekbench.com/Geekbench-4.3.3-Linux.tar.gz | tar xz --strip-components=1 -C $GEEKBENCH_PATH &>/dev/null
# run the Geekbench 4 test and grep the test results URL given at the end of the test
GEEKBENCH_TEST=$($GEEKBENCH_PATH/geekbench4 2>/dev/null | grep "https://browser")
if [[ "$ARCH" == *"x86"* ]]; then
# run the Geekbench 4 test and grep the test results URL given at the end of the test
GEEKBENCH_TEST=$($GEEKBENCH_PATH/geekbench_x86_32 2>/dev/null | grep "https://browser")
else
# run the Geekbench 4 test and grep the test results URL given at the end of the test
GEEKBENCH_TEST=$($GEEKBENCH_PATH/geekbench4 2>/dev/null | grep "https://browser")
fi
# ensure the test ran successfully
if [ -z "$GEEKBENCH_TEST" ]; then
# if the Geekbench 4 test failed for any reason, exit cleanly and print error message
echo -e "\r\033[0KGeekbench 4 test failed. Run manually to determine cause."
if [ -z "$IPV4_CHECK" ]; then
# Geekbench 4 test failed to download because host lacks IPv4 (cdn.geekbench.com = IPv4 only)
echo -e "\r\033[0KGeekbench releases can only be downloaded over IPv4. FTP the Geekbench files and run manually."
else
# if the Geekbench 4 test failed for any reason, exit cleanly and print error message
echo -e "\r\033[0KGeekbench 4 test failed. Run manually to determine cause."
fi
else
# if the Geekbench 4 test succeeded, parse the test results URL
GEEKBENCH_URL=$(echo -e $GEEKBENCH_TEST | head -1)